how to get specific nested-prone block content using regex

Question

I want to extract the infobox block from Wikipedia. Below is a sample input file:

{{some text}}
some other text
{{Infobox President
birth|d/m/y
other_inner_text:{{may contain curly bracket}}
other text}}
some other text
or even another infobox
{{Infobox Cabinet
same structure
{{text}}also can contain {{}}
}}
can be some other text...

I want the parsing result to return the two Infobox blocks:

{{Infobox President
birth|d/m/y
other_inner_text:{{may contain curly bracket}}
other text
}}

and

{{Infobox Cabinet
same structure
{{text}}also can contain {{}}
}}

Any one know how to use regular expression in python to achieve this?

The requirement to have nested structures makes regular expressions the wrong tool for this job. — Pencho Ilchev, May 20 '12 at 21:58
this might help - http://stackoverflow.com/q/10447423/1176601 — Aprillion, May 20 '12 at 22:13

georg · Answer 1 · 2012-05-20T23:31:14.530

To match nested structures, some regexp dialects provide recursive patterns like (?R). The (?R) thing basically says "something that this expression matches".

Standard python re doesn't support this, but the newer regex module, which eventually will replace re, does. Here's a complete example.

text = """
{{some text}}
some other text
{{Infobox President
birth|d/m/y
other_inner_text:{{may contain {curly} bracket}}
other text}}
some other text
or even another infobox
{{Infobox Cabinet
same structure
{{text}}also can contain {{}}
}}
can be some other text...
"""

import regex

rx = r"""
{{                    # open
(                     # this match
    (?:               # contains...
        [^{}]         # no brackets
        |             # or
        }[^}]         # single close bracket
        |             # or
        {[^{]         # single open bracket
        |             # or
        (?R)          # the whole expression once again <-- recursion!
    )*                # zero or more times
)                     # end of match
}}                    # close
"""

rx = regex.compile(rx, regex.X | regex.S)

for p in rx.findall(text):
    print 'FOUND: (((', p, ')))'

Result:

FOUND: ((( some text )))
FOUND: ((( Infobox President
birth|d/m/y
other_inner_text:{{may contain {curly} bracket}}
other text )))
FOUND: ((( Infobox Cabinet
same structure
{{text}}also can contain {{}}
)))

For a great explanation of recursive regexps see this blog entry.

enter image description here

(couldn't resist stealing this one).

That said, you'd be probably better off with a parser-based solution. See for example parsing nested expressions with pyparsing.

score 1 · Answer 2 · edited May 23 '17 at 10:09

1

It's not python, but this answer may help you. It even includes a (not quick but dirty) regex that can handle one-level nested templates.

The general answer is no, regexes can't parse nested structures. See the linked answer for how to obtain a parsetree from a mediawiki api.

edited May 23 '17 at 10:09

Community

1
1

answered May 20 '12 at 21:58

Bergi

630,263
148
957
1,375

score 1 · Accepted Answer · answered May 20 '12 at 22:19

1

Regex

{{Infobox(?:(?!}}|{{).)*(?:{{(?:(?!}}|{{).)*}}(?:(?!}}|{{).)*)*.*?}}

And my try at Perl which I'm not fluent at

while ($subject =~ m/\{\{Infobox(?:(?!\}\}|\{\{).)*(?:\{\{(?:(?!\}\}|\{\{).)*\}\}(?:(?!\}\}|\{\{).)*)*.*?\}\}/sg) {
    # matched text = $&
}

It will work on an unlimited pair of "{{ some text }}" as long as they are balanced. It does not support nested text of that pair but it wasn't required.

Note that it's maybe better to look for an alternative solution if this is not used in a 1 time only solution. Maintaining such a regex is brutal.

answered May 20 '12 at 22:19

buckley

13,690
3
53
61

@Rudie Fully agree. Don't use it unless it's a throw away piece of code – buckley May 20 '12 at 22:24
I will agree with you and other commenters of this question, since nested situation is very tricky to handle. Thanks for this temp regex, if you can come up with an unlimited version, that will be very educational to people including me. – Peiti Li May 20 '12 at 22:40
@Peiti Peter Li The number of {{ }} pairs can be unlimited. The nesting is something that regexes are not good in. At best you can hardcode a specific number of nestings to support but there is no generic solution. But you didn't nest your {{ }} so the given regex will work on your sample input. – buckley May 21 '12 at 12:53

how to get specific nested-prone block content using regex

3 Answers3