Page 3 of 4
Grouping and alternatives
Regular strings often have alternative forms. For example the ISBN designator could be simply ISBN: or it could be ISBN-13: or any of many other reasonable variations.
You can specify an either/or situation using the vertical bar |, the alternation operator as in x|y which will match an x or a y.
matches either ISBN: or ISBN-13:. This is easy enough but what about:
At first glance this seems to match either ISBN: or ISBN-13 followed by any number of white space characters and a single digit – but it doesn’t.
The | operator has the lowest priority and the alternative matches are everything the left and everything to the right, i.e. either ISBN: or ISBN-13:\s*\d.
To match the white space and digit in both forms of the ISBN suffix we would have to write:
Clearly having to repeat everything that is in common on either side of the alternation operator is going to make things difficult and this is where grouping comes in.
Anything grouped between parentheses is treated as a single unit – and grouping has a higher priority than the alternation operator.
So for example:
matches either form of the ISBN suffix followed by any number of white space characters and a single digit because the brackets limit the range of the alternation operator to the substrings to the left and right within the bracket.
The greedy/lazy situation also applies to the alternation operator. For example, suppose you try to match the previous un-grouped expression but without the colon:
In this case the first pattern, i.e. “ISBN”, will match even if the string is “ISBN-13”. It doesn’t matter that the second expression is a “better” match.
No amount of grouping will help with this problem because the shorter match will be tried and succeed first. In this case the solution is to either swap the order of the sub-expressions so that the longer comes first or include something that always marks the end of the target string.
For example, in this case if we add the colon then the
subexpression cannot possibly match the ISBN-13: string.
Capture and backreference
Now that we have explored grouping it is time to introduce the most sophisticated and useful aspect of regular expressions – the idea of “capture”.
You may think that brackets are just about grouping together items that should be matched as a group, but there is more.
A subexpression, i.e. something between brackets, is said to be “captured” if it matches and captured expressions are remembered by the engine during the match.
Notice that a capture can occur before the entire expression has finished matching – indeed a capture can occur even if the entire expression eventually fails to match at all.
You can see the captures as part of the array returned by the exec or string search operation. The first element of the array is the full match and the subsequent array elements are the captures.
Each capture group, i.e. each sub-expression surrounded by brackets, can be associated with one or more captured string. To be clear, the expression:
has two capture groups which by default are numbered from left-to-right with capture group 1 being the (<div>) and capture group 2 being the (</div>). The entire expression can be regarded as capture group 0 and each result is returned in the corresponding element of the result array.
If we try out this expression on a suitable string and get the array of results back with the capture matches as well as the full match.
Then, in this case, we have three capture groups returned as part of the result array – the entire expression returned as result, the first bracket i.e. capture group 1 is returned as result and the final bracket i.e. capture group 2 as result. The first group, i.e. the entire expression, is reported as matching only once at the start of the test string – after all we only asked for the first match.
Now consider the same argument over again but this time with the expression:
In this case there are four capture groups including the entire expression.
Capture group 0 is the entire expression ((<div>)(</div>))* and this is captured once matching the entire string of three repeats.
The next capture group is the first, i.e. outer, bracket ((<div>)(</div>)) and it is captured once and then the remaining two capture groups (<div>) and (</div>)
So far so good but what can you use captures for?
The answer is two-fold: more sophisticated regular expressions and replacements.
Let’s start with their use in building more sophisticated regular expressions.
Using the default numbering system described above you can refer to a previous capture in the regular expression.
That is, if you write \n where n is the number of a capture group the expression will specify that value of the capture group – confused?
It’s easy once you have seen it in action.
Consider the task of checking that html tags occur in the correct opening and closing pairs. That is, if you find a <div> tag the next closing tag to the right should be a <\div>. You can already write a regular expression to detect this condition but captures and back references make it much easier.
If you start the regular expression with a sub expression that captures the string within the brackets then you can check that the same word occurs within the closing bracket using a back reference to the capture group:
Notice the \1 in the final part of the expression tells the regular expression engine to retrieve the last match of the first capture group - which should be div in this case. If you try this out you will find that it matches <div><\div> but not <div><\pr>, say.
You could have done the same thing without using a back reference but its easy to extend the expression to cope with additional tags.
For example :
matches correctly closed div, pr, span and script tags.
If you are still not convinced of the power of capture and back reference try and write a regular expression that detects repeated words without using a back reference to a capture.
The solution using a back reference is almost trivial:
The first part of the expression simply matches a word by the following process –
start at word boundary capture as many word characters as you can, then allow one or more white space characters. Finally check to see if the next word is the same as the capture.
The only tricky bit is remembering to put the word boundary at the end. Without it you will match words that repeat as a suffix as in “the theory”.
If you need to group items together but don’t want to make use of a capture you can use:
This works exactly as it would without the ?: but the bracket is left out of the list of capture groups. This can improve the efficiency of a regular expression but this usually isn’t an issue.