Regular Expressions Syntax

The following description of regular expressions is based in part on documentation for the REGEXP library, which was created by Henry Spencer. Sally uses this library to match regular expressions.

Basic Regular Expression Syntax

• '^' - at the start of the expression matches the start of a line (for example, '^test' matches all strings 'test' placed at the start of a line)
• '$' - at the end of the expression matches the end of a line
• '.' - matches any single character (for example, 'do.' matches 'dog', 'dot', 'doe', etc.; 'd..r' matches 'door', 'deer', etc.)
• '*' - after an expression matches 0 or more occurrences of the expression (for example, 'ad*' matches 'a', 'ad', and 'add' - in other words, 'a' followed by any number of 'd')
• '+' - after an expression matches 1 or more occurrences of the expression (for example, 'do+' matches 'do' and 'doo' - in other words, 'd' followed by at least one 'o')
• '?' - after an expression matches 0 or 1 occurrence of the expression (for example, 'add?' matches 'ad' and 'add' - in other words, 'ad' followed by optional 'd')
• '|' - between expressions matches the first expression, the second expression, etc. (for example, 'red|blue|black' matches 'red', 'blue', and 'black' - in other words, any of the mentioned words)
• '[ ]' - characters in brackets match any one character that appears in the brackets, but no others (for example, 'a[dts]' matches 'ad', 'at', and 'as' - in other words, 'a' followed by 'd', 't', or 's')
• '[^ ]' - characters in brackets match any one character that doesn't appear in the brackets, but no others (for example, '[^abc]' matches any character except 'a', 'b', and 'c')
• '[-]' - signifies a range of characters (for example, '1[0-9]' matches all numbers in the range '10' to '19')
• '( )' - glues expressions in parentheses into one expression (for example, '(very )*well' matches 'well', 'very well', and 'very very well' - in other words, any number of 'very ' followed by 'well')
• '\' - before any character matches this character (for example 'too\.' matches only 'too.', note that 'too.' matches 'tool', 'took', and as well as 'toon' - in other words, 'too' followed by any character)

Regular Expression Syntax

A regular expression is zero or more branches, separated by '|'. It matches anything that matches one of the branches.

A branch is zero or more pieces, concatenated. It matches the first piece, followed by the second, and so on.

A piece is an atom possibly followed by '*', '+', or '?'. An atom followed by '*' matches a sequence of 0 or more matches of the atom. An atom followed by '+' matches a sequence of 1 or more matches of the atom. An atom followed by '?' matches a match of the atom, or the null string.

An atom is a regular expression in parentheses (matching whatever the regular expression matches), a range (see below), '.' (matching any single character), '^' (matching the null string at the beginning of line), '$' (matching the null string at the end of line), a '\' followed by a single character (matching that character), or a single character with no other significance (matching that character).

A range is a sequence of characters enclosed in '[]'. It normally matches any single character from the sequence. If the sequence begins with '^', it matches any single character not from the rest of the sequence. If two characters in the sequence are separated by '-', this is shorthand for the full list of ASCII characters between them (e.g. '[0-9]' matches any decimal digit). To include a literal ']' in the sequence, make it the first character (following a possible '^'). To include a literal '-', make it the first or last character.

Ambiguity

If a regular expression could match two different parts of the text, it will match the one which begins earliest. If both begin in the same place but match different lengths, or match the same length in different ways, life gets messier, as follows.

In general, the possibilities in a list of branches are considered in left-to-right order, the possibilities for '*', '+', and '?' are considered longest-first, nested constructs are considered from the outermost in, and concatenated constructs are considered leftmost-first. The match that will be chosen is the one that uses the earliest possibility.

For example, '(ab|a)b*c' could match 'abc' in one of two ways. The first choice is between 'ab' and 'a'; since 'ab' is earlier, and does lead to a successful overall match, it is chosen. Since the 'b' is already spoken for, the 'b*' must match its last possibility-the empty string-since it must respect the earlier choice.

In the particular case where no '|'s are present and there is only one '*', '+', or '?', the net effect is that the longest possible match will be chosen. So 'ab*', presented with 'xabbbby', will match 'abbbb'. Note that if 'ab*' is tried against 'xabyabbbz', it will match 'ab' just after 'x', due to the begins-earliest rule.