Maximal munch

In computer programming and computer science, "maximal munch" or "longest match" is the principle that when creating some construct, as much of the available input as possible should be consumed.

The earliest known use of this term is by R.G.G. Cattell in his PhD thesis on automatic derivation of code generators for compilers.

Application
For instance, the lexical syntax of many programming languages requires that tokens be built from the maximum possible number of characters from the input stream. This is done to resolve the problem of inherent ambiguity in commonly used regular expressions such as  (one or more lower-case letters).

The term is also used in compilers in the instruction selection stage to describe a method of "tiling" — determining how a structured tree representing a program in an intermediate language should be converted into linear machine code. An entire subtree might be converted into just one machine instruction, and the problem is how to split the tree into non-overlapping "tiles", each representing one machine instruction. An effective strategy is simply to make a tile of the largest subtree possible at any given point, which is called "maximal munch".

Drawbacks
In some situations, "maximal munch" leads to undesirable or unintuitive outcomes. For instance, in the C programming language, the statement  (without any whitespace) will probably lead to a syntax error since the   character sequence (unintentionally) initiates a comment that is either unterminated or terminated by the end token   of some later, unrelated actual comment (comments in C do not nest). What was actually meant in the statement was to assign to the variable  the result of dividing the value in   by the value obtained by dereferencing pointer  ; this would be valid code. It can be stated by making use of whitespace or using.

Another example, in C++, uses the "angle bracket" characters  and   in the syntax for template specialization, but two consecutive   characters are interpreted as the right-shift operator. Prior to C++11, the following code would produce a parse error, because the right-shift operator token is encountered instead of two right-angle-bracket tokens:

The C++11 standard adopted in August 2011 amended the grammar so that a right-shift token is accepted as synonymous with a pair of right angle brackets (as in Java), which complicates the grammar but allows the continued use of the maximal munch principle. An exception to the maximal munch rule had to be added anyway to deal with the sequence  which can appear in templates. In that case, unless the sequence is followed by  or   the character   is interpreted as its own token instead of part of the token.

Alternatives
Programming languages researchers have also responded by replacing or supplementing the principle of maximal munch with other lexical disambiguation tactics. One approach is to utilize "follow restrictions", which instead of directly taking the longest match will put some restrictions on what characters can follow a valid match. For example, stipulating that strings matching  cannot be followed by an alphabetic character achieves the same effect as maximal munch with that regular expression. (In the context of regular expressions, the maximal munch principle is referred to as greediness and contrasted with laziness.) Another approach is to keep the principle of maximal munch but make it subordinate to some other principle, such as context (e.g., the right-shift token in Java would not be matched in the context of a generics expression, where it is syntactically invalid).