Wikipedia:Wikipedia Signpost/2008-01-28/New parser

A new preprocessor was introduced this week (see Migration to the new preprocessor), after several days of testing by the community, and preceding that extensive testing by developers, to try to make sure that it would not cause problems (last week's coverage). The new parser fixes several bugs, and makes new features available; however, it caused some problems as well that were not caught by the testing.

The new preprocessor has actually existed in the software for a while (since 20 November, revision r27667 of the software); however, it caused problems with strip markers (the 'UNIQ bug'; see related story) and was disabled on Wikimedia wikis until the problems could be sorted out. It was re-enabled for the English Wikipedia on 24 January (Village Pump announcement), but some unexpected problems developed, in addition to the changes that were known.

There were some changes that fit to some extent within expected behaviour; for instance, the method previously used to prevent MediaWiki:Ipbreason-dropdown expanding templates within the drop-down list stopped working, but this was easily fixed by Tim Starling (who wrote the new preprocessor) using a different method. However, there were other unexpected problems. One problem that affected certain processes like Templates for deletion was that signature tildes (~&#126;) inside a &lt;!-- comment --&gt; ended up being expanded when the page was saved under the new preprocessor; the new preprocessor was disabled for several hours while the issue was fixed, but now handles such cases correctly. Another problem was caused by the change in semantics of &lt;onlyinclude&gt; (which indirectly affected Portal:Current events, and possibly other pages); it now no longer removes whitespace from the same line, and therefore now behaves the same way as &lt;includeonly&gt; and &lt;noinclude&gt;. However, the change for the most part went smoothly.

Two new pieces of markup are available with the new preprocessor. The new parser function checks to see if another parser function returned an error message, and allows a different error message or other markup instead: There is also the #tag syntax, allowing magic words, template markup and similar preprocessor markup to be used inside tag-like extensions, like &lt;ref&gt; and &lt;imagemap&gt;. There is an example on the technical village pump. One less obvious but important improvement is that branches of parser functions like that are not used are now not parsed, reducing the load on the server for pages which use such constructs; this means that the pages in question will not use up nearly as much of the page's template limits as before.

A large number of bugs were also fixed by the new preprocessor (see related story).