Wikipedia:Wikipedia Signpost/2012-12-10/Technology report

First Wikipedians allowed to opt in to Visual Editor
The Visual Editor project – an attempt to create the first WMF-deployable WYSIWYG editor – will go live on its first Wikipedias imminently following nearly six months of testing on MediaWiki.org. A full explanatory blog post accompanied the news, explaining the project and its setup.

By opting in, an editor can handle basic formatting, headings and lists, while safely ignoring elements the new system is yet to understand, including references, categories, templates, tables and images. At the last count, about 2% of pages would break in some way if a user tried the Visual Editor on them; it is unclear whether any specific protection will be put in place beyond relying on editors to spot problems. Only users with compatible browsers (currently Chrome and Firefox) will be able to take advantage of the Visual Editor at the moment; Internet Explorer 9+ is expected to be supported eventually, as is Safari. The Visual Editor is likely to get much faster as the Parsoid (Parser 2.0) project develops.

WMF developers describe the opt-in process (the same as that used for Vector skin over two years ago) as designed to allow editors to "get familiar, highlight bugs, and help us prioritise". Once enabled, the editor will be updated every two weeks, although that is no guarantee of rapid expansion in feature capability; few headline capabilities have been introduced since the Signpost's last story about the Visual Editor back in June. Instead, the work of recent months has focussed on internal cleanup and documentation. Even though refactoring work has come to an end, the Foundation is likely to miss its target of implementing three plugins (e.g. list, tables and citations) by the end of this month. For many casual users, of course, it will be a case of better late than never whenever the editor arrives.

November engineering report published
The WMF's engineering report for November 2012 was published this week on the Wikimedia blog and on the MediaWiki wiki ("friendly" summary version), giving an overview of all Foundation-sponsored technical operations in that month (as well as brief coverage of progress on Wikimedia Deutschland's Wikidata project, phase 1 of which will soon be trialled on the Hungarian Wikipedia). Of the four headlines picked out for the report, two (the launch of Wikivoyage.org and the TimedMediaHandler extension) have already received Signpost coverage. The third focusses on the create of a cluster devoted to analytics number crunching, and the fourth is an invitation to volunteers to assist not just with development but product management.

The report featured an extended section on performance, an area often neglected in official communications. Much of the news was positive; a problem with caching server stability has been fixed, and freeing up memory on the WMF's application servers "addresses some of the root causes of multiple site outages, and brings with it multiple client improvements including consistent hashing, igbinary serialization, and better timeout handling". On the negative side, the Foundation's image server continued to experience occasional hardware failure, leading to an agreement with the hardware vendor to replace them. The migration of the primary data centre from Tampa to Ashburn is ongoing.

Elsewhere, there was work on developing new UI theming across all skins (primarily with the intention of making the "Save page" button more prominent) and discussion about getting more JavaScript (browser) tests automated following previous broken deployments. (Users interested in the subject may consult a more recent, detailed post on the topic.) The first phase of the Universal Language Selector (ULS) was completed in November, but, as the lack of reporting in the Signpost will attest, there were further delays in launching the Wikidata client to its first test wiki (the Hungarian Wikipedia).

In brief
Not all fixes may have gone live to WMF sites at the time of writing; some may not be scheduled to go live for several weeks.
 * Job queue nightmare: Developers and system administrators, both staff and volunteer, have been working on understanding and resolving an apparent glut of problems with the JobQueue. Bottlenecks in the queue, which handles tasks that do not have to be performed immediately, such as updating Special:WhatLinksHere and category membership, normally manifest themselves as straightforward delays in job handling. These delays then cause the number of jobs in the backlog, which is more easily observable, to grow (as has been the case recently). This time, however, newly enabled code (1, 2) exposed weaknesses in both "core" MediaWiki code and several extensions; for example, the changes prompted several translation notifications to be sent multiple times (detailed analysis), further cluttering the queue. A (presumably related) "logic issue in the job queue" unusually caused an internal database replication lag, resulting in a three hour lock-down for many smaller wikis (wikitech-l mailing list). Several fixes were deployed, though the complexity of the area involved suggests the possibility of ongoing issues, including (but not limited to) a slower processing of the job queue caused by the introduction of randomness of job selection.
 * UploadWizard gets Flickr integration: The UploadWizard has been updated for the easy transfer of suitably licensed content from popular image hosting site Flickr (wikitech-l mailing list). The project, an attempt to improve upon existing Toolserver tools designed to provide a similar service, represents the first of this year's Google Summer of Code projects to go live. Uploaders can select single images for transfer, or multiple images from the same Flickr photoset. Although only administrators can test the feature at present, the plan is for a wider rollout in the near future.
 * Matthew Flaschen hired as a features engineer: Wikimedia insider Matthew Flaschen has joined the WMF as a features engineer (wikitech-l mailing list). Flaschen, whose previous work includes the ProveIt extension, has edited Wikipedia since 2004 and has been an administrator since 2006. His first focus will be feature development for the Editor Engagement Team.