User:PiperNigrum/Research study recruitment of Wikipedians

Past attempts at recruitment of Wikipedians to participate in studies has met with limited success. Specific details surrounding several such attempts are discussed below, along with how WP:Research and WP:SRAG could have improved the situation.

Research of Wikipedia
Wikipedia is an interesting medium for scientific research. It is one of the most visited websites on the internet, serving as an information resource to millions of users every day. Scientists find it remarkable that an encyclopedia in which articles can be edited by anyone anonymously, and in which damage can only be repaired after it occurs, has quality comparable to traditional encyclopedias. They want to understand how the social dynamic of Wikipedia works. Further, Wikipedia is one of the few examples of millions of people working together on a single project. The Wikimedia Foundation also supports the work of researchers by maintaining a public mailing list devoted to scholarly research of Wikimedia projects, employing a Chief Research Officer, organizing and supporting the Wikimania conference for research of Wikimedia projects and releasing periodic database snapshots for analysis.

Recruitment
In order to conduct some types of research, it is necessary to randomly sample a population for statistical validity, however past attempts at recruitment of Wikipedians to participate in studies has met with limited success. One avenue that is often used is posting on various community forums such as Village Pumps, Centralized discussion, and mailing lists. While this method generates participants for studies, there are a number of problems.


 * The samples are non-random
 * The sample draws only from Wikipedians who read the forum
 * As a community forum, readers often ask questions, or comment on study design.

The perferred method of obtaining a random sample is to contact individual editors via talk page posting. Unfortunately, a minority of users considered these postings to be Spam or Canvassing.

Since researchers are not familiar with Wikipedia's norms, they often do not know what is acceptable. In the previous example, the researcher was unaware of the policies around user accounts. In other cases, researchers end up doing something even more absurd.

Also, even Wikipedians disagree on what is appropriate, as can be seen in many of these examples, and also in what occurred to Wikimedia's own usability testers].

Subject Recruitment Approval
By conducting subject recruitment through WP:SRAG, many of the problems seen above can be alleviated, as SRAG provides a centralized place to discuss the research and approve the recruitment, as well as management of the actual recruitment itself (aka bot, but how is immaterial)

Discussions
SRAG brings together a knowledgeable group of researchers and Wikipedians to discuss the merits of conducting recruitment for the research. The community can advise the researchers about the policies and norms of Wikipedia, as well as understand what the research is for, and how it will be conducted.

By having these discussions beforehand, editors who raise concerns once recruitment begins can be pointed at the discussion and consensus reached. It also provides a continuing forum for editors to track other issues about the recruitment which may arise. This dialog can be used to continuously refine the process of recruitment, both for the specific study, as well as SRAG as a whole.

Recruitment
Having recruitment messages sent by/via SRAG also provides a central point of contact for all subject recruitment. Any issues about recruitment can be directed back to the approval group rather than individual researchers, which can be used to refine the recruitment process.

While the posting of recruitment messages can be done manually by an SRAG member, the most likely means for conducting the recruitment is via a bot. Using a bot will allow for better and faster recruitment. Also bots can take advantage of automated opt-out mechanisms (via bots, or a bot-specific template) by which individual editors can control whether or not they wish to be recruited, as well as the frequency of that recruitment -- a process which would be more difficult using manual bookkeeping.