Wikipedia:Research recruitment

This page documents the process that researchers should follow before asking Wikipedia contributors to participate in research studies such as surveys, interviews and experiments. Researchers must first describe their research project and proposed recruitment strategy (as explained below) and obtain consensus from the Wikipedia community for their activities before mass messaging users. This process exists because researchers have a great demand for research participants which the Wikipedia community is unable to meet, and so researchers are requested design their studies to be thoughtful and minimally invasive when they recruit the time of volunteers here.

Scholarly research of Wikipedia is useful for understanding the encyclopedia's content, readers, editors, history, current state, and future. These results also yield important knowledge applicable to other open content communities. In addition to driving scholarly knowledge of such systems, this work can also give results that improve Wikipedia itself. Much valuable research cannot be done without Wikipedia editors who volunteer to participate in studies by being interviewed, taking surveys or testing experimental interfaces. This policy describes an interface between academic researchers and Wikipedia editors that does the following:
 * 1) allows the Wikimedia community to control recruitment campaigns from researchers looking for study participants
 * 2) affords legitimacy to researchers whose plans are successfully vetted by the Wikipedia community

Who needs to go through this process?
Anyone who wants to recruit a group of Wikipedians for a research study must go through the following process.

For the avoidance of doubt, if you want enough Wikipedians to respond that you are posting invites on mailing lists and/or enough talkpages that you might be considered to be spamming people then you need to go through this process first.

Recommendations
Minimize sample size to minimize disruption. Studies that affect a larger number of editors will need wider consensus to move forward.

Consider vulnerable populations Members of the Wikimedia community become more stressed as they are targeted for intensive research more frequently. In general, community members who are highly active or members of highly researched minority populations are more often sought by researchers more often. Consider estimates of user demographics when designing a survey and see the estimates of members of frequently requested populations.

Don't repeat what has already been done. Lots of people have studied the Wikipedia community and some questions have become over familiar. Wikipedians are more likely to agree to a new survey that is likely to find out something new than one which merely replicates surveys that have previously been run. See wikipapers for an index of wiki research and meta:Editor_Survey for a description of surveys run by the Wikimedia Foundation.

Give back to the community. Your research results will likely have value to the community. Editors may be more likely to approve of your work if you can deliver your results in a freely licensed format for their consumption. Consider open licensing your manuscript, publishing your datasets, open sourcing your code and writing a summary for easy consumption.

Keep individual responses confidential Though the community expects aggregated results to be available, the norm is for individual results to be confidential. Please be clear as to where you are drawing the line between confidentiality and openness.

Keep your cool. Discussions can sometimes get heated over misunderstandings. Consider using the documented strategies for consensus building if the discussion ceases to be productive. Ask for help at WikiProject_Research or meta:Research:Committee.

No breaching experiments. Do not violate the community's policies or norms. Damage or disruption, for any reason, is not tolerated and is a very fast way to generate hostilities between the Wikipedia community and the research community.

Operate transparently. Wikipedians tend to appreciate openness. The wiki keeps a public record of the history of all pages and conversations. You will aid Wikipedians in tracking your activities by ensuring that they happen on-wiki. Make sure you are logged in when participating in discussions and sign your comments with four tildes (e.g. ~ ).

Be responsive, but patient. Reviews take time and you can't rush consensus. However, you can make sure that questions, suggestions and concerns posted to your project's talk page are addressed quickly.

For Reviewers
All interested editors should act as reviewers. There are no special titles or rights necessary. Proposal reviews should be closed when there is consensus that the project is adequately documented, not likely to cause a disruption and likely to be successful at producing valuable research results.

Recommendations
Use the checklist. ..and add to it.

Take a constructive role. Researchers may be missing an opportunity that you see. Offer your suggestions.

Assume good faith. Researchers may not understand the norms and terminology of Wikipedia, but it was their interest in understanding that brought them here in the first place.

If you don't understand... that is the researcher's fault, but you'll have to help them know what needs explaining.

Information for Wikipedians
This section gives a brief overview of who researchers are and why they are interested in studying Wikipedia and its editors.

Who are researchers?
There are a wide variety of backgrounds from which people approach studying Wikipedia. They are here to perform scientific analysis of Wikipedia and its users. They often intend to publish the results of their work in academic publications.
 * Academics: the students, professors, and staff of colleges and universities
 * Industry researchers: the staff of private companies
 * Independents: individuals without an affiliation

What do they do?
In the past, research in Wikipedia has built an understanding of how Wikipedia works, how editors interact with each other, what work is discarded and why, how admins are chosen, and how to detect vandalism. This research serves to increase understanding in how Wikipedia works and to improve its functioning. Researchers approach understanding Wikipedia in a few different ways.
 * Field experiment: Determines the limitations and strengths of Wikipedia's functionality or tests new functionality for editing, collaborating, navigating, etc., by developing and distributing modifications to Wikipedia's functionality. Field trials will usually need to recruit users under this policy.
 * Surveys and interviews: Learns various aspects of editors and editing (e.g. demographics, motivations, activities) using pre-written forms or back-and-forth conversations. Requests for participation can be either general (i.e. random), or targeted to specific editors.  Requests for participation require sending unsolicited messages, so surveys and interviews frequently require recruitment under this policy.
 * Participant observation: Gains a close familiarity with the editing community by joining Wikipedia and doing the same work that Wikipedians do on a regular basis. They often casually converse with editors as a part of the collaborative editing process.  Participant observers who do not send unsolicited requests do not usually fall under the SRAG's guidelines.  However, they are recommended to disclose themselves as researchers and consult with the SRAG before starting their research project.  In addition, participant observers who decide to interview or survey users need to recruit users under this policy.
 * Offline analysis: Analyzes database snapshots of publicly available information to examine the history of encyclopedia construction. Offline analyses will seldom need to recruit users under this policy.

Why are they here?
Wikipedia is an interesting medium for scientific research. It is one of the most visited websites on the internet, serving as an information resource to millions of users every day. Many find it remarkable that an encyclopedia in which articles can be edited by anyone anonymously, and in which damage can only be repaired after it occurs, has quality comparable to traditional encyclopedias. They want to understand how the social dynamics of Wikipedia works. Further, Wikipedia is one of the few examples of millions of people working together with a single goal. The Wikimedia Foundation also supports the work of researchers by maintaining public mailing list (e.g. wiki-research-l and rcom-l), releasing public datasets, hiring them on fellowship and releasing periodic database snapshots for analysis.

Resources

 * meta:Research - The research portal for all Wikimedia projects.
 * WikiProject Research - A group of Wikipedians interested in scholarly research of Wikipedia. Many are interested in participating in and closing discussions about recruitment proposals.
 * Research - Essay on scholarly research of Wikipedia
 * Wikipedian checklist - A checklist of concerns for research proposals. Use this to help you review research proposals.
 * Ethically researching Wikipedia


 * Academic studies of Wikipedia - A list of academic studies of Wikipedia.
 * Wikipapers - A semantic mediawiki instance for collecting and organizing academic papers about wikis and wiki-like systems.