Wikipedia:Wikipedia Signpost/2015-05-06/Blog

The month-long "Inspire" campaign seeking ideas for new initiatives to increase gender diversity on Wikipedia recently concluded successfully, with hundreds of new ideas and over 40 proposals entering consideration for funding. During this campaign, there were a lot of questions about the empirical basis for the statement that women are underrepresented among Wikipedia editors, and in particular about the estimate given in the campaign’s invitation banners (which stated that less than 20% of contributors are female). This blog post gives an overview of the existing research on this question, and also includes new results from the most recent general Wikipedia editor survey.

The Wikimedia Foundation conducted four general user surveys that shed light on this issue, in 2008 (in partnership with academic researchers from UNU-MERIT), 2011 (twice) and 2012. These four large surveys, as well as some others mentioned below, share the same basic approach: Wikipedia editors are shown a survey invitation on the site, and volunteer to follow the link to fill out a web-based survey. This has been a successful and widely used method. But there are some general caveats about the data collected through such voluntary web surveys:

 Percentages cannot be compared, due to different survey populations: The overall percentage among respondents from one survey (e.g. the frequently cited 9% from the December 2011 WMF editor survey, or the 13% from the 2008 WMF/UNU-MERIT survey) is often taken as a rough proxy of &#8220;the&#8221; gender ratio among Wikipedia contributors overall. But different surveys cover different populations, e.g. because they were not available in the same set of languages, or because the definition of who counts as &#8220;editor&#8221; varies. This is especially relevant when trying to understand how the gender gap develops over time &#8211; e.g. we can&#8217;t talk about a "drop" from 13% to 9% between the 2008 and April 2011 surveys, because their populations are not comparable. Also, the slightly higher overall percentage in the 2012 survey, compared to the preceding one (see below) should not be interpreted as a rise. However, comparisons are possible for comparable populations, and in this post we present such trend statements for the first time. Participation bias between languages: There is evidence that the participation rates for such surveys vary greatly between editors from different languages. For example, in both the 2008 survey and the 2012 survey, the number of Russian-language participants was much higher than for other languages, compared to the number of active editors in each language. Women editors may be less likely to participate in surveys: A 2013 research paper by Benjamin Mako Hill and Aaron Shaw confirmed the longstanding suspicion that female Wikipedians are less likely to participate in such user surveys. They managed to quantify this participation bias in the case of the 2008 UNU-MERIT Wikipedia user survey, correcting the above mentioned 13% to 16%, and arriving at an estimate of 22.7% female editors in the US (more than a quarter higher than among US respondents in that survey). Hence we now know that the percentages given below are likely to be several percent lower than the real female ratio. Different definitions of "editor": Most of these surveys have focused on logged-in users, but there are also many people contributing as anonymous (IP) editors without logging into an account. What’s more, many users create accounts without ever editing (for this reason, the 2011/12 editor surveys contained a question on whether the respondent had ever edited Wikipedia, and excluded those who said "no". Without this restriction, female percentages are somewhat higher). Because they only reach users who visit the site during the time of the survey, these surveys target active users only. And depending on methodology, users with higher edit frequency (which, as some evidence suggests, are more likely to be male) may be more likely to participate as respondents. Sample size: As usual with surveys, the fact that respondents form only part of the surveyed population gives rise to a degree of statistical uncertainty about the measured percentage, which can be quantified in form of a confidence interval.  Still, these caveats do not change the fact that the results from these web-based surveys remain the best data we have on the problem. And the overall conclusion remains intact that Wikipedia&#8217;s editing community has a large gender gap.

What follows is a list of past surveys, briefly summarizing the targeted population and stating the percentage of respondents who responded to the question about their gender with female in each. In each case, please refer to the linked documentation for further context and caveats. Keep in mind that the stated percentages have not been corrected for the aforementioned participation bias, i.e. that it is likely that many of them are several percent too low, per Hill’s and Shaw’s result.

General user surveys

2012 Editor Survey <ul> <li>Population: Logged-in Wikipedia users who did not respond "no" to the question "Have you EVER edited Wikipedia?"</li> <li>Method: Banners in 17 languages, shown only once per user (October/November 2012)</li> <li>10% female (n=8,716. 11% when including non-editors and users who took the survey on Wikimedia Commons. 14% among Commons users, with n=463)</li> </ul>

<span id="December_2011_Editor_Survey">December 2011 Editor Survey <ul> <li>Population: Logged-in Wikipedia users who did not respond "no" to the question "Have you EVER edited Wikipedia?"</li> <li>Method: Banners in multiple languages, shown only once per user</li> <li>9% female (n=6,503)</li> </ul>

<span id="April_2011_Editor_Survey">April 2011 Editor Survey <ul> <li> Population : Logged-in Wikipedia users who did not say they had only made 0 edits so far</li> <li> Method : Banners in 22 languages, shown only once per user</li> <li>9% female (n=4,930)</li> </ul>

<span id="UNU-MERIT.2FWMF_survey_.282008.29">UNU-MERIT/WMF survey (2008) <ul> <li>Population: Site visitors who described themselves as "Occasional Contributor" or "Regular Contributor"</li> <li>Method: Banners shown to both logged-in and logged-out users, in multiple languages</li> <li>13% female (n=53,888)</li> </ul>

<span id="Other_surveys">Other surveys There have also been several surveys with a more limited focus, for example:

<span id="Global_South_User_Survey_.28WMF.2C_2014.29">Global South User Survey (WMF, 2014) <ul> <li>Population: Site visitors in 11 countries and 16 languages, who selected "Wikipedia" (along other large websites) in response to the question "Which accounts do you most frequently use"?</li> <li>Method: Banners shown to both logged-in and logged-out users</li> <li>20% female (n=10,061)</li> </ul>
 * Note: In this survey, the ratio of female editors was much higher than in the 2011 and 2012 surveys, in those countries where data is available. However, it is plausible that this difference can largely be attributed to different methodologies rather than an actual rise of female participation across the Global South.

<span id="Gender_micro-survey_.28WMF.2C_2013.29">Gender micro-survey (WMF, 2013) <ul> <li>Population: Newly registered users on English Wikipedia</li> <li>Method: Overlay prompt immediately after registration</li> <li>Draft results: 22% female (n=32,199. 25% when not counting "Prefer not to say" responses)</li> </ul>

<span id="JASIS_paper_on_anonymity_.282012.29">JASIS paper on anonymity (2012) <ul> <li>Population: Active editors on English Wikipedia (estimated to number 146,208 users at the time of the survey (2012))</li> <li>Method: User talk page messages sent to a random sample of 250 users</li> <li>9% female (n=106)</li> </ul> <dl> <dd> ''Tsikerdekis, M. (2013), The effects of perceived anonymity and anonymity states on conformity and groupthink in online communities: A Wikipedia study. J. Am. Soc. Inf. Sci.. DOI:10.1002/asi.22795 (preprint, corresponding to published version)'' </dd> </dl>

<span id=".22Grassroots_Survey.22_.28Wikimedia_Nederland.2C_2012.29 &#8220;"> (Wikimedia Nederland, 2012) <ul> <li>Population: Members of the Dutch Wikimedia chapter and logged-in users on the Dutch Wikipedia</li> <li>Method: Banner on Dutch Wikipedia, and letters mailed to chapter members</li> <li>6% female (n=1,089 (completed))</li> </ul>

<span id="Wikibooks_survey_.282009.2F2010.29"> Wikibooks survey (2009/2010) <ul> <li>Population: Wikibookians in English and Arabic</li> <li>Method: Project mailing list postings and sitenotice banners</li> <li>26% female (of 262 respondents, 88% of which described themselves as contributors)</li> </ul> <dl> <dd> Hanna, A. 2014, ‘How to motivate formal students and informal learners to participate in Open Content Educational Resources (OCER)’, International Journal of Research in Open Educational Resources, vol. 1, no. 1, pp. 1-15, PDF </dd> </dl>

Wikipedia Editor Satisfaction Survey (Wikimedia Deutschland with support from WMF, 2009) <ul> <li>Population: Logged-in and anonymous editors on German and English Wikipedia</li> <li>Method: Centralnotice banner displayed after the user&#8217;s first edit on that day, for 15 minutes (all users on dewiki, 1:10 sampled on enwiki)</li> <li>9% female (ca. 2100 respondents &#8211; ca. 1600 on dewiki, ca. 500 on enwiki)</li> </ul> <dl> <dd> ''Merz, Manuel (2011): Understanding Editor Satisfaction and Commitment. First impressions of the Wikipedia Editor Satisfaction Survey. Wikimania 2011, Haifa, Israel, 4-7 August 2011 PDF (archived)'' </dd> </dl>

"What motivates Wikipedians?" (ca. 2006) <ul> <li>Population: English Wikipedia editors</li> <li>Method: Emailed 370 users listed on the (hand-curated, voluntary, since deleted) "Alphabetical List of Wikipedians", inviting them to fill out a web survey</li> <li>7.3% female (n=151)</li> </ul> <dl> <dd> ''Nov, Oded (2007). &#8220;What Motivates Wikipedians?&#8221;. Communications of the ACM 50 (11): 60–64. DOI:10.1145/1297797.1297798, also available here'' </dd> </dl>

"Wikipedians, and Why They Do It" (University of Würzburg, 2005) <ul> <li>Population: Contributors to the German Wikipedia</li> <li>Method: Survey invitation sent to the German Wikipedia mailing list (Wikide-l) ("The sample characteristics of the present study might be [a] limitation because participants were very involved in Wikipedia &#8230; the reported results might not be the same for occasional contributors to Wikipedia.")</li> <li>10% female (n=106)</li> </ul>

For further research on these and other questions, see e.g. the &#8220;Address the gender gap&#8221; FAQ on Meta-wiki, or follow our monthly newsletter about recent academic research on Wikipedia.


 * This post has been condensed. The full version is available on the Wikimedia blog.