Wikipedia:Measuring conflict of interest editing on Wikipedia

As of 2017 there have been no measurements of conflict of interest editing on Wikipedia. In Wikipedia, "Conflict of interest" (COI) editing refers to all the wiki editing which occurs in the context of a conflict of interest and which is said to have a likelihood of being a disruptive engagement, negatively impacting Wikipedia, and greatly stressing Wikipedia community volunteers who come into contact with this activity. COI is a popular topic of discussion on Wikipedia, and yet the community does not have access to basic data about the scope of the issue. Desirable data estimates would include how often conflict of interest editing happens, what percentage of the time it has positive/neutral/negative outcomes, and how much Wikipedia community labor the conflict of interest management procedures are consuming. Various participants in discussions form opinions in debates and take sides in policy discussions. Through all this, there is wide disagreement on the facts and premises from which any discussion should develop. As an example, some people might say that conflict of interest editing causes several serious problems daily which consume hours of labor to address, whereas others might say that conflict of interest editing rarely causes problems and is almost always a net positive.

This page discusses what data the Wikipedia community has to measure conflict of interest and what quality that data is.

Why Wikipedia needs measurements of conflict of interest editing
The Wikipedia community continually raises policy discussions, administers bureaucratic oversight, provides editor counseling, and finds itself in uncontrolled media attention as a consequence of addressing conflict of interest editing. Many Wikipedia community members who engage in these processes find them to be either negative experiences or an unfortunate use of their time.

When users propose policy in an attempt to improve Wikipedia's treatment of conflict of interest editors or the Wikipedia community volunteers who respond to them, recurring discussion points include that conflict of interest editing is a large problem, while others say it is not. Since there is no data personal perception varies wildly.

As of November 2017, these are the opposing sorts of perspectives which different reasonable, experienced Wikipedians might hold:

Determining the effect of conflict of interest editing should have some basis in some facts. There is room for difference in opinion, but there should be less doubt about how often a controversial process causes problems or is welcomed. Wikipedia needs some counts and estimates. Even anecdotes and educated guessing by Wikipedia users with experience and doing informal counts would be better than allowing conversations to proceed with imagined data guesses which might have a range of several orders of magnitude.

Unanswered basic questions

 * 1) How frequently does conflict of interest editing happen on Wikipedia?
 * 2) What are Wikipedia's tools for identifying and counting instances of conflict of interest editing?
 * 3) What ledgers are there of identified editors with a conflict of interest?
 * 4) How often is conflict of interest editing problematic?
 * 5) Considering all identified conflict of interest editing, what percentage of time are those contributions good versus neutral versus bad?
 * 6) How much ambiguity is there in differentiating good versus bad conflict of interest editing?
 * 7) How often does conflict of interest editing result in community pushback, like reverting content, blocking a user, creating a dispute, or escalating the conflict to an admin?
 * 8) When conflict of interest editing is problematic, how much Wikipedia community volunteer labor do volunteers provide to fix it?
 * 9) How many Wikipedia volunteer editors address problematic conflict of interest editing?
 * 10) What communities of Wikipedia editors exist which routinely address conflict of interest editing?
 * 11) Among Wikipedia editors who address conflict of interest editing, what are their opinions on the value of this labor?
 * 12) What skills does a Wikipedia volunteer need to interact with paid editors? To what extent do these interactions require training and expertise, or otherwise more valuable labor?
 * 13) To what extent does conflict of interest editing affect Wikipedia's reputation?
 * 14) How often does the media report stories about conflict of interest editing?
 * 15) What reputation does Wikipedia have among conflict of interest editors?
 * 16) How often do Wikipedia community volunteers interact with conflict of interest editors?
 * 17) How often do Wikipedia readers view content provided by conflict of interest editors?

Technological measurement
Wikipedia's software could automatically count and detect information about conflict of interest editing. As of November 2017 either no such systems exist or none are well known to the community.

Educated guesses
Ask Wikipedians who have experience in dealing with conflict of interest what their educated guesses are to the scope of the issue. These guesses will of course be limited to the narrow part of Wikipedia which they observe, but perhaps what they report could be insightful. Ask these questions:


 * Please volunteer #1 edit here
 * 1) What is your best guess for how often the volunteer team in this project oversees conflict of interest editing events?
 * 2) Imagine all the conflict of interest editing events you have seen in this project. Divide them into two groups, "acceptable or productive contributions" and "unacceptable or unproductive contributions". What is your guess for the percentage of conflict of interest editing events which are "acceptable or productive contributions"?
 * 3) What is your best guess for how many hours Wikipedia community volunteers spend in this project addressing conflict of interest editing events?
 * 4) Imagine that Wikipedians in this project with your level of experience provide 10 hours of support to other volunteer Wikipedians. What is your best guess of how many hours of "acceptable or productive contributions" Wikipedia is likely to get in return for that 10 hour investment?
 * 5) Imagine that Wikipedians in this project with your level of experience provide 10 hours of support to conflict of interest editors. What is your best guess of how many hours of "acceptable or productive contributions" Wikipedia is likely to get in return for that 10 hour investment?
 * 6) In 2017 English Wikipedia has about 40,000 active editors. In terms of Wikipedia experience, where do you think you rank among editors? Bottom 20%, second 20%, third 20%, fourth 20%, and or top 20%?
 * 7) Please sign your username.
 * 1) Imagine that Wikipedians in this project with your level of experience provide 10 hours of support to conflict of interest editors. What is your best guess of how many hours of "acceptable or productive contributions" Wikipedia is likely to get in return for that 10 hour investment?
 * 2) In 2017 English Wikipedia has about 40,000 active editors. In terms of Wikipedia experience, where do you think you rank among editors? Bottom 20%, second 20%, third 20%, fourth 20%, and or top 20%?
 * 3) Please sign your username.
 * 1) In 2017 English Wikipedia has about 40,000 active editors. In terms of Wikipedia experience, where do you think you rank among editors? Bottom 20%, second 20%, third 20%, fourth 20%, and or top 20%?
 * 2) Please sign your username.
 * 1) Please sign your username.

Survey communities
It would be helpful to get multiple guesses and a consensus from communities which routinely address conflict of interest editing. The following communities and forums are likely to have many participants with a lot of experience interacting with conflict of interest editors.
 * Wikipedia talk:Paid-contribution disclosure
 * Conflict of interest/Noticeboard
 * OTRS noticeboard
 * New pages patrol
 * Articles for creation
 * Administrators' noticeboard/Incidents
 * Sock puppetry
 * Recent changes patrol

Human arithmetic
Anyone who has time and interest can go to discussion boards and do simple counts and addition of incidents. For example, anyone can find a place which discusses conflict of interest editing, count all relevant discussions in the past 3 months or year, and classify them in an insightful way. Doing this and sharing the result could provide starting data for determining the scope of the issue.

Seek organizational consensus
Certain WikiProjects, Wikimedia user groups, and teams might be organized enough to confer among themselves and issue a collective statement.