Wikipedia:Arbitration Committee/Requests for comment/Article creation at scale/Closing statement

Closing statement for Article Creation at Scale RFC. Panel close by:
 * -- RoySmith (talk) 20:41, 4 November 2022 (UTC)
 * Katietalk 22:51, 4 November 2022 (UTC)
 * The SandDoctor Talk 20:53, 4 November 2022 (UTC)

Technical assistance from is gratefully acknowledged.

Question 1: Should we develop a noticeboard where mass creations and sources used for them can be discussed?
Fails.

Question 2: Should we require (a) source(s) that plausibly contribute(s) to WP:GNG?
The way this question was worded, it wasn’t clear how the results should be scored, so we needed to improvise. There were 7 items (A, A2, B, B2, C, C2, and D) which were to be ranked in order. We gave 7 points for a first-place ranking, 6 for a 2nd place, etc down to 1 for a last place. If a choice wasn’t ranked at all, we gave it 0 points. Those of you who got creative in your rankings can consider yourself trouted; we did our best to map your prose into numerical scores.

The raw data is available on Google Drive (still trying to figure out how to get WikiMedia to upload an xls file).

A raw summation of the votes for each choice would show that “D” was the most popular, by a substantial margin. However, we’re unsure if that accurately reflects the consensus, per Vote splitting.

It might be reasonable to consider C and C2 as a group (and likewise with the A’s and B’s). It seems likely that somebody who voted for C2 would be happier with C than with D. If you look at things that way, then the answer is “The C group got the most votes, and within that, C was substantially more popular than C2”.

We request that the moderators conduct a 7-day run-off, with the simple binary question, “Should we enact C?”, no commentary allowed.

As an aside, if a multiple-choice question like this is to be used in the future, we strongly recommend that a specific algorithm for scoring be established before the RfC begins. Leaving it to the vote counters to decide how to count the votes is a recipe for disaster. See Ranked voting for a discussion of some possibilities.

Update after the run-off
Enacting option C is the clear consensus of those who responded during the runoff. Looking at just the responses from people who participated in the original RFC, the numbers are closer, but it's still consensus to enact C.


 * -- RoySmith (talk) 00:29, 13 November 2022 (UTC)
 * The SandDoctor Talk 04:12, 13 November 2022 (UTC)
 * Katietalk 01:02, 13 November 2022 (UTC)

Question 2A: Should we require the inclusion of (a) source(s) that plausibly contribute(s) to WP:GNG?
This question got essentially no responses.

Question 3: Should we create a definition of "article creation at scale"? By rate, source, similarity, other?
Fails almost unanimously.

Question 3A: Alternative three-part definition
Fails on numbers, and also the support votes are mostly wishy-washy “Something like this, but not exactly what was proposed”.

Question 3B: Should we create a definition of "article creation at scale"?
Passes by a wide margin. It’s clear people want some sort of definition, even if they can’t agree on what it should be.

Question 4: Should we prohibit the creation of articles at scale?
Fails overwhelmingly.

Question 4A: Should we restrict creation of articles at scale?
Fails by a wide margin.

Question 5: Minimum article quality when created at scale
Fails by a wide margin. The proposal failed to quantify how "quality" would be measured.

Question 6: New mass creator permission
Withdrawn.

Question 7: Should we adopt a new speedy deletion criterion that relates to mass-created articles that lack any sourced claims of importance?
Moved to the second RfC.

Question 7a: Should we instead introduce the following speedy deletion criterion: A12: Unsourced or obviously unreliably sourced mass-created articles.
Moved to the second RfC.

Question 8: Delete ranking of editors by created articles
Fails by a wide margin. Parties noted that it is pointless and consider deletion to not be a net benefit, especially considering that quarry queries can be made to get the same results.

Question 9: Should mass-creators be forced to respond to issues?
Fails overwhelmingly, with reference to WP:NOTCOMPULSORY.

Question 10: add mass-creation as a reason to WP:BUNDLE
Moved to second RfC.

Question 11: Should we allow (or disallow) mass-created articles that lack any sourced claims of importance?
Fails by a wide margin.

Question 12: Editors who mass-create stubs have a duty to expand the articles later
Fails nearly unanimously.

Question 13
There’s no question here.

Question 14: Prohibit mass creation of (GNG) articles sourced only to non-prose-containing databases
Some of the major issues people talked about were We could not discern any consensus position, but closer to “fail” than “pass”.
 * Does it matter if it’s prose or not?
 * What matters is whether it’s reliable, which may not correlate with whether it’s prose.
 * Human-curated vs machine-gathered data?
 * Databases vary widely in quality

Question 15: How about to award the expanders of articles (stubs) with the notifications that usually only the creator receives?
On the surface, a majority for “pass”, but so few people responded here, we’re not sure we can legitimately divine anything.

Question 16: Develop metric to measure editor's engagement in article improvement
This isn’t phrased as a question. We assume it was intended to say, “Should we develop…” In any case, while “support” got more votes, we find several of those to be unenthusiastic: “I wonder about the practicalities” “Difficult to implement, but I support in principle” “I do not think it will have any useful effect” “I don’t think it will solve this problem” We’re calling this a “pass”, but just barely, and noting minimal participation.

Question 17: Amend WP:MASSCREATE
Technically passes on the numbers. The opposition has some valid points about BOTPOL being the wrong venue and the lack of a definition of how many article creations cause this proposal to take effect. A number of those opposed support in spirit but are opposed over the proposal's ill-defined nature.

We acknowledge that we were not adequately clear in our original closing of Q17 and are issuing the following clarification:

No adequately robust consensus was formed by this discussion that can be relied upon to change bot policy. While proponents debated what would trigger this proposal to take effect, those in opposition noted that human editing falls outside of the scope of bot policy and noted that the definition of “mass creation” was sufficiently imprecise/ill-defined so as to cause considerable problems with implementation.

Ultimately, this discussion was too close to adequately give a clear, actionable, consensus. As an aid for future discussion, here's a list of salient points we saw on the two sides:

Support:


 * Early review would have prevented some ARBCOM cases.
 * Even mass creations that are ostensibly human-powered are really semi-automated and don’t get much human oversight, so those processes should be reviewed.
 * Talking about any mass-creation scheme ahead of time will save a lot of effort on the back end cleaning up problems.
 * Eliminate the loophole of copy-pasting from off-wiki automated processes and then claiming it’s not automated.
 * Any large-scale editing (whether automated or not) should have some kind of prior community review.
 * Perhaps allow creating a small number of example articles (10?) and shop that around for approval before proceeding with a larger scale project.
 * Some of the semi-automated schemes that mass-create articles ostensibly by hand would actually be done better if they were fully automated.

Oppose:
 * Controlling human editing is out of scope for WP:BOTPOL
 * No need to make this change, since BOTPOL already references MEATBOT
 * Given that “mass creation” is ill-defined, it’s unworkable to enact policy which depends on that definition.
 * How do you discern the dividing line between similar articles in a series that are truly human authored versus essentially automated processes that are just front-ended by humans who contribute little or no editorial input?
 * This would be contrary to WP:BOLD.
 * We should concentrate on quality not quantity; if somebody is cranking out many well-referenced articles a day, that’s a good thing.
 * If somebody is creating low-quality articles, the solution is to block them.

Statement by:
 * The SandDoctor Talk 06:23, 11 November 2022 (UTC)
 * -- RoySmith (talk) 16:16, 11 November 2022 (UTC)
 * Katietalk 13:25, 11 November 2022 (UTC)

Question 18: Expanding the powers of WP:BRFA
Fails by a wide margin.