User:Bluerasberry/experiment

Scope
To scope an experiment describe the sector of content which could be affected. Make an estimate of how much content, how many pageviews that content gets, and how prominent the change is. It will be possible to get metrics for some of this but not all of this. Make an estimate, and since this is wiki, other people can refine the measurement.

Now based on this information about the environment, choose a subset of content in that environment for an experiment. If the environment is huge, like "affects all biographies" or "affects all articles on a field of art", then an appropriate subset could be 100 Wikipedia articles, and excluding most highly popular or contentious ones. The point of the experiment is to see how an intervention works under routine conditions. Set a time to run the experiment. If paid researchers are involved then greatly shorten the time, perhaps to weeks. If the experiment is mostly conducted by Wikipedia community volunteers in the context of an ongoing conversation, 12-18 months might be appropriate.

The usual outcome of an experiment should be nearly no notice or feedback. Even when Wikipedia experimenters make sweeping changes, typically only other wiki editors remark on this, and even still there are usually not many. If a complaint comes in early that is cause to be wary; if there are a few complaints then consider pausing the experiment. To proceed with the experiment accept a compromise. Opposition should not seek to end an experiment without offering a safe compromise. Making a change in 1 less popular live article is usually okay if the experimenter does minimal documentation and presents the discourse. If anyone can thoughtfully do documentation and present the sides of the discourse, then experimenting in 20 live articles should be an acceptable baseline to get data.

Experiments need not operate with the assumption that Wikipedia will never change. Typical experiments probe to develop policy while imagining that Wikipedia can soon get radical new and power access to media collections, technology, or new communities of contributors editing in new ways. Try to make room for safe compromise. Avoid endless debate and try to write less, with summaries at the top of the page describing the experiment.

Do not surprise or upset the readers, collaborate with each other, be nice, and always advance the wiki.