User:Altsalt/Web-based experiments

A Web-based experiment or "Internet-based experiment" is an experiment that is conducted over the Internet. Psychology and Linguistics are probably the disciplines that have used these experiments most widely, although a range of other disciplines use web-based experiments. Within Psychology most web-based experiments are conducted in the areas of Cognitive and Social Psychology. This form of experimental setup has become increasingly popular because researchers can cheaply collect large amounts of data from a wide range of locations and people. A web-based experiment is a type of online research method.

Introduction
Experiments are an integral part of research, however, their integration with the Internet has been gradual. There are three main categories of experiements:
 * Controlled experiments, done in a labratory setting, attempt to control for all variables then test for a single effect.
 * Natural experiments, conducted after a large scale event which was prohibitively difficult or impossible to control, collect as many variables as possible then draw correlations.
 * Field experiments, observed in a natural setting where less controls can be applied, have the advantage of better external validity.

While natural and field experiments adapt well online, controlled (also known as laboratory) experiments have had some hurdles.

Affordances
In web-based experiments, there is less reliance on data gathered from populations of Western undergraduate students who often are often used as default research subjects in social science disciplines:
 * possibility of reaching more diverse samples
 * recruiting larger subject pools
 * conducting cross-cultural social experiments in real time
 * significantly more affordable

Limitations
Web-based experiments can make it difficult to come-up with procedures that ensure internal validity:
 * difficulty verifying the identity of subjects participating in the experiment
 * experimental instructions may be ignored or read too carelessly
 * significant distractions may occur during the course of the experiment
 * subjects may selectively drop out of the experiment
 * concerns about compensation at the end of the experiment
 * issues of reliably and automatically processing the payment over the Internet in an anonymous fashion

Methodologies
In responding to the call for an "online laboratory", Hergueux and Jacquemet created a platform which:
 * controls for differences in response times
 * deals with the issues of selective attrition, concentration and distraction
 * provides as much control as possible over subjects’ beliefs as regards the experimental instructions

Use in psychology
Web experiments have been used to validate results from laboratory research and field research and to conduct new experiments that are only feasible if done online. Further, the materials created for web experiments can be used in a traditional laboratory setting if later desired.

Interdisciplinary research using web experiments is rising. For example, a number of psychology and law researchers have used the web to collect data. Lora Levett and Margaret Bull Kovera examined whether opposing expert witnesses are effective in educating jurors about unreliable expert evidence. Rather than sensitizing jurors to flaws in the other expert's testimony, the researchers found that jurors became more skeptical of all expert testimony. In her experiment, this led to more guilty verdicts.

Levett and Kovera's research used a written transcript (law) of a trial, which participants then read before making their decision. This type of stimulus has been criticized by some researchers as lacking ecological validity—that is, it does not closely approximate a real-life trial. Many recommend the use of video where possible.

Researchers at New York University are currently conducting a psychology and law study that uses video of a criminal trial. Participants who go to the website can watch the trial (less than one hour long) and act as jurors.

Researchers at University of Salford are currently conducting a number of studies to explore sound perception. Sound experiments over the web are particularly difficult due to lack of control over sound reproduction equipment (see criticisms below).

A wide range of psychology experiments are conducted on the web. The Web Experiment List provides a way to recruit participants and archives past experiments (over 700 and growing). A good resource for designing a web experiment is the free Wextor tool, which "dynamically creates the customized Web pages needed for the experimental procedure" and is remarkably easy to use.

Use in economics
Jerome Hergueux and Nicolas Jacquemet developed an "online laboratory" to compare social preferences and risk aversion online and in person. They administred a Public good game, a Trust game, a Dictator game, and an Ultimatum game.

Salganik, Dodds, and Watts conducted an experiment to measure social influence, specifically in the popularity rating of songs. Their use of the Internet allowed them to collect over 14,000 participants and examine the relationship between individual and collective behavior.

As more experiments have been conducted in economics, the issue of an appropriate methodology and organization has been raised.

Criticisms
Some researchers have expressed concern that Web-based experiments have weaker experimental controls compared to laboratory-based ones. For instance, it may be difficult to be confident that the subjects characteristics are what they claim (e.g., age, race, gender, etc.) and that they are taking the experiment seriously. Others have argued  that brick-and-mortar experiments are just as affected by these problems, if not more so. Schoeffler et al. (2013) compared laboratory- and web-based results (62 and 1168 subjects) of an auditory experiment and found no significant differences.