Reproducibility Project

The Reproducibility Project is a series of crowdsourced collaborations aiming to reproduce published scientific studies, finding high rates of results which could not be replicated. It has resulted in two major initiatives focusing on the fields of psychology and cancer biology. The project has brought attention to the replication crisis, and has contributed to shifts in scientific culture and publishing practices to address it.

The project was led by the Center for Open Science and its co-founder, Brian Nosek, who started the project in November 2011.

Results
Brian Nosek of University of Virginia and colleagues sought out to replicate 100 different studies, all published in 2008. The project pulled these studies from three different journals, Psychological Science, the Journal of Personality and Social Psychology, and the Journal of Experimental Psychology: Learning, Memory, and Cognition, published in 2008 to see if they could get the same results as the initial findings. 97 of the original studies had significant effects, but of those 97, only 36% of the replications yielded significant findings (p value below 0.05), and the effects were often smaller than those in the original papers. The authors emphasized that the findings reflect a problem that affects all of science and not just psychology, and that there is room to improve reproducibility in psychology.

In 2021, the project showed that of 193 experiments from 53 top papers about cancer published between 2010 and 2012, only 50 experiments from 23 papers could be replicated. Moreover, it showed that the effect sizes of that fraction were 85% smaller on average than the original findings. None of the papers had its experimental protocols fully described and 70% of experiments required asking for key reagents.

Impact
The project, along with broader action in response to the replication crisis, has helped spur changes in scientific culture and publishing practices. The results of the Reproducibility Project might also affect public trust in psychology. Lay people who learned about the low replication rate found in the Reproducibility Project subsequently reported a lower trust in psychology, compared to people who were told that a high number of the studies had replicated.