Transparent Replications and Open Science

Come to The Unjournal’s Asia-Pacific online event on March 25, 6:30 PST, March 26, 10:30am Japan. Registration is open.

Often, social scientists are faced with incentives to find “novel” and counter-intuitive results to help their research get into prestigious publications. This can make research less reliable and replicable. To address this issue in recent social science and psychology research, Transparent Replications (by Clearer Thinking) aim to make replications of studies much more common, encouraging researchers to produce high-quality, methodologically rigorous research.

At the online event, Innovations in Research Evaluation, Replicability, and Impact, Amanda Metskas and Clare Harris from Transparent Replications will discuss incentivizing best practices through transparent replications.

We are testing a new approach to promoting open science and replicable research practices in experimental psychology. We replicate studies from randomly-selected, newly-published papers from a predefined set of prestigious psychology journals, plus all newly-published psychology papers in Nature and Science. We rate studies on their transparency, replicability, and clarity in communicating findings, and publish the results on our website. Through these ratings, we aim to reward the many teams already engaged in open science practices and encourage others to follow their lead. In this talk, we will explain our approach and its challenges, report on the first seven replications we completed, and explain how we plan to apply our methods at scale. Though the original studies were mostly transparently conducted, with findings that have mostly replicated so far, we wish to emphasize the importance of authors not only conducting transparent, replicable research, but also unambiguously communicating if and how their conclusions necessarily follow from their results. Finally, we conclude by inviting feedback, ideas, and discussion from the audience. We recognize that to truly shift incentives in psychological science, it will require many scientists working together towards this goal.

Metskas and Harris will present and take questions. The event will feature a range of presentations and interactive discussions, with speakers and including Gary Charness (“Improving Peer Review in Economics”), David Reinstein (“Introducing The Unjournal”, “(How) should we rate research?”)  and Takahiro Kubo (“Banning wildlife trade can boost demand for unregulated threatened species”; winner of The Unjournal’s Impactful Research Prize).

Participants can choose which sessions they want to during the 2.5-hour-long event. Transparent Replications are scheduled to present at 6:55 PM PST.

This event is co-hosted with Center for Open Science and Effective Thesis.