Give Now  »

Noon Edition

When Experiments Fail to Replicate

a blurry, abstract black-and-white photograph

The results of any scientific study should be reproducible, right?

One of the main tenets of scientific inquiry is that we should be able to reach the same conclusion over and over again. So, what happens when the results of a study don‘t replicate?

Fewer Than Forty Percent

The answer may seem simple at first. You might think when a study can‘t be reproduced, its conclusions must be false. And they might be.

But there can also be other explanations. When the Reproducibility Project tried to replicate one hundred psychological studies, they succeeded with fewer than 40%. A shockingly low number.

But that doesn't necessarily mean those studies should be thrown out or discredited. Jay Van Bavel, a professor of psychology from NYU, considered how and why this might have occurred, and formed a testable hypothesis of his own. He considered whether the failure to replicate could have been due to differences between the contexts in which the experiments were performed.


Something as simple as an experiment done on mice can be heavily impacted by contextual factors. Consider that everything from the type of food the mice eat, the materials used to line their cages, the temperature that the lab is kept at, and how much light they see can alter their biological processes, such as metabolism.

Just imagine how complex contextual factors might be among people. Professor Van Bavel‘s investigation found that yes, the studies with topics more sensitive to context – which might mean time, location, or demographic differences – were also the least likely to replicate successfully.

Thank you to Jay Van Bavel of New York University for reviewing this episode's script.

Sources And Further Reading:

Support For Indiana Public Media Comes From

About A Moment of Science