Transparent Replications

Rapid replications for reliable research

Always Conduct the “Simplest Valid Analysis”

This is an opinion piece from our founder, Spencer Greenberg. A significant and pretty common problem I see when reading papers in social science (and psychology in particular) is that they present a fancy analysis but don’t show the results of what we have named the “Simplest Valid Analysis” – which is the simplest possible…
Report #10: Replication of a study from “The illusion of moral decline” (Nature | Mastroianni & Gilbert 2023)

Executive Summary Transparency Replicability Clarity We ran a replication of Study 5b from this paper. This study tested whether people believe that morality is declining over time. The paper noted that people encounter disproportionately negative information about current-day people (e.g., via the media) and people often have weaker emotional responses to negative events from the…
Report #9: Replication of a study from “Loss Aversion, the Endowment Effect, and Gain-Loss Framing Shape Preferences for Noninstrumental Information” (PNAS | Litovsky et al. 2022)

Executive Summary Transparency Replicability Clarity We ran a replication of Study 2 from this paper, which found that participants place greater value on information in situations where they’ve been specifically assigned or “endowed with” that information compared to when they are not endowed with that information. This is the case even if that information is…
Reality Check Podcast Interview with Spencer Greenberg

Transparent Replications Founder Spencer Greenberg recently appeared on The Reality Check podcast. He talks about the state of replication in psychology, incentives in academic research, statistical methods, and how Transparent Replications is working to improve the reliability of research. Check it out!
Year of Open Science Conference Presentation

Transparent Replications presented our project and preliminary results at the Year of Open Science Culminating Conference. This virtual conference was a collaboration between the Open Science Foundation and NASA and was attended by over 1,000 people. Now you can see our presentation too! The Transparent Replications presentation is the first fifteen minutes of this video.…
Report #8: Replication of a study from “The Visual Mandela Effect as Evidence for Shared and Specific False Memories Across People” (Psychological Science | Prasad & Bainbridge 2022)

Executive Summary Transparency Replicability Clarity We ran a replication of Study 1 from this paper, which tested whether a series of popular logos and characters (e.g., Apple logo, Bluetooth symbol, Mr. Monopoly) showed a “Visual Mandela Effect”—a phenomenon where people hold “specific and consistent visual false memories for certain images in popular culture.” For example,…
Report #7: Replication of a study from “The Surprise of Reaching Out: Appreciated More Than We Think” (JPSP | Liu et al. 2023)

Executive Summary Transparency Replicability Clarity We ran a replication of study 4a from this paper, which found that people underestimate how much their acquaintances would appreciate it if they reached out to them. This finding was replicated in our study. The study asked participants to think of an acquaintance with whom they have pleasant interactions,…
Report #6: Replication of a study from “Collective transcendence beliefs shape the sacredness of objects: the case of art” (JPSP | Chen, Ruttan & Feinberg, 2022)

Executive Summary Transparency Replicability Clarity We ran a replication of study 4 from this paper, which found that people’s perceptions of an artwork as sacred are shaped by collective transcendence beliefs (“beliefs that an object links the collective to something larger and more important than the self, spanning space and time”). In the study, participants…
Report #5: Replication of a study from “Relational diversity in social portfolios predicts well-being” (PNAS | Collins et al. 2022)

Transparency Replicability Clarity We ran a replication of study 1 from this paper , which found that the variety in a person’s social interactions predicts greater well-being, even when controlling for their amount of in-person social interaction. This finding was replicated in our study. In this study participants were asked about their well-being over the…
Report #4: Replication of two studies from “Behavioral nudges reduce failure to appear for court” (Science | Fishbane, Ouss & Shah, 2020)

TransparencyReplicabilityClarity We ran replications of studies three (3) and four (4) from this paper. These studies found that: People have less support for behavioral nudges (such as sending reminders about appointment times) to prevent failures to appear in court than to address other kinds of missed appointments People view missing court as more likely to…
Why We Introduced the Clarity Criterion for the Transparent Replications Project

At Transparent Replications we have introduced a study rating criterion that, to our knowledge, has not been used before. We call it our “clarity” rating, and it is an assessment of how unlikely it would be for a reader to misinterpret the results of the study based on the discussion in the paper. When a…
Explaining our Replicability Ratings

Transparent Replications rates the studies that we replicate on three main criteria: transparency, replicability, and clarity. You can read more about our transparency ratings here. The replicability rating is our evaluation of the degree of consistency between the findings that we obtained in our replication study and the findings in the original study. Our goal…
Our Transparency Ratings criteria: contents and rationale

Transparent Replications is an initiative that celebrates and encourages openness, replicability, and clear communication in psychological science. We do this by regularly placing new papers from high-impact journals under the microscope. We select studies from them, run replications, and rate the replicated studies using three key criteria. Each of the three criteria represents a concept…
Transparent Replications in Vox

Sigal Samuel’s article, “Lots of bad science still gets published. Here’s how we can change that.” in Vox today, highlights the work we’re doing at Transparent Replications to address the replication crisis in psychology research.”, in Vox today, highlights the work we’re doing at Transparent Replications to address the replication crisis in psychology research. If…
Transparent Replications on Two Psychologists Four Beers

Listen to project founder, Spencer Greenberg, discuss Transparent Replications on the Two Psychologists Four Beers podcast. If you’ve found your way here from the podcast episode… [continue reading]
Report #3: Replication of a study from “Knowledge about others reduces one’s own sense of anonymity” (Nature | Shah & LaForest 2022)

TransparencyReplicabilityClarity We ran a replication of study 2A from this paper, which tested whether knowing additional information about another person changed what participants thought the other person would know about them. The primary result in the original study failed to replicate. There was no relationship between whether participants were given information about their ‘partner’ and…
Report #2: Replication of a study from “Counterfeit Diversity: How Strategically Misrepresenting Gender Diversity Dampens Organizations’ Perceived Sincerity and Elevates Women’s Identity Threat Concerns” (JPSP | Kroeper, Williams & Murphy 2022)

TransparencyReplicabilityClarity We ran a replication of study 3 from this paper, which assessed men’s and women’s beliefs about hypothetical scenarios in which they imagined applying to and working for technology companies with different ratios of men and women on staff (either 3:1 or 1:1). In the original study, when a company had a male:female staff ratio…
Report #1: Replication of a study from “Lay Theories of Financial Well-being Predict Political and Policy Message Preferences” (JPSP | Krijnen et al. 2022)

TransparencyReplicabilityClarity We ran a replication of study 2 from this paper, which assessed three sets of beliefs (each measured by averaging responses to three self-report questions) about what causes variation in financial well-being. The original authors predicted that people’s agreement with a given set of beliefs would be more positively associated with support for government goals…