2  Introduction

Replication has become an increasingly important topic in economics. As the field has grown more empirical and data-driven, the ability to reproduce published results is essential for scientific credibility. The so-called “replication crisis” –highlighted humorously in this XKCD comic– has affected not only economics but many scientific disciplines.

2.1 Purpose and Reason d’être

The main purpose of replication is to verify that published results can be independently reproduced using the same data and code. This process helps ensure that findings are not the result of errors, selective reporting, or other issues. Replication is a fundamental part of the scientific method, fostering transparency, accountability, and trust in research.

2.1.1 Types of Replication

There are several types of replication in economics, each serving a different purpose (Clemens 2017):

  • Pure (or direct) replication: Reproducing the results of a study using the same data and methods as the original paper.
  • Robustness replication: Testing whether the results hold under alternative methods, model specifications, or subsets of the data.
  • Conceptual replication: Examining whether the main findings hold in different contexts, with different data, or using different approaches.

Understanding these distinctions is important, as each type of replication addresses different aspects of scientific validity and reliability. This course focuses on the first type, namely, how to organize and document research projects to facilitate pure replication.

2.1.2 Limits of Replication

However, replication has its limits:

  • Code errors: Replication only checks if the code produces the reported results, not whether the code itself is correct. For example, see this case of undocumented code alterations.
  • Mismatch between code and text: Sometimes, the code does not follow the procedures described in the paper. See this study for an example.
  • Peer review limitations: Code is often not reviewed by referees, so errors or inconsistencies may go unnoticed.

2.2 The Value of Sharing Data and Methods

By publishing the data and methods used in research, economists facilitate the reuse of data and code. This practice, inspired by the open source movement, enables other researchers to build upon existing work, verify results, and accelerate scientific progress. Open sharing not only increases transparency but also fosters collaboration and innovation, as others can adapt and extend methods for new questions and contexts.

2.3 Additional Considerations

  • Open science movement: Many journals now require authors to share data and code, promoting transparency.
  • Challenges: Replication can be hindered by restricted data access, proprietary software, or poor documentation.