ManyLabs DACH - About The Project

About the Project

ManyLabsDACH (with DACH referring to Germany (D), Austria (A), and Switzerland (CH)) is a large-scale, community-driven crowd-science project spanning Germany, Austria, and Switzerland, in which participating research teams propose and evaluate experimental designs, ensuring that the highest-rated study is implemented consistently across all locations using standardized materials and procedures.

The project brings together leading research institutions across the German-speaking DACH region in a crowd-science initiative. Supported by key organizations and research projects such as Lab² (supported by the Leibniz Association), the CRC Rationality and Competition (funded by the DFG)1, and SOEP (DIW), ManyLabsDACH aims to connect approximately 20 experimental labs to collect a high-powered dataset of about 5,000 observations. The exclusive regional focus is intentional, simplifying logistics and avoiding cross-language heterogeneity in order to maximize data comparability.

The coordinating Project Team will jointly analyze the collected results to develop a comprehensive metascience paper, co-authored by all contributing research teams. The large-scale coordination will enable high-powered meta-analytic tests and the estimation of population heterogeneity across participating laboratories, accelerating the accumulation of robust evidence in the field by coordinating data collection across many labs simultaneously.

1) Support by the Deutsche Forschungsgemeinschaft through CRC TRR 190 (project number 280092119) is gratefully acknowledged.

Primary Aims of the Study

Test a mechanism for selecting experimental designs for multi-lab studies

The project tests a structured, peer-based process for proposing, screening, and selecting an experimental design that meets strict scientific and logistical criteria and can be implemented uniformly across all participating laboratories.

Conduct high-powered meta-analytic tests of treatment effects

By pooling data from all participating labs, the study enables precise estimation of treatment effects and provides strong empirical evidence regarding the validity and robustness of the experimental hypotheses.

Estimate population heterogeneity in treatment effects across laboratories

ManyLabsDACH examines how and why treatment effects vary across different participant pools, institutional settings, and lab environments, thereby offering insights into the generalizability of experimental findings.

Estimate heterogeneity in economic preferences from an incentivized follow-up survey

In the follow-up survey, we will also include incentivized economic games to measure economic and social preferences in a standardized way across all participating laboratories. This design allows for precise estimation of preference heterogeneity across individuals and sites and provides empirical evidence on how such preferences vary across populations and institutional contexts.

Procedure

The research project is organized into three major phases:

Pre-Data Collection

a. Project Initialization

i. Writing the project outline: The Project Team writes the project outline and specifies the general procedures the pre-analysis plan (PAP 1).

ii. Building a lab consortium: Research Teams (RTs) with access to an experimental laboratory (in a German speaking country) are invited
to participate in the study and comment on PAP 1.

b. Design Proposal & Selection

i. Design submission: A call is distributed to the consortium, and study designs are collected using a standardized form.

ii. Commitment to conduct the experiment : All RTs commit to conducting the selected experimental design in their laboratory.

iii. Design evaluation and selection: All participating Research Teams (RTs) evaluate the submitted designs, and the highest-rated
one is selected.

c. Final Preparation of Data Collection

i. Incentivized survey design: The follow-up incentivized survey is prepared by the Project Team and RTs.

ii. Lab Panel Survey Preparation: A short institutional survey for all RTs is designed and administered.

iii. Preparing PAP 2: Part 2 of the pre-analysis plan (PAP 2) is prepared based on the selected experimental design and surveys.

iv. Preparing the experimental software and piloting: The experimental software is prepared and piloted to ensure it runs smoothly.

Data Collection and Merging

a. Execution

i. Piloting: The Project Team conducts a final pilot to test the program and instructions.

ii. Coordination meeting: A training and discussion meeting is organized with all participating labs before data collection.

iii. Data collection: Contributing labs collect the experimental and survey data using the standardized materials and procedures.

iv. Data delivery: Collected data is delivered to the coordinating team to be merged and prepared for analysis.

Post-Data Collection

a. Finalization

i. Data analysis: The Project Team analyzes the merged data as per the registered PAP.

ii. Drafting the paper and review: The first draft of the paper is written by the Project Team and then reviewed by all contributing RTs.

iii. Dissemination: The project results are shared through working papers, presentations, and journal submissions. All members from the contributing RTs will be co-authors.

The Project Team. The project will be coordinated by researchers from WZB, Stockholm School of Economics, Humboldt University, DIW, FU Berlin, University of Innsbruck, LMU Munich, and ifo Institute. Meet the Project Team.

Pre-analysis plans. The project is preregistered through a two-stage Pre-Analysis Plan (PAP). PAP 1 covers the crowdsourcing and design selection, while PAP 2 will detail the specific hypotheses, procedures, and experimental instructions after a design is chosen. All study contributors will review and suggest changes to PAP 2 before it is finalized and formally registered on the project’s OSF repository. The entire OSF repository, including the pre-registered analysis plan, its addendum, the data, and the experimental software packages, will be made publicly available after the publication of the paper.

Budget and financial support. Organizational costs of the project are covered by a joint budget provided by Lab² and the CRC Rationality and Competition. Data collection costs are borne by the participating experimental labs themselves and are expected to range between €3,000 and €5,000 per lab, depending on the specific design and sample requirements. For labs that require financial support and are able to issue an invoice, limited funding is available to cover data collection expenses. (see Budget Guidelines)

Questions. We have set up a detailed FAQ with commonly asked questions. Of course, feel free to contact us via e-mail.

ManyLabsDACH is supported by Deutsche Forschungsgemeinschaft and Leibniz Association.

Page updated

Google Sites

Report abuse