Pre-Analysis Plan Template (Economics and Finance, AEA RCT Registry, OSF)

Pre-Analysis Plan: [Study title]

Pre-analysis plan (PAP) for an economics or finance study: a field experiment, a lab or online experiment, a natural experiment, or an analysis of proprietary data that the team has not yet opened. It fixes the hypotheses, the sample, the outcomes, the estimating equations, the multiple-testing correction and the exclusion rules before anyone sees outcome data, and it keeps a log of every later deviation. The fields follow the registration form of the AEA RCT Registry (as checked 2026-10-25) and the OSF Registries workflow, and the plan can also serve as the Stage 1 protocol of a registered report. Replace every [bracketed] value and every row marked Example. This is a document format, not legal, ethics-review, financial or investment advice. Obtain the journal policy, registry rules and ethics regulations for the normative text.

Document control

Approvals

Revision history

Contents

• 1. Study registration and identifiers
• 2. Background, research questions and hypotheses
• 3. Study design and setting
• 4. Sample, randomisation and power
• 5. Outcomes and measurement
• 6. Estimation strategy
• 7. Multiple hypothesis testing
• 8. Heterogeneity and mechanisms
• 9. Data collection, cleaning and exclusions
• 10. Threats to validity and contingency rules
• 11. Ethics, data protection and disclosure
• 12. Timeline and registration record
• 13. Deviations log and reporting commitments
• 14. Variant for natural experiments and proprietary data
• 15. Glossary and references
• Standard and licence note

1. Study registration and identifiers

Register the study in a public registry and attach this plan. The choice depends on the study type and on the journals you will target.

Fill the registry form from this plan, so the two never disagree.

2. Background, research questions and hypotheses

2.1 Motivation

[Two or three paragraphs: the economic question, what existing evidence shows, the gap this study fills and why the setting is well suited. Cite the closest papers by author and year.]

2.2 Research questions

2.3 Hypotheses

State which test is one-sided, if any, and why. Two-sided tests are the default in economics and finance journals; justify any exception here, before data exist.

2.4 Theory and predictions

[Explain the mechanism you expect (inertia, attention, mental accounting, liquidity constraints) and the prediction each mechanism makes for the secondary outcomes. A table that maps mechanism to prediction makes later interpretation honest: if the data contradict a mechanism, you said so in advance.]

3. Study design and setting

3.1 Setting and partner

[Describe the population, the partner institution, the product and the period. Say what the partner already does for all customers, and what the partner may change during the study (pricing, marketing) that could interfere.]

3.2 Treatment arms

The shares sum to 100%. The 2 x 2 factorial design lets you estimate each main effect and the interaction (H-04). State whether you will pool arms for the main effects; pooling raises power for H-01 but assumes no interaction.

3.3 Design diagram

The diagram shows the flow from the sampling frame to the endline data for the example design.

3.4 Unit of randomisation and spillovers

Randomise at the level at which spillovers would occur. In the example, colleagues at the same employer talk about the feature, so the unit is the payroll-employer group, not the customer. State the expected direction of any remaining spillover and how it would bias the estimate (usually towards zero).

4. Sample, randomisation and power

4.1 Eligibility and sampling frame

4.2 Randomisation procedure

• Method: stratified cluster randomisation, implemented in [software and version] by [@pi-2], with the seed [seed] stored in [repository path].
• Strata: [region x baseline savings tercile], giving [N] strata.
• Assignment within strata: equal shares across the four arms; remainders allocated at random.
• Timing: after baseline data are frozen, before any outreach.
• Balance checks: compare arms on [list baseline covariates]; report a joint F-test. Do not re-randomise on imbalance unless the rule is written here in advance (for example, re-draw when the joint p-value is below 0.05, with all draws logged).

4.3 Power and minimum detectable effect

For a two-arm comparison with share $P$ treated, $N$ units, outcome standard deviation $\sigma$, two-sided size $\alpha$ and power $1-\kappa$, the MDE under individual randomisation is

With clusters of size $m$ and intra-cluster correlation $\rho$, multiply by the square root of the design effect:

Controlling for a baseline covariate that explains a share $R^2$ of outcome variance multiplies the MDE by $\sqrt{1-R^2}$.

Interaction tests (H-04) have much less power than main effects: in a balanced 2x2 design the standard error of the interaction coefficient is about twice that of a pooled main effect, so detecting an interaction of the same size needs roughly four times the sample. Report the MDE for $\beta_3$ separately (Example: about 2 x 0.1237 = 0.25 SD of OUT-001 before any baseline control, against 0.12 SD for the pooled main effect) and treat H-04 as secondary.

Judge whether the MDE is economically meaningful. If the smallest effect worth knowing about is 3 percentage points, the study is underpowered for H-02 and you should add clusters, not hope. Report power for each primary hypothesis separately and keep the script with the plan.

5. Outcomes and measurement

5.1 Outcome definitions

5.2 Indices

For each family, build a summary index as the equally weighted mean of the standardised components (each standardised against the control group, signed so that higher is better), following Kling, Liebman and Katz (2007), or the inverse-covariance weighted index of Anderson (2008). Choose one now and name it here: [index method].

5.3 Covariates

List the baseline covariates used in the primary specification. Pick them now, on prediction grounds (strong predictors of the outcome), not on balance grounds after the fact.

6. Estimation strategy

6.1 Estimand

The primary estimand is the intention-to-treat (ITT) effect: the effect of being assigned to an arm, whatever the customer did with it. Report the effect of actually using the feature (treatment on the treated) as secondary, using assignment as an instrument for take-up.

6.2 Primary specification

For customer $i$ in cluster $c$ and stratum $s$:

where $D_c$ is the default assignment, $R_c$ the reminder assignment, $Y^0$ the baseline outcome, $X$ the covariates in section 5.3 and $\mu_s$ strata fixed effects. H-01 and H-02 are tests of $\beta_1 + \beta_3/2$ (the default effect averaged over reminder status), H-03 of $\beta_2$, and H-04 of $\beta_3$. State the exact linear combination for each hypothesis in a table so there is no doubt later.

6.3 Standard errors and inference

• Cluster standard errors at the unit of randomisation (the employer group). With fewer than about 50 clusters per comparison, add randomisation inference or a wild cluster bootstrap and state which one is primary.
• Report point estimates, confidence intervals and the control mean for every outcome, not only p-values.
• Use linear probability models for binary outcomes in the primary specification; report logit or probit as robustness only.

6.4 Robustness specifications

List them now so they are not chosen for their results: no covariates; alternative winsorisation levels; levels instead of transformed outcomes; excluding the first month; dropping one region at a time.

7. Multiple hypothesis testing

7.1 Families and method

7.2 Worked example of a false discovery rate correction

The Benjamini and Hochberg (1995) procedure orders the $m$ p-values from smallest to largest and rejects every hypothesis up to the largest rank $k$ for which $p_{(k)} \le \alpha k / m$. Sharpened q-values (Anderson 2008), the method chosen in 7.1, are a two-stage refinement of this procedure that first estimates how many null hypotheses are true; they reject at least as many hypotheses as the one-stage version.

The table below is an illustrative family of five outcomes chosen to show the arithmetic; it is not one of the families defined in 7.1, where OUT-001 sits in the primary family.

The largest rank meeting the condition is 2, so the first two hypotheses are rejected. OUT-006 and OUT-004 would each pass at 0.05 on their own; report both the unadjusted and the adjusted result.

8. Heterogeneity and mechanisms

Estimate heterogeneity with an interaction term in the primary specification. If you also plan machine-learning methods for heterogeneous effects (for example generic machine learning or causal forests), name the method, the sample split and the tuning rule now, and label the results exploratory.

Mechanism tests: [list the outcomes that discriminate between the mechanisms in section 2.4, such as opt-out rates, timing of transfers relative to payday and survey measures of attention].

9. Data collection, cleaning and exclusions

9.1 Data sources and timing

9.2 Exclusion and cleaning rules

9.3 Attrition

Administrative outcomes should have little attrition; survey outcomes will not. Test whether response rates differ by arm. If the difference exceeds [5] percentage points or is statistically significant, report Lee (2009) bounds for survey outcomes alongside the main estimates.

10. Threats to validity and contingency rules

11. Ethics, data protection and disclosure

• Ethics review: approved by [committee] on YYYY-MM-DD, reference [number]. For US federally funded human-subjects research, the Common Rule (45 CFR part 46) applies, including the criteria for IRB approval (46.111) and the informed consent requirements (46.116); check whether a waiver of consent was approved and on what grounds.
• Personal data: where the GDPR applies, document the lawful basis and the research safeguards under Article 89(1) (data minimisation and pseudonymisation where the purpose allows), and complete a DPIA if the processing is likely to be high risk. Use DPIA (compliance/dpia).
• Harm review: explain why no arm is expected to harm participants (for example, the default can be switched off in one click and no fees apply), and how a participant can opt out of the research.
• Partner rights: record what the partner may review before publication (factual accuracy and confidentiality only, not results), in writing.
• Funding and interests: list every funder and any financial or advisory relationship with the partner.

12. Timeline and registration record

13. Deviations log and reporting commitments

Reporting commitments:

1. Report every primary hypothesis, whatever the result.
2. Present the pre-specified specification first; place deviations and exploratory analyses in clearly labelled sections.
3. Include a "populated PAP" appendix that shows each pre-specified test next to its result.
4. Publish the code and the data (or access instructions) with a Replication Package README (academic/replication-package-readme).

14. Variant for natural experiments and proprietary data

When the identifying variation comes from a policy, a regulation or a data release rather than your own randomisation, replace sections 3 and 4 with the following.

15. Glossary and references

15.1 Glossary

15.2 References

Standard and licence note

Based on the public registration fields of the AEA RCT Registry and the American Economic Association registry policy for its journals, the OSF Registries preregistration workflow, the published Registered Reports guidelines of the Journal of Behavioral and Experimental Finance, the US Common Rule (45 CFR part 46, a US federal work) and Regulation (EU) 2016/679, together with the methodological papers listed in section 15.2, which are cited and not reproduced. No registry form text, journal guideline wording or template from another organisation is copied; the OSF forms and the AsPredicted questions are referenced by name only. The study, partner and numbers in the examples are fictional. This template is original text written for this collection and is not affiliated with or endorsed by any registry or journal named. Check the current registry rules, journal policies and ethics regulations before use. Template licence: MIT.