# Reproducibility Checklist: [Paper or project title]

> Checklist for computational research, including machine learning and
> data analysis studies, that records the evidence another team needs to
> rerun the work and obtain the same results: code, environment, data,
> randomness, hyperparameters, compute, statistical reporting, evaluation
> and limitations. Based on common practice in research software and
> conference reproducibility checklists. Replace every `[bracketed]` value
> and every row marked *Example*.

<!--
Guidance: How to use this template
- Answer every item with Yes, Partial, No or Not applicable, and give an
  evidence link for every Yes. A Yes without evidence counts as No.
- Fill it in while the work is fresh, then test it: ask someone outside
  the project to rerun one main result from your instructions alone.
- "No" is acceptable when explained (for example, licensed data). Silent
  gaps are not.
- Keep the checklist under version control next to the code and update it
  with each release of the paper or artefact.
-->

| Field | Value |
|---|---|
| Checklist ID | [REPRO-2026-007] |
| Paper / project | [Title, venue or DOI] |
| Repository | [URL and commit hash or release tag] |
| Version | [1.0] |
| Status | [Draft / Complete / Independently verified] |
| Completed by | [@name] |
| Verified by | [@name, or "not yet verified"] |
| Last updated | YYYY-MM-DD |

## 1. Summary checklist

<!-- Guidance: This table is what reviewers read first. Each row points to the section that gives the detail. -->

The table summarises the status of each area; details follow in later
sections.

| ID | Item | Status | Evidence link | Section |
|---|---|---|---|---|
| RC-01 | Source code publicly available | [Yes] | *Example:* [github.com/org/repo at tag v1.2.0] | 2 |
| RC-02 | Licence for code stated | [Yes] | [LICENSE file] | 2 |
| RC-03 | Software environment fully specified | [Partial] | [environment.yml; CUDA version missing] | 3 |
| RC-04 | Data available or access route described | [Yes] | [Zenodo DOI] | 4 |
| RC-05 | Random seeds fixed and reported | [Yes] | [config/seeds.yaml] | 5 |
| RC-06 | All hyperparameters and search ranges reported | [Yes] | [Appendix B] | 6 |
| RC-07 | Compute budget reported | [Yes] | [Section 7 table] | 7 |
| RC-08 | Variance across runs reported | [Yes] | [Table 2, 5 seeds] | 8 |
| RC-09 | Evaluation protocol fully described | [Yes] | [eval/README.md] | 9 |
| RC-10 | Limitations stated | [Yes] | [Paper section 6] | 10 |
| RC-11 | One-command reproduction of main result | [Partial] | [make reproduce-table2] | 2 |

## 2. Code availability

<!-- Guidance: Give a permanent reference (release tag plus archive DOI), not only a branch name. Say which script produces which figure or table. -->

Code is available at [repository URL], release [tag], archived at [DOI].

| Result in paper | Script or command | Expected runtime |
|---|---|---|
| *Example:* Table 2 | `python scripts/train.py --config configs/table2.yaml && python scripts/eval.py` | 6 GPU hours |
| Figure 3 | [Command] | [Runtime] |
| [Result] | [Command] | [Runtime] |

- [ ] Code licence stated: [MIT / Apache-2.0 / other]
- [ ] Pre-trained models or checkpoints released: [link, or reason not]
- [ ] Instructions tested on a clean machine: [date, by whom]

## 3. Environment specification

<!-- Guidance: Pin exact versions. "PyTorch 2" is not reproducible; "torch==2.4.1+cu121" is. A container image digest is the strongest evidence. -->

| Component | Version | How pinned |
|---|---|---|
| Operating system | [Ubuntu 22.04] | [Container base image] |
| Language | [Python 3.11.9] | [.python-version] |
| Key libraries | *Example:* torch 2.4.1, numpy 1.26.4, scikit-learn 1.5.1 | requirements.lock |
| GPU driver / CUDA / cuDNN | [Version] | [Documented] |
| Container | [Image name] | [Digest sha256:...] |

## 4. Data availability

<!-- Guidance: For each dataset, give the version, source, licence and preprocessing. If data cannot be shared, describe how others can obtain access. -->

| Dataset | Version / date | Source and licence | Preprocessing | Splits |
|---|---|---|---|---|
| *Example:* [Dataset name] | v2.1, downloaded YYYY-MM-DD | [URL], CC BY 4.0 | `scripts/prep.py`, removes duplicates | Official train/val/test |
| [Dataset] | [Version] | [Source, licence] | [Steps] | [Split method and seed] |

Restricted data: [what cannot be shared, why, and the access process].
Personal data handling: [anonymisation, consent, applicable law].

## 5. Randomness and seeds

<!-- Guidance: Report every seed and every source of non-determinism you could not remove (for example GPU atomic operations). -->

| Source of randomness | Seed or control | Deterministic? |
|---|---|---|
| Data split | [42] | Yes |
| Weight initialisation | [Seeds 0-4, one per run] | Yes |
| Data loader shuffling | [Seeded generator] | Yes |
| GPU kernels | *Example:* `torch.use_deterministic_algorithms(True)` not used for speed | No; differences below 0.1 points observed |

## 6. Hyperparameters

<!-- Guidance: Report final values and how they were chosen, including the search space, budget and selection metric. Selection on the test set invalidates the evaluation. -->

| Hyperparameter | Search range | Final value | Selection method |
|---|---|---|---|
| Learning rate | [1e-5 to 1e-3, log scale] | [3e-4] | *Example:* Best mean validation accuracy over 3 seeds |
| Batch size | [32, 64, 128] | [64] | [Method] |
| [Hyperparameter] | [Range] | [Value] | [Method] |

Search budget: [number of trials, method such as random search or
Bayesian optimisation].

## 7. Compute budget

<!-- Guidance: Report hardware, time and total compute, including failed and tuning runs, so others can judge the cost of reproduction. -->

| Item | Value |
|---|---|
| Hardware | [8 x GPU model, 80 GB] |
| Time per main run | [6 GPU hours] |
| Number of runs (final) | [5 seeds x 4 configurations] |
| Total including tuning | *Example:* about 1,200 GPU hours |
| Estimated energy or emissions | [Value and method, if reported] |

## 8. Statistical reporting

<!-- Guidance: Report variance across runs, not a single best run. State what the error bars mean. -->

Report the mean and spread over $k$ independent runs:

$$
\bar{x} = \frac{1}{k}\sum_{i=1}^{k} x_i, \qquad
s = \sqrt{\frac{1}{k-1}\sum_{i=1}^{k}\left(x_i - \bar{x}\right)^2}, \qquad
\text{95\% CI} \approx \bar{x} \pm t_{k-1,\,0.975}\,\frac{s}{\sqrt{k}}
$$

| Item | Answer |
|---|---|
| Number of runs per configuration | [5] |
| What error bars show | [Standard deviation / standard error / 95% CI] |
| Significance testing | [Paired bootstrap / permutation test / none] |
| Multiple comparisons | [Correction method or "not applicable"] |

## 9. Evaluation details

<!-- Guidance: Describe metrics exactly, including implementation, averaging and any filtering. Small differences in evaluation code often explain failed reproductions. -->

| Item | Detail |
|---|---|
| Metrics | *Example:* Macro-averaged F1 computed with scikit-learn 1.5.1 |
| Test set | [Name, size, version; never used for tuning] |
| Baselines | [How obtained: rerun with our code / numbers copied from paper] |
| Preprocessing at test time | [Same as training / differences] |
| Post-processing | [Thresholds, calibration] |

## 10. Limitations

<!-- Guidance: State what the results do not show. Include known failure modes and the conditions under which results may not hold. -->

- [Results shown only on [languages / domains / data sizes]]
- [Sensitivity to [hyperparameter or seed] observed]
- [Parts that could not be made deterministic]
- [Known differences between released code and the paper's runs]

## 11. Verification record

<!-- Guidance: Record an independent rerun. It is the strongest evidence that the checklist is true. -->

| Date | Verifier | Result rerun | Outcome | Notes |
|---|---|---|---|---|
| YYYY-MM-DD | [@name] | *Example:* Table 2, row 3 | Reproduced within 0.2 points of reported mean | Used released container |

## Standard and licence note

Based on common practice for computational reproducibility in research
software and machine learning, with topics similar to conference
reproducibility checklists. This template is original text written for
this collection. Template licence: MIT.