How to Reproduce a Research Codebase on a New Workstation

A public research repository can still be surprisingly hard to run. The paper may describe the algorithm well, while the working environment lives only in one lab member’s shell history: a particular commit, an unrecorded parameter, a vendor SDK, a driver version, or a data layout that the code silently assumes.

The fastest way through this is usually not to rebuild the whole project. Pick one stage that answers a real decision, freeze its inputs, and make a small reproduction packet around it. You want to know exactly what ran, what failed, and whether the next step is worth the time.

Name one stage and one observable result

“Reproduce the repository” is too broad. A useful target sounds more like this:

At commit REVISION, run scripts/make_plot.py against one approved sample and produce a non-empty PNG plus an exit code of zero.

The observable result may be a file, a table with a known schema, a test that passes, or a log line tied to a documented output. Name it before changing the environment. Also name what the test does not cover. A visualization script does not validate acquisition hardware; a preprocessing stage does not reproduce a paper’s final metric.

This narrow boundary prevents an installation problem from turning into an open-ended attempt to rediscover the entire research program.

Freeze the source before debugging it

Record the exact revision and working-tree state first:

mkdir -p repro/environment repro/output
git rev-parse --verify HEAD > repro/environment/git-commit.txt
git status --short > repro/environment/git-status.txt
git submodule status --recursive > repro/environment/git-submodules.txt

`git rev-parse –verify` resolves the revision to a concrete object name. The status file matters just as much: a clean commit and a commit plus three local patches are different inputs.

If the repository has no release tag or environment lock, do not quietly invent one and call it upstream. Keep your reproduction patch separate and list every change. A one-line compatibility edit can be legitimate, but it belongs in the failure-and-fix record.

Record the machine without copying secrets

Capture the facts that can change execution:

python -VV > repro/environment/python.txt 2>&1
python -m platform > repro/environment/platform.txt
python -m pip --version > repro/environment/pip.txt
python -m pip freeze --all > repro/environment/packages.txt

if command -v nvidia-smi >/dev/null 2>&1; then
  nvidia-smi --query-gpu=name,driver_version --format=csv,noheader > repro/environment/gpu.txt
fi

The Python `platform` module provides a compact system and interpreter description. The package snapshot is evidence of what was installed, not a perfect lock: the official `pip freeze` documentation explicitly distinguishes an installed-package report from a lockfile or dependency-solver result.

For GPU code, record the GPU, driver, toolkit used to build extensions, and the CUDA runtime reported by the framework. NVIDIA’s CUDA compatibility guide explains why “CUDA 12” alone is not enough: driver, toolkit, compatibility mode, and requested features all affect whether a binary can run.

Before sharing the packet, inspect it for usernames, hostnames, private paths, tokens, network addresses, license data, and device serial numbers. Reproducibility needs technical identity, not personal or credential data.

Write the input contract before moving the data

Many failures that look like dependency problems are actually input-shape problems. Record only the metadata needed to identify the contract:

• file type and array or table keys;
• dimensions, dtype, units, ordering, and coordinate convention;
• expected value ranges and missing-value rules;
• a SHA-256 digest of the exact test fixture;
• whether the sample is synthetic, public, licensed, or supplied privately.

Keep customer or laboratory data out of a public packet. A small synthetic fixture is often enough to prove that the file/interface path works. It is not enough to claim scientific equivalence to the real experiment.

Start with the project’s own environment path

Use the repository’s documented setup before introducing a new package manager or container. If it provides a lockfile, environment YAML, container definition, or exact installation script, preserve that path and its version.

When a container is part of the method, record the image digest as well as the friendly tag. Docker’s `image pull` documentation shows how a digest pins one immutable image version; a tag can move later.

If no lock exists, create a clean environment and keep the command history. Do not “repair” the workstation by upgrading the global Python, replacing its system CUDA installation, or adding unrelated package channels until you know which dependency actually blocks the stage.

Preserve the first real failure

Run the exact command once and capture both output streams and the exit status:

set +e
python -u scripts/make_plot.py --input sample/input.npz --output repro/output/result.png > repro/output/run.log 2>&1
run_status=$?
set -e
echo "$run_status" > repro/output/exit-code.txt

Do not overwrite the first failure with the final successful log. It often contains the most useful information in the whole exercise: the missing shared library, unexpected key, shape mismatch, or unsupported device capability.

For each attempted fix, record four things:

1. the observed error;
2. the smallest change made;
3. the reason for that change;
4. the result of the next run.

This turns “I installed a few things until it worked” into a path another person can inspect and challenge.

Hash the packet, then decide

The final packet should keep the evidence together:

repro/
├── environment/
│   ├── git-commit.txt
│   ├── git-status.txt
│   ├── platform.txt
│   ├── python.txt
│   ├── packages.txt
│   └── gpu.txt
├── output/
│   ├── run.log
│   ├── exit-code.txt
│   └── result.png
├── input-contract.md
├── failure-ledger.md
└── report.md

Hash the selected fixture, scripts, logs, and outputs with SHA-256. Python’s `hashlib` provides SHA-256 on every supported build, and ordinary sha256sum is sufficient on a Linux workstation. The digest does not prove that a scientific result is correct; it proves which bytes the report refers to.

End with a decision rather than a vague success statement:

• GO: the named stage completed on the recorded setup and produced the agreed observable.
• GO WITH LIMITS: the stage completed after documented changes, but portability or data assumptions remain.
• NO-GO: the stage is blocked by a named dependency, unavailable input, license, hardware requirement, or failed acceptance check.

A narrow no-go is valuable. It tells the team what must change before it spends days on the next layer.

A concrete OpenHI example

I used this method on one public stage of OpenHI, an event-based hyperspectral imaging project. At revision 080ad074…, the weighted-cumulative visualization script completed headlessly against a deterministic synthetic fixture containing 4,096 events. The packet records Python 3.10.13, NumPy 2.2.6, Matplotlib 3.10.9, the exact command and log, 120 time bins, the generated PNG, and hashes for the inputs and outputs.

The result is deliberately narrow: GO for that file/interface and visualization path; NO-GO as evidence for acquisition, learned compensation, calibration, reconstruction accuracy, hardware compatibility, or the paper’s scientific result.

You can inspect the complete OpenHI software-stage sample before using the pattern on your own code.

If the decision concerns one OpenHI stage on an existing workstation, start with the metadata-only fit check. The optional fixed USD 500 sprint applies the same environment, command, output, failure-ledger, and go/no-go structure to one agreed stage and one rights-cleared dataset. Hardware and new scientific development remain separate.

The useful order is simple: name the stage, freeze the source, describe the input, record the environment, preserve the first failure, and decide from the evidence.