# PHerc0332 spiral fit: reproduction recipes (repro/)

This directory extends the 2026-11-05 PHerc0332 spiral-fit dataset one level up (`../README.md`, `../eval/DEPENDENCIES.md`).
It holds what you need to re-score, and to refit, the reference fits:

| Reference | Seeds | Dataset arm | Launcher (as recorded) |
|---|---|---|---|
| #81 (`Vb_hub1030h_trim_25k`) | s1, s2 | dataset_tta_hub1030h_trim | s1 `work/chain_trim.sh`; s2 `work/chain_nolas.sh` |
| AW | s1, s2 | dataset_tta_hub1030h_trim_aircrop | `work/chain_whole.sh` (density hook: capture only) |
| SWA | s1, s2 | dataset_tta_hub1030h_trim_aircrop | `work/chain_whole.sh` (hook compares against `launch/work/density/trackhash_Vb_AW_s1_25k.json`) |
| M6 | s1, s2 | dataset_tta_hub1030h_trim_aircrop_m1106 | `work/chain_whole.sh` (capture only) |
| M8 | s1, s2 | dataset_tta_hub1030h_trim_aircrop_m1108 | `work/chain_whole.sh` (capture only) |

Every one of these ran from the frozen code tree `s1_f327248db2c1f694`. That tree is `../eval/frozen_source_s1_f327248db2c1f694.tar.zst`;
in git it is branch `pherc0332-repro-s1_f327` (tag `pherc0332-s1_f327-2026-11-09`) of github.com/ScrollPrize/villa.
Each reference is two stages: Va (10k iterations from scratch), then Vb (resumed from Va's checkpoint, to 25k).
The reference metric is Vb's `scores_trim.json`.

## Contents
- `recipes/<ref>/<seed>/{Va,Vb}/`: the run's `manifest.json` (verbatim; it records the frozen root, the source, venv
  and inputs hashes, the full overrides, the env, and the parent checkpoint hash), `overrides.json` (its sha256 equals
  the manifest's `overrides_sha256`), and the published scores (`scores_trim.json` is the reference metric; `scores.json` is the untrimmed one).
  `recipes/<ref>/<seed>/lineage.json` holds the checkpoint sha256 of both stages, a check that Vb's parent is Va, the launcher, and
  the launch records. `recipes/INDEX.json` indexes all of it.
- `runs/<ref>/<seed>/{Va,Vb}/`: the fitted checkpoints (`checkpoint_fitted.ckpt`), `manifest.json`, `satisfied_fitted.json`
  (the scorer reads it for held_patches), the anchor audit and the satisfaction summary.
- `arms/<arm>/dataset_manifest.json`: every file of each dataset arm, with its path, type (file or symlink plus target), sha256 and size.
  M8's arm lists 1,346,776 entries. These manifests are what `assemble_arm.py` rebuilds from.
  `arms/<arm>/mtimes.json.gz` holds each file's original modification time. The track loaders bind their packed-track
  and crossing caches to the DBM files' (name, size, mtime) signature, so a byte-identical copy with new mtimes would
  silently rebuild those caches. The assembler restores the recorded mtimes and checks them.
- `members.sha256`: a verified index of every published file **and every member of every published .tar.zst**.
  Each line is `<sha256>  <size>  <path>[::<member>]`. Each archive was checked against `../INVENTORY.json` before its members were hashed.
- `delta/`: data the reference arms use that the 2026-11-05 upload did not include. It has two archives:
  - `lasagna_inputs.tar.zst`: the #81-era raw lasagna zarrs.
  - `other.tar.zst`: M6/M8 strip patches and a track index.

  Each archive has a `<name>.members.sha256` listing its members. Every member was sha256-verified against its arm's manifest when packed.
- `launch/`: the launchers and the recorded launches.
  - Scripts: `work/chain_whole.sh`, `work/chain_trim.sh`, `work/chain_nolas.sh`, `work/fiber_jump_map_sheet.py`, the density sampler hook (`work/density/hook/sitecustomize.py`) and the AW s1 fingerprint that SWA compared against.
  - Logs: the `chain_trim` / `chain_nolas` logs.
  - Per invocation (AW, SWA, M6, M8), in `invocations/`: the brief, `steps.log`, `chain.log`, `inner.out` and the receipt.
- New tools, not part of the frozen tree: `assemble_arm.py`, `run_reference.sh`, `compare_scores.py`, `check_track_caches.py`, and `dataset_manifest.py` (a copy of the tool that wrote the arm manifests).

## Environment
Use a complete clone of the git branch above, which is the monorepo at 30b4d409c plus the frozen tree.
`spiral-fitting/pyproject.toml` needs the editable siblings `../vc_object_store` and `vesuvius/`. Then:

    cd spiral-fitting && uv sync --frozen        # Python 3.14 (.python-version), torch 2.11.0+cu128

`vc_object_store` is a C++ extension that is built during the sync. The authoritative list of its native
prerequisites is `env/cmake_configure.log` (every find_package / pkg_check_modules result from a fresh build). In
short: CMake, pkg-config, a C++23 compiler, SQLite, OpenSSL, nlohmann_json, Zstd, OpenCV, OpenMP, TIFF, CURL, and,
through vc_reader, ZLIB, Blosc and the native vc_delta3d it fetches.

Checked on a stock host: `env/minimal_host/` (Dockerfile + run.sh) builds the env on a clean Ubuntu 26.04 image from only these apt packages:
`cmake ninja-build pkg-config g++ make python3.14 python3.14-dev python3.14-venv libsqlite3-dev libssl-dev nlohmann-json3-dev
libzstd-dev libopencv-dev libtiff-dev libcurl4-openssl-dev zlib1g-dev libblosc-dev` (plus ca-certificates, curl, git and uv 0.13.1).
On 2026-11-10 that build passed (`env/minimal_host/minimal_host.log`). Torch 2.11.0+cu128, vc_object_store and vc_spiral import, and
`pip_freeze_minimal.txt` matches the fresh env package for package. Only the editable source paths differ. The list is enough on a stock host;
it was not tested for minimality by removing packages one by one. Use `libopencv-dev`, not `libopencv-core-dev`: the core package lacks
`OpenCVConfig.cmake`, so the first attempt failed at configure (`minimal_host_attempt1_opencv.log`).

`env/pip_freeze.diff` is the fresh environment against the original `../eval/venv.json`. The known difference is that
the original venv had vc-delta3d 0.1.0 while the lock pins 0.2.0. vc_delta3d is used only to pack lasagna sidecars, and
the fit packs sidecars only when they are missing (`lasagna_data.py:50`). Keep the published `../lasagna_sidecars/` in
place, and the fit never touches it.

## Re-score a reference (CPU, minutes)

    tar -xf ../eval/frozen_source_s1_f327248db2c1f694.tar.zst -C SRC      # SRC must be this extracted tar, not a git checkout
    repro/run_reference.sh score --data <this dataset dir> --src SRC --ref M6 --seed s1 --out OUT --py <env>/bin/python

The script:
- copies the frozen `work/` scripts and replaces ONLY the workstation path constants (OUT/relocation.diff). If any `/media/djosey` or `/home/djosey` string remains, it aborts.
- checks every input against its published sha256 in `members.sha256` before using it: the umbilicus, the trimmed anchor records, the gauge, the frozen `work/s1/holdout_eval_set.json`, and the full set of 51 slice-line files. It then runs the launchers' own trimmed-scoring command (`s1_score.py` on CPU).
- compares the result with the published `scores_trim.json`, leaf by leaf, with the same types and exact values. That includes the checkpoint, model and manifest hashes. Only the five relocatable path fields may differ. Exit 0 means identical.

Note: the reference fits were scored with the frozen `holdout_eval_set.json`, not `../eval/window_eval_set.json`.
The window set is for the window-arm (density) runs.

## Rebuild a dataset arm, refit (GPU, hours per stage)

    python3 repro/assemble_arm.py --data <this dataset dir> --manifest repro/arms/<arm>/dataset_manifest.json --root ROOT
    repro/run_reference.sh fit --data <this dataset dir> --src SRC --ref M6 --seed s1 --out OUT --arm-root ROOT --py <env>/bin/python

`assemble_arm.py` rebuilds the arm by sha256 from the loose files and the archive members, then re-lists it.
- **Layout:** ROOT must be new or empty. The script mirrors the original absolute layout under ROOT, and each absolute symlink target T becomes a relative link to ROOT/T. Nothing is written or resolved outside ROOT.
- **mtimes:** it restores each file's recorded mtime.
- **Verification:** the fresh listing must equal the original manifest entry for entry: path, type, sha256, size and mtime, with symlinks compared after that relocation.
- **Failure:** any missing or mismatched entry fails.
- **Track caches:** `check_track_caches.py SRC ROOT/.../<arm>` runs the frozen loader's own signature test on the rebuilt arm.
- **Cost:** about 33 GB and 1.2 to 1.35 million entries per arm. `--verify-only` re-checks an existing ROOT.

The `fit` mode:
- **Stages:** sources the frozen `work/s1/chain_s1.sh` exactly as the launchers did and runs its own `stage()`: Va from scratch, then Vb resumed from this replay's Va (or `--stage Vb` resumes from the published Va). Each stage keeps the recorded contract: a VALID parent, the run manifest, the untrimmed `scores.json`, and the horizon, audit and score-manifest checks. `--dry-run` prints each stage's command and environment without fitting.
- **Caches:** uses the launcher's FIT_CACHE name. The fitter rebuilds its caches from the arm when they are empty.
- **Overrides:** uses the recorded overrides. The one absolute path, `abs_winding_audit_records`, is rewritten to `../anchors/anchors_train_records.json`, which has the same sha256 as the original `work/s1_trim` file.
- **Density hook:** runs in each original stage's mode, then the trimmed scoring.

It is an *equivalent replay of the recorded commands*, not a byte-identical rerun: GPU training is not
bit-reproducible, so compare a refit at seed-noise level against the two published seeds.

## Hard-coded paths
The frozen tree's `work/` scripts hard-code `/media/djosey/nvme4/spiral_0332`, `/media/djosey/nvme4/fibers` and
`/home/djosey/Documents/villa/spiral-fitting` (see `run_reference.sh` for the substitution list). The `spiral-fitting/*.py`
runtime modules have none. Inside the recorded manifests and overrides those paths are kept verbatim. They are records, not inputs.
