thesis-prototype-agregation/scripts/sweep

Sweep analysis scripts

Read-only analysis helpers for the completed seed-0 sweeps. They were originally written inside results/sweep/ and results/sweep_repro/ (both git-ignored); these copies are the version-controlled source.

All four scripts accept --repo-root (default: the repository that contains scripts/sweep/, i.e. two levels up from the script). Data is read from the sweep run directories and, where noted, from the earlier grid runs and artifacts/analysis/. Nothing is retrained.

analyze_sweep.py

Inputs

• results/sweep/runs/<dataset>__alpha<alpha>__seed0__support_weighted__beta<beta>/COMPLETE
• .../results/<run>/val_selection.json
• .../results/<run>/test_global_metrics.json
• results/<run>/val_selection.json and test_global_metrics.json for the earlier grid runs (stage 3)

Outputs

• results/sweep/analysis/selection.csv (validation only)
• results/sweep/analysis/test_at_selected_beta.csv
• results/sweep/analysis/repro_vs_previous.csv

Command

python3 scripts/sweep/analyze_sweep.py
# or, against a different tree:
python3 scripts/sweep/analyze_sweep.py --repo-root /path/to/repo

check_noise.py

Inputs

• results/sweep/runs/<run>/results/<run>/{history.json,val_selection.json,test_global_metrics.json}
• results/<run>/{history.json,val_selection.json,test_global_metrics.json} (earlier grid)

Outputs

• results/sweep/analysis/noise_checks.json

Command

python3 scripts/sweep/check_noise.py
# or:
python3 scripts/sweep/check_noise.py --repo-root /path/to/repo

compare_with_grid.py

Inputs

• results/sweep/analysis/selection.csv (run analyze_sweep.py first)
• artifacts/analysis/final__six_condition_master.csv
• artifacts/analysis/<dataset>__alpha<alpha>__multiseed__validation_per_seed.csv

Outputs

• results/sweep/analysis/compare_with_grid.csv

Command

python3 scripts/sweep/compare_with_grid.py
# or:
python3 scripts/sweep/compare_with_grid.py --repo-root /path/to/repo

compare_repro.py

Inputs

• results/sweep_repro/a_1thread_repeat/results/<run>/
• results/sweep_repro/b_8thread/results/<run>/
• results/sweep/runs/<run>/results/<run>/
• results/<run>/ (earlier grid)
• prototype snapshots under results/sweep_repro/*/analysis/, results/sweep/runs/<run>/analysis/ and artifacts/analysis/

where <run> is xiiotid__alpha0.25__seed0__support_weighted__beta0.

Outputs

• results/sweep_repro/compare_repro.json
• a compact comparison table on stdout

Command

python3 scripts/sweep/compare_repro.py
# or:
python3 scripts/sweep/compare_repro.py --repo-root /path/to/repo

Notes

• results/ is git-ignored. These scripts write only into results/sweep/analysis/ and results/sweep_repro/; they never write into a tracked file.
• analyze_sweep.py opens validation data in stage 1 and only opens test data afterwards, by construction. The scripts do not select or recommend a beta for the thesis; they print descriptive tables.
• The computation is unchanged from the originals. The only change is that the repository root is a --repo-root argument with a relative default instead of being derived from the script's old location inside results/.