willtanoe
9fe584d08e
Copy analyze_sweep.py, check_noise.py, compare_with_grid.py and compare_repro.py out of the git-ignored results/ tree into scripts/sweep/. Replace the implicit script-location repository root with a --repo-root argument (relative default) without changing any computation. Outputs were verified byte-identical to the originals on a scratch copy of the data.
3.2 KiB
3.2 KiB
Sweep analysis scripts
Read-only analysis helpers for the completed seed-0 sweeps. They were originally
written inside results/sweep/ and results/sweep_repro/ (both git-ignored);
these copies are the version-controlled source.
All four scripts accept --repo-root (default: the repository that contains
scripts/sweep/, i.e. two levels up from the script). Data is read from the
sweep run directories and, where noted, from the earlier grid runs and
artifacts/analysis/. Nothing is retrained.
analyze_sweep.py
Inputs
results/sweep/runs/<dataset>__alpha<alpha>__seed0__support_weighted__beta<beta>/COMPLETE.../results/<run>/val_selection.json.../results/<run>/test_global_metrics.jsonresults/<run>/val_selection.jsonandtest_global_metrics.jsonfor the earlier grid runs (stage 3)
Outputs
results/sweep/analysis/selection.csv(validation only)results/sweep/analysis/test_at_selected_beta.csvresults/sweep/analysis/repro_vs_previous.csv
Command
python3 scripts/sweep/analyze_sweep.py
# or, against a different tree:
python3 scripts/sweep/analyze_sweep.py --repo-root /path/to/repo
check_noise.py
Inputs
results/sweep/runs/<run>/results/<run>/{history.json,val_selection.json,test_global_metrics.json}results/<run>/{history.json,val_selection.json,test_global_metrics.json}(earlier grid)
Outputs
results/sweep/analysis/noise_checks.json
Command
python3 scripts/sweep/check_noise.py
# or:
python3 scripts/sweep/check_noise.py --repo-root /path/to/repo
compare_with_grid.py
Inputs
results/sweep/analysis/selection.csv(runanalyze_sweep.pyfirst)artifacts/analysis/final__six_condition_master.csvartifacts/analysis/<dataset>__alpha<alpha>__multiseed__validation_per_seed.csv
Outputs
results/sweep/analysis/compare_with_grid.csv
Command
python3 scripts/sweep/compare_with_grid.py
# or:
python3 scripts/sweep/compare_with_grid.py --repo-root /path/to/repo
compare_repro.py
Inputs
results/sweep_repro/a_1thread_repeat/results/<run>/results/sweep_repro/b_8thread/results/<run>/results/sweep/runs/<run>/results/<run>/results/<run>/(earlier grid)- prototype snapshots under
results/sweep_repro/*/analysis/,results/sweep/runs/<run>/analysis/andartifacts/analysis/
where <run> is xiiotid__alpha0.25__seed0__support_weighted__beta0.
Outputs
results/sweep_repro/compare_repro.json- a compact comparison table on stdout
Command
python3 scripts/sweep/compare_repro.py
# or:
python3 scripts/sweep/compare_repro.py --repo-root /path/to/repo
Notes
results/is git-ignored. These scripts write only intoresults/sweep/analysis/andresults/sweep_repro/; they never write into a tracked file.analyze_sweep.pyopens validation data in stage 1 and only opens test data afterwards, by construction. The scripts do not select or recommend a beta for the thesis; they print descriptive tables.- The computation is unchanged from the originals. The only change is that the
repository root is a
--repo-rootargument with a relative default instead of being derived from the script's old location insideresults/.