docs: add repeat and multi-thread test (section 3.6) for the 1-thread sweep

This commit is contained in:
willtanoe committed 2026-10-29 12:57:21 +01:00
1 parent ad203eb2ee
commit b40c928051
2 files changed
+49 -11

No files matched your search

+11 -4
View File
@@ -336,11 +336,18 @@ Validation-selected beta (seed 0) and test rare-class macro-F1 (global) afterwar
confounded** with host (8 threads / 29 GiB documented earlier vs 12 / 48 GiB now), possibly a
different PyTorch version (not recorded in metadata), and a different checkout path. A sign test
(p = 0.86 / 1.0) is reported only descriptively because the pairs are not independent.
- **Write-up:** `docs/research/SWEEP_1THREAD_SEED0_CHECK.md` (untracked, not yet committed)
summarizes the sweep, the run-to-run difference, the noise checks and the comparison above.
- **Write-up:** `docs/research/SWEEP_1THREAD_SEED0_CHECK.md` (committed in `ad203eb`; section 3.6
was added afterwards and is uncommitted) summarizes the sweep, the grid-vs-sweep difference,
the noise checks and the comparison above.
- **Repeat / multi-thread test** (`results/sweep_repro/`, X-IIoTID alpha 0.25 seed 0 beta 0 only):
a 1-thread repeat is bit-identical to the sweep run (all result JSON and the prototype
snapshot), so the 1-thread sweep has no run-to-run noise. An 8-thread run on this host differs
from the 1-thread run (test rare-F1 1.9e-3) and also does not reproduce the old grid (2.5e-3).
Thread count changes results but does not explain the whole grid-vs-sweep difference; host and
PyTorch version remain uncontrolled. 5G-NIDD and an 8-thread repeat were not tested.
- **Exact next task:** decide with the user whether the 5G-NIDD alpha 0.5 / 0.75 disagreement
warrants investigating the thread-count hypothesis (needs extra runs; not started) and whether
to commit the handoff and the write-up. Do not change the primary pooled selection on the basis of this single-seed re-run.
warrants a 5G-NIDD repeat / multi-thread test (needs extra runs; not started) and whether to
commit section 3.6. Do not change the primary pooled selection on the basis of this single-seed re-run.
- **Housekeeping:** the tmux session `sweep` still exists (windows `control`, `pool`); the
`heartbeat` window was stopped on 2026-10-29. Backups of edited sweep scripts are `*.bak` in
`results/sweep/`; orphaned attempt-1 artifacts are in `results/sweep/_orphans_20261027T011213Z/`.