docs: add repeat and multi-thread test (section 3.6) for the 1-thread sweep
This commit is contained in:
1 parent
ad203eb2ee
commit
b40c928051
2 files changed
+49
-11
No files matched your search
@@ -336,11 +336,18 @@ Validation-selected beta (seed 0) and test rare-class macro-F1 (global) afterwar
|
||||
confounded** with host (8 threads / 29 GiB documented earlier vs 12 / 48 GiB now), possibly a
|
||||
different PyTorch version (not recorded in metadata), and a different checkout path. A sign test
|
||||
(p = 0.86 / 1.0) is reported only descriptively because the pairs are not independent.
|
||||
- **Write-up:** `docs/research/SWEEP_1THREAD_SEED0_CHECK.md` (untracked, not yet committed)
|
||||
summarizes the sweep, the run-to-run difference, the noise checks and the comparison above.
|
||||
- **Write-up:** `docs/research/SWEEP_1THREAD_SEED0_CHECK.md` (committed in `ad203eb`; section 3.6
|
||||
was added afterwards and is uncommitted) summarizes the sweep, the grid-vs-sweep difference,
|
||||
the noise checks and the comparison above.
|
||||
- **Repeat / multi-thread test** (`results/sweep_repro/`, X-IIoTID alpha 0.25 seed 0 beta 0 only):
|
||||
a 1-thread repeat is bit-identical to the sweep run (all result JSON and the prototype
|
||||
snapshot), so the 1-thread sweep has no run-to-run noise. An 8-thread run on this host differs
|
||||
from the 1-thread run (test rare-F1 1.9e-3) and also does not reproduce the old grid (2.5e-3).
|
||||
Thread count changes results but does not explain the whole grid-vs-sweep difference; host and
|
||||
PyTorch version remain uncontrolled. 5G-NIDD and an 8-thread repeat were not tested.
|
||||
- **Exact next task:** decide with the user whether the 5G-NIDD alpha 0.5 / 0.75 disagreement
|
||||
warrants investigating the thread-count hypothesis (needs extra runs; not started) and whether
|
||||
to commit the handoff and the write-up. Do not change the primary pooled selection on the basis of this single-seed re-run.
|
||||
warrants a 5G-NIDD repeat / multi-thread test (needs extra runs; not started) and whether to
|
||||
commit section 3.6. Do not change the primary pooled selection on the basis of this single-seed re-run.
|
||||
- **Housekeeping:** the tmux session `sweep` still exists (windows `control`, `pool`); the
|
||||
`heartbeat` window was stopped on 2026-10-29. Backups of edited sweep scripts are `*.bak` in
|
||||
`results/sweep/`; orphaned attempt-1 artifacts are in `results/sweep/_orphans_20261027T011213Z/`.
|
||||
|
||||
Reference in new issue
Block a user