docs: add 5G-NIDD repeat and multi-thread test (section 3.7)

This commit is contained in:
willtanoe committed 2026-10-29 17:56:16 +01:00
1 parent b40c928051
commit e66ecc4b59
2 files changed
+46 -7

No files matched your search

+6 -3
View File
@@ -344,10 +344,13 @@ Validation-selected beta (seed 0) and test rare-class macro-F1 (global) afterwar
snapshot), so the 1-thread sweep has no run-to-run noise. An 8-thread run on this host differs
from the 1-thread run (test rare-F1 1.9e-3) and also does not reproduce the old grid (2.5e-3).
Thread count changes results but does not explain the whole grid-vs-sweep difference; host and
PyTorch version remain uncontrolled. 5G-NIDD and an 8-thread repeat were not tested.
PyTorch version remain uncontrolled. The same test was also run on 5G-NIDD alpha 0.5 beta 0.75
(the largest disagreement): the 1-thread repeat is again bit-identical to the sweep, and the
8-thread run again reproduces neither (closer to the sweep on test rare-F1, closer to the grid
on the validation metric). Still untested: an 8-thread repeat (its own stability).
- **Exact next task:** decide with the user whether the 5G-NIDD alpha 0.5 / 0.75 disagreement
warrants a 5G-NIDD repeat / multi-thread test (needs extra runs; not started) and whether to
commit section 3.6. Do not change the primary pooled selection on the basis of this single-seed re-run.
warrants an 8-thread repeat (is the multi-thread setup itself stable?) and whether to commit
section 3.7. Do not change the primary pooled selection on the basis of this single-seed re-run.
- **Housekeeping:** the tmux session `sweep` still exists (windows `control`, `pool`); the
`heartbeat` window was stopped on 2026-10-29. Backups of edited sweep scripts are `*.bak` in
`results/sweep/`; orphaned attempt-1 artifacts are in `results/sweep/_orphans_20261027T011213Z/`.