docs: record 1-thread seed-0 sweep, noise checks and grid comparison

docs/handoff/SESSION_HANDOFF.md

@@ -268,3 +268,82 @@ and artifacts/analysis/final__*. Obey CLAIMS_REGISTER.md wording, keep all limit
 and do not run training or add inferential tests by default. Use .venv/bin/python. Do not commit
 or push unless asked.
 ```
+
+---
+
+## 15. Addendum (2026-10-29, WIB / UTC+7): 1-thread seed-0 sweep in `results/sweep/`
+
+This addendum is **separate from the 108-run grid above**, which remains the primary evidence. It
+records an additional deterministic-threading re-run of seed 0 only. Everything lives in the
+git-ignored `results/sweep/`; no tracked source, model or aggregation file was changed.
+
+- **What ran:** 30 runs = 2 datasets x 3 alpha (0.25/0.5/0.75) x 5 beta, seed 0, 1 thread per
+  process, pool P=3. Relaunched 2026-10-27T01:13Z (08:13 WIB); last run finished
+  2026-10-28T18:00Z (2026-10-29 01:00 WIB). 30/30 `completed exit 0`, 30 `COMPLETE` markers,
+  0 failed, retry pass was a no-op, `PHASE=done`.
+- **Gate deadlock (fixed):** `reference_check.py` read a wrong JSON key
+  (`val_rare_macro_f1_per_client_mean` instead of `val_rare_class_macro_f1_per_client_mean`), so the
+  gate never resolved. After the fix the reference check PASSED (max difference to the manual verdict
+  4.3e-10). Details, diffs and raw numbers: `results/sweep/relaunch_report.md` (untracked).
+- **One journal line to keep open:** `Oct 28 12:11:37 systemd[2118]: -.slice: A process of this unit
+  has been killed by the OOM killer` (host-local time, UTC+1 inferred from file mtimes, i.e.
+  18:11 WIB). The killed process is unidentified; no run scope failed and all 30 runs completed, so
+  it is probably not a training process (inference, not verified).
+- **Analysis script:** `results/sweep/analyze_sweep.py` (read-only, deterministic; re-running it
+  reproduced all three CSVs byte-identically). Stage 1 selects beta from validation only
+  (`val_rare_class_macro_f1_per_client_mean`, tie -> smaller beta) and is written before any test
+  file is opened. Outputs in `results/sweep/analysis/`: `selection.csv`,
+  `test_at_selected_beta.csv`, `repro_vs_previous.csv`.
+- **Status of this selection:** it is a **single-seed, per-seed diagnostic**. The primary thesis
+  selection remains pooled validation across clients and seeds (section 6, item 9).
+
+Validation-selected beta (seed 0) and test rare-class macro-F1 (global) afterwards:
+
+| dataset | alpha | selected beta | val margin to runner-up | test @ selected | test @ beta 0 |
+|---|---|---|---|---|---|
+| X-IIoTID | 0.25 | 0.75 | 0.0413 | 0.6692 | 0.6420 |
+| X-IIoTID | 0.50 | 0.50 | 0.0074 | 0.6439 | 0.6131 |
+| X-IIoTID | 0.75 | 0.50 | 0.0004 | 0.5068 | 0.5263 |
+| 5G-NIDD | 0.25 | 0.25 | 0.0155 | 0.8876 | 0.8612 |
+| 5G-NIDD | 0.50 | 0.75 | 0.0053 | 0.8793 | 0.8694 |
+| 5G-NIDD | 0.75 | 0.25 | 0.0132 | 0.9065 | 0.9024 |
+
+- **Reproducibility caveat (important):** the 1-thread re-run is **not bit-identical** to the earlier
+  grid for the same (dataset, alpha, seed 0, beta). Over 30 pairs the absolute test rare-F1
+  difference has median 0.0022 and max 0.0127 (5G-NIDD alpha 0.5 beta 0.75); the validation
+  selection metric differs by median 0.0029, max 0.0328; 0 of 30 pairs are exactly equal. Compared
+  with the median validation difference (0.0029), only X-IIoTID alpha 0.75 (margin 0.0004) is
+  clearly inside the noise. X-IIoTID alpha 0.5 (0.0074) and 5G-NIDD alpha 0.5 (0.0053) are above
+  the median but well below the max, so they are also not clearly separated. The other three
+  margins (0.0132 to 0.0413) are larger than the median difference; the max difference (0.0328)
+  still exceeds all but the X-IIoTID alpha 0.25 margin. None of this is a significance test.
+- **Comparison with the 108-run grid** (`results/sweep/compare_with_grid.py` ->
+  `results/sweep/analysis/compare_with_grid.csv`, validation only, no test file opened):
+  the sweep's seed-0 beta equals the grid's seed-0 beta in 4/6 conditions (all three X-IIoTID,
+  plus 5G-NIDD alpha 0.25). The two differences are both 5G-NIDD: alpha 0.5 (sweep 0.75, grid
+  seed-0 1.0; sweep pick is rank 3 in the grid ranking) and alpha 0.75 (sweep 0.25, grid 0.5;
+  rank 4). These two conditions also have the largest per-beta validation differences
+  (0.0328 and 0.0262). The sweep beta equals the grid **pooled** beta in only 1/6, but the grid's
+  own seed-0 beta equals its pooled beta in only 3/6, so a seed-0 vs pooled mismatch is expected
+  and is not a reproducibility measure.
+- **Not done:** no inferential test; thread count as the cause of the difference is a hypothesis, not
+  verified; the metric files were not cross-checked beyond what `analyze_sweep.py` reads.
+- **Noise checks** (`results/sweep/check_noise.py` -> `analysis/noise_checks.json`, no training):
+  all 30 pairs already differ at round 0 with identical sample counts; test rare-F1 differences
+  scatter around zero (14 higher / 16 lower, mean signed -0.0004, mean absolute 0.0034); 5G-NIDD is
+  about 2.7x noisier than X-IIoTID on the validation metric; beta-ranking Spearman is 0.30 / 0.50
+  for 5G-NIDD alpha 0.75 / 0.5 and >= 0.90 elsewhere. Thread count remains **unproven and
+  confounded** with host (8 threads / 29 GiB documented earlier vs 12 / 48 GiB now), possibly a
+  different PyTorch version (not recorded in metadata), and a different checkout path. A sign test
+  (p = 0.86 / 1.0) is reported only descriptively because the pairs are not independent.
+- **Write-up:** `docs/research/SWEEP_1THREAD_SEED0_CHECK.md` (untracked, not yet committed)
+  summarizes the sweep, the run-to-run difference, the noise checks and the comparison above.
+- **Exact next task:** decide with the user whether the 5G-NIDD alpha 0.5 / 0.75 disagreement
+  warrants investigating the thread-count hypothesis (needs extra runs; not started) and whether
+  to commit the handoff and the write-up. Do not change the primary pooled selection on the basis of this single-seed re-run.
+- **Housekeeping:** the tmux session `sweep` still exists (windows `control`, `pool`); the
+  `heartbeat` window was stopped on 2026-10-29. Backups of edited sweep scripts are `*.bak` in
+  `results/sweep/`; orphaned attempt-1 artifacts are in `results/sweep/_orphans_20261027T011213Z/`.
+  rsync destination was never specified and rsync was never run.
+- **Git:** this addendum is an uncommitted edit on branch `chore/posthoc-audit-and-docs-refresh`;
+  nothing was committed or pushed.