experiment: complete xiiotid three-alpha beta study

docs/handoff/SESSION_HANDOFF.md

@@ -3,34 +3,33 @@
 This is the canonical continuation point for the next agent session. It is written to be
 readable without prior chat history, terminal scrollback, or git history.
 
-Last updated: 2026-09-26 (Session 7, post-experiment audit complete). Session 7 type:
-**protocol-scaling / non-IID-severity experiment**. Session 6 completed X-IIoTID `alpha=0.50`
-for seeds `{0,1,2}`. Session 7 completed X-IIoTID **`alpha=0.25`** for seeds `{0,1,2}` across
-the five frozen beta candidates plus `original_protean`, using the **same pooled-across-seed
-validation-only** selection rule. **alpha=0.75 and 5G-NIDD were not run.** No commit or push
-was made.
+Last updated: 2026-09-26 (Session 8). Session 8 type: **protocol-scaling /
+non-IID-severity experiment**. Session 7 completed X-IIoTID `alpha=0.25`; Session 6 completed
+`alpha=0.50`. Session 8 completed X-IIoTID **`alpha=0.75`** for seeds `{0,1,2}` across the five
+frozen beta candidates plus `original_protean`, using the **same pooled-across-seed
+validation-only** selection rule, and produced the first **three-alpha (0.25/0.50/0.75)
+comparison** for X-IIoTID. **5G-NIDD was not run.** No commit or push was made.
 
-> **Audit status (Session 7).** Independent audit verdict: **PASS WITH MINOR CORRECTIONS**
-> (`docs/research/SESSION7_AUDIT.md`). `beta_star(alpha=0.25) = 0.00` confirmed from validation
-> only; leakage audit PASS; `beta=0` class **B**; source/data integrity PASS; pytest 75 passed,
-> 0 failed, 0 skipped, 0 warnings. Corrections were documentation-only (cross-alpha label
-> disambiguation + one wording nuance); no `beta_star` or scientific interpretation changed.
-> **Session 7 is safe to checkpoint.**
+> **Audit status (Session 8).** Independent audit verdict: **PASS** (`docs/research/SESSION8_AUDIT.md`).
+> `beta_star(alpha=0.75) = 0.50` confirmed from validation only; no corrections required; run
+> inventory 18/18; pairing hash-identical and reproduced from raw data; leakage audit PASS;
+> `beta=0` class **B**; source/data integrity PASS; pytest 75 passed. **Session 8 is safe to
+> checkpoint.**
 
-> **Headline (Session 7).** PRIMARY pooled-validation selected beta for X-IIoTID
-> `alpha=0.25` = **0.00** (validation only). The pooled validation curve is monotonically
-> **decreasing** in beta at alpha=0.25, whereas at alpha=0.50 it peaked at 0.75. Per-seed
-> diagnostics were 0.75 / 0.50 / 0.00. beta=0 is class **B** (semantically equivalent +
-> floating-point divergence) for all three seeds; no semantic discrepancy. The realized
-> alpha=0.25 partitions are materially more heterogeneous (19/27 class-condition cells with
-> `|S_j| < M` vs 7/27 at alpha=0.50). Effect remains small relative to between-seed variance.
+> **Headline (Session 8).** PRIMARY pooled-validation selected beta for X-IIoTID `alpha=0.75`
+> = **0.50** (validation only; per-seed diagnostics 0.50 / 0.50 / 0.00). The pooled validation
+> curve peaks at 0.50 and declines both sides, but the total between-beta spread (≈ 0.0081) is
+> much smaller than the between-seed std (≈ 0.045). `beta=0` is class **B** for all three seeds.
+> Realized alpha=0.75 partitions are the mildest of the three (3/27 cells with `|S_j| < M`,
+> 4 missing cells, client-ratio mean 4.67). **Three-alpha pooled selections: 0.25 → 0.00,
+> 0.50 → 0.75, 0.75 → 0.50 — not monotone in alpha.**
 
 ---
 
 ## 1. Project objective
 
-Build the code and experiment infrastructure for a thesis study that modifies **only the
-server-side prototype aggregation** of PROTEAN (Chennoufi et al., 2025), a federated
+Build the code and experiment infrastructure for a thesis study that modifies **only** the
+server-side prototype aggregation of PROTEAN (Chennoufi et al., 2025), a federated
 intrusion-detection framework. The mechanism weights each contributing client's class
 prototype by its class sample support:
 
@@ -55,327 +54,350 @@ bitwise identical to the baseline (see §7).
   `~/datasets/5G-NIDD/5G-NIDD.csv`
   `fa36f80859585f474504ca69eb951a079b35145537976c86b00aae3aab46ee59`.
 - Frozen baseline (`original_protean`) source unchanged; no `equal_available` mode added;
-  beta candidate set unchanged (`{0, .25, .5, .75, 1}`).
+  beta candidate set unchanged (`{0, .25, .5, .75, 1}`). No tracked source/test/config file
+  differs from HEAD.
 
-## 3. Session 7 objective and outcome
+## 3. Session 8 objective and outcome
 
-Objective: evaluate one additional non-IID severity (X-IIoTID `alpha=0.25`, seeds `{0,1,2}`)
-across the five beta candidates using the exact Session-6 three-seed pooled-validation
-protocol, and test whether the pooled-selected beta changes under a more heterogeneous
-condition. All 18 runs (15 `support_weighted` + 3 `original_protean`) completed with rc=0.
-Report: `docs/research/ALPHA025_BETA_VALIDATION.md`.
+Objective: evaluate the final X-IIoTID non-IID severity (`alpha=0.75`, seeds `{0,1,2}`) across
+the five beta candidates using the exact pooled-validation protocol, then compare alpha=0.25,
+0.50 and 0.75 using realized partition characteristics. All 18 runs (15 `support_weighted` +
+3 `original_protean`) completed with rc=0.
 
-Outcome: **PRIMARY pooled-validation selected beta = 0.00** (validation only);
-`test_used_for_selection = false` for all 15 support_weighted runs. Pooled means: 0.00 →
-0.591704, 0.25 → 0.588691, 0.50 → 0.585851, 0.75 → 0.579463, 1.00 → 0.532099 (monotone
-decreasing). Between-seed std 0.045–0.090; the effect is small relative to seed variance.
+Outcome: **PRIMARY pooled-validation selected beta = 0.50** (validation only);
+`test_used_for_selection = false` for all 15 runs. Pooled means: 0.00 → 0.471706,
+0.25 → 0.474368, 0.50 → 0.476749, 0.75 → 0.472541, 1.00 → 0.468605 (peak at 0.50). Between-seed
+std 0.0447–0.0553; between-beta spread ≈ 0.0081, so the effect is small relative to seed
+variance.
+
+Reports: `docs/research/ALPHA075_BETA_VALIDATION.md`,
+`docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md`, audit `docs/research/SESSION8_AUDIT.md`.
 
 ## 4. Condition and pairing
 
 | Field | Value |
 |---|---|
-| dataset / alpha / seeds | `xiiotid` / 0.25 / `{0,1,2}` |
+| dataset / alpha / seeds | `xiiotid` / 0.75 / `{0,1,2}` |
 | runs | 15 `support_weighted` + 3 `original_protean`, all new this session |
 | training | 10 clients, 10 rounds, 3 local epochs, batch 256, lr 1e-3, mu 0.1, lambda 1.0, Adam |
-| code revision | unchanged from HEAD `6b97a0d38b839c737c49f41006817503d0672871`; only docs/analysis artifacts added |
+| code revision | unchanged from HEAD `ff68d380e23859d21c07c8a3a7b56b1b7f06c91b`; only docs/analysis/scripts added |
 
-Manifests (reused, never regenerated; one content hash per seed across all six runs; each was
-reproduced from the raw dataset with split/`N_ij`/partition hashes verified):
+Manifests (reused, never regenerated; one content hash per seed across all six runs; each
+reproduced from the raw dataset with split/`N_ij`/partition hash verified):
 
 | seed | manifest | content hash | partition hash | split hash | rng seed |
 |---|---|---|---|---|---|
-| 0 | `artifacts/partitions/xiiotid__alpha0.25__seed0.json` | `839ba18e…` | `60faf299…` | `d0acc0c1…` | 1617371214 |
-| 1 | `artifacts/partitions/xiiotid__alpha0.25__seed1.json` | `84c20b4b…` | `6ffb10ae…` | `880a9f82…` | 1670706564 |
-| 2 | `artifacts/partitions/xiiotid__alpha0.25__seed2.json` | `d6e68ee1…` | `c8e1ad31…` | `8e0db0be…` | 795318573 |
+| 0 | `artifacts/partitions/xiiotid__alpha0.75__seed0.json` | `759f7e50…` | `473b1c4a…` | `d0acc0c1…` | 957653162 |
+| 1 | `artifacts/partitions/xiiotid__alpha0.75__seed1.json` | `4a50ba21…` | `a0ec9e9c…` | `880a9f82…` | 762050403 |
+| 2 | `artifacts/partitions/xiiotid__alpha0.75__seed2.json` | `922b7bb1…` | `00345255…` | `8e0db0be…` | 843543396 |
 
-Pairing proof: round-0 snapshot byte-identical across the five betas **and**
-`original_protean` within each seed — seed 0 `cd821beb…`, seed 1 `59d2d2bc…`, seed 2
-`47befeff…`. `train_support_Nij.csv`, `characterization.csv`, `rare_unseen.csv` identical
-across each seed's runs. `manifest_reload_ok = true`, `smoke = false` everywhere.
+Pairing proof: round-0 snapshot byte-identical across the five betas **and** `original_protean`
+within each seed — seed 0 `8c477df6…`, seed 1 `41f0187c…`, seed 2 `be2ad5fb…`.
+`train_support_Nij.csv`, `characterization.csv`, `rare_unseen.csv` identical across each seed's
+runs. `manifest_reload_ok = true`, `smoke = false` everywhere.
 
 ## 5. Validation results (USED FOR SELECTION) and pooled selection
 
 Primary metric: validation per-client-mean rare-class macro-F1. All 10 clients eligible in
-every seed. Full tables: `artifacts/analysis/xiiotid__alpha0.25__multiseed__validation_per_seed.csv`.
+every seed. Full tables:
+`artifacts/analysis/xiiotid__alpha0.75__multiseed__validation_per_seed.csv`.
 
 Per-seed validation per-client-mean rare-class macro-F1:
 
 | beta | seed 0 | seed 1 | seed 2 |
 |---|---|---|---|
-| 0.00 | 0.616380 | 0.492425 | 0.666307 |
-| 0.25 | 0.609913 | 0.514237 | 0.641922 |
-| 0.50 | 0.595247 | 0.537040 | 0.625267 |
-| 0.75 | 0.654033 | 0.535672 | 0.548685 |
-| 1.00 | 0.578385 | 0.530942 | 0.486970 |
+| 0.00 | 0.496803 | 0.408276 | 0.510038 |
+| 0.25 | 0.496892 | 0.421386 | 0.504827 |
+| 0.50 | 0.504388 | 0.422919 | 0.502942 |
+| 0.75 | 0.486868 | 0.421053 | 0.509702 |
+| 1.00 | 0.479146 | 0.419534 | 0.507136 |
 
 Pooled across seeds (`...__validation_pooled.csv`):
 
 | beta | pooled mean | std (sample) | min | max | selected |
 |---|---|---|---|---|---|
-| **0.00** | **0.591704** | 0.089529 | 0.492425 | 0.666307 | **yes** |
-| 0.25 | 0.588691 | 0.066435 | 0.514237 | 0.641922 | |
-| 0.50 | 0.585851 | 0.044858 | 0.537040 | 0.625267 | |
-| 0.75 | 0.579463 | 0.064906 | 0.535672 | 0.654033 | |
-| 1.00 | 0.532099 | 0.045718 | 0.486970 | 0.578385 | |
+| 0.00 | 0.471706 | 0.055329 | 0.408276 | 0.510038 | |
+| 0.25 | 0.474368 | 0.046055 | 0.421386 | 0.504827 | |
+| **0.50** | **0.476749** | 0.046624 | 0.422919 | 0.504388 | **yes** |
+| 0.75 | 0.472541 | 0.046028 | 0.421053 | 0.509702 | |
+| 1.00 | 0.468605 | 0.044742 | 0.419534 | 0.507136 | |
 
-- **PRIMARY selected beta (`beta_star_alpha025`) = 0.00**, max pooled validation
+- **PRIMARY selected beta (`beta_star_alpha075`) = 0.50**, max pooled validation
   per-client-mean rare-class macro-F1; **no tie-break used**; selection is validation-only.
-- **SECONDARY per-seed diagnostics only:** seed 0 → 0.75, seed 1 → 0.50, seed 2 → 0.00.
+- **SECONDARY per-seed diagnostics only:** seed 0 → 0.50, seed 1 → 0.50, seed 2 → 0.00.
 
 ## 6. Final test evaluation of the pooled-selected beta
 
-`beta_star = 0.00` was fixed from pooled validation before test was read.
+`beta_star = 0.50` was fixed from pooled validation before test was read.
 
 | metric (test) | seed 0 | seed 1 | seed 2 | mean ± std (sample) |
 |---|---|---|---|---|
-| rare-class macro-F1 | 0.641362 | 0.580555 | 0.674489 | 0.632136 ± 0.047642 |
-| rare-class recall | 0.830701 | 0.825699 | 0.807370 | 0.821257 ± 0.012283 |
-| macro-F1 | 0.674558 | 0.665039 | 0.704433 | 0.681343 ± 0.020555 |
-| accuracy | 0.874551 | 0.863490 | 0.884002 | 0.874015 ± 0.010267 |
-| weighted-F1 | 0.906156 | 0.899864 | 0.914627 | 0.906882 ± 0.007408 |
+| rare-class macro-F1 | 0.511889 | 0.624144 | 0.679704 | 0.605246 ± 0.085489 |
+| rare-class recall | 0.803887 | 0.876726 | 0.865681 | 0.848765 ± 0.039256 |
+| macro-F1 | 0.741977 | 0.761752 | 0.769179 | 0.757636 ± 0.014060 |
+| accuracy | 0.914365 | 0.932611 | 0.937352 | 0.928110 ± 0.012137 |
+| weighted-F1 | 0.935566 | 0.948344 | 0.951740 | 0.945216 ± 0.008528 |
 
 `original_protean` for the same seeds/condition:
 
 | metric (test) | seed 0 | seed 1 | seed 2 | mean ± std (sample) |
 |---|---|---|---|---|
-| rare-class macro-F1 | 0.639810 | 0.582522 | 0.676039 | 0.632790 ± 0.047152 |
-| rare-class recall | 0.829601 | 0.826700 | 0.809595 | 0.821965 ± 0.010811 |
-| macro-F1 | 0.673103 | 0.666981 | 0.705969 | 0.682018 ± 0.020967 |
-| accuracy | 0.873722 | 0.867341 | 0.885231 | 0.875431 ± 0.009067 |
-| weighted-F1 | 0.905466 | 0.902075 | 0.915416 | 0.907652 ± 0.006934 |
+| rare-class macro-F1 | 0.521540 | 0.614061 | 0.676068 | 0.603890 ± 0.077764 |
+| rare-class recall | 0.816440 | 0.883238 | 0.869835 | 0.856504 ± 0.035338 |
+| macro-F1 | 0.742471 | 0.747395 | 0.765680 | 0.751849 ± 0.012228 |
+| accuracy | 0.913254 | 0.913898 | 0.934576 | 0.920576 ± 0.012129 |
+| weighted-F1 | 0.932563 | 0.935937 | 0.948908 | 0.939136 ± 0.008629 |
 
 Descriptive difference for test rare-class macro-F1 (`beta_star` − `original_protean`) is
-+0.001552 / −0.001967 / −0.001550 (mean −0.000655) — i.e. essentially identical, as expected
-because `beta = 0` is mathematically the equal mean over `S_j`. Test is descriptive only; no
-superiority or generalisation is claimed.
+−0.009651 / +0.010083 / +0.003636 (mean +0.001356). Test is descriptive only; no superiority
+or generalisation claimed.
 
 ## 7. `beta = 0` equivalence result (all three seeds)
 
 For seeds 0, 1, 2, `support_weighted beta=0` vs frozen `original_protean`:
 
 - Same manifest/split/hashes: **yes**.
-- Round-0 snapshot byte-identical: **yes** (`cd821beb…`, `59d2d2bc…`, `47befeff…`).
+- Round-0 snapshot byte-identical: **yes** (`8c477df6…`, `41f0187c…`, `be2ad5fb…`).
 - `train_support_Nij.csv`, `characterization.csv`, `rare_unseen.csv`: **identical**.
-- Independent aggregation (`ΣC/n` vs `(1/n)ΣC`): ≤ 4.4e-16 (1 ULP ordering).
-- Test/history/per-client/per-class: **differ slightly** (history rare-macro-F1 max diff
-  3.6e-3–6.6e-3; test metric max diff 1.6e-3–3.9e-3).
+- Independent aggregation (`ΣC/n` vs `(1/n)ΣC`): 4.4e-16 / 4.4e-16 / 2.2e-16.
+- History rare-macro-F1 max diff (rounds ≥ 1): 1.10e-2 / 8.12e-3 / 3.64e-3; test metric max
+  diff 4.33e-3 / 1.14e-3 / 0.85e-3.
 - Classification: **B — mathematically equivalent + floating-point numerical divergence; no
   semantic discrepancy** in any seed. Machine-readable:
-  `artifacts/analysis/xiiotid__alpha0.25__multiseed__beta0_equivalence.csv`.
+  `artifacts/analysis/xiiotid__alpha0.75__multiseed__beta0_equivalence.csv`.
 
-Conditional on the shared `S_j` reproduction assumption; not proof `beta=0` reproduces
-original PROTEAN.
+Conditional on the shared `S_j` reproduction assumption; not proof `beta=0` reproduces original
+PROTEAN.
 
 ## 8. Partition characterization (realized)
 
-`|S_j| < M` class-condition cells and missing client-class cells (training split):
+`|S_j| < M` class-condition cells, missing client-class cells, and client-size ratios
+(training split):
 
-| condition | `|S_j| < M` cells | missing client-class cells | client-size max/min (seeds) |
-|---|---|---|---|
-| alpha=0.25 | **19 / 27** | **38** | 15.6, 20.8, 7.8 |
-| alpha=0.50 | 7 / 27 | 9 | 5.1, 7.2, 9.9 |
+| condition | `|S_j| < M` cells | missing client-class cells | client-size ratio (seeds) | union unseen | mean class CV |
+|---|---|---|---|---|---|
+| alpha=0.25 | **19 / 27** | **38** | 15.6, 20.8, 7.8 | 6/6/5 | 1.442 |
+| alpha=0.50 | 7 / 27 | 9 | 5.1, 7.2, 9.9 | 3/3/1 | 1.183 |
+| alpha=0.75 | **3 / 27** | **4** | 4.15, 5.07, 4.78 | 2/1/0 | 0.881 |
 
-Union unseen classes per seed: alpha=0.25 = 6/6/5 (max 4 on a client); alpha=0.50 = 3/3/1
-(max 2). Union rare classes per seed: alpha=0.25 = 8/7/8; alpha=0.50 = 6/7/7. All clients have
-exactly two rare classes at both alphas. Full per-class `N_ij`, CV and max/min in
-`docs/research/ALPHA025_BETA_VALIDATION.md` §B. The lower alpha does **not** uniformly increase
-every realized statistic (e.g. alpha=0.25 seed 2 client-size ratio 7.8 < alpha=0.50 seed 2's
-9.9), so interpretation uses the realized partitions, not alpha alone.
+Union rare classes per seed: alpha=0.75 = 4/5/6; all clients have exactly two rare classes.
+Full per-class `N_ij`, CV and max/min in `docs/research/ALPHA075_BETA_VALIDATION.md` §C and
+`artifacts/analysis/xiiotid__cross_alpha__partition_heterogeneity_*.csv`. Heterogeneity
+decreases with alpha overall but is **not uniformly monotone per seed** (alpha=0.75 seed 0
+ratio 4.15 < seed 2's 4.78; alpha=0.50 seed 2 ratio 9.9 > alpha=0.25 seed 2's 7.8), so
+interpretation uses realized partitions, not alpha alone.
 
 ## 9. Direct prototype-effect findings (fixed round-0 snapshots, no retraining)
 
-`||Cbar_j(1) - Cbar_j(0)||`, cross-seed mean (std), alpha=0.25 vs alpha=0.50:
+`||Cbar_j(1) - Cbar_j(0)||`, cross-seed mean, alpha=0.25 vs 0.50 vs 0.75 (mean across classes
+3.422 / 2.099 / **1.305**):
 
-| class | a25 mean (std) | a50 mean (std) |
-|---|---|---|
-| Exfiltration | 4.316 (0.898) | 2.020 (0.502) |
-| Tampering | 4.248 (0.241) | 2.354 (1.165) |
-| C&C | 4.020 (0.290) | 1.694 (0.505) |
-| Lateral_movement | 3.726 (1.383) | 2.724 (1.035) |
-| Exploitation | 3.464 (1.113) | 2.344 (0.412) |
-| Reconnaissance | 3.339 (1.107) | 1.002 (0.261) |
-| Weaponization | 3.124 (0.585) | 2.561 (1.228) |
-| RDOS | 2.566 (1.219) | 2.631 (1.165) |
-| Normal | 1.997 (0.533) | 1.557 (0.548) |
+| class | a25 | a50 | a75 |
+|---|---|---|---|
+| Tampering | 4.248 | 2.354 | 1.934 |
+| Exfiltration | 4.316 | 2.020 | 1.448 |
+| Exploitation | 3.464 | 2.344 | 1.539 |
+| Lateral_movement | 3.726 | 2.724 | 1.369 |
+| C&C | 4.020 | 1.694 | 1.284 |
+| Weaponization | 3.124 | 2.561 | 1.472 |
+| Reconnaissance | 3.339 | 1.002 | 0.983 |
+| Normal | 1.997 | 1.557 | 0.903 |
+| RDOS | 2.566 | 2.631 | 0.814 |
 
-Mean displacement across classes is larger at alpha=0.25 (3.42) than alpha=0.50 (2.10), because
-support is more imbalanced and more classes are missing contributors. Displacement is a
-direct-aggregation measure only, not a detection result. Outputs:
-`artifacts/analysis/xiiotid__alpha0.25__multiseed__prototype_effect_pooled.csv` and per-seed
+Displacement tracks realized support imbalance and is a direct-aggregation measure only, not a
+detection result. Outputs:
+`artifacts/analysis/xiiotid__alpha0.75__multiseed__prototype_effect_pooled.csv` and per-seed
 `...__seed{0,1,2}__support_weighted__beta0__prototype_snapshot__prototype_effect.csv`.
 
 ## 10. Client-level findings (validation, vs beta = 0)
 
-Detail: `...__multiseed__client_validation_detail.csv`; summary:
-`...__client_validation_summary.csv`.
+Detail: `...__client_validation_detail.csv`; summary: `...__client_validation_summary.csv`; and
+`artifacts/analysis/xiiotid__cross_alpha__cross_alpha_client_summary.csv`.
 
-- At alpha=0.25 the higher-beta response is **seed-dependent and often negative**:
-  improve/degrade/unchanged at beta=0.75 = 5/4/1 (seed 0, mean Δ +0.038), 5/4/1 (seed 1,
-  +0.043), 3/5/2 (seed 2, −0.118). At beta=1.00 seed 2 degrades 6/10 clients with a worst
-  client Δ = −0.678, while seed 1 still gains on average.
-- This **contrasts with alpha=0.50**, where beta=0.75 improved the majority of clients in every
-  seed. Client ids are per-seed realisations and are not one population; the analysis is
-  descriptive and non-causal, and `N_ij` vs ΔF1 remains non-monotonic.
+- At alpha=0.75, the response is smaller and seed-dependent. At the selected `beta=0.50`:
+  6/4/0 (seed 0, mean Δ +0.008), 9/0/1 (seed 1, +0.015), 6/4/0 (seed 2, −0.007; worst client
+  −0.152).
+- At a common `beta=0.75`: alpha=0.25 = 5/4/1, 5/4/1, 3/5/2 (often negative); alpha=0.50 =
+  7/3/0, 8/1/1, 8/1/1 (majority improve); alpha=0.75 = 3/7/0, 9/0/1, 5/5/0.
+- `N_ij` vs ΔF1 remains non-monotonic at every alpha. Client ids are per-seed realisations and
+  are not pooled.
 
-## 11. Comparison with alpha=0.50 (RQ3)
+## 11. Three-alpha comparison (RQ3)
 
-| | alpha=0.50 | alpha=0.25 |
-|---|---|---|
-| pooled-selected beta | 0.75 | **0.00** |
-| pooled validation curve | rises to 0.75 | monotone decreasing |
-| pooled mean range | 0.544–0.579 | 0.532–0.592 |
-| pooled std range | 0.075–0.100 | 0.045–0.090 |
-| `|S_j| < M` cells | 7/27 | 19/27 |
-| mean `||Cbar(1)−Cbar(0)||` | 2.10 | 3.42 |
+| | alpha=0.25 | alpha=0.50 | alpha=0.75 |
+|---|---|---|---|
+| pooled-selected beta | **0.00** | **0.75** | **0.50** |
+| pooled validation curve | monotone decreasing | rises to peak at 0.75 | peaks at 0.50 |
+| pooled mean range (spread) | 0.532–0.592 (0.060) | 0.544–0.579 (0.035) | 0.469–0.477 (0.008) |
+| pooled std range | 0.045–0.090 | 0.075–0.100 | 0.045–0.055 |
+| `|S_j| < M` cells | 19/27 | 7/27 | 3/27 |
+| mean `||Cbar(1)−Cbar(0)||` | 3.422 | 2.099 | 1.305 |
 
-The selection **changes** with non-IID severity. This is documented as a descriptive
-association accompanied by the realized partition characteristics; **no causal claim is made
-that alpha by itself caused the difference**. The effect is small relative to between-seed
-variance at both settings.
+The pooled-selected beta **changes** with the non-IID condition (0.00 → 0.75 → 0.50) but is
+**not monotone** in alpha. This is documented as a descriptive association accompanied by the
+realized partition characteristics; **no causal claim** is made. With only three alpha levels,
+three project-defined seeds, one dataset and no inferential trend test, monotonicity is not
+claimed.
 
-> **Audited label (do not confuse).** Condition-level **pooled** selections are alpha=0.50 →
-> **0.75** and alpha=0.25 → **0.00**. The value **0.50** is only the Session-5 **seed-0
-> single-condition** choice and the Session-6 alpha=0.50 **per-seed diagnostic** for seed 0;
-> it is not the alpha=0.50 pooled selection. Any "0.50 → 0.00" phrasing for condition-level
-> pooled selection is wrong. Verified in `docs/research/SESSION7_AUDIT.md` §M.
+> **Audited label (do not confuse).** Condition-level **pooled** selections are alpha=0.25 →
+> **0.00**, alpha=0.50 → **0.75**, alpha=0.75 → **0.50**. The value **0.50** is *not* the
+> alpha=0.50 pooled selection: it is only the Session-5 **seed-0 single-condition** choice, the
+> Session-6 alpha=0.50 **per-seed diagnostic** for seed 0, and now the alpha=0.75 pooled
+> selection. See `docs/research/SESSION7_AUDIT.md` §M and `SESSION8_AUDIT.md` §M.
 
 ## 12. Runtime / resources
 
-CPU only, no GPU, no swap, exit 0 on all 18 runs. Per-run wall-clock 39:49–42:12; CPU
-741–746 %; peak RSS 8.38 GiB (seed 0), 10.80 GiB (seed 1), 7.51 GiB (seed 2). New compute:
-18 runs, ≈ 43,799 s ≈ **12.17 h** wall-clock, sequential (8 cores). Machine-readable:
-`artifacts/analysis/xiiotid__alpha0.25__multiseed__runtime.csv`.
+CPU only, no GPU, no swap, exit 0 on all 18 runs. Per-run wall-clock 40:00–49:02 (two runs
+slower under transient host contention); CPU 714–746 %; peak RSS 6.49 GiB (seed 0), 6.91 GiB
+(seed 1), 7.20 GiB (seed 2). New compute: 18 runs, 44,371 s ≈ **12.33 h** wall-clock,
+sequential (8 cores). Machine-readable:
+`artifacts/analysis/xiiotid__alpha0.75__multiseed__runtime.csv`.
 
 ## 13. Methodological decisions / assumptions
 
-1. **`S_j` participant/denominator rule** (`S_j = {i | N_ij > 0 and valid C_ij}`, equal
-   weights in the baseline) is a **REPRODUCTION ASSUMPTION**; material in 19/27
-   class-condition cells at alpha=0.25 (7/27 at alpha=0.50). Carry this caveat in all reporting.
+1. **`S_j` participant/denominator rule** (`S_j = {i | N_ij > 0 and valid C_ij}`, equal weights
+   in the baseline) is a **REPRODUCTION ASSUMPTION**; material in 19/27 class-condition cells at
+   alpha=0.25, 7/27 at alpha=0.50, 3/27 at alpha=0.75. Carry this caveat in all reporting.
 2. `N_ij` is class-specific sample support, not prototype quality.
 3. Beta is one value for all classes/clients; candidates frozen at `{0,.25,.5,.75,1}`.
 4. **PRIMARY selection = pooled validation across clients and seeds**; per-seed selection is
    secondary stability diagnostics; test used only after the pooled beta is fixed.
-5. The `scripts/analyze_multiseed_beta.py` already supported `--alpha` (default 0.5) and
-   already includes alpha in output names; **no code change was needed**. alpha=0.50 outputs
-   were re-run and are byte-identical to the Session-6 artifacts (reproducibility verified).
+5. Three new read-only analysis scripts were added this session (no training/aggregation code
+   changed): `scripts/analyze_partition_heterogeneity.py`,
+   `scripts/verify_beta0_equivalence.py`, `scripts/analyze_cross_alpha.py`.
 6. `lambda=1.0`, Adam, lr 1e-3, batch 256 remain PROJECT DECISIONS. No `equal_available` mode.
 
 ## 14. Unresolved questions / blockers
 
 1. `S_j` reproduction ambiguity — unresolved; no official PROTEAN code exists.
-2. **Pooled beta selection remains statistically weak at both alphas (descriptive only):**
-   the across-seed sample std (0.045–0.090) is comparable to, or larger than, the total
-   between-beta pooled-mean spread (0.035 at alpha=0.50, 0.060 at alpha=0.25). No inferential
-   test was performed, so this is a descriptive comparison, not evidence of insignificance.
-3. `beta = 0` is not bitwise identical to `original_protean`; documentation decision on how to
-   report this remains open.
+2. **Pooled beta selection remains statistically weak at all three alphas (descriptive only):**
+   the across-seed sample std is comparable to or larger than the between-beta spread. No
+   inferential test was performed.
+3. `beta = 0` is not bitwise identical to `original_protean`; the documentation decision on how
+   to report this remains open.
 4. Session-2 manifests still store `preprocessing_hash = null` / `config_hash = null`.
-5. Only alpha=0.25 and 0.50, X-IIoTID, three seeds measured. No 5G-NIDD beta result.
-6. The alpha=0.25 result (pooled-selected beta = 0) is **one additional condition, not yet a
-   trend across alpha**. Whether any relationship between non-IID severity and the selected beta
-   is monotone would require alpha=0.75 (and is still not implied by two conditions).
+5. Only alpha=0.25/0.50/0.75, X-IIoTID, three seeds measured. No 5G-NIDD beta result.
+6. The three-alpha selected-beta sequence (0.00 / 0.75 / 0.50) is **not monotone**; no trend
+   claim is made and no inference has been run.
 
 ## 15. Git status
 
 No commit or push was made. `git status --short` shows only untracked docs/analysis/run
-artifacts (`results/` is gitignored; `artifacts/analysis/` is intentionally tracked):
+artifacts and the three new scripts (`results/` is gitignored; `artifacts/analysis/` is
+intentionally tracked):
 
 ```
  M docs/handoff/SESSION_HANDOFF.md
-?? docs/research/ALPHA025_BETA_VALIDATION.md
-?? docs/research/SESSION7_AUDIT.md
-?? artifacts/analysis/xiiotid__alpha0.25__*  (runs' snapshots, effect CSVs, multiseed tables)
-?? (pre-existing Session-5/6 untracked alpha=0.5 views, unchanged)
+?? docs/research/ALPHA075_BETA_VALIDATION.md
+?? docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md
+?? docs/research/SESSION8_AUDIT.md
+?? scripts/analyze_cross_alpha.py
+?? scripts/analyze_partition_heterogeneity.py
+?? scripts/verify_beta0_equivalence.py
+?? artifacts/analysis/xiiotid__alpha0.75__*  (runs' snapshots, effect CSVs, multiseed tables)
+?? artifacts/analysis/xiiotid__cross_alpha__*  (heterogeneity, cross-alpha tables)
+?? (pre-existing Session-5/6/7 untracked alpha=0.25/0.5 views, unchanged)
 ```
 
-`planningawal/` and both raw datasets are unchanged (audit re-verified SHA-256:
-X-IIoTID `7b929005…`, 5G-NIDD `fa36f808…`). No model/training/aggregation source changed
-(working-tree blobs equal HEAD). `artifacts/partitions/` unmodified. No alpha=0.75 run and no
-5G-NIDD `support_weighted` run exists. `equal_available` absent.
-`HEAD = 6b97a0d38b839c737c49f41006817503d0672871`. No commit or push was made.
+No tracked source/test/config file differs from HEAD. `planningawal/` and both raw datasets are
+unchanged (audit re-verified SHA-256: X-IIoTID `7b929005…`, 5G-NIDD `fa36f808…`).
+`artifacts/partitions/` unmodified. No 5G-NIDD `support_weighted` run exists. `equal_available`
+absent. `HEAD = ff68d380e23859d21c07c8a3a7b56b1b7f06c91b`. No commit or push was made.
 
-## 15b. Session 7 audit result (independent)
+## 16. Session 8 audit result
 
-- Verdict: **PASS WITH MINOR CORRECTIONS** (`docs/research/SESSION7_AUDIT.md`).
+- Verdict: **PASS** (`docs/research/SESSION8_AUDIT.md`).
 - Run inventory 18/18, pairing hash-identical, manifests reproduced from raw data.
-- Validation recomputed exactly; **pooled `beta_star(alpha=0.25) = 0.00`** (validation only,
-  monotone decreasing in beta). `beta_star` test and `original_protean` test means/stds exact.
-- Leakage audit: selection reads validation only; `test_used_for_selection=false` upheld.
+- Validation recomputed exactly; **pooled `beta_star(alpha=0.75) = 0.50`** (validation only).
+  `beta_star` test and `original_protean` test means/stds exact.
+- Leakage audit PASS; `test_used_for_selection=false` upheld.
 - `beta=0` equivalence: class **B** for all three seeds (no semantic discrepancy).
-- Source/data integrity: PASS. pytest: **75 passed, 0 failed, 0 skipped, 0 warnings**.
-- Corrections: cross-alpha label disambiguation (documentation-only) and one wording nuance.
-  No arithmetic, metric-label, std-convention, or provenance defect was found.
+- Direct-effect, client-level, rare/unseen, cross-alpha labels: PASS.
+- Source/data integrity PASS. pytest: **75 passed, 0 failed, 0 skipped, 0 warnings**.
 
-## 16. Files created / modified this session
+## 17. Files created / modified this session
 
 Created:
 
-- `docs/research/ALPHA025_BETA_VALIDATION.md`
-- `docs/research/SESSION7_AUDIT.md` (independent post-experiment audit)
-- `artifacts/analysis/xiiotid__alpha0.25__seed{0,1,2}__*__prototype_snapshot.json`
-- `artifacts/analysis/xiiotid__alpha0.25__seed{0,1,2}__support_weighted__beta0__prototype_snapshot__{prototype_effect,prototype_weights}.csv`
-- `artifacts/analysis/xiiotid__alpha0.25__multiseed__{validation_per_seed,validation_pooled,client_validation_deltas,client_validation_detail,client_validation_summary,prototype_effect_pooled,test_descriptive,runtime,beta0_equivalence}.csv` and `...__selection_summary.json`
-- `results/xiiotid__alpha0.25__seed{0,1,2}__*` (18 run dirs, gitignored)
+- `docs/research/ALPHA075_BETA_VALIDATION.md`
+- `docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md`
+- `docs/research/SESSION8_AUDIT.md`
+- `scripts/analyze_partition_heterogeneity.py`
+- `scripts/verify_beta0_equivalence.py`
+- `scripts/analyze_cross_alpha.py`
+- `artifacts/analysis/xiiotid__alpha0.75__*` (run snapshots, effect/weights CSVs, multiseed
+  tables, `beta0_equivalence.csv`, `runtime.csv`)
+- `artifacts/analysis/xiiotid__cross_alpha__*` (partition heterogeneity per-seed/per-class/summary,
+  cross-alpha beta table/displacement/client summary/summary)
+- `results/xiiotid__alpha0.75__seed{0,1,2}__*` (18 run dirs, gitignored)
 
 Modified:
 
 - `docs/handoff/SESSION_HANDOFF.md` (this file)
 
-No source or test code was changed. `scripts/analyze_multiseed_beta.py` was re-run for
-alpha=0.50 and produced byte-identical outputs.
+No source or test code was changed.
 
-## 17. Exact recommended next phase
+## 18. Exact recommended next phase
 
-1. **Session 7 is audited and safe to checkpoint** (verdict PASS WITH MINOR CORRECTIONS;
-   see §15b and `docs/research/SESSION7_AUDIT.md`). alpha=0.25 behaved cleanly: pairing valid,
-   beta=0 class B, pooled selection clean, artifacts complete, no test leakage, runtime
-   manageable (≈12 h per alpha). Before checkpointing, stage only the intended docs/analysis
-   artifacts; do not commit `results/` or the raw datasets.
-2. **Next experiment:** run X-IIoTID `alpha=0.75` under the same protocol (seeds `{0,1,2}`,
-   five betas + `original_protean`, sequential, reusing frozen manifests, pooled
-   validation-only selection), then test evaluation, direct-effect, client-level, and
-   cross-alpha comparison. This would complete the three planned non-IID severities. Do **not**
-   infer a monotone alpha→beta relation from two conditions.
-3. Do **not** treat `beta = 0.00` (alpha=0.25) or `0.75` (alpha=0.50) as optima. Report each as
-   "best among tested candidates under pooled validation for that condition".
-4. **Do not launch 5G-NIDD or the full grid** until alpha=0.75 is inspected.
-5. Use `.venv/bin/python`. Run one alpha at a time; measure runtime as before.
-6. Update `docs/research/ALPHA025_BETA_VALIDATION.md` (or add `ALPHA075_BETA_VALIDATION.md`),
-   `EXPERIMENT_PROTOCOL.md` only if a real ambiguity appears, and this handoff. Do not commit or
-   push unless asked.
+1. **Session 8 is audited and safe to checkpoint** (verdict PASS; see §16 and
+   `docs/research/SESSION8_AUDIT.md`). X-IIoTID alpha=0.75 behaved cleanly. Before
+   checkpointing, stage only the intended docs/scripts/analysis artifacts; do not commit
+   `results/` or the raw datasets.
+2. **X-IIoTID three-alpha study is complete** (0.25, 0.50, 0.75 × seeds `{0,1,2}`), with
+   audited condition-level pooled selections 0.00 / 0.75 / 0.50 and a cross-alpha report.
+3. **Next major phase: 5G-NIDD.** Follow the same protocol (the planned alphas `{0.75, 0.50,
+   0.25}`, seeds `{0,1,2}`, five betas + `original_protean`, frozen manifests, pooled
+   validation-only selection). Do **not** launch the full grid blindly: verify 5G-NIDD manifests
+   exist and reproduce, run the same one-condition-at-a-time pacing, and measure runtime first.
+   A single smoke/one-condition run before scaling is recommended.
+4. Do **not** treat any selected beta as an optimum. Report each as "best among tested
+   candidates under pooled validation for that condition".
+5. Do **not** infer a monotone alpha→beta relation from three X-IIoTID conditions.
+6. Update `EXPERIMENT_PROTOCOL.md` only if a real ambiguity appears; update this handoff.
+7. Do not commit or push unless asked.
 
-## 18. Important commands
+## 19. Important commands
 
 ```bash
 # Tests (75 expected)
 .venv/bin/python -m pytest
 
-# One full paired beta run (CPU, ~40 min, ~7.5-11.3 GiB)
-.venv/bin/python scripts/run_baseline.py --dataset xiiotid --alpha 0.25 --seed 1 \
+# One full paired beta run (CPU, ~40 min, ~6.5-7.2 GiB)
+.venv/bin/python scripts/run_baseline.py --dataset xiiotid --alpha 0.75 --seed 1 \
     --clients 10 --rounds 10 --local-epochs 3 \
     --aggregation-mode support_weighted --beta 0.5
 
-# Pooled multi-seed validation analysis (read-only, no retraining; --alpha already supported)
-.venv/bin/python scripts/analyze_multiseed_beta.py --dataset xiiotid --alpha 0.25 --seeds 0 1 2
+# Pooled multi-seed validation analysis (read-only, no retraining)
+.venv/bin/python scripts/analyze_multiseed_beta.py --dataset xiiotid --alpha 0.75 --seeds 0 1 2
 
 # Direct prototype-effect analysis (no retraining)
 .venv/bin/python scripts/analyze_prototype_effect.py \
-    --snapshot artifacts/analysis/xiiotid__alpha0.25__seed0__support_weighted__beta0__prototype_snapshot.json \
+    --snapshot artifacts/analysis/xiiotid__alpha0.75__seed0__support_weighted__beta0__prototype_snapshot.json \
     --class-names "C&C,Exfiltration,Exploitation,Lateral_movement,Normal,RDOS,Reconnaissance,Tampering,Weaponization" \
     --output-dir artifacts/analysis
+
+# Realized partition characterization (all alphas)
+.venv/bin/python scripts/analyze_partition_heterogeneity.py --dataset xiiotid --alphas 0.75 0.5 0.25
+
+# beta=0 equivalence audit
+.venv/bin/python scripts/verify_beta0_equivalence.py --dataset xiiotid --alpha 0.75 --seeds 0 1 2
+
+# Cross-alpha comparison
+.venv/bin/python scripts/analyze_cross_alpha.py --dataset xiiotid --alphas 0.25 0.5 0.75
 ```
 
-## 19. Recommended next-session prompt
+## 20. Recommended next-session prompt
 
 ```
-You are continuing the perdalam-protean research repository (Session 8).
+You are continuing the perdalam-protean research repository (Session 9).
 
 FIRST, read completely:
   1. docs/handoff/SESSION_HANDOFF.md
-  2. docs/research/ALPHA025_BETA_VALIDATION.md
-  3. docs/research/MULTISEED_BETA_VALIDATION.md
-  4. docs/research/EXPERIMENT_PROTOCOL.md
-  5. docs/baseline/BASELINE_VALIDATION.md
+  2. docs/research/ALPHA075_BETA_VALIDATION.md
+  3. docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md
+  4. docs/research/SESSION8_AUDIT.md
+  5. docs/research/EXPERIMENT_PROTOCOL.md
+  6. docs/baseline/BASELINE_VALIDATION.md
 
 Critical rules:
   - planningawal/ is immutable.
@@ -383,19 +405,18 @@ Critical rules:
   - The S_j participant rule is a REPRODUCTION ASSUMPTION (material in 19/27 cells at alpha=0.25).
   - Do NOT add an executable equal_available mode.
   - Beta selection uses validation only; PRIMARY is pooled across clients and seeds.
-  - Do NOT launch 5G-NIDD or the full grid.
+  - Do NOT launch the full grid blindly; verify manifests and pace one condition at a time.
 
-Context: Session 7 completed X-IIoTID alpha=0.25 for seeds {0,1,2} (18 runs, ~12.17 h) and was
-independently audited (PASS WITH MINOR CORRECTIONS; docs/research/SESSION7_AUDIT.md). Pooled
-validation selected beta=0.00 (pooled mean 0.591704; monotone decreasing in beta), unlike
-alpha=0.50 which selected 0.75. Per-seed diagnostics 0.75/0.50/0.00. beta=0 equivalence class B
-(no semantic discrepancy). Realized partitions more heterogeneous (19/27 |S_j|<M vs 7/27).
-Condition-level pooled labels: alpha=0.50 -> 0.75, alpha=0.25 -> 0.00 (the value 0.50 is only the
-Session-5 seed-0 choice, not a condition-level selection).
+Context: Session 8 completed X-IIoTID alpha=0.75 for seeds {0,1,2} (18 runs, ~12.33 h) and was
+independently audited (PASS; docs/research/SESSION8_AUDIT.md). Pooled validation selected
+beta=0.50. Three-alpha X-IIoTID pooled selections: alpha=0.25 -> 0.00, alpha=0.50 -> 0.75,
+alpha=0.75 -> 0.50 (NOT monotone). beta=0 equivalence class B. Realized heterogeneity: 19/27,
+7/27, 3/27 cells |S_j|<M; mean displacement 3.42 / 2.10 / 1.31.
 
-Task: run X-IIoTID alpha=0.75 for seeds {0,1,2} across the five frozen betas plus
-original_protean, sequential, reusing frozen manifests, pooled validation-only selection, then
-test evaluation, direct-effect, client-level, and cross-alpha comparison. Produce
-docs/research/ALPHA075_BETA_VALIDATION.md and update the handoff. Use .venv/bin/python. Do not
-commit or push unless asked.
+Task: X-IIoTID three-alpha study is complete. Recommended next phase is 5G-NIDD under the same
+frozen protocol (alphas {0.75, 0.50, 0.25}, seeds {0,1,2}, five betas + original_protean,
+frozen manifests, pooled validation-only selection). Before scaling, verify 5G-NIDD manifests
+exist and reproduce, run one paired condition, inspect it, then proceed. Produce a 5G-NIDD beta
+validation report and a cross-dataset comparison, update the handoff. Use .venv/bin/python. Do
+not commit or push unless asked.
 ```