experiment: validate 5gnidd beta selection at alpha 0.50
This commit is contained in:
1 parent
2d27abc350
commit
a7f2b0049e
28 files changed
+38378
-256
No files matched your search
+231
-256
@@ -3,26 +3,26 @@
|
||||
This is the canonical continuation point for the next agent session. It is written to be
|
||||
readable without prior chat history, terminal scrollback, or git history.
|
||||
|
||||
Last updated: 2026-09-26 (Session 8). Session 8 type: **protocol-scaling /
|
||||
non-IID-severity experiment**. Session 7 completed X-IIoTID `alpha=0.25`; Session 6 completed
|
||||
`alpha=0.50`. Session 8 completed X-IIoTID **`alpha=0.75`** for seeds `{0,1,2}` across the five
|
||||
frozen beta candidates plus `original_protean`, using the **same pooled-across-seed
|
||||
validation-only** selection rule, and produced the first **three-alpha (0.25/0.50/0.75)
|
||||
comparison** for X-IIoTID. **5G-NIDD was not run.** No commit or push was made.
|
||||
Last updated: 2026-09-28 (Session 9). Session 9 type: **first cross-dataset (5G-NIDD)
|
||||
beta-validation condition**. Sessions 6–8 completed the X-IIoTID three-alpha study
|
||||
(`alpha=0.25/0.50/0.75`, seeds `{0,1,2}`). Session 9 ran **only** 5G-NIDD `alpha=0.50` for seeds
|
||||
`{0,1,2}` across the five frozen beta candidates plus `original_protean`, using the same
|
||||
pooled-across-seed validation-only selection rule. **5G-NIDD alpha=0.25 and alpha=0.75 were NOT
|
||||
run.** No commit or push was made.
|
||||
|
||||
> **Audit status (Session 8).** Independent audit verdict: **PASS** (`docs/research/SESSION8_AUDIT.md`).
|
||||
> `beta_star(alpha=0.75) = 0.50` confirmed from validation only; no corrections required; run
|
||||
> inventory 18/18; pairing hash-identical and reproduced from raw data; leakage audit PASS;
|
||||
> `beta=0` class **B**; source/data integrity PASS; pytest 75 passed. **Session 8 is safe to
|
||||
> checkpoint.**
|
||||
> **Audit status (Session 9).** Independent audit verdict: **PASS**
|
||||
> (`docs/research/SESSION9_AUDIT.md`). `beta_star(5gnidd, alpha=0.50) = 1.00` confirmed from
|
||||
> validation only; reuse decision resolved; run inventory 18/18; pairing hash-identical and
|
||||
> reproduced from raw data; rare/unseen 0 mismatches over 270 cells; leakage audit PASS; `beta=0`
|
||||
> class **B**; source/data integrity PASS; pytest 75 passed. **Session 9 is safe to checkpoint.**
|
||||
|
||||
> **Headline (Session 8).** PRIMARY pooled-validation selected beta for X-IIoTID `alpha=0.75`
|
||||
> = **0.50** (validation only; per-seed diagnostics 0.50 / 0.50 / 0.00). The pooled validation
|
||||
> curve peaks at 0.50 and declines both sides, but the total between-beta spread (≈ 0.0081) is
|
||||
> much smaller than the between-seed std (≈ 0.045). `beta=0` is class **B** for all three seeds.
|
||||
> Realized alpha=0.75 partitions are the mildest of the three (3/27 cells with `|S_j| < M`,
|
||||
> 4 missing cells, client-ratio mean 4.67). **Three-alpha pooled selections: 0.25 → 0.00,
|
||||
> 0.50 → 0.75, 0.75 → 0.50 — not monotone in alpha.**
|
||||
> **Headline (Session 9).** PRIMARY pooled-validation selected beta for 5G-NIDD `alpha=0.50` =
|
||||
> **1.00** (validation only; per-seed diagnostics 1.00 / 0.00 / 1.00). The pooled validation curve
|
||||
> is **non-monotone** (0.687 → 0.645 → 0.675 → 0.688 → 0.692) and the total between-beta spread
|
||||
> (≈ 0.047) is comparable to the between-seed std (0.053–0.139), so the selection is statistically
|
||||
> weak. `beta=0` is class **B** for all three seeds. The pre-existing seed-0 baseline was **not
|
||||
> reused** (missing validation artifacts); the fresh rerun reproduces its training outputs
|
||||
> byte-identically. X-IIoTID alpha=0.50 selected 0.75; the two datasets differ descriptively.
|
||||
|
||||
---
|
||||
|
||||
@@ -30,8 +30,8 @@ comparison** for X-IIoTID. **5G-NIDD was not run.** No commit or push was made.
|
||||
|
||||
Build the code and experiment infrastructure for a thesis study that modifies **only** the
|
||||
server-side prototype aggregation of PROTEAN (Chennoufi et al., 2025), a federated
|
||||
intrusion-detection framework. The mechanism weights each contributing client's class
|
||||
prototype by its class sample support:
|
||||
intrusion-detection framework. The mechanism weights each contributing client's class prototype
|
||||
by its class sample support:
|
||||
|
||||
```
|
||||
Cbar_j(beta) = ( sum_{i in S_j} N_ij^beta * C_ij ) / ( sum_{i in S_j} N_ij^beta )
|
||||
@@ -49,287 +49,267 @@ bitwise identical to the baseline (see §7).
|
||||
- `planningawal/` is immutable. Re-verified unchanged this session: `git status --short
|
||||
planningawal/` and `git diff -- planningawal/` are both empty.
|
||||
- Raw datasets re-verified unchanged (SHA-256):
|
||||
`~/datasets/X-IIoTID/X-IIoTID-dataset.csv`
|
||||
`7b9290057ee42e784da3c0d84b781815502c9205c74175c96374e71a5ffd98a0`;
|
||||
`~/datasets/5G-NIDD/5G-NIDD.csv`
|
||||
`fa36f80859585f474504ca69eb951a079b35145537976c86b00aae3aab46ee59`.
|
||||
- Frozen baseline (`original_protean`) source unchanged; no `equal_available` mode added;
|
||||
beta candidate set unchanged (`{0, .25, .5, .75, 1}`). No tracked source/test/config file
|
||||
differs from HEAD.
|
||||
`fa36f80859585f474504ca69eb951a079b35145537976c86b00aae3aab46ee59`;
|
||||
`~/datasets/X-IIoTID/X-IIoTID-dataset.csv`
|
||||
`7b9290057ee42e784da3c0d84b781815502c9205c74175c96374e71a5ffd98a0`.
|
||||
- Frozen baseline (`original_protean`) source unchanged; no `equal_available` mode added; beta
|
||||
candidate set unchanged (`{0, .25, .5, .75, 1}`). No tracked source/test/config file differs
|
||||
from HEAD. The only modified tracked file is this handoff.
|
||||
|
||||
## 3. Session 8 objective and outcome
|
||||
## 3. Session 9 objective and outcome
|
||||
|
||||
Objective: evaluate the final X-IIoTID non-IID severity (`alpha=0.75`, seeds `{0,1,2}`) across
|
||||
the five beta candidates using the exact pooled-validation protocol, then compare alpha=0.25,
|
||||
0.50 and 0.75 using realized partition characteristics. All 18 runs (15 `support_weighted` +
|
||||
3 `original_protean`) completed with rc=0.
|
||||
Objective: begin the 5G-NIDD phase conservatively — Stage A verification, then (if clean) **one**
|
||||
condition: 5G-NIDD `alpha=0.50`, seeds `{0,1,2}`, `original_protean` + five betas.
|
||||
|
||||
Outcome: **PRIMARY pooled-validation selected beta = 0.50** (validation only);
|
||||
`test_used_for_selection = false` for all 15 runs. Pooled means: 0.00 → 0.471706,
|
||||
0.25 → 0.474368, 0.50 → 0.476749, 0.75 → 0.472541, 1.00 → 0.468605 (peak at 0.50). Between-seed
|
||||
std 0.0447–0.0553; between-beta spread ≈ 0.0081, so the effect is small relative to seed
|
||||
variance.
|
||||
Stage A result: raw hash valid; all nine 5G-NIDD manifests (alpha 0.25/0.50/0.75 × seeds) reproduce
|
||||
exactly from raw data; label semantics and rare/unseen logic valid; pre-existing seed-0 baseline
|
||||
not reusable. Gate = **GO**.
|
||||
|
||||
Reports: `docs/research/ALPHA075_BETA_VALIDATION.md`,
|
||||
`docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md`, audit `docs/research/SESSION8_AUDIT.md`.
|
||||
Outcome: 18/18 runs rc=0. **PRIMARY pooled-validation selected beta = 1.00** (validation only);
|
||||
`test_used_for_selection = false` for all 15 runs. Pooled means: 0.00 → 0.687230,
|
||||
0.25 → 0.645078, 0.50 → 0.674669, 0.75 → 0.688368, 1.00 → 0.692370. Reports:
|
||||
`docs/research/5GNIDD_ALPHA050_BETA_VALIDATION.md`, audit `docs/research/SESSION9_AUDIT.md`.
|
||||
|
||||
## 4. Condition and pairing
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| dataset / alpha / seeds | `xiiotid` / 0.75 / `{0,1,2}` |
|
||||
| dataset / alpha / seeds | `5gnidd` / 0.50 / `{0,1,2}` |
|
||||
| runs | 15 `support_weighted` + 3 `original_protean`, all new this session |
|
||||
| training | 10 clients, 10 rounds, 3 local epochs, batch 256, lr 1e-3, mu 0.1, lambda 1.0, Adam |
|
||||
| code revision | unchanged from HEAD `ff68d380e23859d21c07c8a3a7b56b1b7f06c91b`; only docs/analysis/scripts added |
|
||||
| code revision | unchanged from HEAD `2d27abc35026456de765c929dbcf0503a72ac572`; only docs/analysis added |
|
||||
|
||||
Manifests (reused, never regenerated; one content hash per seed across all six runs; each
|
||||
reproduced from the raw dataset with split/`N_ij`/partition hash verified):
|
||||
|
||||
| seed | manifest | content hash | partition hash | split hash | rng seed |
|
||||
| seed | manifest content hash | partition hash | split hash | rng seed | round-0 snapshot SHA-256 |
|
||||
|---|---|---|---|---|---|
|
||||
| 0 | `artifacts/partitions/xiiotid__alpha0.75__seed0.json` | `759f7e50…` | `473b1c4a…` | `d0acc0c1…` | 957653162 |
|
||||
| 1 | `artifacts/partitions/xiiotid__alpha0.75__seed1.json` | `4a50ba21…` | `a0ec9e9c…` | `880a9f82…` | 762050403 |
|
||||
| 2 | `artifacts/partitions/xiiotid__alpha0.75__seed2.json` | `922b7bb1…` | `00345255…` | `8e0db0be…` | 843543396 |
|
||||
| 0 | `0732582cf4510089…` | `8352d4ff…` | `18f89f48…` | 673621352 | `eb0839f2595727…` |
|
||||
| 1 | `f43906e111a8342c…` | `0815c91e…` | `3d05b7fe…` | 1403944623 | `6cbd04e0dd97bb…` |
|
||||
| 2 | `fe8b4fc6f0486cf4…` | `fdcf388e…` | `fc758d06…` | 633461056 | `f012a02b543b92…` |
|
||||
|
||||
Pairing proof: round-0 snapshot byte-identical across the five betas **and** `original_protean`
|
||||
within each seed — seed 0 `8c477df6…`, seed 1 `41f0187c…`, seed 2 `be2ad5fb…`.
|
||||
`train_support_Nij.csv`, `characterization.csv`, `rare_unseen.csv` identical across each seed's
|
||||
runs. `manifest_reload_ok = true`, `smoke = false` everywhere.
|
||||
Splits: train/validation/test = 851,123 / 121,589 / 243,177 for every seed (10 clients, 9 classes,
|
||||
`Benign`). `train_support_Nij.csv`, `characterization.csv`, `rare_unseen.csv` identical across each
|
||||
seed's runs. `manifest_reload_ok = true`, `smoke = false` everywhere. Pairing proven by content
|
||||
hashes.
|
||||
|
||||
## 5. Validation results (USED FOR SELECTION) and pooled selection
|
||||
|
||||
Primary metric: validation per-client-mean rare-class macro-F1. All 10 clients eligible in
|
||||
every seed. Full tables:
|
||||
`artifacts/analysis/xiiotid__alpha0.75__multiseed__validation_per_seed.csv`.
|
||||
Primary metric: validation per-client-mean rare-class macro-F1. All 10 clients eligible in every
|
||||
seed. Full tables: `artifacts/analysis/5gnidd__alpha0.5__multiseed__validation_per_seed.csv`.
|
||||
|
||||
Per-seed validation per-client-mean rare-class macro-F1:
|
||||
|
||||
| beta | seed 0 | seed 1 | seed 2 |
|
||||
|---|---|---|---|
|
||||
| 0.00 | 0.496803 | 0.408276 | 0.510038 |
|
||||
| 0.25 | 0.496892 | 0.421386 | 0.504827 |
|
||||
| 0.50 | 0.504388 | 0.422919 | 0.502942 |
|
||||
| 0.75 | 0.486868 | 0.421053 | 0.509702 |
|
||||
| 1.00 | 0.479146 | 0.419534 | 0.507136 |
|
||||
| 0.00 | 0.738597 | 0.633243 | 0.689849 |
|
||||
| 0.25 | 0.661461 | 0.573717 | 0.700055 |
|
||||
| 0.50 | 0.764487 | 0.563388 | 0.696131 |
|
||||
| 0.75 | 0.750661 | 0.543319 | 0.771124 |
|
||||
| 1.00 | 0.768877 | 0.531764 | 0.776468 |
|
||||
|
||||
Pooled across seeds (`...__validation_pooled.csv`):
|
||||
|
||||
| beta | pooled mean | std (sample) | min | max | selected |
|
||||
|---|---|---|---|---|---|
|
||||
| 0.00 | 0.471706 | 0.055329 | 0.408276 | 0.510038 | |
|
||||
| 0.25 | 0.474368 | 0.046055 | 0.421386 | 0.504827 | |
|
||||
| **0.50** | **0.476749** | 0.046624 | 0.422919 | 0.504388 | **yes** |
|
||||
| 0.75 | 0.472541 | 0.046028 | 0.421053 | 0.509702 | |
|
||||
| 1.00 | 0.468605 | 0.044742 | 0.419534 | 0.507136 | |
|
||||
| 0.00 | 0.687230 | 0.052726 | 0.633243 | 0.738597 | |
|
||||
| 0.25 | 0.645078 | 0.064743 | 0.573717 | 0.700055 | |
|
||||
| 0.50 | 0.674669 | 0.102253 | 0.563388 | 0.764487 | |
|
||||
| 0.75 | 0.688368 | 0.126032 | 0.543319 | 0.771124 | |
|
||||
| **1.00** | **0.692370** | 0.139140 | 0.531764 | 0.776468 | **yes** |
|
||||
|
||||
- **PRIMARY selected beta (`beta_star_alpha075`) = 0.50**, max pooled validation
|
||||
per-client-mean rare-class macro-F1; **no tie-break used**; selection is validation-only.
|
||||
- **SECONDARY per-seed diagnostics only:** seed 0 → 0.50, seed 1 → 0.50, seed 2 → 0.00.
|
||||
- **PRIMARY selected beta (`beta_star_5gnidd_alpha050`) = 1.00**, max pooled validation
|
||||
per-client-mean rare-class macro-F1; **no tie-break used**; validation-only.
|
||||
- **SECONDARY per-seed diagnostics only:** seed 0 → 1.00, seed 1 → 0.00, seed 2 → 1.00.
|
||||
- Between-beta spread ≈ 0.047; between-seed std 0.053–0.139 → statistically weak, descriptive.
|
||||
|
||||
## 6. Final test evaluation of the pooled-selected beta
|
||||
|
||||
`beta_star = 0.50` was fixed from pooled validation before test was read.
|
||||
`beta_star = 1.00` was fixed from pooled validation before test was read.
|
||||
|
||||
| metric (test) | seed 0 | seed 1 | seed 2 | mean ± std (sample) |
|
||||
|---|---|---|---|---|
|
||||
| rare-class macro-F1 | 0.511889 | 0.624144 | 0.679704 | 0.605246 ± 0.085489 |
|
||||
| rare-class recall | 0.803887 | 0.876726 | 0.865681 | 0.848765 ± 0.039256 |
|
||||
| macro-F1 | 0.741977 | 0.761752 | 0.769179 | 0.757636 ± 0.014060 |
|
||||
| accuracy | 0.914365 | 0.932611 | 0.937352 | 0.928110 ± 0.012137 |
|
||||
| weighted-F1 | 0.935566 | 0.948344 | 0.951740 | 0.945216 ± 0.008528 |
|
||||
| rare-class macro-F1 | 0.880962 | 0.840075 | 0.894914 | 0.871984 ± 0.028501 |
|
||||
| rare-class recall | 0.974355 | 0.968341 | 0.958616 | 0.967104 ± 0.007942 |
|
||||
| macro-F1 | 0.850470 | 0.815941 | 0.849378 | 0.838596 ± 0.019627 |
|
||||
| accuracy | 0.742838 | 0.744026 | 0.742907 | 0.743257 ± 0.000667 |
|
||||
| weighted-F1 | 0.716950 | 0.720106 | 0.717530 | 0.718195 ± 0.001680 |
|
||||
|
||||
`original_protean` for the same seeds/condition:
|
||||
|
||||
| metric (test) | seed 0 | seed 1 | seed 2 | mean ± std (sample) |
|
||||
|---|---|---|---|---|
|
||||
| rare-class macro-F1 | 0.521540 | 0.614061 | 0.676068 | 0.603890 ± 0.077764 |
|
||||
| rare-class recall | 0.816440 | 0.883238 | 0.869835 | 0.856504 ± 0.035338 |
|
||||
| macro-F1 | 0.742471 | 0.747395 | 0.765680 | 0.751849 ± 0.012228 |
|
||||
| accuracy | 0.913254 | 0.913898 | 0.934576 | 0.920576 ± 0.012129 |
|
||||
| weighted-F1 | 0.932563 | 0.935937 | 0.948908 | 0.939136 ± 0.008629 |
|
||||
| rare-class macro-F1 | 0.871964 | 0.921136 | 0.848525 | 0.880542 ± 0.037058 |
|
||||
| rare-class recall | 0.974314 | 0.968566 | 0.975859 | 0.972913 ± 0.003843 |
|
||||
| macro-F1 | 0.836881 | 0.865437 | 0.824867 | 0.842395 ± 0.020840 |
|
||||
| accuracy | 0.734416 | 0.747377 | 0.750133 | 0.743975 ± 0.008393 |
|
||||
| weighted-F1 | 0.708765 | 0.722802 | 0.724622 | 0.718730 ± 0.008678 |
|
||||
|
||||
Descriptive difference for test rare-class macro-F1 (`beta_star` − `original_protean`) is
|
||||
−0.009651 / +0.010083 / +0.003636 (mean +0.001356). Test is descriptive only; no superiority
|
||||
or generalisation claimed.
|
||||
+0.008998 / −0.081061 / +0.046389 (mean −0.008558). Test is descriptive only; no superiority or
|
||||
generalisation claimed. Note 5G-NIDD test weighted-F1 (`≈0.718`) sits **below** accuracy
|
||||
(`≈0.744`) — the opposite of X-IIoTID — because the high-support classes are the harder ones for
|
||||
nearest-prototype inference.
|
||||
|
||||
## 7. `beta = 0` equivalence result (all three seeds)
|
||||
|
||||
For seeds 0, 1, 2, `support_weighted beta=0` vs frozen `original_protean`:
|
||||
|
||||
- Same manifest/split/hashes: **yes**.
|
||||
- Round-0 snapshot byte-identical: **yes** (`8c477df6…`, `41f0187c…`, `be2ad5fb…`).
|
||||
- `train_support_Nij.csv`, `characterization.csv`, `rare_unseen.csv`: **identical**.
|
||||
- Independent aggregation (`ΣC/n` vs `(1/n)ΣC`): 4.4e-16 / 4.4e-16 / 2.2e-16.
|
||||
- History rare-macro-F1 max diff (rounds ≥ 1): 1.10e-2 / 8.12e-3 / 3.64e-3; test metric max
|
||||
diff 4.33e-3 / 1.14e-3 / 0.85e-3.
|
||||
- Classification: **B — mathematically equivalent + floating-point numerical divergence; no
|
||||
semantic discrepancy** in any seed. Machine-readable:
|
||||
`artifacts/analysis/xiiotid__alpha0.75__multiseed__beta0_equivalence.csv`.
|
||||
- Same manifest/split/hashes, round-0 snapshot byte-identical, support/char/rare_unseen
|
||||
identical: **yes** for all three.
|
||||
- Independent aggregation (`ΣC/n` vs `(1/n)ΣC`): 4.4e-16 / 4.4e-16 / 4.4e-16.
|
||||
- History rare-macro-F1 max diff (rounds ≥ 1): 1.93e-2 / 6.01e-2 / 2.60e-2; test metric max diff
|
||||
2.33e-3 / 1.86e-4 / 8.70e-3.
|
||||
- Classification: **B** in all seeds — semantic equivalence + floating-point divergence; no
|
||||
semantic discrepancy. Machine-readable:
|
||||
`artifacts/analysis/5gnidd__alpha0.5__multiseed__beta0_equivalence.csv`.
|
||||
|
||||
Conditional on the shared `S_j` reproduction assumption; not proof `beta=0` reproduces original
|
||||
PROTEAN.
|
||||
|
||||
## 8. Partition characterization (realized)
|
||||
## 8. Partition characterization (realized) and dataset structure
|
||||
|
||||
`|S_j| < M` class-condition cells, missing client-class cells, and client-size ratios
|
||||
(training split):
|
||||
5G-NIDD `alpha=0.50` (training split), `M = 10`:
|
||||
|
||||
| condition | `|S_j| < M` cells | missing client-class cells | client-size ratio (seeds) | union unseen | mean class CV |
|
||||
|---|---|---|---|---|---|
|
||||
| alpha=0.25 | **19 / 27** | **38** | 15.6, 20.8, 7.8 | 6/6/5 | 1.442 |
|
||||
| alpha=0.50 | 7 / 27 | 9 | 5.1, 7.2, 9.9 | 3/3/1 | 1.183 |
|
||||
| alpha=0.75 | **3 / 27** | **4** | 4.15, 5.07, 4.78 | 2/1/0 | 0.881 |
|
||||
| seed | `|S_j| < M` cells | missing client-class cells | client-size ratio | union rare | union unseen | max unseen/client |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 0 | 2 / 9 | 2 | 11.63 | 6 | 2 | 1 |
|
||||
| 1 | 0 / 9 | 0 | 4.43 | 5 | 0 | 0 |
|
||||
| 2 | 2 / 9 | 2 | 12.72 | 6 | 2 | 1 |
|
||||
| **sum/mean** | **4 / 27** | **4** | mean **9.59** | 6/5/6 | 2/0/2 | 1 |
|
||||
|
||||
Union rare classes per seed: alpha=0.75 = 4/5/6; all clients have exactly two rare classes.
|
||||
Full per-class `N_ij`, CV and max/min in `docs/research/ALPHA075_BETA_VALIDATION.md` §C and
|
||||
`artifacts/analysis/xiiotid__cross_alpha__partition_heterogeneity_*.csv`. Heterogeneity
|
||||
decreases with alpha overall but is **not uniformly monotone per seed** (alpha=0.75 seed 0
|
||||
ratio 4.15 < seed 2's 4.78; alpha=0.50 seed 2 ratio 9.9 > alpha=0.25 seed 2's 7.8), so
|
||||
interpretation uses realized partitions, not alpha alone.
|
||||
All 10 clients have exactly two rare classes and are eligible in every seed; mean class-support
|
||||
CV = **1.225**. Full tables:
|
||||
`artifacts/analysis/5gnidd__alpha0.5__partition_heterogeneity_*.csv` and
|
||||
`...__partition_characterization.csv`.
|
||||
|
||||
### Comparison with X-IIoTID alpha=0.50 (descriptive)
|
||||
|
||||
| statistic | 5G-NIDD a0.50 | X-IIoTID a0.50 |
|
||||
|---|---|---|
|
||||
| pooled-selected beta | **1.00** | **0.75** |
|
||||
| pooled validation mean range (spread) | 0.645–0.692 (0.047) | 0.544–0.579 (0.035) |
|
||||
| pooled validation std range | 0.053–0.139 | 0.075–0.100 |
|
||||
| `|S_j| < M` cells (sum/27) | **4/27** | 7/27 |
|
||||
| missing client-class cells | **4** | 9 |
|
||||
| client-size ratio mean | **9.59** | 7.41 |
|
||||
| mean class-support CV | 1.225 | 1.183 |
|
||||
| union rare per seed | 6/5/6 | 6/7/7 |
|
||||
| union unseen per seed | 2/0/2 | 3/3/1 |
|
||||
| mean displacement ‖Cbar(1)−Cbar(0)‖ | 2.340 | 2.099 |
|
||||
| selected-beta client improve/degrade | 6/3, 3/5, 7/2 | 7/3, 8/1, 8/1 |
|
||||
|
||||
5G-NIDD has fewer missing class cells but more imbalanced client sizes; the selected beta differs
|
||||
descriptively. No causal or monotone claim; only two datasets, one condition each.
|
||||
|
||||
## 9. Direct prototype-effect findings (fixed round-0 snapshots, no retraining)
|
||||
|
||||
`||Cbar_j(1) - Cbar_j(0)||`, cross-seed mean, alpha=0.25 vs 0.50 vs 0.75 (mean across classes
|
||||
3.422 / 2.099 / **1.305**):
|
||||
`||Cbar_j(1) − Cbar_j(0)||`, cross-seed mean (std), 5G-NIDD alpha=0.50:
|
||||
|
||||
| class | a25 | a50 | a75 |
|
||||
|---|---|---|---|
|
||||
| Tampering | 4.248 | 2.354 | 1.934 |
|
||||
| Exfiltration | 4.316 | 2.020 | 1.448 |
|
||||
| Exploitation | 3.464 | 2.344 | 1.539 |
|
||||
| Lateral_movement | 3.726 | 2.724 | 1.369 |
|
||||
| C&C | 4.020 | 1.694 | 1.284 |
|
||||
| Weaponization | 3.124 | 2.561 | 1.472 |
|
||||
| Reconnaissance | 3.339 | 1.002 | 0.983 |
|
||||
| Normal | 1.997 | 1.557 | 0.903 |
|
||||
| RDOS | 2.566 | 2.631 | 0.814 |
|
||||
| class | mean disp | std |
|
||||
|---|---|---|
|
||||
| UDPScan | 3.461 | 2.011 |
|
||||
| HTTPFlood | 3.120 | 1.289 |
|
||||
| SYNScan | 2.875 | 0.959 |
|
||||
| SlowrateDoS | 2.763 | 1.454 |
|
||||
| SYNFlood | 2.262 | 0.748 |
|
||||
| TCPConnectScan | 2.131 | 0.213 |
|
||||
| Benign | 1.690 | 0.699 |
|
||||
| UDPFlood | 1.598 | 0.428 |
|
||||
| ICMPFlood | 1.159 | 0.093 |
|
||||
|
||||
Displacement tracks realized support imbalance and is a direct-aggregation measure only, not a
|
||||
detection result. Outputs:
|
||||
`artifacts/analysis/xiiotid__alpha0.75__multiseed__prototype_effect_pooled.csv` and per-seed
|
||||
`...__seed{0,1,2}__support_weighted__beta0__prototype_snapshot__prototype_effect.csv`.
|
||||
Mean across classes **2.340** at beta=1.0 (monotone: 0.953 / 1.567 / 2.003 / 2.340 for beta
|
||||
0.25/0.50/0.75/1.00). Seed-sensitive (e.g. UDPScan 1.42 / 3.52 / 5.44). Displacement is a
|
||||
direct-aggregation measure only, not a detection result. Outputs:
|
||||
`...__prototype_effect_pooled.csv` and per-seed `...__prototype_effect.csv` / `...__prototype_weights.csv`.
|
||||
|
||||
## 10. Client-level findings (validation, vs beta = 0)
|
||||
|
||||
Detail: `...__client_validation_detail.csv`; summary: `...__client_validation_summary.csv`; and
|
||||
`artifacts/analysis/xiiotid__cross_alpha__cross_alpha_client_summary.csv`.
|
||||
Detail: `...__client_validation_detail.csv`; summary: `...__client_validation_summary.csv`.
|
||||
|
||||
- At alpha=0.75, the response is smaller and seed-dependent. At the selected `beta=0.50`:
|
||||
6/4/0 (seed 0, mean Δ +0.008), 9/0/1 (seed 1, +0.015), 6/4/0 (seed 2, −0.007; worst client
|
||||
−0.152).
|
||||
- At a common `beta=0.75`: alpha=0.25 = 5/4/1, 5/4/1, 3/5/2 (often negative); alpha=0.50 =
|
||||
7/3/0, 8/1/1, 8/1/1 (majority improve); alpha=0.75 = 3/7/0, 9/0/1, 5/5/0.
|
||||
- `N_ij` vs ΔF1 remains non-monotonic at every alpha. Client ids are per-seed realisations and
|
||||
are not pooled.
|
||||
At the selected `beta=1.00`: **6/3/1** (seed 0, mean Δ +0.030), **3/5/2** (seed 1, mean Δ −0.101;
|
||||
worst client −0.656), **7/2/1** (seed 2, mean Δ +0.087). The response is seed-dependent and
|
||||
`N_ij` vs ΔF1 is non-monotonic. Client ids are per-seed realisations and are not pooled.
|
||||
|
||||
## 11. Three-alpha comparison (RQ3)
|
||||
## 11. Runtime / resources
|
||||
|
||||
| | alpha=0.25 | alpha=0.50 | alpha=0.75 |
|
||||
|---|---|---|---|
|
||||
| pooled-selected beta | **0.00** | **0.75** | **0.50** |
|
||||
| pooled validation curve | monotone decreasing | rises to peak at 0.75 | peaks at 0.50 |
|
||||
| pooled mean range (spread) | 0.532–0.592 (0.060) | 0.544–0.579 (0.035) | 0.469–0.477 (0.008) |
|
||||
| pooled std range | 0.045–0.090 | 0.075–0.100 | 0.045–0.055 |
|
||||
| `|S_j| < M` cells | 19/27 | 7/27 | 3/27 |
|
||||
| mean `||Cbar(1)−Cbar(0)||` | 3.422 | 2.099 | 1.305 |
|
||||
CPU only, no GPU, no swap, exit 0 on all 18 runs. Per-run wall-clock 62–68 min; CPU 734–745 %;
|
||||
peak RSS 10.36 GiB (seed 0), 8.67 GiB (seed 1), 14.04 GiB (seed 2). New compute: 18 runs,
|
||||
69,336 s ≈ **19.26 h** wall-clock, sequential (8 cores). Machine-readable:
|
||||
`artifacts/analysis/5gnidd__alpha0.5__multiseed__runtime.csv`.
|
||||
|
||||
The pooled-selected beta **changes** with the non-IID condition (0.00 → 0.75 → 0.50) but is
|
||||
**not monotone** in alpha. This is documented as a descriptive association accompanied by the
|
||||
realized partition characteristics; **no causal claim** is made. With only three alpha levels,
|
||||
three project-defined seeds, one dataset and no inferential trend test, monotonicity is not
|
||||
claimed.
|
||||
## 12. Methodological decisions / assumptions
|
||||
|
||||
> **Audited label (do not confuse).** Condition-level **pooled** selections are alpha=0.25 →
|
||||
> **0.00**, alpha=0.50 → **0.75**, alpha=0.75 → **0.50**. The value **0.50** is *not* the
|
||||
> alpha=0.50 pooled selection: it is only the Session-5 **seed-0 single-condition** choice, the
|
||||
> Session-6 alpha=0.50 **per-seed diagnostic** for seed 0, and now the alpha=0.75 pooled
|
||||
> selection. See `docs/research/SESSION7_AUDIT.md` §M and `SESSION8_AUDIT.md` §M.
|
||||
|
||||
## 12. Runtime / resources
|
||||
|
||||
CPU only, no GPU, no swap, exit 0 on all 18 runs. Per-run wall-clock 40:00–49:02 (two runs
|
||||
slower under transient host contention); CPU 714–746 %; peak RSS 6.49 GiB (seed 0), 6.91 GiB
|
||||
(seed 1), 7.20 GiB (seed 2). New compute: 18 runs, 44,371 s ≈ **12.33 h** wall-clock,
|
||||
sequential (8 cores). Machine-readable:
|
||||
`artifacts/analysis/xiiotid__alpha0.75__multiseed__runtime.csv`.
|
||||
|
||||
## 13. Methodological decisions / assumptions
|
||||
|
||||
1. **`S_j` participant/denominator rule** (`S_j = {i | N_ij > 0 and valid C_ij}`, equal weights
|
||||
in the baseline) is a **REPRODUCTION ASSUMPTION**; material in 19/27 class-condition cells at
|
||||
alpha=0.25, 7/27 at alpha=0.50, 3/27 at alpha=0.75. Carry this caveat in all reporting.
|
||||
1. **`S_j` participant/denominator rule** (`S_j = {i | N_ij > 0 and valid C_ij}`, equal weights in
|
||||
the baseline) is a **REPRODUCTION ASSUMPTION**; material in only **4/27** class-condition cells
|
||||
at 5G-NIDD alpha=0.50. Carry this caveat in all reporting.
|
||||
2. `N_ij` is class-specific sample support, not prototype quality.
|
||||
3. Beta is one value for all classes/clients; candidates frozen at `{0,.25,.5,.75,1}`.
|
||||
4. **PRIMARY selection = pooled validation across clients and seeds**; per-seed selection is
|
||||
secondary stability diagnostics; test used only after the pooled beta is fixed.
|
||||
5. Three new read-only analysis scripts were added this session (no training/aggregation code
|
||||
changed): `scripts/analyze_partition_heterogeneity.py`,
|
||||
`scripts/verify_beta0_equivalence.py`, `scripts/analyze_cross_alpha.py`.
|
||||
5. No new source scripts this session; analysis used the existing read-only
|
||||
`analyze_partition_heterogeneity.py`, `analyze_prototype_effect.py`, `analyze_multiseed_beta.py`,
|
||||
`verify_beta0_equivalence.py`. Ad-hoc Stage-A/audit helpers live only under
|
||||
`/tmp/opencode/s9/` (outside the repo).
|
||||
6. `lambda=1.0`, Adam, lr 1e-3, batch 256 remain PROJECT DECISIONS. No `equal_available` mode.
|
||||
|
||||
## 14. Unresolved questions / blockers
|
||||
## 13. Unresolved questions / blockers
|
||||
|
||||
1. `S_j` reproduction ambiguity — unresolved; no official PROTEAN code exists.
|
||||
2. **Pooled beta selection remains statistically weak at all three alphas (descriptive only):**
|
||||
the across-seed sample std is comparable to or larger than the between-beta spread. No
|
||||
inferential test was performed.
|
||||
3. `beta = 0` is not bitwise identical to `original_protean`; the documentation decision on how
|
||||
to report this remains open.
|
||||
2. Pooled beta selection remains statistically weak at all measured conditions: across-seed std is
|
||||
comparable to or larger than the between-beta spread. No inferential test was performed.
|
||||
3. `beta = 0` is not bitwise identical to `original_protean`; documentation decision still open.
|
||||
4. Session-2 manifests still store `preprocessing_hash = null` / `config_hash = null`.
|
||||
5. Only alpha=0.25/0.50/0.75, X-IIoTID, three seeds measured. No 5G-NIDD beta result.
|
||||
6. The three-alpha selected-beta sequence (0.00 / 0.75 / 0.50) is **not monotone**; no trend
|
||||
claim is made and no inference has been run.
|
||||
5. Only 5G-NIDD alpha=0.50 (one condition) measured; no 5G-NIDD alpha=0.25/0.75 result; no
|
||||
cross-alpha 5G-NIDD or cross-dataset causal claim.
|
||||
6. The selected beta differs across datasets (5G-NIDD 1.00 vs X-IIoTID 0.75) but no trend or
|
||||
causality is claimed.
|
||||
|
||||
## 15. Git status
|
||||
## 14. Git status
|
||||
|
||||
No commit or push was made. `git status --short` shows only untracked docs/analysis/run
|
||||
artifacts and the three new scripts (`results/` is gitignored; `artifacts/analysis/` is
|
||||
intentionally tracked):
|
||||
No commit or push was made. `HEAD = 2d27abc35026456de765c929dbcf0503a72ac572`, branch `main`,
|
||||
in sync with `origin/main`. `git status --short` shows only untracked docs/analysis artifacts and
|
||||
the modified handoff; `results/` is gitignored:
|
||||
|
||||
```
|
||||
M docs/handoff/SESSION_HANDOFF.md
|
||||
?? docs/research/ALPHA075_BETA_VALIDATION.md
|
||||
?? docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md
|
||||
?? docs/research/SESSION8_AUDIT.md
|
||||
?? scripts/analyze_cross_alpha.py
|
||||
?? scripts/analyze_partition_heterogeneity.py
|
||||
?? scripts/verify_beta0_equivalence.py
|
||||
?? artifacts/analysis/xiiotid__alpha0.75__* (runs' snapshots, effect CSVs, multiseed tables)
|
||||
?? artifacts/analysis/xiiotid__cross_alpha__* (heterogeneity, cross-alpha tables)
|
||||
?? (pre-existing Session-5/6/7 untracked alpha=0.25/0.5 views, unchanged)
|
||||
?? docs/research/5GNIDD_ALPHA050_BETA_VALIDATION.md
|
||||
?? docs/research/SESSION9_AUDIT.md
|
||||
?? artifacts/analysis/5gnidd__alpha0.5__* (46 files: snapshots, multiseed tables, effect/weights,
|
||||
partition characterization, beta0 equivalence, runtime,
|
||||
audit_recomputation)
|
||||
?? (pre-existing Session-5/6/7/8 untracked views, unchanged)
|
||||
```
|
||||
|
||||
No tracked source/test/config file differs from HEAD. `planningawal/` and both raw datasets are
|
||||
unchanged (audit re-verified SHA-256: X-IIoTID `7b929005…`, 5G-NIDD `fa36f808…`).
|
||||
`artifacts/partitions/` unmodified. No 5G-NIDD `support_weighted` run exists. `equal_available`
|
||||
absent. `HEAD = ff68d380e23859d21c07c8a3a7b56b1b7f06c91b`. No commit or push was made.
|
||||
unchanged. `artifacts/partitions/` unmodified. `equal_available` absent. No 5G-NIDD
|
||||
`support_weighted` run before this session; no 5G-NIDD alpha=0.25/0.75 run. The pre-existing
|
||||
`5gnidd__alpha0.5__seed0__original_protean` was overwritten in place by the fresh rerun (backed up
|
||||
to `/tmp/opencode/s9/old_baseline_backup/`).
|
||||
|
||||
## 16. Session 8 audit result
|
||||
## 15. Session 9 audit result
|
||||
|
||||
- Verdict: **PASS** (`docs/research/SESSION8_AUDIT.md`).
|
||||
- Verdict: **PASS** (`docs/research/SESSION9_AUDIT.md`).
|
||||
- Run inventory 18/18, pairing hash-identical, manifests reproduced from raw data.
|
||||
- Validation recomputed exactly; **pooled `beta_star(alpha=0.75) = 0.50`** (validation only).
|
||||
`beta_star` test and `original_protean` test means/stds exact.
|
||||
- Validation recomputed exactly; **pooled `beta_star(5gnidd, alpha=0.50) = 1.00`** (validation
|
||||
only). Selected-beta test and `original_protean` test means/stds exact.
|
||||
- Leakage audit PASS; `test_used_for_selection=false` upheld.
|
||||
- `beta=0` equivalence: class **B** for all three seeds (no semantic discrepancy).
|
||||
- Direct-effect, client-level, rare/unseen, cross-alpha labels: PASS.
|
||||
- Rare/unseen integrity PASS (0 mismatches / 270 cells). `beta=0` equivalence class **B**.
|
||||
- Reuse decision PASS (not reused; rerun reproduces old baseline exactly).
|
||||
- Direct-effect, client-level, cross-dataset labels: PASS.
|
||||
- Source/data integrity PASS. pytest: **75 passed, 0 failed, 0 skipped, 0 warnings**.
|
||||
|
||||
## 17. Files created / modified this session
|
||||
## 16. Files created / modified this session
|
||||
|
||||
Created:
|
||||
|
||||
- `docs/research/ALPHA075_BETA_VALIDATION.md`
|
||||
- `docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md`
|
||||
- `docs/research/SESSION8_AUDIT.md`
|
||||
- `scripts/analyze_partition_heterogeneity.py`
|
||||
- `scripts/verify_beta0_equivalence.py`
|
||||
- `scripts/analyze_cross_alpha.py`
|
||||
- `artifacts/analysis/xiiotid__alpha0.75__*` (run snapshots, effect/weights CSVs, multiseed
|
||||
tables, `beta0_equivalence.csv`, `runtime.csv`)
|
||||
- `artifacts/analysis/xiiotid__cross_alpha__*` (partition heterogeneity per-seed/per-class/summary,
|
||||
cross-alpha beta table/displacement/client summary/summary)
|
||||
- `results/xiiotid__alpha0.75__seed{0,1,2}__*` (18 run dirs, gitignored)
|
||||
- `docs/research/5GNIDD_ALPHA050_BETA_VALIDATION.md`
|
||||
- `docs/research/SESSION9_AUDIT.md`
|
||||
- `artifacts/analysis/5gnidd__alpha0.5__*` (46 files: 18 prototype snapshots, 6 prototype
|
||||
effect/weights CSVs, multiseed validation/client/test/effect/beta0/runtime tables,
|
||||
partition heterogeneity + characterization, audit recomputation)
|
||||
- `results/5gnidd__alpha0.5__seed{0,1,2}__*` (18 run dirs, gitignored)
|
||||
|
||||
Modified:
|
||||
|
||||
@@ -337,86 +317,81 @@ Modified:
|
||||
|
||||
No source or test code was changed.
|
||||
|
||||
## 18. Exact recommended next phase
|
||||
## 17. Exact recommended next phase
|
||||
|
||||
1. **Session 8 is audited and safe to checkpoint** (verdict PASS; see §16 and
|
||||
`docs/research/SESSION8_AUDIT.md`). X-IIoTID alpha=0.75 behaved cleanly. Before
|
||||
checkpointing, stage only the intended docs/scripts/analysis artifacts; do not commit
|
||||
`results/` or the raw datasets.
|
||||
2. **X-IIoTID three-alpha study is complete** (0.25, 0.50, 0.75 × seeds `{0,1,2}`), with
|
||||
audited condition-level pooled selections 0.00 / 0.75 / 0.50 and a cross-alpha report.
|
||||
3. **Next major phase: 5G-NIDD.** Follow the same protocol (the planned alphas `{0.75, 0.50,
|
||||
0.25}`, seeds `{0,1,2}`, five betas + `original_protean`, frozen manifests, pooled
|
||||
validation-only selection). Do **not** launch the full grid blindly: verify 5G-NIDD manifests
|
||||
exist and reproduce, run the same one-condition-at-a-time pacing, and measure runtime first.
|
||||
A single smoke/one-condition run before scaling is recommended.
|
||||
4. Do **not** treat any selected beta as an optimum. Report each as "best among tested
|
||||
candidates under pooled validation for that condition".
|
||||
5. Do **not** infer a monotone alpha→beta relation from three X-IIoTID conditions.
|
||||
6. Update `EXPERIMENT_PROTOCOL.md` only if a real ambiguity appears; update this handoff.
|
||||
7. Do not commit or push unless asked.
|
||||
1. **Session 9 is audited and safe to checkpoint** (verdict PASS; see §15 and
|
||||
`docs/research/SESSION9_AUDIT.md`). Before checkpointing, stage only the intended
|
||||
docs/analysis artifacts; do not commit `results/` or the raw datasets.
|
||||
2. **5G-NIDD alpha=0.50 is complete.** Next 5G-NIDD condition (alpha=0.75 or alpha=0.25) is
|
||||
**safe to run** — its manifest already exists and reproduces from raw data, the pipeline is
|
||||
stable, and the same frozen protocol applies. Budget **≈19 h per condition** (18 runs × ~64 min)
|
||||
and run **one condition at a time**; do not launch the full grid blindly. A single condition,
|
||||
then inspect, then proceed is recommended.
|
||||
3. Follow the same pooled validation-only selection rule; report each selected beta as "best among
|
||||
the tested candidates under pooled validation for that condition" — never a global optimum.
|
||||
4. Do not infer a monotone alpha→beta relation from three X-IIoTID conditions or from a single
|
||||
5G-NIDD condition; do not claim dataset causality from two datasets.
|
||||
5. Update `EXPERIMENT_PROTOCOL.md` only if a real ambiguity appears; update this handoff.
|
||||
6. Do not commit or push unless asked.
|
||||
|
||||
## 19. Important commands
|
||||
## 18. Important commands
|
||||
|
||||
```bash
|
||||
# Tests (75 expected)
|
||||
.venv/bin/python -m pytest
|
||||
|
||||
# One full paired beta run (CPU, ~40 min, ~6.5-7.2 GiB)
|
||||
.venv/bin/python scripts/run_baseline.py --dataset xiiotid --alpha 0.75 --seed 1 \
|
||||
# One full paired beta run (CPU, ~63-68 min, ~8.7-14.1 GiB for 5G-NIDD)
|
||||
.venv/bin/python scripts/run_baseline.py --dataset 5gnidd --alpha 0.5 --seed 1 \
|
||||
--clients 10 --rounds 10 --local-epochs 3 \
|
||||
--aggregation-mode support_weighted --beta 0.5
|
||||
--aggregation-mode support_weighted --beta 1.0
|
||||
|
||||
# Pooled multi-seed validation analysis (read-only, no retraining)
|
||||
.venv/bin/python scripts/analyze_multiseed_beta.py --dataset xiiotid --alpha 0.75 --seeds 0 1 2
|
||||
.venv/bin/python scripts/analyze_multiseed_beta.py --dataset 5gnidd --alpha 0.5 --seeds 0 1 2
|
||||
|
||||
# Direct prototype-effect analysis (no retraining)
|
||||
.venv/bin/python scripts/analyze_prototype_effect.py \
|
||||
--snapshot artifacts/analysis/xiiotid__alpha0.75__seed0__support_weighted__beta0__prototype_snapshot.json \
|
||||
--class-names "C&C,Exfiltration,Exploitation,Lateral_movement,Normal,RDOS,Reconnaissance,Tampering,Weaponization" \
|
||||
--snapshot artifacts/analysis/5gnidd__alpha0.5__seed0__support_weighted__beta0__prototype_snapshot.json \
|
||||
--class-names "Benign,HTTPFlood,ICMPFlood,SYNFlood,SYNScan,SlowrateDoS,TCPConnectScan,UDPFlood,UDPScan" \
|
||||
--output-dir artifacts/analysis
|
||||
|
||||
# Realized partition characterization (all alphas)
|
||||
.venv/bin/python scripts/analyze_partition_heterogeneity.py --dataset xiiotid --alphas 0.75 0.5 0.25
|
||||
# Realized partition characterization
|
||||
.venv/bin/python scripts/analyze_partition_heterogeneity.py --dataset 5gnidd --alphas 0.5
|
||||
|
||||
# beta=0 equivalence audit
|
||||
.venv/bin/python scripts/verify_beta0_equivalence.py --dataset xiiotid --alpha 0.75 --seeds 0 1 2
|
||||
|
||||
# Cross-alpha comparison
|
||||
.venv/bin/python scripts/analyze_cross_alpha.py --dataset xiiotid --alphas 0.25 0.5 0.75
|
||||
.venv/bin/python scripts/verify_beta0_equivalence.py --dataset 5gnidd --alpha 0.5 --seeds 0 1 2
|
||||
```
|
||||
|
||||
## 20. Recommended next-session prompt
|
||||
## 19. Recommended next-session prompt
|
||||
|
||||
```
|
||||
You are continuing the perdalam-protean research repository (Session 9).
|
||||
You are continuing the perdalam-protean research repository (Session 10).
|
||||
|
||||
FIRST, read completely:
|
||||
1. docs/handoff/SESSION_HANDOFF.md
|
||||
2. docs/research/ALPHA075_BETA_VALIDATION.md
|
||||
3. docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md
|
||||
4. docs/research/SESSION8_AUDIT.md
|
||||
5. docs/research/EXPERIMENT_PROTOCOL.md
|
||||
6. docs/baseline/BASELINE_VALIDATION.md
|
||||
2. docs/research/5GNIDD_ALPHA050_BETA_VALIDATION.md
|
||||
3. docs/research/SESSION9_AUDIT.md
|
||||
4. docs/research/EXPERIMENT_PROTOCOL.md
|
||||
5. docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md
|
||||
6. docs/research/MULTISEED_BETA_VALIDATION.md
|
||||
|
||||
Critical rules:
|
||||
- planningawal/ is immutable.
|
||||
- ~/datasets/X-IIoTID/X-IIoTID-dataset.csv and ~/datasets/5G-NIDD/5G-NIDD.csv are read-only.
|
||||
- The S_j participant rule is a REPRODUCTION ASSUMPTION (material in 19/27 cells at alpha=0.25).
|
||||
- ~/datasets/5G-NIDD/5G-NIDD.csv and ~/datasets/X-IIoTID/X-IIoTID-dataset.csv are read-only.
|
||||
- The S_j participant rule is a REPRODUCTION ASSUMPTION.
|
||||
- Do NOT add an executable equal_available mode.
|
||||
- Beta selection uses validation only; PRIMARY is pooled across clients and seeds.
|
||||
- Do NOT launch the full grid blindly; verify manifests and pace one condition at a time.
|
||||
- Do NOT launch the full grid; run one condition at a time (~19 h per 5G-NIDD condition).
|
||||
|
||||
Context: Session 8 completed X-IIoTID alpha=0.75 for seeds {0,1,2} (18 runs, ~12.33 h) and was
|
||||
independently audited (PASS; docs/research/SESSION8_AUDIT.md). Pooled validation selected
|
||||
beta=0.50. Three-alpha X-IIoTID pooled selections: alpha=0.25 -> 0.00, alpha=0.50 -> 0.75,
|
||||
alpha=0.75 -> 0.50 (NOT monotone). beta=0 equivalence class B. Realized heterogeneity: 19/27,
|
||||
7/27, 3/27 cells |S_j|<M; mean displacement 3.42 / 2.10 / 1.31.
|
||||
Context: Session 9 completed 5G-NIDD alpha=0.50 for seeds {0,1,2} (18 runs, ~19.26 h) and was
|
||||
independently audited (PASS; docs/research/SESSION9_AUDIT.md). Pooled validation selected
|
||||
beta=1.00 (per-seed diagnostics 1.00 / 0.00 / 1.00). beta=0 equivalence class B. Realized
|
||||
heterogeneity: 4/27 cells |S_j|<M, 4 missing cells, client ratio mean 9.59, mean displacement
|
||||
2.340. X-IIoTID alpha=0.50 selected 0.75. The pre-existing seed-0 baseline was not reusable and
|
||||
was re-run; it reproduces exactly.
|
||||
|
||||
Task: X-IIoTID three-alpha study is complete. Recommended next phase is 5G-NIDD under the same
|
||||
frozen protocol (alphas {0.75, 0.50, 0.25}, seeds {0,1,2}, five betas + original_protean,
|
||||
frozen manifests, pooled validation-only selection). Before scaling, verify 5G-NIDD manifests
|
||||
exist and reproduce, run one paired condition, inspect it, then proceed. Produce a 5G-NIDD beta
|
||||
validation report and a cross-dataset comparison, update the handoff. Use .venv/bin/python. Do
|
||||
not commit or push unless asked.
|
||||
Task: run the next single 5G-NIDD condition (recommended alpha=0.75, then alpha=0.25) under the
|
||||
same frozen protocol (seeds {0,1,2}, five betas + original_protean, frozen manifests, pooled
|
||||
validation-only selection). Verify the manifest reproduces, run the 18 runs sequentially with
|
||||
runtime/status files, produce a condition report + audit, and extend the cross-dataset/cross-alpha
|
||||
comparison. Use .venv/bin/python. Do not commit or push unless asked.
|
||||
```
|
||||
Reference in new issue
Block a user