experiment: complete two-dataset beta study
This commit is contained in:
willtanoe committed
1 parent
bbd3508506
commit
d30e9831e6
37 files changed
+36038
-195
No files matched your search
+223
-195
@@ -3,26 +3,30 @@
|
||||
This is the canonical continuation point for the next agent session. It is written to be
|
||||
readable without prior chat history, terminal scrollback, or git history.
|
||||
|
||||
Last updated: 2026-09-29 (Session 10). Session 10 type: **second 5G-NIDD beta-validation
|
||||
condition**. Sessions 6–8 completed the X-IIoTID three-alpha study (`alpha=0.25/0.50/0.75`,
|
||||
seeds `{0,1,2}`); Session 9 ran 5G-NIDD `alpha=0.50` for seeds `{0,1,2}`. Session 10 ran **only**
|
||||
5G-NIDD `alpha=0.75` for seeds `{0,1,2}` across the five frozen beta candidates plus
|
||||
`original_protean`, using the same pooled-across-seed validation-only selection rule.
|
||||
**5G-NIDD alpha=0.25 was NOT run.** No commit or push was made.
|
||||
Last updated: 2026-09-30 (Session 11). Session 11 type: **final 5G-NIDD beta-validation condition**
|
||||
(`alpha=0.25`), completing the planned experiment grid. Sessions 6–8 completed the X-IIoTID
|
||||
three-alpha study (`alpha=0.25/0.50/0.75`, seeds `{0,1,2}`); Session 9 ran 5G-NIDD `alpha=0.50`;
|
||||
Session 10 ran 5G-NIDD `alpha=0.75`; Session 11 ran **only** 5G-NIDD `alpha=0.25` for seeds
|
||||
`{0,1,2}` across the five frozen beta candidates plus `original_protean`, using the same
|
||||
pooled-across-seed validation-only selection rule. **No condition was run after alpha=0.25.** No
|
||||
commit or push was made.
|
||||
|
||||
> **Audit status (Session 10).** Independent audit verdict: **PASS**
|
||||
> (`docs/research/SESSION10_AUDIT.md`). `beta_star(5gnidd, alpha=0.75) = 0.50` confirmed from
|
||||
> **Audit status (Session 11).** Independent audit verdict: **PASS**
|
||||
> (`docs/research/SESSION11_AUDIT.md`). `beta_star(5gnidd, alpha=0.25) = 0.00` confirmed from
|
||||
> validation only; run inventory 18/18; pairing hash-identical and reproduced from raw data;
|
||||
> rare/unseen 0 mismatches over 1,620 cells; leakage audit PASS; `beta=0` class **B**; source/data
|
||||
> integrity PASS; pytest 75 passed. **Session 10 is safe to checkpoint.**
|
||||
> integrity PASS; pytest 75 passed. **Session 11 is safe to checkpoint.**
|
||||
|
||||
> **Headline (Session 10).** PRIMARY pooled-validation selected beta for 5G-NIDD `alpha=0.75` =
|
||||
> **0.50** (validation only; per-seed diagnostics 0.50 / 1.00 / 1.00). The pooled validation curve
|
||||
> is **non-monotone** (0.778 → 0.790 → 0.809 → 0.792 → 0.804) and the total between-beta spread
|
||||
> (≈ 0.031) is smaller than the between-seed std (0.051–0.065), so the selection is statistically
|
||||
> weak. `beta=0` is class **B** for all three seeds. At alpha=0.75 the rare-class union is
|
||||
> **identical across seeds** and realized support imbalance is lower than at alpha=0.50, yet the
|
||||
> number of `|S_j|<M` cells is unchanged (4/27) and missing cells are slightly higher (5 vs 4).
|
||||
> **Headline (Session 11).** PRIMARY pooled-validation selected beta for 5G-NIDD `alpha=0.25` =
|
||||
> **0.00** (validation only; per-seed diagnostics 0.25 / 0.00 / 0.00). The pooled validation curve
|
||||
> declines from 0.761 (beta=0) to 0.722 (beta=0.75) and rises slightly to 0.738 (beta=1.0), with
|
||||
> the between-beta spread (0.039) exceeding the across-seed std (0.003–0.022). This is the first
|
||||
> condition where the selected beta is 0, i.e. the equal-`S_j` mean; the selected mode is therefore
|
||||
> numerically the baseline and shows no support-weighting benefit. `beta=0` is class **B** for all
|
||||
> three seeds. Realized heterogeneity is the highest of the three 5G-NIDD conditions
|
||||
> (16/27 `|S_j|<M` cells, 30 missing cells, client ratio mean 17.60, mean CV 1.302), and the
|
||||
> rare-class union differs across seeds (6/6/8), so this condition's test rare support is not
|
||||
> constant.
|
||||
|
||||
---
|
||||
|
||||
@@ -46,8 +50,8 @@ bitwise identical to the baseline (see §7).
|
||||
## 2. Authoritative source and immutability
|
||||
|
||||
- `planningawal/PROTEAN_Revisi_Lurus.md` — authoritative research specification.
|
||||
- `planningawal/` is immutable. Re-verified unchanged this session: `git status --short
|
||||
planningawal/` and `git diff -- planningawal/` are both empty.
|
||||
- `planningawal/` is immutable. Re-verified unchanged this session: `git status --porcelain
|
||||
planningawal/` empty.
|
||||
- Raw datasets re-verified unchanged (SHA-256):
|
||||
`~/datasets/5G-NIDD/5G-NIDD.csv`
|
||||
`fa36f80859585f474504ca69eb951a079b35145537976c86b00aae3aab46ee59`;
|
||||
@@ -55,40 +59,42 @@ bitwise identical to the baseline (see §7).
|
||||
`7b9290057ee42e784da3c0d84b781815502c9205c74175c96374e71a5ffd98a0`.
|
||||
- Frozen baseline (`original_protean`) source unchanged; no `equal_available` mode added; beta
|
||||
candidate set unchanged (`{0, .25, .5, .75, 1}`). No tracked source/test/config file differs
|
||||
from HEAD. The only new tracked-class file candidates are the new docs/artifacts listed in §16.
|
||||
from HEAD. Only new docs/analysis artifacts are untracked.
|
||||
|
||||
## 3. Session 10 objective and outcome
|
||||
## 3. Session 11 objective and outcome
|
||||
|
||||
Objective: run exactly **one** new condition — 5G-NIDD `alpha=0.75`, seeds `{0,1,2}`,
|
||||
`original_protean` + five betas — after a Stage-A verification gate. Do not run alpha=0.25, do not
|
||||
launch the grid, do not change methodology.
|
||||
Objective: run exactly the **final** condition — 5G-NIDD `alpha=0.25`, seeds `{0,1,2}`,
|
||||
`original_protean` + five betas — after a Stage-A verification gate; then build the 5G-NIDD
|
||||
three-alpha comparison and the cross-dataset comparison. Do not run anything after it, do not
|
||||
change methodology.
|
||||
|
||||
Stage A result: raw hash valid; all three 5G-NIDD `alpha=0.75` manifests reproduce exactly from raw
|
||||
data (support match, partition-hash match); label semantics and rare/unseen logic valid; pairing
|
||||
framework valid. Gate = **GO**.
|
||||
Stage A result: raw hash valid; all three 5G-NIDD `alpha=0.25` manifests reproduce exactly from raw
|
||||
data (support match, partition-hash match, split-hash match); label semantics and rare/unseen logic
|
||||
valid; pairing framework valid. Gate = **GO**.
|
||||
|
||||
Outcome: 18/18 runs rc=0. **PRIMARY pooled-validation selected beta = 0.50** (validation only);
|
||||
`test_used_for_selection = false` for all 15 runs. Pooled means: 0.00 → 0.778094, 0.25 → 0.790483,
|
||||
0.50 → 0.809081, 0.75 → 0.791530, 1.00 → 0.804306. Reports:
|
||||
`docs/research/5GNIDD_ALPHA075_BETA_VALIDATION.md`, audit `docs/research/SESSION10_AUDIT.md`.
|
||||
Outcome: 18/18 runs rc=0. **PRIMARY pooled-validation selected beta = 0.00** (validation only);
|
||||
`test_used_for_selection = false` for all 15 runs. Pooled means: 0.00 → 0.761453, 0.25 → 0.749286,
|
||||
0.50 → 0.725715, 0.75 → 0.722235, 1.00 → 0.737636. Reports:
|
||||
`docs/research/5GNIDD_ALPHA025_BETA_VALIDATION.md`, `docs/research/5GNIDD_CROSS_ALPHA_ANALYSIS.md`,
|
||||
`docs/research/CROSS_DATASET_BETA_ANALYSIS.md`, audit `docs/research/SESSION11_AUDIT.md`.
|
||||
|
||||
## 4. Condition and pairing
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| dataset / alpha / seeds | `5gnidd` / 0.75 / `{0,1,2}` |
|
||||
| dataset / alpha / seeds | `5gnidd` / 0.25 / `{0,1,2}` |
|
||||
| runs | 15 `support_weighted` + 3 `original_protean`, all new this session |
|
||||
| training | 10 clients, 10 rounds, 3 local epochs, batch 256, lr 1e-3, mu 0.1, lambda 1.0, Adam |
|
||||
| code revision | unchanged from HEAD `a7f2b0049e99fb66021b4660b82a48d5dbbef072`; only docs/analysis added |
|
||||
| code revision | only docs/analysis added; HEAD `bbd35085063194db58e4472e70b204d4be4a3607` |
|
||||
|
||||
Manifests (reused, never regenerated; one content hash per seed across all six runs; each
|
||||
reproduced from the raw dataset with split/`N_ij`/partition hash verified):
|
||||
|
||||
| seed | manifest content hash | partition hash | split hash | rng seed | round-0 snapshot SHA-256 |
|
||||
|---|---|---|---|---|---|
|
||||
| 0 | `96cd9b90b71895da…` | `74a91845627dbe57…` | `18f89f48…` | 892370747 | `266a6e9976d5c138…` |
|
||||
| 1 | `007c23564a44919b…` | `fc04ba00f4942bf9…` | `3d05b7fe…` | 1817661572 | `acecab9a60852e45…` |
|
||||
| 2 | `3134bea2d47a7336…` | `4ee33d0a28326ae1…` | `fc758d06…` | 306220443 | `ef90c51aa9998ec8…` |
|
||||
| 0 | `ab6d8a8b63d478c4…` | `e628c17369f25e35…` | `18f89f4807de5493…` | 708356900 | `e2243ac4ed4391ff…` |
|
||||
| 1 | `74df102e4edb4509…` | `05aec12ceb4c94bd…` | `3d05b7fe48d5a06c…` | 902143161 | `26a162991b7cbde9…` |
|
||||
| 2 | `061c094d1913f942…` | `d9631ea6b2d3907e…` | `fc758d060dc51dec…` | 218373793 | `e287bff5ca71f8d2…` |
|
||||
|
||||
Splits: train/validation/test = 851,123 / 121,589 / 243,177 for every seed (10 clients, 9 classes,
|
||||
`Benign`). `train_support_Nij.csv`, `characterization.csv`, `rare_unseen.csv` identical across each
|
||||
@@ -98,61 +104,62 @@ hashes.
|
||||
## 5. Validation results (USED FOR SELECTION) and pooled selection
|
||||
|
||||
Primary metric: validation per-client-mean rare-class macro-F1. All 10 clients eligible in every
|
||||
seed. Full tables: `artifacts/analysis/5gnidd__alpha0.75__multiseed__validation_per_seed.csv`.
|
||||
seed. Full tables: `artifacts/analysis/5gnidd__alpha0.25__multiseed__validation_per_seed.csv`.
|
||||
|
||||
Per-seed validation per-client-mean rare-class macro-F1:
|
||||
|
||||
| beta | seed 0 | seed 1 | seed 2 |
|
||||
|---|---|---|---|
|
||||
| 0.00 | 0.834681 | 0.763787 | 0.735812 |
|
||||
| 0.25 | 0.823388 | 0.821190 | 0.726871 |
|
||||
| 0.50 | **0.836792** | 0.848836 | 0.741617 |
|
||||
| 0.75 | 0.824667 | 0.833255 | 0.716668 |
|
||||
| 1.00 | 0.815774 | **0.852469** | **0.744677** |
|
||||
| 0.00 | 0.736360 | **0.771268** | **0.776732** |
|
||||
| 0.25 | **0.747299** | 0.767300 | 0.733259 |
|
||||
| 0.50 | 0.717450 | 0.738223 | 0.721471 |
|
||||
| 0.75 | 0.713262 | 0.728478 | 0.724966 |
|
||||
| 1.00 | 0.734455 | 0.739297 | 0.739157 |
|
||||
|
||||
Pooled across seeds (`...__validation_pooled.csv`):
|
||||
|
||||
| beta | pooled mean | std (sample) | min | max | selected |
|
||||
|---|---|---|---|---|---|
|
||||
| 0.00 | 0.778094 | 0.050963 | 0.735812 | 0.834681 | |
|
||||
| 0.25 | 0.790483 | 0.055100 | 0.726871 | 0.823388 | |
|
||||
| **0.50** | **0.809081** | 0.058735 | 0.741617 | 0.848836 | **yes** |
|
||||
| 0.75 | 0.791530 | 0.064974 | 0.716668 | 0.833255 | |
|
||||
| 1.00 | 0.804306 | 0.054803 | 0.744677 | 0.852469 | |
|
||||
| **0.00** | **0.761453** | 0.021903 | 0.736360 | 0.776732 | **yes** |
|
||||
| 0.25 | 0.749286 | 0.017107 | 0.733259 | 0.767300 | |
|
||||
| 0.50 | 0.725715 | 0.011017 | 0.717450 | 0.738223 | |
|
||||
| 0.75 | 0.722235 | 0.007967 | 0.713262 | 0.728478 | |
|
||||
| 1.00 | 0.737636 | 0.002756 | 0.734455 | 0.739297 | |
|
||||
|
||||
- **PRIMARY selected beta (`beta_star_5gnidd_alpha075`) = 0.50**, max pooled validation
|
||||
per-client-mean rare-class macro-F1; **no tie-break used**; validation-only.
|
||||
- **SECONDARY per-seed diagnostics only:** seed 0 → 0.50, seed 1 → 1.00, seed 2 → 1.00.
|
||||
- Between-beta spread ≈ 0.031; between-seed std 0.051–0.065 → statistically weak, descriptive.
|
||||
- **PRIMARY selected beta (`beta_star_5gnidd_alpha025`) = 0.00**, max pooled validation
|
||||
per-client-mean rare-class macro-F1; **no tie-break used**; validation-only. Top-two gap ≈ 1.2e-2.
|
||||
- **SECONDARY per-seed diagnostics only:** seed 0 → 0.25, seed 1 → 0.00, seed 2 → 0.00.
|
||||
- Between-beta spread ≈ 0.039; between-seed std 0.003–0.022. At this condition the spread exceeds
|
||||
the seed noise, but the sample is only 3 project-defined seeds.
|
||||
|
||||
## 6. Final test evaluation of the pooled-selected beta
|
||||
|
||||
`beta_star = 0.50` was fixed from pooled validation before test was read.
|
||||
`beta_star = 0.00` was fixed from pooled validation before test was read.
|
||||
|
||||
| metric (test) | seed 0 | seed 1 | seed 2 | mean ± std (sample) |
|
||||
|---|---|---|---|---|
|
||||
| rare-class macro-F1 | 0.904447 | 0.905721 | 0.880045 | 0.896738 ± 0.014470 |
|
||||
| rare-class recall | 0.958666 | 0.959723 | 0.960715 | 0.959701 ± 0.001025 |
|
||||
| macro-F1 | 0.857309 | 0.859939 | 0.842201 | 0.853150 ± 0.009572 |
|
||||
| accuracy | 0.749508 | 0.754381 | 0.752933 | 0.752274 ± 0.002503 |
|
||||
| weighted-F1 | 0.723885 | 0.730109 | 0.728408 | 0.727467 ± 0.003217 |
|
||||
| rare-class macro-F1 | 0.859893 | 0.813889 | 0.849925 | 0.841236 ± 0.024202 |
|
||||
| rare-class recall | 0.985234 | 0.972659 | 0.925756 | 0.961216 ± 0.031347 |
|
||||
| macro-F1 | 0.829680 | 0.802987 | 0.817254 | 0.816640 ± 0.013357 |
|
||||
| accuracy | 0.744844 | 0.737779 | 0.708052 | 0.730225 ± 0.019525 |
|
||||
| weighted-F1 | 0.717404 | 0.712302 | 0.683684 | 0.704463 ± 0.018175 |
|
||||
|
||||
`original_protean` for the same seeds/condition:
|
||||
|
||||
| metric (test) | seed 0 | seed 1 | seed 2 | mean ± std (sample) |
|
||||
|---|---|---|---|---|
|
||||
| rare-class macro-F1 | 0.905589 | 0.860758 | 0.873356 | 0.879901 ± 0.023121 |
|
||||
| rare-class recall | 0.961285 | 0.959036 | 0.963321 | 0.961214 ± 0.002144 |
|
||||
| macro-F1 | 0.857109 | 0.829867 | 0.837238 | 0.841405 ± 0.014091 |
|
||||
| accuracy | 0.747048 | 0.754072 | 0.751745 | 0.750955 ± 0.003578 |
|
||||
| weighted-F1 | 0.721059 | 0.729950 | 0.727133 | 0.726047 ± 0.004544 |
|
||||
| rare-class macro-F1 | 0.864903 | 0.814848 | 0.847072 | 0.842274 ± 0.025370 |
|
||||
| rare-class recall | 0.985444 | 0.972333 | 0.923890 | 0.960556 ± 0.032423 |
|
||||
| macro-F1 | 0.835043 | 0.808373 | 0.814800 | 0.819405 ± 0.013919 |
|
||||
| accuracy | 0.745280 | 0.741295 | 0.706329 | 0.730968 ± 0.021431 |
|
||||
| weighted-F1 | 0.718041 | 0.716327 | 0.681939 | 0.705435 ± 0.020367 |
|
||||
|
||||
Descriptive difference for test rare-class macro-F1 (`beta_star` − `original_protean`) is
|
||||
−0.001141 / +0.044963 / +0.006689 (mean +0.016837). Test is descriptive only; no superiority or
|
||||
generalisation claimed. The global rare evaluation support is **28,000 for every seed** (rare-class
|
||||
union identical across seeds at alpha=0.75), so cross-seed test levels are directly comparable at
|
||||
this condition. Test weighted-F1 (`≈0.727`) again sits **below** accuracy (`≈0.752`), as at
|
||||
alpha=0.50.
|
||||
−0.005010 / −0.000959 / +0.002853 (mean −0.001039). Because beta_star = 0, this difference is just
|
||||
floating-point divergence (see §7). The global rare evaluation support differs by seed
|
||||
(**141,677 / 129,824 / 147,630**), so cross-seed test levels are **not** directly comparable at
|
||||
this condition. Test weighted-F1 (`≈0.704`) again sits **below** accuracy (`≈0.730`), as at
|
||||
alpha=0.50/0.75.
|
||||
|
||||
## 7. `beta = 0` equivalence result (all three seeds)
|
||||
|
||||
@@ -160,219 +167,243 @@ For seeds 0, 1, 2, `support_weighted beta=0` vs frozen `original_protean`:
|
||||
|
||||
- Same manifest/split/hashes, round-0 snapshot byte-identical, support/char/rare_unseen
|
||||
identical: **yes** for all three.
|
||||
- Independent aggregation (`ΣC/n` vs `(1/n)ΣC`): 4.4e-16 / 4.4e-16 / 4.4e-16.
|
||||
- History rare-macro-F1 max diff (rounds ≥ 1): 3.22e-2 / 5.04e-2 / 2.72e-2; test metric max diff
|
||||
5.22e-3 / 5.32e-3 / 2.74e-4.
|
||||
- Independent aggregation (`ΣC/n` vs `(1/n)ΣC`): 4.4e-16 / 4.4e-16 / 6.7e-16.
|
||||
- History rare-macro-F1 max diff (rounds ≥ 1): 2.17e-2 / 6.85e-2 / 2.74e-2; test metric max diff
|
||||
5.36e-3 / 5.39e-3 / 2.85e-3.
|
||||
- Classification: **B** in all seeds — semantic equivalence + floating-point divergence; no
|
||||
semantic discrepancy. Machine-readable:
|
||||
`artifacts/analysis/5gnidd__alpha0.75__multiseed__beta0_equivalence.csv`.
|
||||
`artifacts/analysis/5gnidd__alpha0.25__multiseed__beta0_equivalence.csv`.
|
||||
|
||||
Conditional on the shared `S_j` reproduction assumption; not proof `beta=0` reproduces original
|
||||
PROTEAN.
|
||||
PROTEAN. Since the selected beta is 0, the selected-run test metrics differ from original only by
|
||||
this divergence.
|
||||
|
||||
## 8. Partition characterization (realized) and comparison with alpha=0.50
|
||||
## 8. Partition characterization (realized) and comparison with alpha=0.50/0.75
|
||||
|
||||
5G-NIDD `alpha=0.75` (training split), `M = 10`:
|
||||
5G-NIDD `alpha=0.25` (training split), `M = 10`:
|
||||
|
||||
| seed | `|S_j| < M` cells | missing client-class cells | client-size ratio | union rare | union unseen | max unseen/client |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| 0 | 2 / 9 | 3 | 3.70 | 6 | 2 | 1 |
|
||||
| 1 | 2 / 9 | 2 | 11.36 | 6 | 2 | 1 |
|
||||
| 2 | 0 / 9 | 0 | 3.87 | 6 | 0 | 0 |
|
||||
| **sum/mean** | **4 / 27** | **5** | mean **6.31** | 6/6/6 | 2/2/0 | 1 |
|
||||
| 0 | 6 / 9 | 10 | 9.93 | 6 | 6 | 3 |
|
||||
| 1 | 5 / 9 | 10 | 27.83 | 6 | 5 | 3 |
|
||||
| 2 | 5 / 9 | 10 | 15.05 | 8 | 5 | 2 |
|
||||
| **sum/mean** | **16 / 27** | **30** | mean **17.60** | 6/6/8 | 6/5/5 | 3 |
|
||||
|
||||
All 10 clients have exactly two rare classes and are eligible in every seed. **The rare-class union
|
||||
is identical across all three seeds** (`{ICMPFlood, SYNFlood, SYNScan, SlowrateDoS, TCPConnectScan,
|
||||
UDPScan}`), so test rare support is constant (28,000). Mean class-support CV = **0.916**. Missing
|
||||
cells are single-client gaps: seed 0 (`ICMPFlood` c0/c3, `SYNFlood` c4), seed 1 (`ICMPFlood` c3,
|
||||
`SYNScan` c6). Full tables:
|
||||
`artifacts/analysis/5gnidd__alpha0.75__partition_heterogeneity_*` and
|
||||
Mean class-support CV = **1.302**; max per-class `max/min` = 185,504. All 10 clients have exactly
|
||||
two rare classes and are eligible. The rare-class union **differs across seeds** (6/6/8), so test
|
||||
rare support is not constant (141,677 / 129,824 / 147,630). This is the most heterogeneous of the
|
||||
three 5G-NIDD conditions. Full tables:
|
||||
`artifacts/analysis/5gnidd__cross_alpha__partition_heterogeneity_*` and
|
||||
`...__partition_characterization.csv`.
|
||||
|
||||
### Comparison with 5G-NIDD alpha=0.50 (descriptive)
|
||||
### 5G-NIDD three-alpha comparison
|
||||
|
||||
| statistic | 5G-NIDD a0.50 | 5G-NIDD a0.75 |
|
||||
|---|---|---|
|
||||
| pooled-selected beta | **1.00** | **0.50** |
|
||||
| pooled validation mean range (spread) | 0.645–0.692 (0.047) | 0.778–0.809 (0.031) |
|
||||
| pooled validation std range | 0.053–0.139 | 0.051–0.065 |
|
||||
| `|S_j| < M` cells (sum/27) | 4/27 | 4/27 |
|
||||
| missing client-class cells | 4 | 5 |
|
||||
| client-size ratio mean | 9.59 | 6.31 |
|
||||
| mean class-support CV | 1.225 | 0.916 |
|
||||
| union rare per seed | 6/5/6 | 6/6/6 |
|
||||
| union unseen per seed | 2/0/2 | 2/2/0 |
|
||||
| test rare support per seed | 104,843 / 13,375 / 128,995 | 28,000 / 28,000 / 28,000 |
|
||||
| mean displacement `‖Cbar(1)−Cbar(0)‖` | 2.340 | 1.385 |
|
||||
| selected-beta client improve/degrade | 6/3, 3/5, 7/2 | 4/6, 6/3, 4/2 |
|
||||
| `beta=0` class | B/B/B | B/B/B |
|
||||
| statistic | a0.25 | a0.50 | a0.75 |
|
||||
|---|---|---|---|
|
||||
| pooled-selected beta | **0.00** | **1.00** | **0.50** |
|
||||
| `|S_j| < M` cells (sum/27) | **16/27** | 4/27 | 4/27 |
|
||||
| missing client-class cells | **30** | 4 | 5 |
|
||||
| client-size ratio mean | **17.60** | 9.59 | 6.31 |
|
||||
| mean class-support CV | **1.302** | 1.225 | 0.916 |
|
||||
| union rare per seed | 6/6/8 | 6/5/6 | 6/6/6 |
|
||||
| union unseen per seed | 6/5/5 | 2/0/2 | 2/2/0 |
|
||||
| test rare support per seed | 141,677 / 129,824 / 147,630 | 104,843 / 13,375 / 128,995 | 28,000 / 28,000 / 28,000 |
|
||||
| mean displacement `‖Cbar(1)−Cbar(0)‖` | **2.919** | 2.340 | 1.385 |
|
||||
| `beta=0` class | B/B/B | B/B/B | B/B/B |
|
||||
|
||||
Higher alpha lowers **mean** support imbalance and mean displacement, but the number of
|
||||
`|S_j|<M` cells is unchanged and missing cells rise. No causal or monotone claim (two alpha levels).
|
||||
Machine-readable cross-alpha artifacts: `artifacts/analysis/5gnidd__cross_alpha__*`.
|
||||
Realized imbalance and mean displacement decrease as alpha rises; the selected beta is **not
|
||||
monotone** in alpha (0.00 → 1.00 → 0.50). No causal/monotone claim (three alphas, three seeds, one
|
||||
dataset).
|
||||
|
||||
## 9. Direct prototype-effect findings (fixed round-0 snapshots, no retraining)
|
||||
|
||||
`||Cbar_j(1) − Cbar_j(0)||`, cross-seed mean (std), 5G-NIDD alpha=0.75:
|
||||
`||Cbar_j(1) − Cbar_j(0)||`, cross-seed mean (std), 5G-NIDD alpha=0.25:
|
||||
|
||||
| class | mean disp | std |
|
||||
|---|---|---|
|
||||
| UDPScan | 2.445 | 0.484 |
|
||||
| SYNFlood | 1.897 | 0.171 |
|
||||
| SlowrateDoS | 1.565 | 0.654 |
|
||||
| SYNScan | 1.563 | 0.640 |
|
||||
| HTTPFlood | 1.366 | 0.544 |
|
||||
| TCPConnectScan | 1.226 | 0.476 |
|
||||
| UDPFlood | 0.903 | 0.103 |
|
||||
| Benign | 0.824 | 0.272 |
|
||||
| ICMPFlood | 0.673 | 0.331 |
|
||||
| SYNScan | 3.815 | 1.402 |
|
||||
| HTTPFlood | 3.760 | 1.539 |
|
||||
| TCPConnectScan | 3.258 | 2.574 |
|
||||
| SlowrateDoS | 3.179 | 1.157 |
|
||||
| SYNFlood | 3.176 | 1.007 |
|
||||
| UDPScan | 2.816 | 1.604 |
|
||||
| Benign | 2.648 | 0.519 |
|
||||
| UDPFlood | 2.408 | 0.578 |
|
||||
| ICMPFlood | 1.215 | 0.151 |
|
||||
|
||||
Mean across classes **1.385** at beta=1.0 (monotone: 0.497 / 0.875 / 1.163 / 1.385 for beta
|
||||
0.25/0.50/0.75/1.00), smaller than at alpha=0.50 (2.340). Displacement is a direct-aggregation
|
||||
measure only, not a detection result. Outputs:
|
||||
`artifacts/analysis/5gnidd__alpha0.75__multiseed__prototype_effect_pooled.csv` and per-seed
|
||||
Mean across classes **2.919** at beta=1.0 (monotone: 1.295 / 2.055 / 2.559 / 2.919 for beta
|
||||
0.25/0.50/0.75/1.00), the largest of the three 5G-NIDD alphas (a0.50 = 2.340, a0.75 = 1.385).
|
||||
Displacement is a direct-aggregation measure only, not a detection result. Outputs:
|
||||
`artifacts/analysis/5gnidd__alpha0.25__multiseed__prototype_effect_pooled.csv` and per-seed
|
||||
`...__prototype_effect.csv` / `...__prototype_weights.csv`.
|
||||
|
||||
## 10. Client-level findings (validation, vs beta = 0)
|
||||
|
||||
Detail: `...__client_validation_detail.csv`; summary: `...__client_validation_summary.csv`.
|
||||
|
||||
At the selected `beta=0.50`: **4/6/0** (seed 0, mean Δ +0.0021, range −0.0434…+0.0744),
|
||||
**6/3/1** (seed 1, mean Δ +0.0850, range −0.0172…+0.2667), **4/2/4** (seed 2, mean Δ +0.0058).
|
||||
The response is seed-dependent and `N_ij` vs ΔF1 is non-monotonic. Client ids are per-seed
|
||||
realisations and are not pooled.
|
||||
At the selected `beta=0.00` the delta is trivially 0/0/10 in every seed. At a common
|
||||
`beta=0.75`: **3/4/3** (seed 0, mean Δ −0.023), **2/5/3** (seed 1, mean −0.043), **1/4/5** (seed 2,
|
||||
mean −0.052): at this low-alpha condition support weighting at beta=0.75 **degrades** the
|
||||
rare-class validation metric for most clients, consistent with the pooled selection of beta=0.
|
||||
`N_ij` vs ΔF1 is non-monotonic. Client ids are per-seed realisations and are not pooled.
|
||||
|
||||
## 11. Runtime / resources
|
||||
|
||||
CPU only, no GPU, no swap, exit 0 on all 18 runs. Per-run wall-clock 61:55–66:37; CPU 738–745 %;
|
||||
peak RSS ~8.69 GiB (seed 0), ~11.78 GiB (seed 1), ~11.00 GiB (seed 2). New compute: 18 runs,
|
||||
**69,064 s ≈ 19.18 h** wall-clock, sequential (8 cores). Machine-readable:
|
||||
`artifacts/analysis/5gnidd__alpha0.75__multiseed__runtime.csv`.
|
||||
CPU only, no GPU, no swap, exit 0 on all 18 runs. Per-run wall-clock 62:00–64:48; CPU 739–743 %;
|
||||
peak RSS ~13.2 GiB (seed 0), ~11.9 GiB (seed 1), ~16.1 GiB (seed 2). New compute: 18 runs,
|
||||
**68,798 s ≈ 19.11 h** wall-clock, sequential (8 cores). Machine-readable:
|
||||
`artifacts/analysis/5gnidd__alpha0.25__multiseed__runtime.csv`.
|
||||
|
||||
## 12. Methodological decisions / assumptions
|
||||
## 12. Cross-dataset comparison (descriptive; completed this session)
|
||||
|
||||
Six-condition, validation-only pooled selections:
|
||||
|
||||
| dataset | alpha=0.25 | alpha=0.50 | alpha=0.75 |
|
||||
|---|---|---|---|
|
||||
| 5G-NIDD | **0.00** (0.7615) | **1.00** (0.6924) | **0.50** (0.8091) |
|
||||
| X-IIoTID | **0.00** (0.5917) | **0.75** (0.5794) | **0.50** (0.4767) |
|
||||
|
||||
- Matched-alpha selections coincide at alpha=0.25 (both 0.00) and alpha=0.75 (both 0.50) and
|
||||
differ at alpha=0.50 (1.00 vs 0.75). The selected beta takes values `{0.00, 0.50, 1.00}` across
|
||||
the six conditions — **not universal, not a function of alpha alone, not monotone**.
|
||||
- Realized heterogeneity is close but not identical at matched alpha (X-IIoTID has more missing
|
||||
cells at alpha=0.25/0.50; 5G-NIDD has a higher client-size ratio). Dataset structural
|
||||
differences and **rare-set composition differences are kept explicit**: 5G-NIDD rare classes are
|
||||
low-volume flood/scan types; X-IIoTID rare classes are low-volume behavioural categories. A
|
||||
"rare class" is not a comparable object across datasets.
|
||||
- Mean displacement at beta=1 falls with alpha on both datasets (5G-NIDD 2.919/2.340/1.385;
|
||||
X-IIoTID 3.422/2.099/1.305).
|
||||
- No causality, no universal beta, no cross-dataset superiority, no monotone law. Machine-readable:
|
||||
`artifacts/analysis/cross_dataset__matched_alpha_summary.csv` / `.json` and
|
||||
`cross_dataset__beta_curve.csv`. Report: `docs/research/CROSS_DATASET_BETA_ANALYSIS.md`.
|
||||
|
||||
## 13. Methodological decisions / assumptions
|
||||
|
||||
1. **`S_j` participant/denominator rule** (`S_j = {i | N_ij > 0 and valid C_ij}`, equal weights in
|
||||
the baseline) is a **REPRODUCTION ASSUMPTION**; material in **4/27** class-condition cells at
|
||||
5G-NIDD alpha=0.75. Carry this caveat in all reporting.
|
||||
the baseline) is a **REPRODUCTION ASSUMPTION**; material in **16/27** class-condition cells at
|
||||
5G-NIDD alpha=0.25 (and 19/27 at X-IIoTID alpha=0.25). Carry this caveat in all reporting.
|
||||
2. `N_ij` is class-specific sample support, not prototype quality.
|
||||
3. Beta is one value for all classes/clients; candidates frozen at `{0,.25,.5,.75,1}`.
|
||||
4. **PRIMARY selection = pooled validation across clients and seeds**; per-seed selection is
|
||||
secondary stability diagnostics; test used only after the pooled beta is fixed.
|
||||
5. No new source scripts this session; analysis used the existing read-only
|
||||
`analyze_partition_heterogeneity.py`, `analyze_prototype_effect.py`, `analyze_multiseed_beta.py`,
|
||||
`verify_beta0_equivalence.py`, `analyze_cross_alpha.py`. Ad-hoc Stage-A/audit helpers live only
|
||||
under `/tmp/opencode/s10/` (outside the repo).
|
||||
`verify_beta0_equivalence.py`, `analyze_cross_alpha.py`, plus the ad-hoc Session-11 helpers only
|
||||
under `/tmp/opencode/s11/` (outside the repo).
|
||||
6. `lambda=1.0`, Adam, lr 1e-3, batch 256 remain PROJECT DECISIONS. No `equal_available` mode.
|
||||
|
||||
## 13. Unresolved questions / blockers
|
||||
## 14. Unresolved questions / blockers
|
||||
|
||||
1. `S_j` reproduction ambiguity — unresolved; no official PROTEAN code exists.
|
||||
2. Pooled beta selection remains statistically weak at all measured conditions: across-seed std is
|
||||
comparable to the between-beta spread. No inferential test was performed.
|
||||
2. Pooled beta selection remains statistically weak at all six conditions; only three seeds.
|
||||
3. `beta = 0` is not bitwise identical to `original_protean`; documentation decision still open.
|
||||
4. Session-2 manifests still store `preprocessing_hash = null` / `config_hash = null`.
|
||||
5. 5G-NIDD now has alpha=0.50 (1.00) and alpha=0.75 (0.50); **alpha=0.25 remains unmeasured**, so
|
||||
no 5G-NIDD alpha→beta trend can be stated.
|
||||
6. The selected beta differs across datasets/conditions (5G-NIDD a0.50 = 1.00, a0.75 = 0.50;
|
||||
X-IIoTID a0.75 = 0.50) but no trend or causality is claimed.
|
||||
5. At both alpha=0.25 conditions the selected beta is 0, so those conditions cannot demonstrate a
|
||||
support-weighting benefit.
|
||||
6. No inferential trend test; selected beta differs across conditions but no monotone/causal claim.
|
||||
|
||||
## 14. Git status
|
||||
## 15. Git status
|
||||
|
||||
No commit or push was made. `HEAD = a7f2b0049e99fb66021b4660b82a48d5dbbef072`, branch `main`,
|
||||
in sync with `origin/main`. `git status --short` shows only untracked docs/analysis artifacts (no
|
||||
tracked modification); `results/` is gitignored. `planningawal/` and both raw datasets are
|
||||
unchanged. `equal_available` absent. No test-based selection. The 18 runs are the only
|
||||
`results/5gnidd__alpha0.75__*` directories.
|
||||
No commit or push was made. `HEAD = bbd35085063194db58e4472e70b204d4be4a3607`, branch `main`, in
|
||||
sync with `origin/main`. Only untracked docs/analysis artifacts exist (no tracked modification);
|
||||
`results/` is gitignored. `planningawal/` and both raw datasets are unchanged. `equal_available`
|
||||
absent. No test-based selection. The 18 runs are the only `results/5gnidd__alpha0.25__*`
|
||||
directories.
|
||||
|
||||
## 15. Session 10 audit result
|
||||
## 16. Session 11 audit result
|
||||
|
||||
- Verdict: **PASS** (`docs/research/SESSION10_AUDIT.md`).
|
||||
- Verdict: **PASS** (`docs/research/SESSION11_AUDIT.md`).
|
||||
- Run inventory 18/18, pairing hash-identical, manifests reproduced from raw data.
|
||||
- Validation recomputed exactly; **pooled `beta_star(5gnidd, alpha=0.75) = 0.50`** (validation
|
||||
- Validation recomputed exactly; **pooled `beta_star(5gnidd, alpha=0.25) = 0.00`** (validation
|
||||
only). Selected-beta test and `original_protean` test means/stds exact.
|
||||
- Leakage audit PASS; `test_used_for_selection=false` upheld.
|
||||
- Rare/unseen integrity PASS (0 mismatches / 1,620 cells). `beta=0` equivalence class **B**.
|
||||
- Direct-effect, client-level, cross-condition labels: PASS.
|
||||
- Direct-effect, client-level, cross-alpha, cross-dataset labels: PASS.
|
||||
- Source/data integrity PASS. pytest: **75 passed, 0 failed, 0 skipped**.
|
||||
|
||||
## 16. Files created / modified this session
|
||||
## 17. Files created / modified this session
|
||||
|
||||
Created:
|
||||
|
||||
- `docs/research/5GNIDD_ALPHA075_BETA_VALIDATION.md`
|
||||
- `docs/research/SESSION10_AUDIT.md`
|
||||
- `artifacts/analysis/5gnidd__alpha0.75__*` — 18 prototype snapshots + 6 per-seed
|
||||
effect/weights CSVs + `5gnidd__alpha0.75__multiseed__*` (validation per-seed/pooled,
|
||||
- `docs/research/5GNIDD_ALPHA025_BETA_VALIDATION.md`
|
||||
- `docs/research/5GNIDD_CROSS_ALPHA_ANALYSIS.md`
|
||||
- `docs/research/CROSS_DATASET_BETA_ANALYSIS.md`
|
||||
- `docs/research/SESSION11_AUDIT.md`
|
||||
- `artifacts/analysis/5gnidd__alpha0.25__*` — 18 prototype snapshots + 3 per-seed
|
||||
effect/weights CSVs + `5gnidd__alpha0.25__multiseed__*` (validation per-seed/pooled,
|
||||
client detail/summary/deltas, test descriptive, prototype effect pooled, beta0 equivalence,
|
||||
runtime, selection summary, audit recomputation) + `5gnidd__alpha0.75__partition_heterogeneity_*`
|
||||
+ `5gnidd__alpha0.75__partition_characterization.csv` + `5gnidd__cross_alpha__*`
|
||||
- `results/5gnidd__alpha0.75__seed{0,1,2}__*` (18 run dirs, gitignored)
|
||||
runtime, selection summary, audit recomputation) + `5gnidd__alpha0.25__partition_characterization.csv`
|
||||
- `artifacts/analysis/5gnidd__cross_alpha__*` (regenerated to include alpha=0.25)
|
||||
- `artifacts/analysis/cross_dataset__matched_alpha_summary.{csv,json}`,
|
||||
`artifacts/analysis/cross_dataset__beta_curve.csv`
|
||||
- `results/5gnidd__alpha0.25__seed{0,1,2}__*` (18 run dirs, gitignored)
|
||||
|
||||
Modified:
|
||||
|
||||
- `docs/handoff/SESSION_HANDOFF.md` (this file)
|
||||
- `artifacts/analysis/5gnidd__cross_alpha__*` (extended from 2 to 3 alphas; previous 2-alpha
|
||||
content was superseded, not silently lost — it is reproducible from the frozen manifests)
|
||||
|
||||
No source or test code was changed.
|
||||
|
||||
## 17. Exact recommended next phase
|
||||
## 18. Exact recommended next phase
|
||||
|
||||
1. **Session 10 is audited and safe to checkpoint** (verdict PASS; see §15 and
|
||||
`docs/research/SESSION10_AUDIT.md`). Before checkpointing, stage only the intended
|
||||
docs/analysis artifacts; do not commit `results/` or the raw datasets.
|
||||
2. **5G-NIDD alpha=0.50 and alpha=0.75 are complete.** The final 5G-NIDD condition, **alpha=0.25**,
|
||||
is **safe to run** — its manifest already exists and reproduces from raw data, the pipeline is
|
||||
stable, and the same frozen protocol applies. Budget **≈19 h** (18 runs × ~64 min) and run
|
||||
**one condition at a time**; do not launch the grid blindly.
|
||||
3. Follow the same pooled validation-only selection rule; report each selected beta as "best among
|
||||
the tested candidates under pooled validation for that condition" — never a global optimum.
|
||||
4. After alpha=0.25, build the 5G-NIDD cross-alpha comparison (alpha=0.25/0.50/0.75) analogous to
|
||||
`XIIOTID_CROSS_ALPHA_ANALYSIS.md`, and only then consider cross-dataset discussion.
|
||||
5. Do not infer a monotone alpha→beta relation; do not claim dataset causality.
|
||||
6. Update `EXPERIMENT_PROTOCOL.md` only if a real ambiguity appears; update this handoff.
|
||||
7. Do not commit or push unless asked.
|
||||
1. **Session 11 is audited and safe to checkpoint** (verdict PASS). Before checkpointing, stage
|
||||
only the intended docs/analysis artifacts; do not commit `results/` or the raw datasets.
|
||||
2. **The planned experiment grid is COMPLETE** (two datasets × three alphas × three seeds × five
|
||||
betas + baseline = 108 runs). Do **not** run any new condition.
|
||||
3. Remaining work is analysis/writing, not new training:
|
||||
- optionally add an inferential comparison (e.g. paired sign/permutation test over seeds) to
|
||||
formalise the weak selection effects — clearly labelled as post-hoc if added;
|
||||
- freeze the final thesis tables/report from the six-condition artifacts;
|
||||
- decide and document the `beta=0` vs `original_protean` reporting convention.
|
||||
4. Do not infer a monotone alpha→beta law; do not claim dataset causality or superiority.
|
||||
5. Update `EXPERIMENT_PROTOCOL.md` only if a real ambiguity appears; update this handoff.
|
||||
6. Do not commit or push unless asked.
|
||||
|
||||
## 18. Important commands
|
||||
## 19. Important commands
|
||||
|
||||
```bash
|
||||
# Tests (75 expected)
|
||||
.venv/bin/python -m pytest
|
||||
|
||||
# One full paired beta run (CPU, ~62-67 min, ~8.7-11.8 GiB for 5G-NIDD)
|
||||
.venv/bin/python scripts/run_baseline.py --dataset 5gnidd --alpha 0.75 --seed 1 \
|
||||
# One full paired beta run (CPU, ~62-65 min, ~12-17 GiB for 5G-NIDD)
|
||||
.venv/bin/python scripts/run_baseline.py --dataset 5gnidd --alpha 0.25 --seed 1 \
|
||||
--clients 10 --rounds 10 --local-epochs 3 \
|
||||
--aggregation-mode support_weighted --beta 1.0
|
||||
|
||||
# Pooled multi-seed validation analysis (read-only, no retraining)
|
||||
.venv/bin/python scripts/analyze_multiseed_beta.py --dataset 5gnidd --alpha 0.75 --seeds 0 1 2
|
||||
.venv/bin/python scripts/analyze_multiseed_beta.py --dataset 5gnidd --alpha 0.25 --seeds 0 1 2
|
||||
|
||||
# Direct prototype-effect analysis (no retraining)
|
||||
.venv/bin/python scripts/analyze_prototype_effect.py \
|
||||
--snapshot artifacts/analysis/5gnidd__alpha0.75__seed0__support_weighted__beta0__prototype_snapshot.json \
|
||||
--snapshot artifacts/analysis/5gnidd__alpha0.25__seed0__support_weighted__beta0__prototype_snapshot.json \
|
||||
--class-names "Benign,HTTPFlood,ICMPFlood,SYNFlood,SYNScan,SlowrateDoS,TCPConnectScan,UDPFlood,UDPScan" \
|
||||
--output-dir artifacts/analysis
|
||||
|
||||
# Realized partition characterization (cross-alpha)
|
||||
.venv/bin/python scripts/analyze_partition_heterogeneity.py --dataset 5gnidd --alphas 0.5 0.75
|
||||
.venv/bin/python scripts/analyze_partition_heterogeneity.py --dataset 5gnidd --alphas 0.25 0.5 0.75
|
||||
|
||||
# beta=0 equivalence audit
|
||||
.venv/bin/python scripts/verify_beta0_equivalence.py --dataset 5gnidd --alpha 0.75 --seeds 0 1 2
|
||||
.venv/bin/python scripts/verify_beta0_equivalence.py --dataset 5gnidd --alpha 0.25 --seeds 0 1 2
|
||||
|
||||
# Cross-alpha comparison table (5G-NIDD)
|
||||
.venv/bin/python scripts/analyze_cross_alpha.py --dataset 5gnidd --alphas 0.5 0.75 --tag 5gnidd__cross_alpha
|
||||
.venv/bin/python scripts/analyze_cross_alpha.py --dataset 5gnidd --alphas 0.25 0.5 0.75
|
||||
```
|
||||
|
||||
## 19. Recommended next-session prompt
|
||||
## 20. Recommended next-session prompt
|
||||
|
||||
```
|
||||
You are continuing the perdalam-protean research repository (Session 11).
|
||||
You are continuing the perdalam-protean research repository (Session 12).
|
||||
|
||||
FIRST, read completely:
|
||||
1. docs/handoff/SESSION_HANDOFF.md
|
||||
2. docs/research/5GNIDD_ALPHA075_BETA_VALIDATION.md
|
||||
3. docs/research/SESSION10_AUDIT.md
|
||||
4. docs/research/5GNIDD_ALPHA050_BETA_VALIDATION.md
|
||||
5. docs/research/EXPERIMENT_PROTOCOL.md
|
||||
2. docs/research/SESSION11_AUDIT.md
|
||||
3. docs/research/5GNIDD_ALPHA025_BETA_VALIDATION.md
|
||||
4. docs/research/5GNIDD_CROSS_ALPHA_ANALYSIS.md
|
||||
5. docs/research/CROSS_DATASET_BETA_ANALYSIS.md
|
||||
6. docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md
|
||||
7. docs/research/EXPERIMENT_PROTOCOL.md
|
||||
|
||||
Critical rules:
|
||||
- planningawal/ is immutable.
|
||||
@@ -380,18 +411,15 @@ Critical rules:
|
||||
- The S_j participant rule is a REPRODUCTION ASSUMPTION.
|
||||
- Do NOT add an executable equal_available mode.
|
||||
- Beta selection uses validation only; PRIMARY is pooled across clients and seeds.
|
||||
- Run one condition at a time (~19 h per 5G-NIDD condition).
|
||||
- The planned 2-dataset x 3-alpha experiment grid is COMPLETE. Do NOT run new conditions.
|
||||
|
||||
Context: Session 10 completed 5G-NIDD alpha=0.75 for seeds {0,1,2} (18 runs, ~19.18 h) and was
|
||||
independently audited (PASS; docs/research/SESSION10_AUDIT.md). Pooled validation selected
|
||||
beta=0.50 (per-seed diagnostics 0.50 / 1.00 / 1.00). beta=0 equivalence class B. Realized
|
||||
heterogeneity: 4/27 cells |S_j|<M, 5 missing cells, client ratio mean 6.31, mean displacement
|
||||
1.385, rare-class union identical across seeds. 5G-NIDD alpha=0.50 selected 1.00; X-IIoTID
|
||||
alpha=0.75 selected 0.50.
|
||||
Context: Session 11 ran the final condition 5G-NIDD alpha=0.25 (18 runs, ~19.11 h), audited PASS,
|
||||
and selected beta_star=0.00 (validation only; per-seed 0.25/0.00/0.00). The six-condition pooled
|
||||
selections are 5G-NIDD 0.25/0.50/0.75 = 0.00/1.00/0.50 and X-IIoTID = 0.00/0.75/0.50; no universal
|
||||
or monotone beta. beta=0 equivalence class B. pytest 75 passed.
|
||||
|
||||
Task: run the final 5G-NIDD condition alpha=0.25 under the same frozen protocol (seeds {0,1,2},
|
||||
five betas + original_protean, frozen manifests, pooled validation-only selection). Verify the
|
||||
manifest reproduces, run the 18 runs sequentially with runtime/status files, produce a condition
|
||||
report + audit, and extend the 5G-NIDD cross-alpha comparison to alpha=0.25/0.50/0.75. Use
|
||||
.venv/bin/python. Do not commit or push unless asked.
|
||||
Task: no new training. Consolidate the final thesis-ready six-condition tables/report from the
|
||||
existing audited artifacts, optionally add a clearly post-hoc inferential comparison of the weak
|
||||
pooled selections, and keep all limitations (S_j assumption, three seeds, no global optimum)
|
||||
explicit. Use .venv/bin/python. Do not commit or push unless asked.
|
||||
```
|
||||
Reference in new issue
Block a user