From 8a9bdcf63c2e29ef42142aebaa70c6c0d90b5277 Mon Sep 17 00:00:00 2001 From: willtanoe Date: Fri, 9 Oct 2026 00:20:24 +0700 Subject: [PATCH] docs: document completed PROTEAN experiments and findings --- README.md | 806 ++++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 632 insertions(+), 174 deletions(-) diff --git a/README.md b/README.md index 14e25ea..0c33191 100644 --- a/README.md +++ b/README.md @@ -1,227 +1,685 @@ # PROTEAN Prototype Aggregation Study -Research code and audited artifacts for a master's thesis on **server-side prototype aggregation in -PROTEAN** (Chennoufi et al., 2025), a federated intrusion-detection framework, under non-IID client -data. The study investigates a **beta-controlled, class-specific sample-support weighting** of local -class prototypes for **rare attack detection**. +Research code and audited artifacts for a master's thesis on **server-side prototype aggregation +in PROTEAN** (Chennoufi et al., 2025), a federated intrusion-detection framework, under non-IID +client data. The study evaluates a **beta-controlled, class-specific sample-support weighting** of +local class prototypes for **rare attack detection**. -The experimental study is **complete** (108 full runs, six non-IID conditions, both datasets) and the -thesis proposal Markdown is **finalized**. No further training is planned unless new academic -feedback explicitly requires it. +**Research status.** The experimental grid is **complete** (108 full federated runs), the results +are synthesized and audited, and the thesis proposal Markdown is finalized. No further training is +planned unless new academic feedback explicitly requires it. The repository is in a documentation +and thesis-writing stage. -## Final Research Title +**Final title.** *Optimasi Agregasi Prototype pada PROTEAN melalui Pembobotan Berbasis Dukungan +Sampel Kelas untuk Deteksi Serangan Langka pada Data Non-IID.* Here "Optimasi" means empirical +selection of `beta` among the predefined candidate values, not a global mathematical optimum. -**Optimasi Agregasi Prototype pada PROTEAN melalui Pembobotan Berbasis Dukungan Sampel Kelas untuk -Deteksi Serangan Langka pada Data Non-IID** +--- -*Optimizing Prototype Aggregation in PROTEAN through Class-Sample-Support Weighting for Rare Attack -Detection under Non-IID Data Distributions.* +## 1. Research Overview -Here **"Optimasi" means empirical tuning/selection of `beta` among the predefined candidate values**, -not a global mathematical optimum and not a universal beta. The selected value is the best among the -tested candidates under the defined validation procedure for each condition. +PROTEAN is a federated intrusion-detection method that shares **class prototypes** between clients +and the server in addition to model parameters. Each client `i` builds a local prototype `C_ij` for +class `j` from its own training samples; the server aggregates these into a global prototype +`Cbar_j` that is used both for local prototype alignment during training and as the classifier at +inference (nearest prototype in embedding space). -## Research Scope +This thesis modifies **only the server-side prototype aggregation**. The original PROTEAN +aggregation gives every contributing client equal weight. The proposed aggregation weights each +client's prototype by `N_ij^beta`, where `N_ij` is that client's class-specific training support and +`beta` is a single exponent in `[0, 1]`. Everything else — local prototype formation, model +aggregation, architecture, loss, and inference rule — is unchanged relative to the reproduced +baseline. -- **Base method:** PROTEAN, a federated intrusion-detection framework for non-IID data. -- **Focus:** the **server-side prototype aggregation** step only. Client training, model - architecture, loss, and inference are unchanged. -- **Proposed mechanism:** weight each contributing client's class prototype by `N_ij^beta`, where - `N_ij` is that client's class-specific training support and `beta` is a single tunable exponent - in `[0, 1]`. -- **Analysis chain:** support imbalance -> contribution weight -> prototype displacement -> - rare-class detection -> per-client effect. +`N_ij` is **class-specific sample support, not prototype quality**. A client with few samples is +not assumed to have a worse prototype. -`N_ij` is **class-specific sample support, not prototype quality**. A client with few samples is not -assumed to have a worse prototype; low-support clients may carry variation unavailable at -high-support clients. +## 2. Problem Statement and Motivation -## Why This Problem Matters +Federated IDS data is non-IID: clients observe different traffic mixtures and different attack +types, so class support varies strongly across clients. Rare attack classes appear in little of +the data, so aggregate metrics can look acceptable while the model fails on exactly the classes +that matter for detection. -Federated IDS data is naturally non-IID: each client observes different traffic and different attack -types. Rare attack classes appear in little of the data, so aggregate metrics can look good while -the model fails on them. PROTEAN builds a global prototype per class on the server and uses it both -for local prototype alignment and for nearest-prototype inference. Its reference aggregation gives -every contributing client equal weight, even though the local prototypes may be built from very -different numbers of samples. This study asks whether, and under which realized non-IID conditions, -weighting contributions by class sample support changes the global prototype and rare-class -detection behavior. +Under the original equal-weight prototype aggregation, a prototype built from a handful of samples +contributes as much as one built from thousands. This thesis investigates whether — and under which +realized non-IID conditions — weighting contributions by class-specific sample support changes the +global prototype and rare-class detection behavior. -## Proposed Aggregation +The study deliberately isolates one mechanism so that any observed change can be attributed to the +aggregation weighting rather than to a confounded set of architectural or training changes. + +## 3. Original PROTEAN versus Proposed Modification + +**A. Local prototype (unchanged).** For class `j` with `N_ij > 0` samples on client `i`, the local +prototype is the mean embedding of that client's class-`j` samples: + +$$ +C_{ij} = \frac{1}{N_{ij}} \sum_{k=1}^{N_{ij}} z_{ijk} +$$ + +where `z_ijk` is the embedding of the k-th sample of class `j` on client `i`. + +**B. Original server-side prototype aggregation (reference).** PROTEAN Eq. (2) averages the `M` +contributing clients equally (equal contribution, no `N_ij` weight): + +$$ +\bar{C}_j = \frac{1}{M} \sum_{i=1}^{M} C_{ij} +$$ + +**C. Proposed beta-controlled aggregation.** Only the prototype aggregation is modified: + +$$ +\bar{C}_j(\beta) = \frac{\sum_{i \in S_j} N_{ij}^{\beta}\, C_{ij}}{\sum_{i \in S_j} N_{ij}^{\beta}}, \qquad \beta \in \{0,\ 0.25,\ 0.50,\ 0.75,\ 1.00\} +$$ + +**D. Normalized contribution weight.** Client `i`'s normalized weight for class `j` is + +$$ +w_{ij}(\beta) = \frac{N_{ij}^{\beta}}{\sum_{k \in S_j} N_{kj}^{\beta}} +$$ + +**E. Symbol meanings.** + +| Symbol | Meaning | +|---|---| +| `N_ij` | number of local **training** samples of class `j` on client `i` (class-specific sample support, not prototype quality) | +| `C_ij` | client `i`'s local class-`j` prototype (mean embedding) | +| `S_j` | participant set: clients with `N_ij > 0` and a valid local prototype for class `j` | +| `M` | number of participants in the reference Eq. (2) (equal average) | +| `beta` | scalar weighting exponent in `[0, 1]`, one value for all classes and clients | +| `w_ij` | normalized contribution weight (sums to 1 over `S_j`) | +| global prototype | `Cbar_j(beta)`; used for alignment and nearest-prototype inference | + +**F. Exact difference between original and modified PROTEAN.** + +- With `beta = 0`, `N_ij^0 = 1`, so the proposed aggregator becomes the **equal mean over `S_j`**. +- With `beta = 1`, the contribution is proportional to class support (equivalent to the mean of + all included local class embeddings when the assumptions hold). +- `beta = 0` is equal weighting over `S_j`; it is equivalent to the original formulation **only + if** the participant and missing-class rules are also equivalent. That equivalence is not + verified against official PROTEAN code. +- Model aggregation, local prototype formation, architecture, loss, and inference are **not** + modified. + +**Novelty boundary.** Sample-count prototype weighting is not new by itself; related work such as +**FedProto** (Tan et al., 2022) already involves class sample counts in prototype-based federated +learning. The contribution here is the **controlled beta parameterization inside PROTEAN's +server-side prototype aggregation**, its condition-wise validation-based selection, the paired +multi-seed multi-condition evaluation, and the resulting behavioral findings — not the invention of +support weighting. + +**Analysis chain (not a causal proof by itself).** ``` -Cbar_j(beta) = ( sum_{i in S_j} N_ij^beta * C_ij ) / ( sum_{i in S_j} N_ij^beta ) -S_j = { i | N_ij > 0 and a valid local prototype C_ij exists } -beta in {0.00, 0.25, 0.50, 0.75, 1.00} +support imbalance -> contribution weight -> global-prototype displacement + -> rare-class detection -> per-client effect ``` -- `beta = 0`: equal weighting over the contributors `S_j`. -- `beta = 1`: contribution proportional to class-specific support. -- `0 < beta < 1`: intermediate weighting. +## 4. Research Questions and Objectives -`S_j` (participant set and denominator rule) is a **documented reproduction assumption**: no verified -official PROTEAN code is available. `beta = 0` is semantically the equal mean over `S_j`, but is not -guaranteed to be bitwise identical to the frozen `original_protean` baseline because of -floating-point operation ordering. +| ID | Research question | +|---|---| +| RQ1 | How are class-support imbalance, differences between local prototypes, and rare-class detection performance related in PROTEAN? | +| RQ2 | How does changing `beta` affect the global prototype and rare-class detection performance under different non-IID conditions? | +| RQ3 | Does the validation-selected `beta` change across non-IID levels, and what is its effect on overall and per-client performance? | -## Experimental Setup +Objectives: (1) relate support imbalance to prototype behavior and rare-class detection (RQ1); +(2) characterize the effect of `beta` on the global prototype and rare-class detection across +conditions (RQ2); (3) select `beta` by validation at different imbalance levels and quantify +overall and per-client consequences (RQ3). + +## 5. Experimental Design | Item | Value | |---|---| +| Base method | PROTEAN federated intrusion detection | +| Modified component | server-side prototype aggregation only | +| Mechanism | `w_ij = N_ij^beta`, one `beta` for all classes/clients | | Datasets | X-IIoTID (`class2`, benign `Normal`, K=9); 5G-NIDD (`Attack Type`, benign `Benign`, K=9) | -| Non-IID conditions | symmetric Dirichlet `alpha` = 0.75, 0.50, 0.25 | +| Clients | 10 | +| Non-IID conditions | symmetric Dirichlet `alpha` = 0.25, 0.50, 0.75 | | Seeds | 0, 1, 2 (project-defined) | -| Clients / rounds / local epochs | 10 / 10 / 3 | -| Batch size / optimizer / learning rate | 256 / Adam / 1e-3 | -| FedProx `mu` / prototype-alignment `lambda` | 0.1 / 1.0 | -| Compared configurations | `original_protean` + `support_weighted` at beta 0, 0.25, 0.50, 0.75, 1.00 | +| Beta candidates | 0, 0.25, 0.50, 0.75, 1.00 | +| Baseline | `original_protean` (equal-weight prototype aggregation) | +| Rounds / local epochs / batch | 10 / 3 / 256 | +| Optimizer / lr / mu / lambda | Adam / 1e-3 / 0.1 / 1.0 | | Splits | stratified train / validation / test = 70 / 10 / 20 | -| Selection | pooled validation per-client-mean rare-class macro-F1 across seeds | -| Test | read only after the selected beta is fixed | +| Selection metric | pooled **validation** per-client-mean rare-class macro-F1 across seeds | +| Test usage | read only after the selected `beta` is fixed | -Each `(dataset, alpha, seed)` uses one deterministic frozen partition; all compared configurations -within it share the same manifest, split, class mapping, and initialization. +Expected experimental matrix: `2 datasets × 3 alpha × 3 seeds × (1 baseline + 5 betas) = 108 runs`. +Within each `(dataset, alpha, seed)` all six configurations share one manifest, split, class mapping, +and round-0 initialization; only the aggregation mode changes. -## Experiment Status +## 6. Completed Research Phases -- **Complete: 108 full runs** = 2 datasets × 3 alpha conditions × 3 seeds × - (1 original PROTEAN + 5 beta settings). -- All conditions are trained, audited, and synthesized. -- Smoke runs are excluded and are not scientific experiments. -- **No further training planned** unless new academic feedback requires it. +Documented phases, in order. Session numbers are used only where the repository documents them. -## Final Scientific Position +| Phase / session | Objective | Key outputs | Validation | +|---|---|---|---| +| Planning and architecture | freeze scope, protocol, and project structure | `AGENTS.md`, `planningawal/PROTEAN_Revisi_Lurus.md`, `docs/research/IMPLEMENTATION_PLAN.md` | checkpoint commit | +| Baseline reproduction | implement reproducible PROTEAN baseline pipeline | `docs/baseline/BASELINE_IMPLEMENTATION.md`, `src/protean_research/` | `docs/baseline/BASELINE_VALIDATION.md` | +| Baseline validation + regression tests | verify baseline behavior and invariants | `tests/`, `docs/baseline/PROTEAN_BASELINE_INVESTIGATION.md` | unit/integration tests | +| Support-weighted aggregation | implement beta-controlled aggregator | `docs/research/BETA_AGGREGATION_IMPLEMENTATION.md`, `src/protean_research/aggregation/` | aggregation invariance tests | +| Session 5 — one paired condition | first full paired beta run (single seed) | `docs/research/PAIRED_BETA_VALIDATION.md` | `val_selection.json` | +| Session 6 — multi-seed (X-IIoTID a0.50) | pooled validation across seeds `{0,1,2}` | `docs/research/MULTISEED_BETA_VALIDATION.md`, `SESSION6_AUDIT.md` | audit PASS with minor corrections | +| Session 7 — X-IIoTID a0.25 | third X-IIoTID alpha condition | `docs/research/ALPHA025_BETA_VALIDATION.md`, `SESSION7_AUDIT.md` | audit PASS with minor corrections | +| Session 8 — X-IIoTID a0.75 | complete X-IIoTID three-alpha study | `docs/research/ALPHA075_BETA_VALIDATION.md`, `SESSION8_AUDIT.md` | audit PASS | +| Session 9 — 5G-NIDD a0.50 | first 5G-NIDD condition | `docs/research/5GNIDD_ALPHA050_BETA_VALIDATION.md`, `SESSION9_AUDIT.md` | audit PASS | +| Session 10 — 5G-NIDD a0.75 | second 5G-NIDD condition | `docs/research/5GNIDD_ALPHA075_BETA_VALIDATION.md`, `SESSION10_AUDIT.md` | audit PASS | +| Session 11 — 5G-NIDD a0.25 | final condition; completes the grid | `docs/research/5GNIDD_ALPHA025_BETA_VALIDATION.md`, `SESSION11_AUDIT.md` | audit PASS | +| Session 12 — synthesis | consolidate all six conditions | `docs/research/FINAL_EXPERIMENTAL_SYNTHESIS.md`, `CROSS_DATASET_BETA_ANALYSIS.md`, `docs/thesis/*`, `artifacts/analysis/final__*`, `artifacts/figures/final/*` | independent recomputation, zero mismatches | +| Proposal phase | write the thesis proposal and audit aids | `docs/proposal/*` | committed `e27c5a8` | +| Handoff / README | keep repository state current | `docs/handoff/SESSION_HANDOFF.md`, `README.md` | this document | -High-level, evidence-safe conclusions only (full tables live in the synthesis documents): +## 7. Experiment Coverage and Reproducibility -- Realized support imbalance, missing client-class cells, and client-size ratio vary by condition; - Dirichlet `alpha` is a design knob, not the realized heterogeneity itself. -- Increasing `beta` changes the normalized contribution weights and therefore the position of each - global prototype; for a fixed snapshot, displacement grows with `beta`. -- The rare-class detection response is **condition-dependent and non-monotone** in `beta`; the direct - aggregation effect and the end-to-end training effect are distinct. -- **No universal beta exists.** The validation-selected beta differs across datasets and non-IID - conditions and is not monotone in `alpha`. -- Stronger support weighting can **help, have limited effect, or hurt** rare-class behavior, - depending on the condition, and can redistribute benefit across clients within one condition. +**Verified run inventory.** Counted from `results/` metadata, not from filenames alone: -Not supported by the evidence (must not be claimed): support weighting always improves rare-class -detection; the selected beta universally beats `original_protean`; `alpha` causally determines beta; -`N_ij` measures prototype quality; larger prototype displacement means better detection; a global -optimum exists; results generalize beyond the two evaluated datasets. +- non-smoke run directories: **108** +- distinct `(dataset, alpha, seed)` cells: **18** (2 × 3 × 3) +- configurations per cell: **1 `original_protean` + 5 `support_weighted`** (beta 0/0.25/0.50/0.75/1.00) +- paired comparison: within every cell, all six runs share **one manifest content hash** (verified); + `original_protean` and `support_weighted beta=0` additionally share a byte-identical round-0 snapshot +- failed runs: **none**; one unrelated `smoke` directory is excluded -## Repository Structure +**Known artifact gap.** One run, `xiiotid__alpha0.5__seed0__original_protean`, lacks its entire +validation-side artifact set (`val_global_metrics.json`, `val_confusion_matrix.json`, +`val_per_class.csv`, `val_per_client.csv`, `val_selection.json`). Its test artifacts and history are +present, and it is not a selection run, so all 108 test evaluations and all six selections are +unaffected. Every other run has the complete test and validation artifact set. This is recorded as +an evidence gap rather than silently omitted. + +**Reproducibility controls.** + +- **Frozen protocol:** `docs/research/EXPERIMENT_PROTOCOL.md` (machine-readable + `configs/experiments/experiment_protocol.yaml`). Values are either FROZEN from the paper or + explicit PROJECT DECISIONs. +- **Frozen manifests:** 18 deterministic Dirichlet partition manifests under `artifacts/partitions/`; + generated once and reused. Regenerating a fresh split for a run is a protocol violation. +- **Raw data:** external and read-only (`~/datasets/…`); never modified, copied, or committed. + SHA-256 hashes are recorded in the audit documents. +- **Validation/test separation:** `beta` is selected from validation only; every run records + `test_used_for_selection = false`. +- **Seeds** control split, Dirichlet partition, model initialization, and batch order. + +## 8. Evaluation Metrics + +All metrics below are implemented in `src/protean_research/evaluation/metrics.py` and persisted per +run. No scikit-learn `classification_report` is used; metrics are pure NumPy. + +| Metric | Definition | Level | Split | Where stored | +|---|---|---|---|---| +| Accuracy | `trace(CM) / sum(CM)` | global | val, test | `{split}_global_metrics.json` | +| Macro-F1 | unweighted mean of per-class F1 over classes with support > 0 | global | val, test | `{split}_global_metrics.json` | +| Weighted-F1 | support-weighted mean of per-class F1 | global | val, test | `{split}_global_metrics.json` | +| Per-class precision/recall/F1/support | one-vs-rest from the confusion matrix | per class (global) | val, test | `{split}_per_class.csv` | +| Rare-class macro-F1 | mean per-class F1 over the **evaluated rare classes** (union of client rare classes with support) | global | val, test | `{split}_global_metrics.json` | +| Rare-class recall | mean recall over the evaluated rare classes | global | val, test | `{split}_global_metrics.json` | +| Rare support | number of eval samples in evaluated rare classes | global | val, test | `{split}_global_metrics.json` | +| Per-client accuracy/macro-F1 | standard metrics on each client's own eval partition | per client | val, test | `per_client.csv`, `val_per_client.csv` | +| Per-client rare-class macro-F1 / recall | rare metrics using that client's own rare class ids | per client | val, test | `per_client.csv`, `val_per_client.csv` | +| Selection score | mean over eligible clients of per-client validation rare-class macro-F1, pooled across seeds | per condition | validation | `val_selection.json`, aggregation artifacts | +| Prototype displacement | `‖Cbar_j(beta) − Cbar_j(0)‖` and relative displacement, fixed round-0 snapshot | per class | n/a | `*__prototype_effect*.csv` | +| Normalized weight | `N_ij^beta / Σ_k N_kj^beta` | per client-class | n/a | `*__prototype_weights.csv` | +| Support CV / missing cells / client-size ratio | realized non-IID statistics from `N_ij` | per class / partition | training | `characterization.csv`, `*partition_heterogeneity*` | + +**Averaging-level distinctions that must not be interchanged.** + +- **Global rare-class macro-F1** pools all rare-class evaluation samples; **per-client mean + rare-class macro-F1** averages each client's own score. The **selection metric is the per-client + mean, pooled across seeds**. +- **Validation** metrics choose `beta`; **test** metrics are reported only after selection. +- **Per-seed** values and **pooled** values are different quantities; the pooled score averages + per-seed per-client-mean scores. + +**Not computed by this repository:** micro-F1, ROC/AUC, statistical significance tests, and +persisted per-class unseen-class recall (unseen-class evaluation is defined in code but only the +unseen flags and union counts are saved). + +## 9. Quantitative Experimental Results + +All test values are means over seeds `{0,1,2}` for the selected `beta`, and the corresponding +`original_protean` test means. Values are from `artifacts/analysis/final__six_condition_master.csv` +and `final__selected_vs_original.csv`, and were independently re-checked against the per-run +`test_global_metrics.json`. + +### 9.1 Pooled validation selection (validation-only) + +Pooled validation per-client-mean rare-class macro-F1 (selected in bold): + +| beta | X-IIoTID a0.25 | X-IIoTID a0.50 | X-IIoTID a0.75 | 5G-NIDD a0.25 | 5G-NIDD a0.50 | 5G-NIDD a0.75 | +|---|---|---|---|---|---|---| +| 0.00 | **0.5917** | 0.5444 | 0.4717 | **0.7615** | 0.6872 | 0.7781 | +| 0.25 | 0.5887 | 0.5554 | 0.4744 | 0.7493 | 0.6451 | 0.7905 | +| 0.50 | 0.5859 | 0.5673 | **0.4767** | 0.7257 | 0.6747 | **0.8091** | +| 0.75 | 0.5795 | **0.5794** | 0.4725 | 0.7222 | 0.6884 | 0.7915 | +| 1.00 | 0.5321 | 0.5735 | 0.4686 | 0.7376 | **0.6924** | 0.8043 | + +### 9.2 Selected beta versus original PROTEAN (test) + +Test rare-class macro-F1 (mean over 3 seeds), selected `beta` vs `original_protean`: + +| Dataset | alpha | selected beta | test rare-F1 (selected) | test rare-F1 (original) | delta | test macro-F1 (selected) | test accuracy (selected) | +|---|---|---|---|---|---|---|---| +| X-IIoTID | 0.25 | 0.00 | 0.6321 | 0.6328 | −0.0007 | 0.6813 | 0.8740 | +| X-IIoTID | 0.50 | 0.75 | 0.6655 | 0.6425 | **+0.0230** | 0.7312 | 0.9064 | +| X-IIoTID | 0.75 | 0.50 | 0.6052 | 0.6039 | +0.0014 | 0.7576 | 0.9281 | +| 5G-NIDD | 0.25 | 0.00 | 0.8412 | 0.8423 | −0.0010 | 0.8166 | 0.7302 | +| 5G-NIDD | 0.50 | 1.00 | 0.8720 | 0.8805 | −0.0086 | 0.8386 | 0.7433 | +| 5G-NIDD | 0.75 | 0.50 | 0.8967 | 0.8799 | +0.0168 | 0.8531 | 0.7523 | + +Reading of the table: + +- The six conditions split into **three mutually exclusive categories that total six**: + **three positive deltas** (X-IIoTID a0.50, X-IIoTID a0.75, 5G-NIDD a0.75), **one negative delta** + (5G-NIDD a0.50), and **two effectively unchanged** (both `alpha = 0.25` conditions, where the + selected `beta` is 0, so the selected run is the equal-weight run and the difference is + floating-point divergence only). +- The **largest positive delta (+0.0230, X-IIoTID a0.50) is still smaller than that condition's + between-seed sample std (0.0267)**. No statistical superiority is claimed. +- **No single beta dominates** across conditions, and the selection depends on both dataset and + condition. + +Cross-dataset metric levels are not comparable: 5G-NIDD rare classes are low-volume flood/scan +types, X-IIoTID rare classes are low-volume behavioural categories, and their rare support differs. + +### 9.3 Selected betas + +| Dataset | alpha=0.25 | alpha=0.50 | alpha=0.75 | +|---|---|---|---| +| X-IIoTID | 0.00 | 0.75 | 0.50 | +| 5G-NIDD | 0.00 | 1.00 | 0.50 | + +## 10. Beta Selection by Dataset and Alpha + +Selection is by pooled validation per-client-mean rare-class macro-F1 (validation only); the test +set is never used. `artifacts/analysis/final__selection_strength.csv` quantifies separation: + +| Dataset | alpha | selected beta | best | 2nd best | abs gap | top-2 gap / selected std | per-seed diagnostic betas | +|---|---|---|---|---|---|---|---| +| X-IIoTID | 0.25 | 0.00 | 0.5917 | 0.5887 (0.25) | 0.0030 | 0.034 | 0.75 / 0.50 / 0.00 | +| X-IIoTID | 0.50 | 0.75 | 0.5794 | 0.5735 (1.00) | 0.0059 | 0.078 | 0.50 / 1.00 / 0.75 | +| X-IIoTID | 0.75 | 0.50 | 0.4767 | 0.4744 (0.25) | 0.0024 | 0.051 | 0.50 / 0.50 / 0.00 | +| 5G-NIDD | 0.25 | 0.00 | 0.7615 | 0.7493 (0.25) | 0.0122 | 0.556 | 0.25 / 0.00 / 0.00 | +| 5G-NIDD | 0.50 | 1.00 | 0.6924 | 0.6884 (0.75) | 0.0040 | 0.029 | 1.00 / 0.00 / 1.00 | +| 5G-NIDD | 0.75 | 0.50 | 0.8091 | 0.8043 (1.00) | 0.0048 | 0.081 | 0.50 / 1.00 / 1.00 | + +In **all six conditions the top-two gap is smaller than the selected beta's across-seed sample std** +(ratios 0.03–0.56), so each selection is a *small numerical separation relative to seed variability*. +No tie-break was ever exercised. Per-seed diagnostic betas are shown only for stability; they are not +the selection rule. + +## 11. Prototype-Level Analysis + +Direct aggregation effect, measured on fixed round-0 snapshots (`C_ij`, `N_ij` held constant, no +retraining). `artifacts/analysis/final__prototype_displacement_by_beta.csv`. + +Mean `‖Cbar_j(beta) − Cbar_j(0)‖` across classes: + +| Dataset | alpha | beta=0.25 | beta=0.50 | beta=0.75 | beta=1.00 | +|---|---|---|---|---|---| +| X-IIoTID | 0.25 | 1.476 | 2.403 | 3.003 | **3.422** | +| X-IIoTID | 0.50 | 0.826 | 1.374 | 1.780 | **2.099** | +| X-IIoTID | 0.75 | 0.477 | 0.829 | 1.097 | **1.305** | +| 5G-NIDD | 0.25 | 1.295 | 2.055 | 2.559 | **2.919** | +| 5G-NIDD | 0.50 | 0.953 | 1.567 | 2.003 | **2.340** | +| 5G-NIDD | 0.75 | 0.497 | 0.875 | 1.163 | **1.385** | + +- Displacement is **monotone increasing in `beta`** for a fixed snapshot in all six conditions + (mechanically expected: larger `beta` shifts weight toward high-`N_ij` clients). +- Displacement at `beta = 1` **decreases as alpha rises**, tracking realized support imbalance. +- At matched alpha, X-IIoTID has larger displacement at a0.25 (3.422 vs 2.919) while 5G-NIDD is + slightly larger at a0.50 (2.340 vs 2.099) and a0.75 (1.385 vs 1.305); per-class ordering is + seed- and dataset-specific, and rare-class identities differ. +- **Displacement is a representation-aggregation metric, not detection quality.** No + displacement-to-accuracy inference is made. + +Normalized contribution weights are stored per client-class in +`artifacts/analysis/*__prototype_weights.csv`; per-class displacement is in +`final__prototype_displacement_by_class.csv`. + +## 12. Rare-Class Detection Results + +Rare classes are defined from each client's **training** support (the two attack classes with the +smallest positive `N_ij`; one if only one; the client is excluded if none). Unseen classes +(`N_ij = 0`) are treated separately and never merged with rare classes. + +Key observations from the test results (Section 9.2) and the synthesis: + +- Selected rare-class macro-F1 deltas are **mixed in sign and small**; the largest positive + difference is within seed noise. +- Selected rare-class recall is **slightly lower than original in most conditions** while macro-F1 + may be higher: weighting can trade a little rare-class recall for precision. +- At both `alpha = 0.25` conditions the selected `beta` is 0, so those conditions cannot demonstrate + any support-weighting benefit. +- Full rare-class curves and per-condition reports: `docs/research/FINAL_EXPERIMENTAL_SYNTHESIS.md`, + the per-condition `*_BETA_VALIDATION.md` reports, and `final__selected_vs_original.csv`. + +## 13. Per-Client Analysis + +Per-client effects are computed on each client's own evaluation partition relative to `beta = 0` +(`artifacts/analysis/final__client_effect_summary.csv`; detail in the per-condition +`*__multiseed__client_validation_*.csv`). + +Improve / degrade / unchanged clients at the **selected** beta (validation, vs `beta=0`), by seed: + +| Condition (selected beta) | seed 0 | seed 1 | seed 2 | +|---|---|---|---| +| X-IIoTID a0.25 (0.00) | 0 / 0 / 10 | 0 / 0 / 10 | 0 / 0 / 10 | +| X-IIoTID a0.50 (0.75) | 7 / 3 / 0 | 8 / 1 / 1 | 8 / 1 / 1 | +| X-IIoTID a0.75 (0.50) | 6 / 4 / 0 | 9 / 0 / 1 | 6 / 4 / 0 | +| 5G-NIDD a0.25 (0.00) | 0 / 0 / 10 | 0 / 0 / 10 | 0 / 0 / 10 | +| 5G-NIDD a0.50 (1.00) | 6 / 3 / 1 | 3 / 5 / 2 | 7 / 2 / 1 | +| 5G-NIDD a0.75 (0.50) | 4 / 6 / 0 | 6 / 3 / 1 | 4 / 2 / 4 | + +- Weighting **redistributes benefit across clients**: within a condition some clients improve while + others degrade. +- At `beta = 1`, mean per-client delta is most negative at both `alpha = 0.25` conditions + (e.g. X-IIoTID a0.25 ≈ −0.060, 5G-NIDD a0.25 ≈ −0.024), consistent with those conditions + selecting `beta = 0`. +- `N_ij` versus per-client delta is **non-monotonic**: the worst-degrading client is not always the + lowest-support client. +- Client identities are per-seed realizations and are **not pooled** across seeds. + +## 14. Cross-Alpha and Cross-Dataset Findings + +- **Matched-alpha selections agree at alpha=0.25 (both 0.00) and alpha=0.75 (both 0.50)** and differ + at alpha=0.50 (X-IIoTID 0.75 vs 5G-NIDD 1.00). Across the six conditions the selected beta takes + values `{0.00, 0.50, 1.00}` — **not universal, not monotone in alpha, not a function of alpha + alone**. +- **Realized heterogeneity** (support CV, missing client-class cells, client-size ratio) decreases as + alpha rises within each dataset, but not strictly per seed. Dirichlet alpha is a design knob, not + the realized heterogeneity itself. +- Displacement at `beta = 1` falls with alpha on both datasets. +- **Rare-set composition differs by dataset**, so absolute metric levels are **not comparable across + datasets**; only within-condition differences are meaningful. +- **No cross-dataset causality, superiority, or dataset-independent beta law is claimed.** + +Reports: `docs/research/CROSS_DATASET_BETA_ANALYSIS.md`, `XIIOTID_CROSS_ALPHA_ANALYSIS.md`, +`5GNIDD_CROSS_ALPHA_ANALYSIS.md`. + +## 15. Confusion Matrix and Classification Breakdown + +**Confusion matrices do exist in the saved artifacts.** They are written per run by the evaluation +code (`confusion_matrix` in `src/protean_research/evaluation/metrics.py`) and saved as: + +- `results//test_confusion_matrix.json` — **108 / 108 runs** +- `results//val_confusion_matrix.json` — **107 / 108 runs** (missing only for + `xiiotid__alpha0.5__seed0__original_protean`, which also lacks the other validation artifacts) + +**Properties of the stored matrices.** + +| Property | Value | +|---|---| +| Shape | 9 × 9 (K = 9 classes) | +| Values | **raw integer counts** (not normalized) | +| Scope | **global** over the evaluation split, not per-client | +| Restriction | full multiclass; **not** restricted to rare classes | +| Split | separate `test_` and `val_` matrices | +| Class order | X-IIoTID: `C&C, Exfiltration, Exploitation, Lateral_movement, Normal, RDOS, Reconnaissance, Tampering, Weaponization`; 5G-NIDD: `Benign, HTTPFlood, ICMPFlood, SYNFlood, SYNScan, SlowrateDoS, TCPConnectScan, UDPFlood, UDPScan` (also in each run's `metadata.json`) | +| Aggregation | each client's eval partition concatenated; no seed/client mixing | + +Representative interpretation — 5G-NIDD, `alpha=0.75`, `seed=0`, **test**, original vs selected +(`beta=0.50`) — using actual counts: + +| Class | eval support | original recall | selected-beta recall | +|---|---|---|---| +| Benign | 95,547 | 0.3795 | 0.3825 | +| HTTPFlood | 28,162 | 0.9729 | 0.9923 | +| ICMPFlood | 231 | 1.0000 | 1.0000 | +| SYNFlood | 1,944 | 0.8570 | 0.8570 | +| SYNScan | 4,009 | 0.9968 | 0.9968 | +| SlowrateDoS | 14,625 | 0.9217 | 0.9054 | +| TCPConnectScan | 4,010 | 0.9948 | 0.9948 | +| UDPFlood | 91,468 | 1.0000 | 1.0000 | +| UDPScan | 3,181 | 0.9975 | 0.9981 | + +Largest off-diagonal cells (true -> predicted): + +| true | predicted | original count | selected-beta count | +|---|---|---|---| +| Benign | UDPFlood | 56,899 | 57,054 | +| Benign | TCPConnectScan | 2,015 | 1,495 | +| SlowrateDoS | HTTPFlood | 1,079 | 1,308 | +| HTTPFlood | SlowrateDoS | 759 | 212 | +| SYNFlood | TCPConnectScan | 278 | 278 | + +What these counts support: + +- The dominant error mode is **Benign misclassified as UDPFlood** (about 57k samples), which + dominates the off-diagonal mass. It is essentially unchanged by `beta`, so `beta` does not fix + this confusion. This is visible in low Benign recall (≈0.38) with very high UDPFlood recall + (1.0000), i.e. UDPFlood is an attractor for Benign. +- The change from original to selected `beta` shifts a few specific off-diagonal cells + (`Benign → TCPConnectScan` decreases; `SlowrateDoS → HTTPFlood` increases), consistent with the + small, mixed-sign aggregate deltas, **not** with a systematic improvement. +- Diagonal versus off-diagonal entries must be read together: high diagonal on the rare/scan classes + coexists with the large Benign confusion, which is why accuracy is moderate while macro-F1 is + higher. + +Reconstruction note: because only aggregate per-class metrics (precision/recall/F1/support) are +persisted, a confusion matrix cannot be uniquely reconstructed from them; the raw `y_true`/`y_pred` +arrays are not saved. Reconstruction is unnecessary here because the matrices themselves are already +persisted as JSON. + +Representative confusion-matrix files: + +- `results/5gnidd__alpha0.75__seed0__original_protean/test_confusion_matrix.json` +- `results/5gnidd__alpha0.75__seed0__support_weighted__beta0.5/test_confusion_matrix.json` +- `results/xiiotid__alpha0.25__seed0__original_protean/test_confusion_matrix.json` + +## 16. Final Figures and Visualizations + +All figures are under `artifacts/figures/final/` and load from the `final__*` analysis tables. They +were generated with matplotlib; no training was involved. + +**Fig A — pooled validation rare-class macro-F1 vs beta** (per dataset). Axes: `beta` (x) vs pooled +validation per-client-mean rare-class macro-F1 (y); one line per alpha. Evidence: the validation +curve is non-monotone and the selected `beta` differs by condition. + +![Pooled validation beta curve, 5G-NIDD](artifacts/figures/final/figA_5gnidd_pooled_validation_beta_curve.png) + +![Pooled validation beta curve, X-IIoTID](artifacts/figures/final/figA_xiiotid_pooled_validation_beta_curve.png) + +**Fig B — selected beta by condition.** Bar chart of the six validation-selected betas; shows that +selections take `{0.00, 0.50, 1.00}` and are not universal. + +![Selected beta by condition](artifacts/figures/final/figB_selected_beta_by_condition.png) + +**Fig C — realized support CV.** Condition-level mean class-support CV; shows realized imbalance +decreases as alpha rises. + +![Realized support CV](artifacts/figures/final/figC_realized_support_cv.png) + +**Fig D — prototype displacement at beta=0 vs beta=1.** Fixed-snapshot mean displacement per class; +shows displacement grows with `beta` and larger where support imbalance is larger. + +![Prototype displacement beta0 vs beta1](artifacts/figures/final/figD_prototype_displacement_beta0_vs_beta1.png) + +**Fig E — support CV vs displacement.** Scatter of realized support CV (x) vs beta=1 displacement +(y) across conditions; a descriptive co-variation, not a causal relation. + +![Support CV vs displacement](artifacts/figures/final/figE_support_cv_vs_displacement.png) + +**Fig F — selected vs original test rare-class macro-F1.** Paired comparison per condition; shows +small, mixed-sign differences. + +![Selected vs original test rare macro-F1](artifacts/figures/final/figF_selected_vs_original_test_rare_macro_f1.png) + +**Fig G — per-client effect at the selected beta.** Per-client validation delta relative to +`beta = 0`; shows within-condition heterogeneity across clients. + +![Client effect at selected beta](artifacts/figures/final/figG_client_effect_at_selected_beta.png) + +## 17. Main Scientific Findings + +1. **Changing `beta` changes the global prototype.** Contribution weights become proportional to + support, and for a fixed snapshot mean prototype displacement grows monotonically with `beta` in + every condition. +2. **The rare-class response is condition-dependent and non-monotone in `beta`.** The direct + aggregation effect (geometry) and the end-to-end training effect (detection) are distinct. +3. **No universal beta was found.** The validation-selected `beta` differs across datasets and + conditions and is not monotone in `alpha`. Every selection is weak relative to seed variability. +4. **Selected-vs-original differences are small and mixed in sign.** No statistical superiority is + established; the largest positive difference is within seed noise. +5. **Weighting redistributes benefit across clients.** Within a condition some clients improve while + others degrade; the effect is not simply "more support is better". +6. **Where weighting does not help:** both `alpha = 0.25` conditions select `beta = 0` (equal + weighting), and at `beta = 1` the per-client effect is most damaging there. +7. **Realized support imbalance and prototype displacement co-vary** descriptively; displacement is + not a proxy for detection quality. +8. **The dominant classification error (Benign → UDPFlood) is not addressed by `beta`.** + +## 18. Limitations and Novelty Boundary + +**Limitations.** + +- The `S_j` participant/denominator rule is a **documented reproduction assumption** (no verified + official PROTEAN code). It is material in up to 19/27 class-condition cells; all results are + conditional on it. +- `beta = 0` is only arithmetically the equal mean over `S_j`; it is **not bitwise identical** to + `original_protean` (floating-point divergence; equivalence class B in every condition). +- Two datasets, three alpha values, three project-defined seeds, 10 clients, one architecture. +- The `beta` grid is discrete; intermediate values were not tested. +- Three seeds are underpowered for inferential testing; no significance test is claimed. +- Prototype displacement is a representation metric, not detection quality. +- One validation artifact set is missing (Section 7); rare-class and unseen-class results cannot be + merged, and cross-dataset absolute metric levels are not comparable. + +**Novelty boundary.** Support-weighted prototype aggregation is not new (e.g. FedProto). The +contribution is the controlled `beta` parameterization inside PROTEAN, the validation-only +condition-wise selection, the paired multi-seed design, and the behavioral findings. No claim is +made that the method universally improves PROTEAN. + +## 19. Repository Structure ``` -src/protean_research/ data, models, aggregation, federated, evaluation, utils -scripts/ training, partition, and analysis scripts -configs/ datasets, experiments, model configuration -docs/proposal/ thesis proposal and its internal audit aids -docs/research/ frozen protocol, per-condition reports, cross-alpha / cross-dataset synthesis +src/protean_research/ data loading/preprocessing/partitioning, models, aggregation, federated + round logic, evaluation metrics, analysis, utils +scripts/ training runner, partition generation, and read-only analysis scripts +configs/ dataset, experiment protocol, and model configuration (YAML) +docs/proposal/ thesis proposal and internal audit aids +docs/research/ frozen protocol, per-condition reports, cross-alpha/cross-dataset synthesis, + session audits +docs/baseline/ PROTEAN baseline investigation, implementation, and validation docs/thesis/ Chapter 4 blueprint and claims register -docs/baseline/ PROTEAN baseline investigation and validation docs/handoff/ session handoff (start here for a new session) planningawal/ immutable research specification (do not modify) +artifacts/partitions/ deterministic frozen Dirichlet partition manifests (18) artifacts/analysis/ committed machine-readable analysis outputs artifacts/figures/final/ thesis-ready figures -artifacts/partitions/ deterministic frozen Dirichlet partition manifests -results/ local run outputs (git-ignored, intentionally not committed) +results/ local run outputs (git-ignored; 108 non-smoke run dirs + 1 smoke) ``` -## Key Scripts - -| Script | Role | -|---|---| -| `scripts/run_baseline.py` | full baseline / support-weighted training runner (frozen; grid complete) | -| `scripts/generate_partitions.py` | deterministic frozen Dirichlet manifests | -| `scripts/analyze_multiseed_beta.py` | pooled validation beta selection | -| `scripts/analyze_partition_heterogeneity.py` | realized non-IID characterization | -| `scripts/analyze_prototype_effect.py` | fixed-snapshot prototype displacement | -| `scripts/analyze_cross_alpha.py` | cross-alpha synthesis | -| `scripts/verify_beta0_equivalence.py` | beta=0 vs original semantic/numerical audit | -| `scripts/validate_sj_participant_rule.py` | `S_j` reproduction-assumption analysis | - -## Important Project Documents - -**Proposal** - -- `docs/proposal/THESIS_PROPOSAL_DRAFT.md` — final Markdown proposal source for the current scope -- `docs/proposal/PROPOSAL_SOURCE_MAP.md` — claim-to-evidence map (internal) -- `docs/proposal/LITERATURE_MATRIX.md` — literature matrix (internal) -- `docs/proposal/REFERENCES_VERIFIED.md` — bibliography verification record (internal) - -**Research synthesis** - -- `docs/research/FINAL_EXPERIMENTAL_SYNTHESIS.md` — authoritative six-condition synthesis -- `docs/research/CROSS_DATASET_BETA_ANALYSIS.md` — cross-dataset comparison -- `docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md` — X-IIoTID cross-alpha analysis -- `docs/research/5GNIDD_CROSS_ALPHA_ANALYSIS.md` — 5G-NIDD cross-alpha analysis -- `docs/research/EXPERIMENT_PROTOCOL.md` — frozen reproduction protocol - -**Thesis support** - -- `docs/thesis/CHAPTER4_RESULTS_OUTLINE.md` — Chapter 4 blueprint -- `docs/thesis/CLAIMS_REGISTER.md` — claim / evidence / status register - -**Handoff and planning** - -- `docs/handoff/SESSION_HANDOFF.md` — current state and next steps (start here) -- `planningawal/PROTEAN_Revisi_Lurus.md` — immutable research specification - -> `planningawal/` is **immutable**. Read it, cite it, but never edit, rename, move, or add files to -> it. - -## Testing and Reproducibility - -```bash -# unit + integration tests (expected: 75 passed) -.venv/bin/python -m pytest -``` - -- **Paired comparison:** within each `(dataset, alpha, seed)`, every compared configuration reuses - the same partition, split, class mapping, and model initialization. -- **Frozen manifests:** partitions are generated once and reused; regenerating a fresh Dirichlet - split is a protocol violation. -- **Immutable raw data:** datasets live outside the repository and are never modified or committed. -- **Validation-only selection:** beta is chosen from validation data only; test data is never used - for any model-selection decision. -- **Machine-readable outputs:** analysis is stored as CSV/JSON under `artifacts/analysis/`, and the - final figures load from those artifacts. - -## Frozen / Do Not Modify - -Treat the following as finished unless there is an explicit, documented reason: - -- `planningawal/` — immutable research source. -- Raw datasets under `~/datasets/` — read-only, external, never copied into the repository. -- Completed scientific results and the final synthesis documents. -- Frozen experiment artifacts: partition manifests, analysis outputs, and final figures. -- `src/`, model, and aggregation code used for the 108-run grid. -- The finalized proposal Markdown. - -## Limitations - -- The `S_j` participant/denominator rule remains a reproduction assumption and is materially - exercised in a substantial fraction of class-condition cells. -- Two datasets, three alpha values, three project-defined seeds, 10 clients, one architecture. -- The beta grid is discrete; intermediate values were not tested. -- No inferential trend testing (three seeds are underpowered for it). -- `beta = 0` is not bitwise identical to `original_protean` (floating-point ordering). -- Prototype displacement is a representation metric, not detection quality. - -## Quickstart +## 20. Reproduction and Tests ```bash # unit + integration tests (expected: 75 passed) .venv/bin/python -m pytest -# read-only pooled validation analysis (no retraining required) +# read-only pooled validation analysis (no retraining) .venv/bin/python scripts/analyze_multiseed_beta.py --dataset 5gnidd --alpha 0.25 --seeds 0 1 2 -# read-only cross-alpha / cross-dataset synthesis +# read-only realized-partition characterization .venv/bin/python scripts/analyze_partition_heterogeneity.py --dataset 5gnidd --alphas 0.25 0.5 0.75 + +# read-only cross-alpha synthesis .venv/bin/python scripts/analyze_cross_alpha.py --dataset 5gnidd --alphas 0.25 0.5 0.75 + +# beta=0 vs original equivalence audit (read-only) +.venv/bin/python scripts/verify_beta0_equivalence.py --dataset 5gnidd --alpha 0.25 --seeds 0 1 2 ``` -See `docs/handoff/SESSION_HANDOFF.md` before making any methodological change. The next work is -academic writing and formatting, not new experiments. +A full training run (not needed; the grid is complete) is: + +```bash +.venv/bin/python scripts/run_baseline.py --dataset 5gnidd --alpha 0.25 --seed 1 \ + --clients 10 --rounds 10 --local-epochs 3 \ + --aggregation-mode support_weighted --beta 1.0 +``` + +Runtime context: the 108 runs consumed ≈ 94.5 wall-clock hours on CPU (8 cores); per-run wall clock +was ≈ 62–65 minutes for 5G-NIDD and ≈ 40 minutes for X-IIoTID (`*__multiseed__runtime.csv`). Do not +rerun the grid. + +**Repository safety.** + +- `planningawal/` is immutable; raw datasets are external and read-only. +- Do not modify or regenerate completed results, manifests, or figures. +- Do not add an `equal_available` mode; it is arithmetically `support_weighted` at `beta = 0` under + the shared `S_j`. +- `beta` selection uses validation only; the test set is never used for selection. + +## 21. Complete Artifact Index + +Per-run outputs (`results/__alpha__seed__/`): + +| File | Content | +|---|---| +| `metadata.json` | run config, dataset/alpha/seed, mode, class names, hashes | +| `history.json` | per-round global accuracy/macro-F1/rare-class metrics | +| `characterization.csv` | per-class training support statistics (`N_ij`) | +| `train_support_Nij.csv` | `N_ij` matrix (client × class) | +| `rare_unseen.csv` | per client-class rare/unseen flags from training support | +| `test_global_metrics.json`, `val_global_metrics.json` | global metrics per split | +| `test_per_class.csv`, `val_per_class.csv` | per-class precision/recall/F1/support | +| `per_client.csv`, `val_per_client.csv` | per-client metrics | +| `test_confusion_matrix.json`, `val_confusion_matrix.json` | 9×9 raw-count confusion matrices | +| `val_selection.json` | validation selection metric and `test_used_for_selection` flag | + +For the one incomplete validation run, see Section 7. + +Analysis outputs (`artifacts/analysis/`), by function: + +| Category | Representative files | +|---|---| +| Final condition summary | `final__six_condition_master.csv/.json` | +| Beta-selection curves | `final__pooled_validation_curves.csv`, `final__selection_strength.csv` | +| Selected vs original | `final__selected_vs_original.csv` | +| Realized heterogeneity | `final__realized_heterogeneity.csv`, `*__partition_heterogeneity_*.csv` | +| Prototype displacement | `final__prototype_displacement_by_beta.csv`, `..._by_class.csv` | +| Per-client effects | `final__client_effect_summary.csv`, `*__multiseed__client_validation_*.csv` | +| Per-condition validation | `*__multiseed__validation_per_seed.csv`, `..._validation_pooled.csv` | +| Per-condition test | `*__multiseed__test_descriptive.csv` | +| Cross-alpha | `{xiiotid,5gnidd}__cross_alpha__*.csv/.json` | +| Cross-dataset | `cross_dataset__matched_alpha_summary.csv/.json`, `cross_dataset__beta_curve.csv` | +| beta=0 equivalence | `*__multiseed__beta0_equivalence.csv` | +| Runtime | `*__multiseed__runtime.csv` | +| Prototype snapshots & weights | `*__prototype_snapshot.json`, `*__prototype_weights.csv`, `*__prototype_effect*.csv` | + +Partition manifests: `artifacts/partitions/__alpha__seed.json` (18 files). +Figures: `artifacts/figures/final/figA–figG*.png` (8 files). + +Key documents: + +| Document | Purpose | +|---|---| +| `docs/research/EXPERIMENT_PROTOCOL.md` | frozen reproduction protocol | +| `docs/research/FINAL_EXPERIMENTAL_SYNTHESIS.md` | authoritative six-condition synthesis | +| `docs/research/CROSS_DATASET_BETA_ANALYSIS.md` | cross-dataset comparison | +| `docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md` | X-IIoTID cross-alpha analysis | +| `docs/research/5GNIDD_CROSS_ALPHA_ANALYSIS.md` | 5G-NIDD cross-alpha analysis | +| `docs/thesis/CHAPTER4_RESULTS_OUTLINE.md` | Chapter 4 blueprint | +| `docs/thesis/CLAIMS_REGISTER.md` | claim / evidence / status register | +| `docs/proposal/THESIS_PROPOSAL_DRAFT.md` | final proposal Markdown source | +| `docs/proposal/PROPOSAL_SOURCE_MAP.md` | claim-to-evidence map | +| `docs/proposal/LITERATURE_MATRIX.md` | literature matrix | +| `docs/proposal/REFERENCES_VERIFIED.md` | bibliography verification record | +| `docs/handoff/SESSION_HANDOFF.md` | current state and next steps | +| `planningawal/PROTEAN_Revisi_Lurus.md` | immutable research specification | + +## 22. References + +- Chennoufi, S., Han, Y., Blanc, G., De Cristofaro, E., & Kiennert, C. (2025). PROTEAN: Federated + Intrusion Detection in Non-IID Environments through Prototype-Based Knowledge Sharing. ESORICS + 2025, pp. 103–125. DOI: 10.1007/978-3-032-07884-1_6. arXiv:2507.05524. +- Tan, Y., Long, G., Liu, L., Zhou, T., Lu, Q., Jiang, J., & Zhang, C. (2022). FedProto: Federated + Prototype Learning across Heterogeneous Clients. AAAI 36(8), 8432–8440. + DOI: 10.1609/aaai.v36i8.20819. arXiv:2105.00243. + +The complete, verified bibliography (including the FL foundations, non-IID, class-imbalance, and +evaluation-metric references used in the proposal) is maintained in +`docs/proposal/REFERENCES_VERIFIED.md` and the proposal bibliography. This README cites only the two +works directly relevant to the mechanism and novelty boundary.