757 lines
43 KiB
Markdown
757 lines
43 KiB
Markdown
# PROTEAN Prototype Aggregation Study
|
||
|
||
Research code and audited artifacts for a master's thesis on **server-side prototype aggregation
|
||
in PROTEAN** (Chennoufi et al., 2025), a federated intrusion-detection framework, under non-IID
|
||
client data. The study evaluates a **beta-controlled, class-specific sample-support weighting** of
|
||
local class prototypes for **rare attack detection**.
|
||
|
||
**Research status.** The experimental grid is **complete** (108 full federated runs), the results
|
||
are synthesized and audited, and the thesis proposal Markdown is finalized. No further training is
|
||
planned unless new academic feedback explicitly requires it. The repository is in a documentation
|
||
and thesis-writing stage.
|
||
|
||
**Final title.** *Optimasi Agregasi Prototype pada PROTEAN melalui Pembobotan Berbasis Dukungan
|
||
Sampel Kelas untuk Deteksi Serangan Langka pada Data Non-IID.* Here "Optimasi" means empirical
|
||
selection of `beta` among the predefined candidate values, not a global mathematical optimum.
|
||
|
||
---
|
||
|
||
## 1. Research Overview
|
||
|
||
PROTEAN is a federated intrusion-detection method that shares **class prototypes** between clients
|
||
and the server in addition to model parameters. Each client `i` builds a local prototype `C_ij` for
|
||
class `j` from its own training samples; the server aggregates these into a global prototype
|
||
`Cbar_j` that is used both for local prototype alignment during training and as the classifier at
|
||
inference (nearest prototype in embedding space).
|
||
|
||
This thesis modifies **only the server-side prototype aggregation**. The original PROTEAN
|
||
aggregation gives every contributing client equal weight. The proposed aggregation weights each
|
||
client's prototype by `N_ij^beta`, where `N_ij` is that client's class-specific training support and
|
||
`beta` is a single exponent in `[0, 1]`. Everything else — local prototype formation, model
|
||
aggregation, architecture, loss, and inference rule — is unchanged relative to the reproduced
|
||
baseline.
|
||
|
||
`N_ij` is **class-specific sample support, not prototype quality**. A client with few samples is
|
||
not assumed to have a worse prototype.
|
||
|
||
## 2. Problem Statement and Motivation
|
||
|
||
Federated IDS data is non-IID: clients observe different traffic mixtures and different attack
|
||
types, so class support varies strongly across clients. Rare attack classes appear in little of
|
||
the data, so aggregate metrics can look acceptable while the model fails on exactly the classes
|
||
that matter for detection.
|
||
|
||
Under the original equal-weight prototype aggregation, a prototype built from a handful of samples
|
||
contributes as much as one built from thousands. This thesis investigates whether — and under which
|
||
realized non-IID conditions — weighting contributions by class-specific sample support changes the
|
||
global prototype and rare-class detection behavior.
|
||
|
||
The study deliberately isolates one mechanism so that any observed change can be attributed to the
|
||
aggregation weighting rather than to a confounded set of architectural or training changes.
|
||
|
||
## 3. Original PROTEAN versus Proposed Modification
|
||
|
||
**A. Local prototype (unchanged).** For class `j` with `N_ij > 0` samples on client `i`, the local
|
||
prototype is the mean embedding of that client's class-`j` samples:
|
||
|
||
$$
|
||
C_{ij} = \frac{1}{N_{ij}} \sum_{k=1}^{N_{ij}} z_{ijk}
|
||
$$
|
||
|
||
where `z_ijk` is the embedding of the k-th sample of class `j` on client `i`.
|
||
|
||
**B. Original server-side prototype aggregation (reference).** PROTEAN Eq. (2) averages the `M`
|
||
contributing clients equally (equal contribution, no `N_ij` weight):
|
||
|
||
$$
|
||
\bar{C}_j = \frac{1}{M} \sum_{i=1}^{M} C_{ij}
|
||
$$
|
||
|
||
**C. Proposed beta-controlled aggregation.** Only the prototype aggregation is modified:
|
||
|
||
$$
|
||
\bar{C}_j(\beta) = \frac{\sum_{i \in S_j} N_{ij}^{\beta}\, C_{ij}}{\sum_{i \in S_j} N_{ij}^{\beta}}, \qquad \beta \in \{0,\ 0.25,\ 0.50,\ 0.75,\ 1.00\}
|
||
$$
|
||
|
||
**D. Normalized contribution weight.** Client `i`'s normalized weight for class `j` is
|
||
|
||
$$
|
||
w_{ij}(\beta) = \frac{N_{ij}^{\beta}}{\sum_{k \in S_j} N_{kj}^{\beta}}
|
||
$$
|
||
|
||
**E. Symbol meanings.**
|
||
|
||
| Symbol | Meaning |
|
||
|---|---|
|
||
| `N_ij` | number of local **training** samples of class `j` on client `i` (class-specific sample support, not prototype quality) |
|
||
| `C_ij` | client `i`'s local class-`j` prototype (mean embedding) |
|
||
| `S_j` | participant set: clients with `N_ij > 0` and a valid local prototype for class `j` |
|
||
| `M` | number of participants in the reference Eq. (2) (equal average) |
|
||
| `beta` | scalar weighting exponent in `[0, 1]`, one value for all classes and clients |
|
||
| `w_ij` | normalized contribution weight (sums to 1 over `S_j`) |
|
||
| global prototype | `Cbar_j(beta)`; used for alignment and nearest-prototype inference |
|
||
|
||
**F. Exact difference between original and modified PROTEAN.**
|
||
|
||
- With `beta = 0`, `N_ij^0 = 1`, so the proposed aggregator becomes the **equal mean over `S_j`**.
|
||
- With `beta = 1`, the contribution is proportional to class support (equivalent to the mean of
|
||
all included local class embeddings when the assumptions hold).
|
||
- `beta = 0` is equal weighting over `S_j`; it is equivalent to the original formulation **only
|
||
if** the participant and missing-class rules are also equivalent. That equivalence is not
|
||
verified against official PROTEAN code.
|
||
- Model aggregation, local prototype formation, architecture, loss, and inference are **not**
|
||
modified.
|
||
|
||
**Novelty boundary.** Sample-count prototype weighting is not new by itself; related work such as
|
||
**FedProto** (Tan et al., 2022) already involves class sample counts in prototype-based federated
|
||
learning. The contribution here is the **controlled beta parameterization inside PROTEAN's
|
||
server-side prototype aggregation**, its condition-wise validation-based selection, the paired
|
||
multi-seed multi-condition evaluation, and the resulting behavioral findings — not the invention of
|
||
support weighting.
|
||
|
||
**Analysis chain (not a causal proof by itself).**
|
||
|
||
```
|
||
support imbalance -> contribution weight -> global-prototype displacement
|
||
-> rare-class detection -> per-client effect
|
||
```
|
||
|
||
## 4. Research Questions and Objectives
|
||
|
||
| ID | Research question |
|
||
|---|---|
|
||
| RQ1 | How are class-support imbalance, differences between local prototypes, and rare-class detection performance related in PROTEAN? |
|
||
| RQ2 | How does changing `beta` affect the global prototype and rare-class detection performance under different non-IID conditions? |
|
||
| RQ3 | Does the validation-selected `beta` change across non-IID levels, and what is its effect on overall and per-client performance? |
|
||
|
||
Objectives: (1) relate support imbalance to prototype behavior and rare-class detection (RQ1);
|
||
(2) characterize the effect of `beta` on the global prototype and rare-class detection across
|
||
conditions (RQ2); (3) select `beta` by validation at different imbalance levels and quantify
|
||
overall and per-client consequences (RQ3).
|
||
|
||
## 5. Experimental Design
|
||
|
||
| Item | Value |
|
||
|---|---|
|
||
| Base method | PROTEAN federated intrusion detection |
|
||
| Modified component | server-side prototype aggregation only |
|
||
| Mechanism | `w_ij = N_ij^beta`, one `beta` for all classes/clients |
|
||
| Datasets | X-IIoTID (`class2`, benign `Normal`, K=9); 5G-NIDD (`Attack Type`, benign `Benign`, K=9) |
|
||
| Clients | 10 |
|
||
| Non-IID conditions | symmetric Dirichlet `alpha` = 0.25, 0.50, 0.75 |
|
||
| Seeds | 0, 1, 2 (project-defined) |
|
||
| Beta candidates | 0, 0.25, 0.50, 0.75, 1.00 |
|
||
| Baseline | `original_protean` (equal-weight prototype aggregation) |
|
||
| Rounds / local epochs / batch | 10 / 3 / 256 |
|
||
| Optimizer / lr / mu / lambda | Adam / 1e-3 / 0.1 / 1.0 |
|
||
| Splits | stratified train / validation / test = 70 / 10 / 20 |
|
||
| Selection metric | pooled **validation** per-client-mean rare-class macro-F1 across seeds |
|
||
| Test usage | read only after the selected `beta` is fixed |
|
||
|
||
Expected experimental matrix: `2 datasets × 3 alpha × 3 seeds × (1 baseline + 5 betas) = 108 runs`.
|
||
Within each `(dataset, alpha, seed)` all six configurations share one manifest, split, class mapping,
|
||
and round-0 initialization; only the aggregation mode changes.
|
||
|
||
## 6. Completed Research Phases
|
||
|
||
Documented phases, in order. Session numbers are used only where the repository documents them.
|
||
|
||
| Phase / session | Objective | Key outputs | Validation |
|
||
|---|---|---|---|
|
||
| Planning and architecture | freeze scope, protocol, and project structure | `AGENTS.md`, `planningawal/PROTEAN_Revisi_Lurus.md`, `docs/research/IMPLEMENTATION_PLAN.md` | checkpoint commit |
|
||
| Baseline reproduction | implement reproducible PROTEAN baseline pipeline | `docs/baseline/BASELINE_IMPLEMENTATION.md`, `src/protean_research/` | `docs/baseline/BASELINE_VALIDATION.md` |
|
||
| Baseline validation + regression tests | verify baseline behavior and invariants | `tests/`, `docs/baseline/PROTEAN_BASELINE_INVESTIGATION.md` | unit/integration tests |
|
||
| Support-weighted aggregation | implement beta-controlled aggregator | `docs/research/BETA_AGGREGATION_IMPLEMENTATION.md`, `src/protean_research/aggregation/` | aggregation invariance tests |
|
||
| Session 5 — one paired condition | first full paired beta run (single seed) | `docs/research/PAIRED_BETA_VALIDATION.md` | `val_selection.json` |
|
||
| Session 6 — multi-seed (X-IIoTID a0.50) | pooled validation across seeds `{0,1,2}` | `docs/research/MULTISEED_BETA_VALIDATION.md`, `SESSION6_AUDIT.md` | audit PASS with minor corrections |
|
||
| Session 7 — X-IIoTID a0.25 | third X-IIoTID alpha condition | `docs/research/ALPHA025_BETA_VALIDATION.md`, `SESSION7_AUDIT.md` | audit PASS with minor corrections |
|
||
| Session 8 — X-IIoTID a0.75 | complete X-IIoTID three-alpha study | `docs/research/ALPHA075_BETA_VALIDATION.md`, `SESSION8_AUDIT.md` | audit PASS |
|
||
| Session 9 — 5G-NIDD a0.50 | first 5G-NIDD condition | `docs/research/5GNIDD_ALPHA050_BETA_VALIDATION.md`, `SESSION9_AUDIT.md` | audit PASS |
|
||
| Session 10 — 5G-NIDD a0.75 | second 5G-NIDD condition | `docs/research/5GNIDD_ALPHA075_BETA_VALIDATION.md`, `SESSION10_AUDIT.md` | audit PASS |
|
||
| Session 11 — 5G-NIDD a0.25 | final condition; completes the grid | `docs/research/5GNIDD_ALPHA025_BETA_VALIDATION.md`, `SESSION11_AUDIT.md` | audit PASS |
|
||
| Session 12 — synthesis | consolidate all six conditions | `docs/research/FINAL_EXPERIMENTAL_SYNTHESIS.md`, `CROSS_DATASET_BETA_ANALYSIS.md`, `docs/thesis/*`, `artifacts/analysis/final__*`, `artifacts/figures/final/*` | independent recomputation, zero mismatches |
|
||
| Proposal phase | write the thesis proposal and audit aids | `docs/proposal/*` | committed `e27c5a8` |
|
||
| Handoff / README | keep repository state current | `docs/handoff/SESSION_HANDOFF.md`, `README.md` | this document |
|
||
|
||
## 7. Experiment Coverage and Reproducibility
|
||
|
||
**Verified run inventory.** Counted from `results/` metadata, not from filenames alone:
|
||
|
||
- non-smoke run directories: **108**
|
||
- distinct `(dataset, alpha, seed)` cells: **18** (2 × 3 × 3)
|
||
- configurations per cell: **1 `original_protean` + 5 `support_weighted`** (beta 0/0.25/0.50/0.75/1.00)
|
||
- paired comparison: within every cell, all six runs share **one manifest content hash** (verified);
|
||
`original_protean` and `support_weighted beta=0` additionally share a byte-identical round-0 snapshot
|
||
- failed runs: **none**; one unrelated `smoke` directory is excluded
|
||
|
||
**Known artifact gap.** One run, `xiiotid__alpha0.5__seed0__original_protean`, lacks its entire
|
||
validation-side artifact set (`val_global_metrics.json`, `val_confusion_matrix.json`,
|
||
`val_per_class.csv`, `val_per_client.csv`, `val_selection.json`). Its test artifacts and history are
|
||
present, and it is not a selection run, so all 108 test evaluations and all six selections are
|
||
unaffected. Every other run has the complete test and validation artifact set. This is recorded as
|
||
an evidence gap rather than silently omitted.
|
||
|
||
**Reproducibility controls.**
|
||
|
||
- **Frozen protocol:** `docs/research/EXPERIMENT_PROTOCOL.md` (machine-readable
|
||
`configs/experiments/experiment_protocol.yaml`). Values are either FROZEN from the paper or
|
||
explicit PROJECT DECISIONs.
|
||
- **Frozen manifests:** 18 deterministic Dirichlet partition manifests under `artifacts/partitions/`;
|
||
generated once and reused. Regenerating a fresh split for a run is a protocol violation.
|
||
- **Raw data:** external and read-only (`~/datasets/…`); never modified, copied, or committed.
|
||
SHA-256 hashes are recorded in the audit documents.
|
||
- **Validation/test separation:** `beta` is selected from validation only; every run records
|
||
`test_used_for_selection = false`.
|
||
- **Seeds** control split, Dirichlet partition, model initialization, and batch order.
|
||
|
||
## 8. Evaluation Metrics
|
||
|
||
All metrics below are implemented in `src/protean_research/evaluation/metrics.py` and persisted per
|
||
run. No scikit-learn `classification_report` is used; metrics are pure NumPy.
|
||
|
||
| Metric | Definition | Level | Split | Where stored |
|
||
|---|---|---|---|---|
|
||
| Accuracy | `trace(CM) / sum(CM)` | global | val, test | `{split}_global_metrics.json` |
|
||
| Macro-F1 | unweighted mean of per-class F1 over classes with support > 0 | global | val, test | `{split}_global_metrics.json` |
|
||
| Weighted-F1 | support-weighted mean of per-class F1 | global | val, test | `{split}_global_metrics.json` |
|
||
| Per-class precision/recall/F1/support | one-vs-rest from the confusion matrix | per class (global) | val, test | `{split}_per_class.csv` |
|
||
| Rare-class macro-F1 | mean per-class F1 over the **evaluated rare classes** (union of client rare classes with support) | global | val, test | `{split}_global_metrics.json` |
|
||
| Rare-class recall | mean recall over the evaluated rare classes | global | val, test | `{split}_global_metrics.json` |
|
||
| Rare support | number of eval samples in evaluated rare classes | global | val, test | `{split}_global_metrics.json` |
|
||
| Per-client accuracy/macro-F1 | standard metrics on each client's own eval partition | per client | val, test | `per_client.csv`, `val_per_client.csv` |
|
||
| Per-client rare-class macro-F1 / recall | rare metrics using that client's own rare class ids | per client | val, test | `per_client.csv`, `val_per_client.csv` |
|
||
| Selection score | mean over eligible clients of per-client validation rare-class macro-F1, pooled across seeds | per condition | validation | `val_selection.json`, aggregation artifacts |
|
||
| Prototype displacement | `‖Cbar_j(beta) − Cbar_j(0)‖` and relative displacement, fixed round-0 snapshot | per class | n/a | `*__prototype_effect*.csv` |
|
||
| Normalized weight | `N_ij^beta / Σ_k N_kj^beta` | per client-class | n/a | `*__prototype_weights.csv` |
|
||
| Support CV / missing cells / client-size ratio | realized non-IID statistics from `N_ij` | per class / partition | training | `characterization.csv`, `*partition_heterogeneity*` |
|
||
|
||
**Averaging-level distinctions that must not be interchanged.**
|
||
|
||
- **Global rare-class macro-F1** pools all rare-class evaluation samples; **per-client mean
|
||
rare-class macro-F1** averages each client's own score. The **selection metric is the per-client
|
||
mean, pooled across seeds**.
|
||
- **Validation** metrics choose `beta`; **test** metrics are reported only after selection.
|
||
- **Per-seed** values and **pooled** values are different quantities; the pooled score averages
|
||
per-seed per-client-mean scores.
|
||
|
||
**Not computed by this repository:** micro-F1, ROC/AUC, statistical significance tests, and
|
||
persisted per-class unseen-class recall (unseen-class evaluation is defined in code but only the
|
||
unseen flags and union counts are saved).
|
||
|
||
## 9. Quantitative Experimental Results
|
||
|
||
All test values are means over seeds `{0,1,2}` for the selected `beta`, and the corresponding
|
||
`original_protean` test means. Values are from `artifacts/analysis/final__six_condition_master.csv`
|
||
and `final__selected_vs_original.csv`, and were independently re-checked against the per-run
|
||
`test_global_metrics.json`.
|
||
|
||
### 9.1 Pooled validation selection (validation-only)
|
||
|
||
Pooled validation per-client-mean rare-class macro-F1 (selected in bold):
|
||
|
||
| beta | X-IIoTID a0.25 | X-IIoTID a0.50 | X-IIoTID a0.75 | 5G-NIDD a0.25 | 5G-NIDD a0.50 | 5G-NIDD a0.75 |
|
||
|---|---|---|---|---|---|---|
|
||
| 0.00 | **0.5917** | 0.5444 | 0.4717 | **0.7615** | 0.6872 | 0.7781 |
|
||
| 0.25 | 0.5887 | 0.5554 | 0.4744 | 0.7493 | 0.6451 | 0.7905 |
|
||
| 0.50 | 0.5859 | 0.5673 | **0.4767** | 0.7257 | 0.6747 | **0.8091** |
|
||
| 0.75 | 0.5795 | **0.5794** | 0.4725 | 0.7222 | 0.6884 | 0.7915 |
|
||
| 1.00 | 0.5321 | 0.5735 | 0.4686 | 0.7376 | **0.6924** | 0.8043 |
|
||
|
||
### 9.2 Selected beta versus original PROTEAN (test)
|
||
|
||
Test rare-class macro-F1 (mean over 3 seeds), selected `beta` vs `original_protean`:
|
||
|
||
| Dataset | alpha | selected beta | test rare-F1 (selected) | test rare-F1 (original) | delta | test macro-F1 (selected) | test accuracy (selected) |
|
||
|---|---|---|---|---|---|---|---|
|
||
| X-IIoTID | 0.25 | 0.00 | 0.6321 | 0.6328 | −0.0007 | 0.6813 | 0.8740 |
|
||
| X-IIoTID | 0.50 | 0.75 | 0.6655 | 0.6425 | **+0.0230** | 0.7312 | 0.9064 |
|
||
| X-IIoTID | 0.75 | 0.50 | 0.6052 | 0.6039 | +0.0014 | 0.7576 | 0.9281 |
|
||
| 5G-NIDD | 0.25 | 0.00 | 0.8412 | 0.8423 | −0.0010 | 0.8166 | 0.7302 |
|
||
| 5G-NIDD | 0.50 | 1.00 | 0.8720 | 0.8805 | −0.0086 | 0.8386 | 0.7433 |
|
||
| 5G-NIDD | 0.75 | 0.50 | 0.8967 | 0.8799 | +0.0168 | 0.8531 | 0.7523 |
|
||
|
||
Reading of the table:
|
||
|
||
- The six conditions split into **three mutually exclusive categories that total six**:
|
||
**three positive deltas** (X-IIoTID a0.50, X-IIoTID a0.75, 5G-NIDD a0.75), **one negative delta**
|
||
(5G-NIDD a0.50), and **two effectively unchanged** (both `alpha = 0.25` conditions, where the
|
||
selected `beta` is 0, so the selected run is the equal-weight run and the difference is
|
||
floating-point divergence only).
|
||
- The **largest positive delta (+0.0230, X-IIoTID a0.50) is still smaller than that condition's
|
||
between-seed sample std (0.0267)**. No statistical superiority is claimed.
|
||
- **No single beta dominates** across conditions, and the selection depends on both dataset and
|
||
condition.
|
||
|
||
Cross-dataset metric levels are not comparable: 5G-NIDD rare classes are low-volume flood/scan
|
||
types, X-IIoTID rare classes are low-volume behavioural categories, and their rare support differs.
|
||
|
||
### 9.3 Selected betas
|
||
|
||
| Dataset | alpha=0.25 | alpha=0.50 | alpha=0.75 |
|
||
|---|---|---|---|
|
||
| X-IIoTID | 0.00 | 0.75 | 0.50 |
|
||
| 5G-NIDD | 0.00 | 1.00 | 0.50 |
|
||
|
||
## 10. Beta Selection by Dataset and Alpha
|
||
|
||
Selection is by pooled validation per-client-mean rare-class macro-F1 (validation only); the test
|
||
set is never used. `artifacts/analysis/final__selection_strength.csv` quantifies separation:
|
||
|
||
| Dataset | alpha | selected beta | best | 2nd best | abs gap | top-2 gap / selected std | per-seed diagnostic betas |
|
||
|---|---|---|---|---|---|---|---|
|
||
| X-IIoTID | 0.25 | 0.00 | 0.5917 | 0.5887 (0.25) | 0.0030 | 0.034 | 0.75 / 0.50 / 0.00 |
|
||
| X-IIoTID | 0.50 | 0.75 | 0.5794 | 0.5735 (1.00) | 0.0059 | 0.078 | 0.50 / 1.00 / 0.75 |
|
||
| X-IIoTID | 0.75 | 0.50 | 0.4767 | 0.4744 (0.25) | 0.0024 | 0.051 | 0.50 / 0.50 / 0.00 |
|
||
| 5G-NIDD | 0.25 | 0.00 | 0.7615 | 0.7493 (0.25) | 0.0122 | 0.556 | 0.25 / 0.00 / 0.00 |
|
||
| 5G-NIDD | 0.50 | 1.00 | 0.6924 | 0.6884 (0.75) | 0.0040 | 0.029 | 1.00 / 0.00 / 1.00 |
|
||
| 5G-NIDD | 0.75 | 0.50 | 0.8091 | 0.8043 (1.00) | 0.0048 | 0.081 | 0.50 / 1.00 / 1.00 |
|
||
|
||
In **all six conditions the top-two gap is smaller than the selected beta's across-seed sample std**
|
||
(ratios 0.03–0.56), so each selection is a *small numerical separation relative to seed variability*.
|
||
No tie-break was ever exercised. Per-seed diagnostic betas are shown only for stability; they are not
|
||
the selection rule.
|
||
|
||
## 11. Prototype-Level Analysis
|
||
|
||
Direct aggregation effect, measured on fixed round-0 snapshots (`C_ij`, `N_ij` held constant, no
|
||
retraining). `artifacts/analysis/final__prototype_displacement_by_beta.csv`.
|
||
|
||
Mean `‖Cbar_j(beta) − Cbar_j(0)‖` across classes:
|
||
|
||
| Dataset | alpha | beta=0.25 | beta=0.50 | beta=0.75 | beta=1.00 |
|
||
|---|---|---|---|---|---|
|
||
| X-IIoTID | 0.25 | 1.476 | 2.403 | 3.003 | **3.422** |
|
||
| X-IIoTID | 0.50 | 0.826 | 1.374 | 1.780 | **2.099** |
|
||
| X-IIoTID | 0.75 | 0.477 | 0.829 | 1.097 | **1.305** |
|
||
| 5G-NIDD | 0.25 | 1.295 | 2.055 | 2.559 | **2.919** |
|
||
| 5G-NIDD | 0.50 | 0.953 | 1.567 | 2.003 | **2.340** |
|
||
| 5G-NIDD | 0.75 | 0.497 | 0.875 | 1.163 | **1.385** |
|
||
|
||
- Displacement is **monotone increasing in `beta`** for a fixed snapshot in all six conditions
|
||
(mechanically expected: larger `beta` shifts weight toward high-`N_ij` clients).
|
||
- Displacement at `beta = 1` **decreases as alpha rises**, tracking realized support imbalance.
|
||
- At matched alpha, X-IIoTID has larger displacement at a0.25 (3.422 vs 2.919) while 5G-NIDD is
|
||
slightly larger at a0.50 (2.340 vs 2.099) and a0.75 (1.385 vs 1.305); per-class ordering is
|
||
seed- and dataset-specific, and rare-class identities differ.
|
||
- **Displacement is a representation-aggregation metric, not detection quality.** No
|
||
displacement-to-accuracy inference is made.
|
||
|
||
Normalized contribution weights are stored per client-class in
|
||
`artifacts/analysis/*__prototype_weights.csv`; per-class displacement is in
|
||
`final__prototype_displacement_by_class.csv`.
|
||
|
||
## 12. Rare-Class Detection Results
|
||
|
||
Rare classes are defined from each client's **training** support (the two attack classes with the
|
||
smallest positive `N_ij`; one if only one; the client is excluded if none). Unseen classes
|
||
(`N_ij = 0`) are treated separately and never merged with rare classes.
|
||
|
||
Key observations from the test results (Section 9.2) and the synthesis:
|
||
|
||
- Selected rare-class macro-F1 deltas are **mixed in sign and small**; the largest positive
|
||
difference is within seed noise.
|
||
- Selected rare-class recall is **slightly lower than original in most conditions** while macro-F1
|
||
may be higher: weighting can trade a little rare-class recall for precision.
|
||
- At both `alpha = 0.25` conditions the selected `beta` is 0, so those conditions cannot demonstrate
|
||
any support-weighting benefit.
|
||
- Full rare-class curves and per-condition reports: `docs/research/FINAL_EXPERIMENTAL_SYNTHESIS.md`,
|
||
the per-condition `*_BETA_VALIDATION.md` reports, and `final__selected_vs_original.csv`.
|
||
|
||
## 13. Per-Client Analysis
|
||
|
||
Per-client effects are computed on each client's own evaluation partition relative to `beta = 0`
|
||
(`artifacts/analysis/final__client_effect_summary.csv`; detail in the per-condition
|
||
`*__multiseed__client_validation_*.csv`).
|
||
|
||
Improve / degrade / unchanged clients at the **selected** beta (validation, vs `beta=0`), by seed:
|
||
|
||
| Condition (selected beta) | seed 0 | seed 1 | seed 2 |
|
||
|---|---|---|---|
|
||
| X-IIoTID a0.25 (0.00) | 0 / 0 / 10 | 0 / 0 / 10 | 0 / 0 / 10 |
|
||
| X-IIoTID a0.50 (0.75) | 7 / 3 / 0 | 8 / 1 / 1 | 8 / 1 / 1 |
|
||
| X-IIoTID a0.75 (0.50) | 6 / 4 / 0 | 9 / 0 / 1 | 6 / 4 / 0 |
|
||
| 5G-NIDD a0.25 (0.00) | 0 / 0 / 10 | 0 / 0 / 10 | 0 / 0 / 10 |
|
||
| 5G-NIDD a0.50 (1.00) | 6 / 3 / 1 | 3 / 5 / 2 | 7 / 2 / 1 |
|
||
| 5G-NIDD a0.75 (0.50) | 4 / 6 / 0 | 6 / 3 / 1 | 4 / 2 / 4 |
|
||
|
||
- Weighting **redistributes benefit across clients**: within a condition some clients improve while
|
||
others degrade.
|
||
- At `beta = 1`, mean per-client delta is most negative at both `alpha = 0.25` conditions
|
||
(e.g. X-IIoTID a0.25 ≈ −0.060, 5G-NIDD a0.25 ≈ −0.024), consistent with those conditions
|
||
selecting `beta = 0`.
|
||
- `N_ij` versus per-client delta is **non-monotonic**: the worst-degrading client is not always the
|
||
lowest-support client.
|
||
- Client identities are per-seed realizations and are **not pooled** across seeds.
|
||
|
||
## 14. Cross-Alpha and Cross-Dataset Findings
|
||
|
||
- **Matched-alpha selections agree at alpha=0.25 (both 0.00) and alpha=0.75 (both 0.50)** and differ
|
||
at alpha=0.50 (X-IIoTID 0.75 vs 5G-NIDD 1.00). Across the six conditions the selected beta takes
|
||
values `{0.00, 0.50, 1.00}` — **not universal, not monotone in alpha, not a function of alpha
|
||
alone**.
|
||
- **Realized heterogeneity** (support CV, missing client-class cells, client-size ratio) decreases as
|
||
alpha rises within each dataset, but not strictly per seed. Dirichlet alpha is a design knob, not
|
||
the realized heterogeneity itself.
|
||
- Displacement at `beta = 1` falls with alpha on both datasets.
|
||
- **Rare-set composition differs by dataset**, so absolute metric levels are **not comparable across
|
||
datasets**; only within-condition differences are meaningful.
|
||
- **No cross-dataset causality, superiority, or dataset-independent beta law is claimed.**
|
||
|
||
Reports: `docs/research/CROSS_DATASET_BETA_ANALYSIS.md`, `XIIOTID_CROSS_ALPHA_ANALYSIS.md`,
|
||
`5GNIDD_CROSS_ALPHA_ANALYSIS.md`.
|
||
|
||
## 15. Confusion Matrix and Classification Breakdown
|
||
|
||
**Confusion matrices do exist in the saved artifacts.** They are written per run by the evaluation
|
||
code (`confusion_matrix` in `src/protean_research/evaluation/metrics.py`) and saved as:
|
||
|
||
- `results/<run>/test_confusion_matrix.json` — **108 / 108 runs**
|
||
- `results/<run>/val_confusion_matrix.json` — **107 / 108 runs** (missing only for
|
||
`xiiotid__alpha0.5__seed0__original_protean`, which also lacks the other validation artifacts)
|
||
|
||
**Properties of the stored matrices.**
|
||
|
||
| Property | Value |
|
||
|---|---|
|
||
| Shape | 9 × 9 (K = 9 classes) |
|
||
| Values | **raw integer counts** (not normalized) |
|
||
| Scope | **global** over the evaluation split, not per-client |
|
||
| Restriction | full multiclass; **not** restricted to rare classes |
|
||
| Split | separate `test_` and `val_` matrices |
|
||
| Class order | X-IIoTID: `C&C, Exfiltration, Exploitation, Lateral_movement, Normal, RDOS, Reconnaissance, Tampering, Weaponization`; 5G-NIDD: `Benign, HTTPFlood, ICMPFlood, SYNFlood, SYNScan, SlowrateDoS, TCPConnectScan, UDPFlood, UDPScan` (also in each run's `metadata.json`) |
|
||
| Aggregation | each client's eval partition concatenated; no seed/client mixing |
|
||
|
||
Representative interpretation — 5G-NIDD, `alpha=0.75`, `seed=0`, **test**, original vs selected
|
||
(`beta=0.50`) — using actual counts:
|
||
|
||
| Class | eval support | original recall | selected-beta recall |
|
||
|---|---|---|---|
|
||
| Benign | 95,547 | 0.3795 | 0.3825 |
|
||
| HTTPFlood | 28,162 | 0.9729 | 0.9923 |
|
||
| ICMPFlood | 231 | 1.0000 | 1.0000 |
|
||
| SYNFlood | 1,944 | 0.8570 | 0.8570 |
|
||
| SYNScan | 4,009 | 0.9968 | 0.9968 |
|
||
| SlowrateDoS | 14,625 | 0.9217 | 0.9054 |
|
||
| TCPConnectScan | 4,010 | 0.9948 | 0.9948 |
|
||
| UDPFlood | 91,468 | 1.0000 | 1.0000 |
|
||
| UDPScan | 3,181 | 0.9975 | 0.9981 |
|
||
|
||
Largest off-diagonal cells (true -> predicted):
|
||
|
||
| true | predicted | original count | selected-beta count |
|
||
|---|---|---|---|
|
||
| Benign | UDPFlood | 56,899 | 57,054 |
|
||
| Benign | TCPConnectScan | 2,015 | 1,495 |
|
||
| SlowrateDoS | HTTPFlood | 1,079 | 1,308 |
|
||
| HTTPFlood | SlowrateDoS | 759 | 212 |
|
||
| SYNFlood | TCPConnectScan | 278 | 278 |
|
||
|
||
What these counts support:
|
||
|
||
- The dominant error mode is **Benign misclassified as UDPFlood** (about 57k samples), which
|
||
dominates the off-diagonal mass. It is essentially unchanged by `beta`, so `beta` does not fix
|
||
this confusion. This is visible in low Benign recall (≈0.38) with very high UDPFlood recall
|
||
(1.0000), i.e. UDPFlood is an attractor for Benign.
|
||
- The change from original to selected `beta` shifts a few specific off-diagonal cells
|
||
(`Benign → TCPConnectScan` decreases; `SlowrateDoS → HTTPFlood` increases), consistent with the
|
||
small, mixed-sign aggregate deltas, **not** with a systematic improvement.
|
||
- Diagonal versus off-diagonal entries must be read together: high diagonal on the rare/scan classes
|
||
coexists with the large Benign confusion, which is why accuracy is moderate while macro-F1 is
|
||
higher.
|
||
|
||
Reconstruction note: because only aggregate per-class metrics (precision/recall/F1/support) are
|
||
persisted, a confusion matrix cannot be uniquely reconstructed from them; the raw `y_true`/`y_pred`
|
||
arrays are not saved. Reconstruction is unnecessary here because the matrices themselves are already
|
||
persisted as JSON.
|
||
|
||
Representative confusion-matrix files:
|
||
|
||
- `results/5gnidd__alpha0.75__seed0__original_protean/test_confusion_matrix.json`
|
||
- `results/5gnidd__alpha0.75__seed0__support_weighted__beta0.5/test_confusion_matrix.json`
|
||
- `results/xiiotid__alpha0.25__seed0__original_protean/test_confusion_matrix.json`
|
||
|
||
## 16. Final Figures and Visualizations
|
||
|
||
All figures live in `artifacts/figures/final/` and are regenerated from the audited
|
||
`artifacts/analysis/final__*` tables by `scripts/make_final_figures.py` (no training; every number is
|
||
loaded from an artifact). Each figure is saved as a high-resolution PNG (220 dpi) and a vector PDF.
|
||
|
||
**Naming and legend conventions used throughout.**
|
||
|
||
- **Charcoal (`original_protean`)**: the frozen reproduced baseline.
|
||
- **Teal (`support_weighted`, validation-selected beta)**: the proposed method at the beta selected by
|
||
pooled validation for that condition.
|
||
- **Gray (`beta = 0` control)**: the equal-weight `support_weighted` run; a control, *not*
|
||
automatically the same recorded configuration as `original_protean`.
|
||
- Alpha conditions are distinguished by colour **and** marker **and** line style, so colour is never
|
||
the only cue.
|
||
|
||
**Comparison types (so figures are not misread).**
|
||
|
||
| Figure | Comparison type | Baseline actually plotted |
|
||
|---|---|---|
|
||
| A (both) | Beta candidate comparison within `support_weighted` | none (support_weighted candidates only) |
|
||
| B | Beta selection surface | none (selection, not performance) |
|
||
| C | Dataset heterogeneity | none (data distribution) |
|
||
| D, E | Fixed-snapshot comparison vs `beta = 0` | `beta = 0` control |
|
||
| F | Direct comparison vs `original_protean` | `original_protean` |
|
||
| G | Per-client comparison vs `beta = 0` | `beta = 0` control |
|
||
| H | Paired `original_protean` vs selected beta | `original_protean` |
|
||
|
||
### Fig A1 / A2 — Validation-based beta selection (support_weighted)
|
||
|
||
**Compared:** the five `support_weighted` beta candidates, separately for each alpha. **Level/split:**
|
||
pooled validation, per-client mean across seeds `{0,1,2}`. **Metric:** rare-class macro-F1
|
||
(per-client mean). **Observation:** the validation response is non-monotone and the selected beta
|
||
differs by condition. **Limits:** this is a selection figure; no `original_protean` run is plotted,
|
||
and the error bars are ±1 sample std across three project-defined seeds, not a confidence interval.
|
||
|
||
|
||
|
||
|
||
|
||
### Fig B — Validation-selected beta by dataset and alpha
|
||
|
||
**Compared:** none (selection only). **Level:** pooled validation. **Metric:** selected beta under
|
||
the selection rule. **Observation:** selections take `{0.00, 0.50, 1.00}` and are not universal.
|
||
**Limits:** selection surface, not a performance comparison; a higher beta is not automatically
|
||
better. No `original_protean` comparison is implied.
|
||
|
||
|
||
|
||
### Fig C — Realized class-support imbalance
|
||
|
||
**Compared:** none (data distribution). **Level:** training partitions. **Metric:** mean class-support
|
||
CV of `N_ij` across contributing clients (std/mean), averaged over classes and seeds. **Observation:**
|
||
realized imbalance decreases as the configured alpha rises. **Limits:** this describes the data,
|
||
not the relative performance of original vs modified PROTEAN; Dirichlet alpha is the configuration
|
||
knob and is not the same as the realized imbalance.
|
||
|
||
|
||
|
||
### Fig D — Support-weighted aggregation: beta=1 vs beta=0 control
|
||
|
||
**Compared:** `beta = 1` against the `beta = 0` control (not `original_protean`). **Level:**
|
||
fixed round-0 snapshots, same local prototypes and `N_ij`, no retraining. **Metric:** mean
|
||
`‖Cbar_j(1) − Cbar_j(0)‖` across classes. **Observation:** displacement increases with beta and is
|
||
largest where realized imbalance is largest. **Limits:** displacement is a representation metric,
|
||
not detection quality.
|
||
|
||
|
||
|
||
### Fig E — Realized support CV vs prototype displacement
|
||
|
||
**Compared:** `beta = 1` displacement against the `beta = 0` control, across conditions. **Level:**
|
||
fixed snapshots. **Metric:** realized support CV (x) vs `beta = 1` displacement (y). **Observation:**
|
||
larger realized imbalance accompanies larger displacement. **Limits:** descriptive association only;
|
||
the imbalance-displacement link is mechanical under the beta formula and is not causal evidence.
|
||
|
||
|
||
|
||
### Fig F — Selected-beta PROTEAN vs ORIGINAL PROTEAN (test)
|
||
|
||
**Compared:** `original_protean` (charcoal) against `support_weighted` at the validation-selected
|
||
beta (teal), per condition. **Level/split:** test, mean over seeds `{0,1,2}`. **Metric:** test
|
||
rare-class macro-F1; error bars = ±1 sample std across seeds. **Observation:** signed deltas are
|
||
`+0.0230`, `+0.0014`, `+0.0168` (positive), `−0.0086` (negative), and `−0.0007`/`−0.0010` at the two
|
||
`beta = 0` conditions (floating-point ~0). **Limits:** no statistical significance test (three
|
||
seeds); the largest gain is within seed noise; the weighting does not consistently outperform
|
||
`original_protean`.
|
||
|
||
|
||
|
||
### Fig G — Per-client effect of selected beta relative to the beta=0 control
|
||
|
||
**Compared:** selected-beta `support_weighted` against the `beta = 0` control (not
|
||
`original_protean`). **Level/split:** validation, per client, per seed. **Metric:** per-client
|
||
rare-class macro-F1 delta; a client is "improved"/"degraded" if its delta is above/below 0,
|
||
"unchanged" otherwise. **Observation:** within a condition some clients improve and others degrade;
|
||
at both `alpha = 0.25` conditions the selected beta is 0, so all clients are unchanged. **Limits:**
|
||
client ids are per-seed realizations and are not pooled.
|
||
|
||
|
||
|
||
### Fig H (supplemental) — Paired test confusion matrices
|
||
|
||
**Compared:** `original_protean` (left) against selected-beta `support_weighted` (right) for a valid
|
||
paired condition (5G-NIDD, `alpha = 0.75`, `seed = 0`). **Level/split:** test; global counts over the
|
||
evaluation split. **Metric:** raw 9×9 counts (log colour scale), identical class order and colour
|
||
scale. **Observation:** the dominant off-diagonal error is `Benign → UDPFlood` (~56.9k vs ~57.1k) and
|
||
is essentially unchanged by beta. **Limits:** no statistical significance is inferred from the visual
|
||
difference; the matrices come from the same manifest, split, seed, and class order.
|
||
|
||
|
||
|
||
|
||
## 17. Main Scientific Findings
|
||
|
||
1. **Changing `beta` changes the global prototype.** Contribution weights become proportional to
|
||
support, and for a fixed snapshot mean prototype displacement grows monotonically with `beta` in
|
||
every condition.
|
||
2. **The rare-class response is condition-dependent and non-monotone in `beta`.** The direct
|
||
aggregation effect (geometry) and the end-to-end training effect (detection) are distinct.
|
||
3. **No universal beta was found.** The validation-selected `beta` differs across datasets and
|
||
conditions and is not monotone in `alpha`. Every selection is weak relative to seed variability.
|
||
4. **Selected-vs-original differences are small and mixed in sign.** No statistical superiority is
|
||
established; the largest positive difference is within seed noise.
|
||
5. **Weighting redistributes benefit across clients.** Within a condition some clients improve while
|
||
others degrade; the effect is not simply "more support is better".
|
||
6. **Where weighting does not help:** both `alpha = 0.25` conditions select `beta = 0` (equal
|
||
weighting), and at `beta = 1` the per-client effect is most damaging there.
|
||
7. **Realized support imbalance and prototype displacement co-vary** descriptively; displacement is
|
||
not a proxy for detection quality.
|
||
8. **The dominant classification error (Benign → UDPFlood) is not addressed by `beta`.**
|
||
|
||
## 18. Limitations and Novelty Boundary
|
||
|
||
**Limitations.**
|
||
|
||
- The `S_j` participant/denominator rule is a **documented reproduction assumption** (no verified
|
||
official PROTEAN code). It is material in up to 19/27 class-condition cells; all results are
|
||
conditional on it.
|
||
- `beta = 0` is only arithmetically the equal mean over `S_j`; it is **not bitwise identical** to
|
||
`original_protean` (floating-point divergence; equivalence class B in every condition).
|
||
- Two datasets, three alpha values, three project-defined seeds, 10 clients, one architecture.
|
||
- The `beta` grid is discrete; intermediate values were not tested.
|
||
- Three seeds are underpowered for inferential testing; no significance test is claimed.
|
||
- Prototype displacement is a representation metric, not detection quality.
|
||
- One validation artifact set is missing (Section 7); rare-class and unseen-class results cannot be
|
||
merged, and cross-dataset absolute metric levels are not comparable.
|
||
|
||
**Novelty boundary.** Support-weighted prototype aggregation is not new (e.g. FedProto). The
|
||
contribution is the controlled `beta` parameterization inside PROTEAN, the validation-only
|
||
condition-wise selection, the paired multi-seed design, and the behavioral findings. No claim is
|
||
made that the method universally improves PROTEAN.
|
||
|
||
## 19. Repository Structure
|
||
|
||
```
|
||
src/protean_research/ data loading/preprocessing/partitioning, models, aggregation, federated
|
||
round logic, evaluation metrics, analysis, utils
|
||
scripts/ training runner, partition generation, and read-only analysis scripts
|
||
configs/ dataset, experiment protocol, and model configuration (YAML)
|
||
docs/proposal/ thesis proposal and internal audit aids
|
||
docs/research/ frozen protocol, per-condition reports, cross-alpha/cross-dataset synthesis,
|
||
session audits
|
||
docs/baseline/ PROTEAN baseline investigation, implementation, and validation
|
||
docs/thesis/ Chapter 4 blueprint and claims register
|
||
docs/handoff/ session handoff (start here for a new session)
|
||
planningawal/ immutable research specification (do not modify)
|
||
artifacts/partitions/ deterministic frozen Dirichlet partition manifests (18)
|
||
artifacts/analysis/ committed machine-readable analysis outputs
|
||
artifacts/figures/final/ thesis-ready figures
|
||
results/ local run outputs (git-ignored; 108 non-smoke run dirs + 1 smoke)
|
||
```
|
||
|
||
## 20. Reproduction and Tests
|
||
|
||
```bash
|
||
# unit + integration tests (expected: 75 passed)
|
||
.venv/bin/python -m pytest
|
||
|
||
# read-only pooled validation analysis (no retraining)
|
||
.venv/bin/python scripts/analyze_multiseed_beta.py --dataset 5gnidd --alpha 0.25 --seeds 0 1 2
|
||
|
||
# read-only realized-partition characterization
|
||
.venv/bin/python scripts/analyze_partition_heterogeneity.py --dataset 5gnidd --alphas 0.25 0.5 0.75
|
||
|
||
# read-only cross-alpha synthesis
|
||
.venv/bin/python scripts/analyze_cross_alpha.py --dataset 5gnidd --alphas 0.25 0.5 0.75
|
||
|
||
# beta=0 vs original equivalence audit (read-only)
|
||
.venv/bin/python scripts/verify_beta0_equivalence.py --dataset 5gnidd --alpha 0.25 --seeds 0 1 2
|
||
```
|
||
|
||
A full training run (not needed; the grid is complete) is:
|
||
|
||
```bash
|
||
.venv/bin/python scripts/run_baseline.py --dataset 5gnidd --alpha 0.25 --seed 1 \
|
||
--clients 10 --rounds 10 --local-epochs 3 \
|
||
--aggregation-mode support_weighted --beta 1.0
|
||
```
|
||
|
||
Runtime context: the 108 runs consumed ≈ 94.5 wall-clock hours on CPU (8 cores); per-run wall clock
|
||
was ≈ 62–65 minutes for 5G-NIDD and ≈ 40 minutes for X-IIoTID (`*__multiseed__runtime.csv`). Do not
|
||
rerun the grid.
|
||
|
||
**Repository safety.**
|
||
|
||
- `planningawal/` is immutable; raw datasets are external and read-only.
|
||
- Do not modify or regenerate completed results, manifests, or figures.
|
||
- Do not add an `equal_available` mode; it is arithmetically `support_weighted` at `beta = 0` under
|
||
the shared `S_j`.
|
||
- `beta` selection uses validation only; the test set is never used for selection.
|
||
|
||
## 21. Complete Artifact Index
|
||
|
||
Per-run outputs (`results/<dataset>__alpha<alpha>__seed<seed>__<config>/`):
|
||
|
||
| File | Content |
|
||
|---|---|
|
||
| `metadata.json` | run config, dataset/alpha/seed, mode, class names, hashes |
|
||
| `history.json` | per-round global accuracy/macro-F1/rare-class metrics |
|
||
| `characterization.csv` | per-class training support statistics (`N_ij`) |
|
||
| `train_support_Nij.csv` | `N_ij` matrix (client × class) |
|
||
| `rare_unseen.csv` | per client-class rare/unseen flags from training support |
|
||
| `test_global_metrics.json`, `val_global_metrics.json` | global metrics per split |
|
||
| `test_per_class.csv`, `val_per_class.csv` | per-class precision/recall/F1/support |
|
||
| `per_client.csv`, `val_per_client.csv` | per-client metrics |
|
||
| `test_confusion_matrix.json`, `val_confusion_matrix.json` | 9×9 raw-count confusion matrices |
|
||
| `val_selection.json` | validation selection metric and `test_used_for_selection` flag |
|
||
|
||
For the one incomplete validation run, see Section 7.
|
||
|
||
Analysis outputs (`artifacts/analysis/`), by function:
|
||
|
||
| Category | Representative files |
|
||
|---|---|
|
||
| Final condition summary | `final__six_condition_master.csv/.json` |
|
||
| Beta-selection curves | `final__pooled_validation_curves.csv`, `final__selection_strength.csv` |
|
||
| Selected vs original | `final__selected_vs_original.csv` |
|
||
| Realized heterogeneity | `final__realized_heterogeneity.csv`, `*__partition_heterogeneity_*.csv` |
|
||
| Prototype displacement | `final__prototype_displacement_by_beta.csv`, `..._by_class.csv` |
|
||
| Per-client effects | `final__client_effect_summary.csv`, `*__multiseed__client_validation_*.csv` |
|
||
| Per-condition validation | `*__multiseed__validation_per_seed.csv`, `..._validation_pooled.csv` |
|
||
| Per-condition test | `*__multiseed__test_descriptive.csv` |
|
||
| Cross-alpha | `{xiiotid,5gnidd}__cross_alpha__*.csv/.json` |
|
||
| Cross-dataset | `cross_dataset__matched_alpha_summary.csv/.json`, `cross_dataset__beta_curve.csv` |
|
||
| beta=0 equivalence | `*__multiseed__beta0_equivalence.csv` |
|
||
| Runtime | `*__multiseed__runtime.csv` |
|
||
| Prototype snapshots & weights | `*__prototype_snapshot.json`, `*__prototype_weights.csv`, `*__prototype_effect*.csv` |
|
||
|
||
Partition manifests: `artifacts/partitions/<dataset>__alpha<alpha>__seed<seed>.json` (18 files).
|
||
Figures: `artifacts/figures/final/` — 9 figures (figA–figH) as PNG (220 dpi) plus matching vector PDF;
|
||
regenerated reproducibly by `scripts/make_final_figures.py`.
|
||
|
||
Key documents:
|
||
|
||
| Document | Purpose |
|
||
|---|---|
|
||
| `docs/research/EXPERIMENT_PROTOCOL.md` | frozen reproduction protocol |
|
||
| `docs/research/FINAL_EXPERIMENTAL_SYNTHESIS.md` | authoritative six-condition synthesis |
|
||
| `docs/research/CROSS_DATASET_BETA_ANALYSIS.md` | cross-dataset comparison |
|
||
| `docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md` | X-IIoTID cross-alpha analysis |
|
||
| `docs/research/5GNIDD_CROSS_ALPHA_ANALYSIS.md` | 5G-NIDD cross-alpha analysis |
|
||
| `docs/thesis/CHAPTER4_RESULTS_OUTLINE.md` | Chapter 4 blueprint |
|
||
| `docs/thesis/CLAIMS_REGISTER.md` | claim / evidence / status register |
|
||
| `docs/proposal/THESIS_PROPOSAL_DRAFT.md` | final proposal Markdown source |
|
||
| `docs/proposal/PROPOSAL_SOURCE_MAP.md` | claim-to-evidence map |
|
||
| `docs/proposal/LITERATURE_MATRIX.md` | literature matrix |
|
||
| `docs/proposal/REFERENCES_VERIFIED.md` | bibliography verification record |
|
||
| `docs/handoff/SESSION_HANDOFF.md` | current state and next steps |
|
||
| `planningawal/PROTEAN_Revisi_Lurus.md` | immutable research specification |
|
||
|
||
## 22. References
|
||
|
||
- Chennoufi, S., Han, Y., Blanc, G., De Cristofaro, E., & Kiennert, C. (2025). PROTEAN: Federated
|
||
Intrusion Detection in Non-IID Environments through Prototype-Based Knowledge Sharing. ESORICS
|
||
2025, pp. 103–125. DOI: 10.1007/978-3-032-07884-1_6. arXiv:2507.05524.
|
||
- Tan, Y., Long, G., Liu, L., Zhou, T., Lu, Q., Jiang, J., & Zhang, C. (2022). FedProto: Federated
|
||
Prototype Learning across Heterogeneous Clients. AAAI 36(8), 8432–8440.
|
||
DOI: 10.1609/aaai.v36i8.20819. arXiv:2105.00243.
|
||
|
||
The complete, verified bibliography (including the FL foundations, non-IID, class-imbalance, and
|
||
evaluation-metric references used in the proposal) is maintained in
|
||
`docs/proposal/REFERENCES_VERIFIED.md` and the proposal bibliography. This README cites only the two
|
||
works directly relevant to the mechanism and novelty boundary.
|