# PROTEAN Prototype Aggregation Study

Research code and audited artifacts for a master's thesis on **server-side prototype aggregation in
PROTEAN** (Chennoufi et al., 2025), a federated intrusion-detection framework, under non-IID client
data. The study investigates a **beta-controlled, class-specific sample-support weighting** of local
class prototypes for **rare attack detection**.

The experimental study is **complete** (108 full runs, six non-IID conditions, both datasets) and the
thesis proposal Markdown is **finalized**. No further training is planned unless new academic
feedback explicitly requires it.

## Final Research Title

**Optimasi Agregasi Prototype pada PROTEAN melalui Pembobotan Berbasis Dukungan Sampel Kelas untuk
Deteksi Serangan Langka pada Data Non-IID**

*Optimizing Prototype Aggregation in PROTEAN through Class-Sample-Support Weighting for Rare Attack
Detection under Non-IID Data Distributions.*

Here **"Optimasi" means empirical tuning/selection of `beta` among the predefined candidate values**,
not a global mathematical optimum and not a universal beta. The selected value is the best among the
tested candidates under the defined validation procedure for each condition.

## Research Scope

- **Base method:** PROTEAN, a federated intrusion-detection framework for non-IID data.
- **Focus:** the **server-side prototype aggregation** step only. Client training, model
  architecture, loss, and inference are unchanged.
- **Proposed mechanism:** weight each contributing client's class prototype by `N_ij^beta`, where
  `N_ij` is that client's class-specific training support and `beta` is a single tunable exponent
  in `[0, 1]`.
- **Analysis chain:** support imbalance -> contribution weight -> prototype displacement ->
  rare-class detection -> per-client effect.

`N_ij` is **class-specific sample support, not prototype quality**. A client with few samples is not
assumed to have a worse prototype; low-support clients may carry variation unavailable at
high-support clients.

## Why This Problem Matters

Federated IDS data is naturally non-IID: each client observes different traffic and different attack
types. Rare attack classes appear in little of the data, so aggregate metrics can look good while
the model fails on them. PROTEAN builds a global prototype per class on the server and uses it both
for local prototype alignment and for nearest-prototype inference. Its reference aggregation gives
every contributing client equal weight, even though the local prototypes may be built from very
different numbers of samples. This study asks whether, and under which realized non-IID conditions,
weighting contributions by class sample support changes the global prototype and rare-class
detection behavior.

## Proposed Aggregation

```
Cbar_j(beta) = ( sum_{i in S_j} N_ij^beta * C_ij ) / ( sum_{i in S_j} N_ij^beta )
S_j = { i | N_ij > 0 and a valid local prototype C_ij exists }
beta in {0.00, 0.25, 0.50, 0.75, 1.00}
```

- `beta = 0`: equal weighting over the contributors `S_j`.
- `beta = 1`: contribution proportional to class-specific support.
- `0 < beta < 1`: intermediate weighting.

`S_j` (participant set and denominator rule) is a **documented reproduction assumption**: no verified
official PROTEAN code is available. `beta = 0` is semantically the equal mean over `S_j`, but is not
guaranteed to be bitwise identical to the frozen `original_protean` baseline because of
floating-point operation ordering.

## Experimental Setup

| Item | Value |
|---|---|
| Datasets | X-IIoTID (`class2`, benign `Normal`, K=9); 5G-NIDD (`Attack Type`, benign `Benign`, K=9) |
| Non-IID conditions | symmetric Dirichlet `alpha` = 0.75, 0.50, 0.25 |
| Seeds | 0, 1, 2 (project-defined) |
| Clients / rounds / local epochs | 10 / 10 / 3 |
| Batch size / optimizer / learning rate | 256 / Adam / 1e-3 |
| FedProx `mu` / prototype-alignment `lambda` | 0.1 / 1.0 |
| Compared configurations | `original_protean` + `support_weighted` at beta 0, 0.25, 0.50, 0.75, 1.00 |
| Splits | stratified train / validation / test = 70 / 10 / 20 |
| Selection | pooled validation per-client-mean rare-class macro-F1 across seeds |
| Test | read only after the selected beta is fixed |

Each `(dataset, alpha, seed)` uses one deterministic frozen partition; all compared configurations
within it share the same manifest, split, class mapping, and initialization.

## Experiment Status

- **Complete: 108 full runs** = 2 datasets × 3 alpha conditions × 3 seeds ×
  (1 original PROTEAN + 5 beta settings).
- All conditions are trained, audited, and synthesized.
- Smoke runs are excluded and are not scientific experiments.
- **No further training planned** unless new academic feedback requires it.

## Final Scientific Position

High-level, evidence-safe conclusions only (full tables live in the synthesis documents):

- Realized support imbalance, missing client-class cells, and client-size ratio vary by condition;
  Dirichlet `alpha` is a design knob, not the realized heterogeneity itself.
- Increasing `beta` changes the normalized contribution weights and therefore the position of each
  global prototype; for a fixed snapshot, displacement grows with `beta`.
- The rare-class detection response is **condition-dependent and non-monotone** in `beta`; the direct
  aggregation effect and the end-to-end training effect are distinct.
- **No universal beta exists.** The validation-selected beta differs across datasets and non-IID
  conditions and is not monotone in `alpha`.
- Stronger support weighting can **help, have limited effect, or hurt** rare-class behavior,
  depending on the condition, and can redistribute benefit across clients within one condition.

Not supported by the evidence (must not be claimed): support weighting always improves rare-class
detection; the selected beta universally beats `original_protean`; `alpha` causally determines beta;
`N_ij` measures prototype quality; larger prototype displacement means better detection; a global
optimum exists; results generalize beyond the two evaluated datasets.

## Repository Structure

```
src/protean_research/   data, models, aggregation, federated, evaluation, utils
scripts/                training, partition, and analysis scripts
configs/                datasets, experiments, model configuration
docs/proposal/          thesis proposal and its internal audit aids
docs/research/          frozen protocol, per-condition reports, cross-alpha / cross-dataset synthesis
docs/thesis/            Chapter 4 blueprint and claims register
docs/baseline/          PROTEAN baseline investigation and validation
docs/handoff/           session handoff (start here for a new session)
planningawal/           immutable research specification (do not modify)
artifacts/analysis/     committed machine-readable analysis outputs
artifacts/figures/final/ thesis-ready figures
artifacts/partitions/   deterministic frozen Dirichlet partition manifests
results/                local run outputs (git-ignored, intentionally not committed)
```

## Key Scripts

| Script | Role |
|---|---|
| `scripts/run_baseline.py` | full baseline / support-weighted training runner (frozen; grid complete) |
| `scripts/generate_partitions.py` | deterministic frozen Dirichlet manifests |
| `scripts/analyze_multiseed_beta.py` | pooled validation beta selection |
| `scripts/analyze_partition_heterogeneity.py` | realized non-IID characterization |
| `scripts/analyze_prototype_effect.py` | fixed-snapshot prototype displacement |
| `scripts/analyze_cross_alpha.py` | cross-alpha synthesis |
| `scripts/verify_beta0_equivalence.py` | beta=0 vs original semantic/numerical audit |
| `scripts/validate_sj_participant_rule.py` | `S_j` reproduction-assumption analysis |

## Important Project Documents

**Proposal**

- `docs/proposal/THESIS_PROPOSAL_DRAFT.md` — final Markdown proposal source for the current scope
- `docs/proposal/PROPOSAL_SOURCE_MAP.md` — claim-to-evidence map (internal)
- `docs/proposal/LITERATURE_MATRIX.md` — literature matrix (internal)
- `docs/proposal/REFERENCES_VERIFIED.md` — bibliography verification record (internal)

**Research synthesis**

- `docs/research/FINAL_EXPERIMENTAL_SYNTHESIS.md` — authoritative six-condition synthesis
- `docs/research/CROSS_DATASET_BETA_ANALYSIS.md` — cross-dataset comparison
- `docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md` — X-IIoTID cross-alpha analysis
- `docs/research/5GNIDD_CROSS_ALPHA_ANALYSIS.md` — 5G-NIDD cross-alpha analysis
- `docs/research/EXPERIMENT_PROTOCOL.md` — frozen reproduction protocol

**Thesis support**

- `docs/thesis/CHAPTER4_RESULTS_OUTLINE.md` — Chapter 4 blueprint
- `docs/thesis/CLAIMS_REGISTER.md` — claim / evidence / status register

**Handoff and planning**

- `docs/handoff/SESSION_HANDOFF.md` — current state and next steps (start here)
- `planningawal/PROTEAN_Revisi_Lurus.md` — immutable research specification

> `planningawal/` is **immutable**. Read it, cite it, but never edit, rename, move, or add files to
> it.

## Testing and Reproducibility

```bash
# unit + integration tests (expected: 75 passed)
.venv/bin/python -m pytest
```

- **Paired comparison:** within each `(dataset, alpha, seed)`, every compared configuration reuses
  the same partition, split, class mapping, and model initialization.
- **Frozen manifests:** partitions are generated once and reused; regenerating a fresh Dirichlet
  split is a protocol violation.
- **Immutable raw data:** datasets live outside the repository and are never modified or committed.
- **Validation-only selection:** beta is chosen from validation data only; test data is never used
  for any model-selection decision.
- **Machine-readable outputs:** analysis is stored as CSV/JSON under `artifacts/analysis/`, and the
  final figures load from those artifacts.

## Frozen / Do Not Modify

Treat the following as finished unless there is an explicit, documented reason:

- `planningawal/` — immutable research source.
- Raw datasets under `~/datasets/` — read-only, external, never copied into the repository.
- Completed scientific results and the final synthesis documents.
- Frozen experiment artifacts: partition manifests, analysis outputs, and final figures.
- `src/`, model, and aggregation code used for the 108-run grid.
- The finalized proposal Markdown.

## Limitations

- The `S_j` participant/denominator rule remains a reproduction assumption and is materially
  exercised in a substantial fraction of class-condition cells.
- Two datasets, three alpha values, three project-defined seeds, 10 clients, one architecture.
- The beta grid is discrete; intermediate values were not tested.
- No inferential trend testing (three seeds are underpowered for it).
- `beta = 0` is not bitwise identical to `original_protean` (floating-point ordering).
- Prototype displacement is a representation metric, not detection quality.

## Quickstart

```bash
# unit + integration tests (expected: 75 passed)
.venv/bin/python -m pytest

# read-only pooled validation analysis (no retraining required)
.venv/bin/python scripts/analyze_multiseed_beta.py --dataset 5gnidd --alpha 0.25 --seeds 0 1 2

# read-only cross-alpha / cross-dataset synthesis
.venv/bin/python scripts/analyze_partition_heterogeneity.py --dataset 5gnidd --alphas 0.25 0.5 0.75
.venv/bin/python scripts/analyze_cross_alpha.py --dataset 5gnidd --alphas 0.25 0.5 0.75
```

See `docs/handoff/SESSION_HANDOFF.md` before making any methodological change. The next work is
academic writing and formatting, not new experiments.