thesis-prototype-agregation/README.md

PROTEAN Prototype Aggregation Study

Research code and audited artifacts for a master's thesis on server-side prototype aggregation in PROTEAN (Chennoufi et al., 2025), a federated intrusion-detection framework, under non-IID client data. The study investigates a beta-controlled, class-specific sample-support weighting of local class prototypes for rare attack detection.

The experimental study is complete (108 full runs, six non-IID conditions, both datasets) and the thesis proposal Markdown is finalized. No further training is planned unless new academic feedback explicitly requires it.

Final Research Title

Optimasi Agregasi Prototype pada PROTEAN melalui Pembobotan Berbasis Dukungan Sampel Kelas untuk Deteksi Serangan Langka pada Data Non-IID

Optimizing Prototype Aggregation in PROTEAN through Class-Sample-Support Weighting for Rare Attack Detection under Non-IID Data Distributions.

Here "Optimasi" means empirical tuning/selection of beta among the predefined candidate values, not a global mathematical optimum and not a universal beta. The selected value is the best among the tested candidates under the defined validation procedure for each condition.

Research Scope

• Base method: PROTEAN, a federated intrusion-detection framework for non-IID data.
• Focus: the server-side prototype aggregation step only. Client training, model architecture, loss, and inference are unchanged.
• Proposed mechanism: weight each contributing client's class prototype by N_ij^beta, where N_ij is that client's class-specific training support and beta is a single tunable exponent in [0, 1].
• Analysis chain: support imbalance -> contribution weight -> prototype displacement -> rare-class detection -> per-client effect.

N_ij is class-specific sample support, not prototype quality. A client with few samples is not assumed to have a worse prototype; low-support clients may carry variation unavailable at high-support clients.

Why This Problem Matters

Federated IDS data is naturally non-IID: each client observes different traffic and different attack types. Rare attack classes appear in little of the data, so aggregate metrics can look good while the model fails on them. PROTEAN builds a global prototype per class on the server and uses it both for local prototype alignment and for nearest-prototype inference. Its reference aggregation gives every contributing client equal weight, even though the local prototypes may be built from very different numbers of samples. This study asks whether, and under which realized non-IID conditions, weighting contributions by class sample support changes the global prototype and rare-class detection behavior.

Proposed Aggregation

Cbar_j(beta) = ( sum_{i in S_j} N_ij^beta * C_ij ) / ( sum_{i in S_j} N_ij^beta )
S_j = { i | N_ij > 0 and a valid local prototype C_ij exists }
beta in {0.00, 0.25, 0.50, 0.75, 1.00}

• beta = 0: equal weighting over the contributors S_j.
• beta = 1: contribution proportional to class-specific support.
• 0 < beta < 1: intermediate weighting.

S_j (participant set and denominator rule) is a documented reproduction assumption: no verified official PROTEAN code is available. beta = 0 is semantically the equal mean over S_j, but is not guaranteed to be bitwise identical to the frozen original_protean baseline because of floating-point operation ordering.

Experimental Setup

Item	Value
Datasets	X-IIoTID (class2, benign Normal, K=9); 5G-NIDD (Attack Type, benign Benign, K=9)
Non-IID conditions	symmetric Dirichlet alpha = 0.75, 0.50, 0.25
Seeds	0, 1, 2 (project-defined)
Clients / rounds / local epochs	10 / 10 / 3
Batch size / optimizer / learning rate	256 / Adam / 1e-3
FedProx mu / prototype-alignment lambda	0.1 / 1.0
Compared configurations	original_protean + support_weighted at beta 0, 0.25, 0.50, 0.75, 1.00
Splits	stratified train / validation / test = 70 / 10 / 20
Selection	pooled validation per-client-mean rare-class macro-F1 across seeds
Test	read only after the selected beta is fixed

Each (dataset, alpha, seed) uses one deterministic frozen partition; all compared configurations within it share the same manifest, split, class mapping, and initialization.

Experiment Status

• Complete: 108 full runs = 2 datasets × 3 alpha conditions × 3 seeds × (1 original PROTEAN + 5 beta settings).
• All conditions are trained, audited, and synthesized.
• Smoke runs are excluded and are not scientific experiments.
• No further training planned unless new academic feedback requires it.

Final Scientific Position

High-level, evidence-safe conclusions only (full tables live in the synthesis documents):

• Realized support imbalance, missing client-class cells, and client-size ratio vary by condition; Dirichlet alpha is a design knob, not the realized heterogeneity itself.
• Increasing beta changes the normalized contribution weights and therefore the position of each global prototype; for a fixed snapshot, displacement grows with beta.
• The rare-class detection response is condition-dependent and non-monotone in beta; the direct aggregation effect and the end-to-end training effect are distinct.
• No universal beta exists. The validation-selected beta differs across datasets and non-IID conditions and is not monotone in alpha.
• Stronger support weighting can help, have limited effect, or hurt rare-class behavior, depending on the condition, and can redistribute benefit across clients within one condition.

Not supported by the evidence (must not be claimed): support weighting always improves rare-class detection; the selected beta universally beats original_protean; alpha causally determines beta; N_ij measures prototype quality; larger prototype displacement means better detection; a global optimum exists; results generalize beyond the two evaluated datasets.

Repository Structure

src/protean_research/   data, models, aggregation, federated, evaluation, utils
scripts/                training, partition, and analysis scripts
configs/                datasets, experiments, model configuration
docs/proposal/          thesis proposal and its internal audit aids
docs/research/          frozen protocol, per-condition reports, cross-alpha / cross-dataset synthesis
docs/thesis/            Chapter 4 blueprint and claims register
docs/baseline/          PROTEAN baseline investigation and validation
docs/handoff/           session handoff (start here for a new session)
planningawal/           immutable research specification (do not modify)
artifacts/analysis/     committed machine-readable analysis outputs
artifacts/figures/final/ thesis-ready figures
artifacts/partitions/   deterministic frozen Dirichlet partition manifests
results/                local run outputs (git-ignored, intentionally not committed)

Key Scripts

Script	Role
scripts/run_baseline.py	full baseline / support-weighted training runner (frozen; grid complete)
scripts/generate_partitions.py	deterministic frozen Dirichlet manifests
scripts/analyze_multiseed_beta.py	pooled validation beta selection
scripts/analyze_partition_heterogeneity.py	realized non-IID characterization
scripts/analyze_prototype_effect.py	fixed-snapshot prototype displacement
scripts/analyze_cross_alpha.py	cross-alpha synthesis
scripts/verify_beta0_equivalence.py	beta=0 vs original semantic/numerical audit
scripts/validate_sj_participant_rule.py	S_j reproduction-assumption analysis

Important Project Documents

Proposal

• docs/proposal/THESIS_PROPOSAL_DRAFT.md — final Markdown proposal source for the current scope
• docs/proposal/PROPOSAL_SOURCE_MAP.md — claim-to-evidence map (internal)
• docs/proposal/LITERATURE_MATRIX.md — literature matrix (internal)
• docs/proposal/REFERENCES_VERIFIED.md — bibliography verification record (internal)

Research synthesis

• docs/research/FINAL_EXPERIMENTAL_SYNTHESIS.md — authoritative six-condition synthesis
• docs/research/CROSS_DATASET_BETA_ANALYSIS.md — cross-dataset comparison
• docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md — X-IIoTID cross-alpha analysis
• docs/research/5GNIDD_CROSS_ALPHA_ANALYSIS.md — 5G-NIDD cross-alpha analysis
• docs/research/EXPERIMENT_PROTOCOL.md — frozen reproduction protocol

Thesis support

• docs/thesis/CHAPTER4_RESULTS_OUTLINE.md — Chapter 4 blueprint
• docs/thesis/CLAIMS_REGISTER.md — claim / evidence / status register

Handoff and planning

• docs/handoff/SESSION_HANDOFF.md — current state and next steps (start here)
• planningawal/PROTEAN_Revisi_Lurus.md — immutable research specification

planningawal/ is immutable. Read it, cite it, but never edit, rename, move, or add files to it.

Testing and Reproducibility

# unit + integration tests (expected: 75 passed)
.venv/bin/python -m pytest

• Paired comparison: within each (dataset, alpha, seed), every compared configuration reuses the same partition, split, class mapping, and model initialization.
• Frozen manifests: partitions are generated once and reused; regenerating a fresh Dirichlet split is a protocol violation.
• Immutable raw data: datasets live outside the repository and are never modified or committed.
• Validation-only selection: beta is chosen from validation data only; test data is never used for any model-selection decision.
• Machine-readable outputs: analysis is stored as CSV/JSON under artifacts/analysis/, and the final figures load from those artifacts.

Frozen / Do Not Modify

Treat the following as finished unless there is an explicit, documented reason:

• planningawal/ — immutable research source.
• Raw datasets under ~/datasets/ — read-only, external, never copied into the repository.
• Completed scientific results and the final synthesis documents.
• Frozen experiment artifacts: partition manifests, analysis outputs, and final figures.
• src/, model, and aggregation code used for the 108-run grid.
• The finalized proposal Markdown.

Limitations

• The S_j participant/denominator rule remains a reproduction assumption and is materially exercised in a substantial fraction of class-condition cells.
• Two datasets, three alpha values, three project-defined seeds, 10 clients, one architecture.
• The beta grid is discrete; intermediate values were not tested.
• No inferential trend testing (three seeds are underpowered for it).
• beta = 0 is not bitwise identical to original_protean (floating-point ordering).
• Prototype displacement is a representation metric, not detection quality.

Quickstart

# unit + integration tests (expected: 75 passed)
.venv/bin/python -m pytest

# read-only pooled validation analysis (no retraining required)
.venv/bin/python scripts/analyze_multiseed_beta.py --dataset 5gnidd --alpha 0.25 --seeds 0 1 2

# read-only cross-alpha / cross-dataset synthesis
.venv/bin/python scripts/analyze_partition_heterogeneity.py --dataset 5gnidd --alphas 0.25 0.5 0.75
.venv/bin/python scripts/analyze_cross_alpha.py --dataset 5gnidd --alphas 0.25 0.5 0.75

See docs/handoff/SESSION_HANDOFF.md before making any methodological change. The next work is academic writing and formatting, not new experiments.