chore: checkpoint PROTEAN research planning and architecture

2026-09-24 17:08:56 +00:00
commit 2cc6a9859e
@@ -0,0 +1,346 @@
+# Session Handoff
+
+This is the canonical continuation point for the next agent session. It is written to be
+readable without any prior chat history, terminal scrollback, or git history.
+
+Session date: 2026-09-24
+Session type: research study, baseline investigation, architecture planning, dataset
+inspection, minimal bootstrap. **No training was run and no experimental sweep was launched.**
+
+---
+
+## 1. Project objective
+
+Build the code and experiment infrastructure for a thesis study that modifies **only the
+server-side prototype aggregation** of PROTEAN (Chennoufi et al., 2025), a federated
+intrusion-detection framework. The proposed mechanism weights each contributing client's
+class prototype by its class sample support:
+
+```
+Cbar_j(beta) = ( sum_{i in S_j} N_ij^beta * C_ij ) / ( sum_{i in S_j} N_ij^beta )
+S_j = { i | N_ij > 0 and a valid local prototype C_ij exists }
+beta in {0.0, 0.25, 0.5, 0.75, 1.0}
+```
+
+Goal: characterise when support weighting helps or hurts rare attack detection under
+non-IID data, selecting beta on a validation set. `beta = 0` is a control, **not** assumed
+identical to the original baseline.
+
+## 2. Authoritative source
+
+- `planningawal/PROTEAN_Revisi_Lurus.md` — the authoritative research specification.
+- The entire `planningawal/` directory is **immutable / read-only**. Do not edit, create,
+  rename, move, delete, reformat, overwrite, or generate reports inside it. It must not
+  appear in any "files modified" list.
+- Verified unchanged this session: SHA-256
+  `9ba1d7beb94354db0c4b588199fb1e2823aec8a23be1625ec09a756507fbcb89` (12,050 bytes).
+
+## 3. Dataset locations (read-only source data)
+
+- `~/datasets/X-IIoTID/X-IIoTID-dataset.csv`
+- `~/datasets/5G-NIDD/5G-NIDD.csv`
+
+Do not move, rename, overwrite, modify, or copy the full raw files into the repository. Use
+configurable external paths (`configs/datasets/datasets.yaml`, with env overrides).
+
+Recorded SHA-256:
+
+- X-IIoTID: `7b9290057ee42e784da3c0d84b781815502c9205c74175c96374e71a5ffd98a0`
+- 5G-NIDD: `fa36f80859585f474504ca69eb951a079b35145537976c86b00aae3aab46ee59`
+
+## 4. Current repository tree
+
+```
+perdalam-protean/
+├── README.md
+├── pyproject.toml
+├── .gitignore
+├── planningawal/                         # IMMUTABLE (read-only)
+│   └── PROTEAN_Revisi_Lurus.md
+├── configs/
+│   ├── datasets/datasets.yaml
+│   ├── experiments/experiment_protocol.yaml
+│   └── models/protean_cnn.yaml
+├── docs/
+│   ├── research/
+│   │   ├── RESEARCH_SPEC_INTERPRETATION.md
+│   │   ├── DATASET_INSPECTION.md
+│   │   ├── IMPLEMENTATION_PLAN.md
+│   │   └── EXPERIMENT_PROTOCOL.md
+│   ├── baseline/
+│   │   └── PROTEAN_BASELINE_INVESTIGATION.md
+│   └── handoff/
+│       └── SESSION_HANDOFF.md
+├── src/protean_research/
+│   ├── __init__.py
+│   ├── aggregation/
+│   │   ├── __init__.py
+│   │   └── prototype_aggregation.py
+│   └── utils/
+│       ├── __init__.py
+│       └── config.py
+├── tests/
+│   └── test_prototype_aggregation.py
+├── scripts/                              # .gitkeep only (tracked)
+├── artifacts/
+│   ├── partitions/                       # tracked: deterministic manifests (.gitkeep now)
+│   ├── analysis/                         # tracked: small analysis outputs (.gitkeep now)
+│   ├── processed/                        # ignored (regenerable)
+│   └── checkpoints/                      # ignored (regenerable)
+└── results/                              # ignored (regenerable)
+```
+
+`.gitignore` deliberately keeps `artifacts/partitions/` and `artifacts/analysis/`
+version-controlled (partition manifests are deterministic, content-hashed, and required for
+paired comparisons), while ignoring `artifacts/processed/`, `artifacts/checkpoints/`,
+`results/`, `logs/`, and `*.log`. `scripts/` is preserved with a `.gitkeep` placeholder.
+`planningawal/` is never ignored.
+
+Note: the repository was **not** empty at session start. A partial scaffold already existed
+(`configs/*`, `src/protean_research/aggregation/*`, `src/protean_research/utils/config.py`,
+`__init__.py` files, and empty `docs/`, `scripts/`, `tests/`, `artifacts/`, `results/`
+directories). `docs/` was empty and there was no handoff. That scaffold was kept and verified;
+it is not the product of this session.
+
+## 5. Work completed this session
+
+1. Read the full specification and produced a tagged interpretation
+   (`docs/research/RESEARCH_SPEC_INTERPRETATION.md`).
+2. Investigated authoritative PROTEAN sources (paper, ESORICS/Springer, HAL, author PhD
+   thesis) and searched for official code.
+3. Established baseline aggregation semantics question-by-question with evidence tags
+   (`docs/baseline/PROTEAN_BASELINE_INVESTIGATION.md`).
+4. Inspected both raw datasets in full (read-only) and recorded shapes, label distributions,
+   missingness, duplicates, identifiers, and mismatches
+   (`docs/research/DATASET_INSPECTION.md`).
+5. Verified the pre-existing config/path-resolution scaffold and aggregation module by running
+   them.
+6. Wrote the implementation architecture (`docs/research/IMPLEMENTATION_PLAN.md`) and the
+   experiment protocol (`docs/research/EXPERIMENT_PROTOCOL.md`).
+7. Added a minimal bootstrap: `pyproject.toml`, `README.md`, and dependency-free unit tests
+   for the aggregation math (`tests/test_prototype_aggregation.py`, 9 tests, all passing).
+8. Recorded dataset and source hashes for integrity verification.
+9. Wrote this handoff.
+
+## 6. Evidence and findings
+
+### 6.1 Official PROTEAN source status
+
+- **No official code exists publicly (as of 2026-09-24).**
+- Author GitHub `sarachn` (Sara Chennoufi) has only `FATE-LLM` (fork) and `SecLLMHolmes` —
+  neither is PROTEAN.
+- GitHub repository search `PROTEAN federated prototype` → 0 results; `grep.app` → none.
+- The unrelated GitHub user `Chennoufi` has only a `septembre` repo — do not chase it.
+- No code/artifact/supplementary link in the arXiv HTML, HAL record, or thesis text.
+- **Consequence: the baseline must be reproduced from the paper/thesis.**
+
+Primary sources:
+
+- arXiv:2507.05524v1 (7 Jul 2025), ESORICS 2025 pp. 103–125,
+  DOI `10.1007/978-3-032-07884-1_6`.
+- HAL `hal-05379203`.
+- Author PhD thesis `tel-05490927` (195 pages, 2025) — restates PROTEAN with extra detail.
+
+### 6.2 Baseline semantics discovered
+
+Confirmed from the paper/thesis:
+
+- Local prototype (Eq. 1 / 5.4): mean embedding over the client's class-`j` samples.
+- Server aggregation (Eq. 2 / 5.5): `omega = (1/M) sum_i omega_i`,
+  `Cbar_j = (1/M) sum_i C_ij` (equal, no `N_ij`).
+- Local loss (Eq. 3 / 5.6): cross-entropy + `lambda * ||C_ij - Cbar_j||^2` +
+  `(mu/2) * ||omega_i - omega^{t-1}||^2`, with `mu = 0.1`.
+- Inference: nearest global prototype by squared L2 (thesis Eqs. 5.11–5.13).
+- Model: 2 Conv1D (64, 128 filters) + ReLU + maxpool + dropout (0.2, 0.5), flatten,
+  dense 128, log-softmax head.
+- Setup: 10 clients, 80/20 train/test, 10 rounds, 3 local epochs,
+  `alpha in {0.75, 0.5, 0.25}`.
+- Thesis (Sec. 5.3.6): "prototypes are shared only for locally observed classes".
+
+### 6.3 Missing-class findings (critical)
+
+- Equation (2)/5.5 sums over all `M` participants, but the paper never states how a missing
+  class prototype is handled. The thesis implies absent classes are not shared, but the
+  server denominator (`M` vs number of contributors) is undefined.
+- Therefore `beta = 0` over `S_j` is **not** automatically equal to the original baseline;
+  equality requires the same participant rule. This is the central baseline ambiguity.
+- `original_protean` is deliberately **not implemented** in
+  `src/protean_research/aggregation/prototype_aggregation.py`; it raises
+  `NotImplementedError` with an explanatory message.
+- Additional unknowns: round-0 global-prototype initialisation; how the alignment loss sums
+  over `j = 1..K` when a client has `N_ij = 0`; per-batch vs end-of-round prototypes;
+  whether model aggregation is equal `1/M` or FedAvg sample-weighted (the thesis intro says
+  FedAvg is a weighted average, contradicting Eq. 5.5); numeric value of `lambda`; seed values;
+  feature preprocessing.
+
+### 6.4 Dataset findings
+
+- X-IIoTID: 820,834 × 68; labels `class1` (19), `class2` (10 general), `class3` (binary).
+  `class2` benign = `Normal`. After dropping `crypto-ransomware`: 9 classes. `'-'`
+  placeholders in 42 columns; 6,222 duplicate rows; label typo `Lateral _movement`.
+  Identifiers: `Date`, `Timestamp`, `Scr_IP`, `Scr_port`, `Des_IP`, `Des_port`.
+- 5G-NIDD: 1,215,890 × 52; `Attack Type` (9 values, benign = `Benign`), `Label` binary,
+  `Attack Tool` (6). 8 attack classes + Benign = 9. Heavy structural missingness (e.g. `dVid`
+  99.83%, `dTos` 77.56%); 1 duplicate row. Identifiers: empty-header index, `Seq`, `Offset`.
+- Both map to PROTEAN's 9-class setup, but PROTEAN's exact preprocessing is unspecified and
+  must be frozen here.
+
+## 7. Decisions made
+
+1. **Treat `planningawal/` as immutable** and keep all outputs elsewhere.
+2. **Do not implement `original_protean` until its semantics are verified.** Encode only the
+   unambiguous modes (`support_weighted`, `equal_available`). Rationale: avoid silently
+   assuming baseline behaviour.
+3. **Keep `equal_available` as a dormant control**, introduced only if baseline investigation
+   shows the participant rule differs from `S_j` (per the specification).
+4. **Centralise dataset paths** in `configs/datasets/datasets.yaml` with env overrides; never
+   hardcode in source.
+5. **Separate raw and generated data**; generated outputs under `artifacts/`.
+6. **Use `class2` for X-IIoTID and `Attack Type` for 5G-NIDD** as the multi-class labels, drop
+   `crypto-ransomware`, treat `Normal`/`Benign` as benign. Matches the paper.
+7. **Keep the pre-existing scaffold** (configs + aggregation + utils) and extend it minimally
+   rather than restructuring; no evidence justified a different layout.
+8. **Add dependency-free unit tests** for aggregation invariants now; defer data/federated
+   tests until those modules exist.
+9. **Do not initialize git during the planning session** (not requested at the time). A
+   follow-up checkpoint task then initialized Git and added remote `origin`; see Section 11.
+10. **Mark seeds, `lambda`, validation fraction, Dirichlet allocation, and preprocessing as
+    PROVISIONAL**; they must be frozen before any sweep.
+
+## 8. Unresolved questions
+
+1. Baseline missing-class rule and prototype-aggregation denominator (`M` vs `|S_j|`).
+2. Round-0 initialisation of global prototypes for all `K` classes.
+3. `lambda` value (paper gives `mu = 0.1` only).
+4. Whether model-parameter aggregation is equal or FedAvg sample-weighted.
+5. How the prototype alignment loss treats `N_ij = 0` classes.
+6. Per-batch vs end-of-round local prototype computation.
+7. Validation split fraction and procedure (the paper has no validation set).
+8. Exact Dirichlet allocation algorithm (affects realised `N_ij`).
+9. Exact seed values and whether metrics are averaged per client then per seed.
+10. Feature preprocessing: encoding, scaling, `'-'`/NaN handling, duplicate-row policy.
+11. ~~Whether to initialize git to protect the immutable source.~~ Resolved: Git was
+    initialized in the follow-up checkpoint task (see Section 11).
+12. Compute budget: no timing measured; a profiling smoke test is required before any sweep.
+
+## 9. Files created this session
+
+- `docs/research/RESEARCH_SPEC_INTERPRETATION.md`
+- `docs/research/DATASET_INSPECTION.md`
+- `docs/research/IMPLEMENTATION_PLAN.md`
+- `docs/research/EXPERIMENT_PROTOCOL.md`
+- `docs/baseline/PROTEAN_BASELINE_INVESTIGATION.md`
+- `docs/handoff/SESSION_HANDOFF.md`
+- `README.md`
+- `pyproject.toml`
+- `.gitignore`
+- `tests/test_prototype_aggregation.py`
+
+## 10. Files modified this session
+
+- None. No existing file was edited. `planningawal/` was not touched (and must not be).
+
+Pre-existing files (present at session start, verified but not modified):
+`configs/datasets/datasets.yaml`, `configs/experiments/experiment_protocol.yaml`,
+`configs/models/protean_cnn.yaml`, `src/protean_research/__init__.py`,
+`src/protean_research/aggregation/__init__.py`,
+`src/protean_research/aggregation/prototype_aggregation.py`,
+`src/protean_research/utils/__init__.py`, `src/protean_research/utils/config.py`.
+
+## 11. Git status
+
+The planning session ran with **no Git repository**. A follow-up checkpoint task then
+initialized Git:
+
+- Branch: `main`
+- Remote `origin`: `git@github.com:willtanoe/prototype-agregation.git`
+- Initial checkpoint commit: `chore: checkpoint PROTEAN research planning and architecture`
+  (exact hash reported in the checkpoint task's terminal summary; deliberately not embedded
+  here to avoid a recursive "hash inside the commit" loop).
+- `.gitkeep` placeholders were added for `scripts/`, `artifacts/partitions/`, and
+  `artifacts/analysis/`. `artifacts/processed/`, `artifacts/checkpoints/`, `results/`,
+  `logs/`, and `*.log` are ignored.
+- `planningawal/PROTEAN_Revisi_Lurus.md` is tracked exactly as-is and was not modified.
+- Raw datasets are outside the repository and are not tracked.
+
+Because this is the initial commit, `planningawal/` was verified with SHA-256 hashing and
+directory listing rather than `git diff`.
+
+## 12. Verification performed
+
+- Read the full specification (118 lines).
+- Read the full paper HTML and extracted the author PhD thesis text (195 pages).
+- GitHub API searches (author account, repositories), DuckDuckGo, `grep.app`.
+- Full pandas load of both datasets; recorded shape, dtypes, label counts, missingness,
+  duplicates, identifier columns.
+- SHA-256 of both datasets and of `planningawal/PROTEAN_Revisi_Lurus.md`.
+- Ran the pre-existing aggregation module and config resolver (modes, weights, path
+  resolution, `original_protean` raises `NotImplementedError`).
+- Ran `python -m pytest -q` in a temporary venv: **9 passed**.
+- Confirmed no training or sweep was launched.
+- Final check: `planningawal/` contains only `PROTEAN_Revisi_Lurus.md` with the same hash;
+  raw datasets retain the same hashes and mtimes.
+
+Checkpoint task (follow-up):
+
+- `git rev-parse --is-inside-work-tree` before init: not a repository.
+- Searched the whole tree for `*.csv`, `*.parquet`, `*.feather`, `*.pkl`, `*.pickle`, model
+  checkpoints: none present; largest file is the 12 KB spec.
+- Scanned tracked text for secret patterns (`API_KEY`, `SECRET`, `TOKEN`, `PASSWORD`,
+  `PRIVATE KEY`, `BEGIN OPENSSH`, `BEGIN RSA`, `sk-`, `ghp_`, `github_pat_`): none found.
+- `find planningawal -maxdepth 2 -type f` → only `PROTEAN_Revisi_Lurus.md`.
+- Re-ran `pytest`: 9 passed.
+
+## 13. Exact next task
+
+**Implement the deterministic data foundation, and freeze the remaining protocol choices
+before coding.**
+
+Concretely, the next session should:
+
+1. In `docs/research/EXPERIMENT_PROTOCOL.md`, replace the PROVISIONAL entries with explicit
+   frozen decisions and rationale for: preprocessing recipe, duplicate-row policy,
+   `'-'`/NaN handling, the Dirichlet allocation algorithm, the train/validation/test split
+   procedure and fraction, the seed set, `lambda`, batch size, and learning rate. These are
+   project decisions because the paper does not specify them; label them as reproduction
+   deviations.
+2. Create `src/protean_research/data/` with `loading.py`, `preprocessing.py`, `splits.py`,
+   `partition.py`, `manifest.py`, `characterization.py`, `rare_unseen.py` implementing the
+   flow in `docs/research/IMPLEMENTATION_PLAN.md` Section 3.
+3. Persist one partition manifest per `(dataset, alpha, seed)` under `artifacts/partitions/`
+   with a content hash, and derive rare/unseen metadata from the training split.
+4. Add tests for determinism of partition generation (strategy item 8) and identical manifest
+   reload (item 9), plus the non-IID characterization fields.
+
+Do **not** start model training or a beta sweep. The baseline missing-class rule is still
+unresolved, so `original_protean` remains unimplemented.
+
+## 14. Recommended next-session prompt
+
+```
+You are continuing the perdalam-protean research repository.
+
+FIRST, read these two files completely:
+  1. docs/handoff/SESSION_HANDOFF.md
+  2. planningawal/PROTEAN_Revisi_Lurus.md
+
+Then read, as needed:
+  - docs/research/IMPLEMENTATION_PLAN.md
+  - docs/research/EXPERIMENT_PROTOCOL.md
+  - docs/research/DATASET_INSPECTION.md
+  - docs/baseline/PROTEAN_BASELINE_INVESTIGATION.md
+
+Critical rules:
+  - planningawal/ is immutable. Do not edit/create/move/rename/delete anything inside it.
+  - ~/datasets/X-IIoTID/X-IIoTID-dataset.csv and ~/datasets/5G-NIDD/5G-NIDD.csv are
+    read-only external source data. Do not modify or copy them into the repo.
+  - Do NOT run full training or an experimental sweep.
+
+Your task is Section 13 of the handoff: first freeze the PROVISIONAL protocol decisions in
+docs/research/EXPERIMENT_PROTOCOL.md, then implement the deterministic data foundation in
+src/protean_research/data/ with partition-manifest persistence, rare/unseen metadata, and
+tests for partition determinism and manifest reload.
+
+Start by reading the handoff, then continue from the documented state. Update
+docs/handoff/SESSION_HANDOFF.md at the end of your session.
+```