thesis-prototype-agregation/AGENTS.md

# PROTEAN Prototype Aggregation Research

## Project Purpose

This repository contains a master's research implementation based on:

`planningawal/PROTEAN_Revisi_Lurus.md`

Research focus:

**Server-side prototype aggregation in PROTEAN using class-specific sample-support weighting for rare attack detection under non-IID federated data.**

The proposed aggregation is:

$$
\bar{C}_j(\beta)
=
\frac{
\sum_{i \in S_j} N_{ij}^{\beta} C_{ij}
}{
\sum_{i \in S_j} N_{ij}^{\beta}
}
$$

with planned candidate values:

```text
beta ∈ {0.00, 0.25, 0.50, 0.75, 1.00}
```

where:

* `C_ij` is client `i`'s local prototype for class `j`
* `N_ij` is the number of local training samples of class `j`
* `S_j` is the set of clients contributing a valid prototype for class `j`

This repository is a research codebase. Scientific traceability and experimental isolation are more important than implementation convenience.

---

# 1. AUTHORITATIVE RESEARCH SOURCE

The primary research specification is:

```text
planningawal/PROTEAN_Revisi_Lurus.md
```

Read this file before making methodological decisions.

Do not silently reinterpret or broaden the research objective.

If implementation evidence conflicts with an assumption in the planning document:

1. preserve the source document unchanged
2. document the discrepancy
3. distinguish paper behavior, code behavior, inference, and project decision
4. do not silently choose one interpretation

---

# 2. `planningawal/` IS IMMUTABLE

The entire directory:

```text
planningawal/
```

is read-only research source material.

Agents MAY:

* read files
* cite them internally
* use them to guide implementation

Agents MUST NOT:

* edit files inside `planningawal/`
* create files inside `planningawal/`
* rename files inside `planningawal/`
* move files into or out of it
* delete files
* autoformat its contents
* rewrite Markdown formatting
* place reports, logs, code, or handoffs there

All generated project material must live elsewhere.

Before major checkpoints, verify that `planningawal/` remains unchanged.

---

# 3. RAW DATASETS ARE EXTERNAL AND IMMUTABLE

Raw datasets exist outside the repository:

```text
~/datasets/5G-NIDD/5G-NIDD.csv
~/datasets/X-IIoTID/X-IIoTID-dataset.csv
```

These are source data.

Do NOT:

* modify them
* overwrite them
* rename them
* move them
* copy full raw datasets into the repository
* commit them to Git

Dataset locations must be supplied through configuration, environment variables, or CLI arguments rather than duplicated throughout source code.

Generated derivatives may be stored inside the repository under appropriate ignored artifact directories.

Raw data and generated data must always remain distinguishable.

---

# 4. PROJECT STRUCTURE

Prefer the following separation of concerns:

```text
perdalam-protean/
├── AGENTS.md
├── README.md
├── planningawal/                # immutable research source
├── docs/
│   ├── research/
│   ├── baseline/
│   └── handoff/
├── configs/
│   ├── datasets/
│   ├── experiments/
│   └── models/
├── src/
│   └── protean_research/
│       ├── data/
│       ├── federated/
│       ├── aggregation/
│       ├── models/
│       ├── evaluation/
│       └── utils/
├── scripts/
├── tests/
├── artifacts/
│   ├── partitions/
│   ├── processed/
│   ├── checkpoints/
│   └── analysis/
└── results/
```

This is a design guideline, not an instruction to create empty directories without purpose.

Do not introduce new top-level structures without a clear reason.

---

# 5. SESSION CONTINUITY

The canonical session continuation file is:

```text
docs/handoff/SESSION_HANDOFF.md
```

At the beginning of a new substantial session:

1. read `AGENTS.md`
2. read `docs/handoff/SESSION_HANDOFF.md` if it exists
3. read relevant project documentation
4. consult `planningawal/PROTEAN_Revisi_Lurus.md` when research semantics matter
5. inspect actual repository state before making assumptions

Do not reconstruct state from chat history when repository documentation is available.

The handoff must reflect actual repository state.

Do not copy stale plans forward as if they were completed work.

---

# 6. HANDOFF REQUIREMENT

At the end of any substantial development/research session, update:

```text
docs/handoff/SESSION_HANDOFF.md
```

The handoff should contain:

* current objective
* completed work
* important findings
* files created
* files modified
* commands/tests performed
* current architecture
* methodological decisions
* unresolved questions
* known failures or incomplete work
* Git state where relevant
* exact next task
* a concise ready-to-paste next-session prompt

Do not write huge terminal-only reports that future sessions cannot recover.

Long analysis should be written into project Markdown files.

---

# 7. BASELINE FIRST

Do not implement the proposed beta weighting as a replacement for PROTEAN before baseline semantics are understood.

The project must preserve a faithful original baseline.

Important unresolved or verifiable baseline properties include:

* local prototype construction
* missing-class handling
* server aggregation participant set
* aggregation denominator
* class support/count availability
* global prototype redistribution
* prototype alignment behavior
* inference behavior

Never assume:

```text
beta = 0 == original PROTEAN
```

This equivalence is only valid if aggregation participants and missing-class handling are also identical.

If needed, distinguish:

```text
original_protean
equal_available
support_weighted
```

`equal_available` should exist only if methodological evidence shows it is required as a control.

---

# 8. RESEARCH SCOPE

The intended modification is narrowly scoped to:

```text
server-side prototype aggregation
```

Do NOT independently add research dimensions such as:

* client rejection
* Byzantine or poisoning defense
* adaptive loss functions
* prototype clustering
* alternative distance metrics
* cosine similarity experiments
* dynamic architectures
* personalized FL
* attention modules
* client scoring
* prototype-quality scoring
* additional aggregation algorithms

unless required to faithfully reproduce the verified baseline.

A technically interesting addition is not automatically relevant to this thesis.

---

# 9. SCIENTIFIC INTERPRETATION

`N_ij` represents:

```text
class-specific sample support
```

It does NOT represent:

```text
ground-truth prototype quality
```

Do not state or encode assumptions that a larger `N_ij` automatically means a better prototype.

Low-support clients may contain useful variation unavailable at higher-support clients.

Use neutral terminology such as:

```text
support_weighted
sample_support_weighting
beta_weighted
```

Avoid names implying proven superiority, such as:

```text
improved_aggregation
optimal_aggregation
better_prototype
```

until supported by evidence.

---

# 10. BETA SEMANTICS

The proposed weight is:

```text
w_ij = N_ij ** beta
```

with:

```text
0 <= beta <= 1
```

Expected properties:

## beta = 0

For an identical contributor set `S_j`:

```text
N_ij ** 0 = 1
```

therefore aggregation becomes equal weighting.

This must not automatically be labeled the original PROTEAN baseline.

## beta = 1

Weight is directly proportional to class-specific support.

Because each local prototype is a local class embedding mean, this should correspond to the combined mean of included embeddings when all assumptions match.

## One contributor

If `|S_j| = 1`, aggregation is unchanged for every beta.

## Equal support

If all `N_ij` values are equal for fixed local prototypes, changing beta must not change the aggregate.

These invariants should eventually have tests.

---

# 11. MISSING CLASS SEMANTICS

For the proposed formulation:

```text
S_j = clients with N_ij > 0 and a valid local prototype C_ij
```

Do not fabricate a prototype for a missing class.

Do not use a zero vector unless this is specifically required to reproduce the original baseline in `original_protean` mode.

Missing-class handling must remain explicit in code.

Do not hide it inside generic tensor padding.

---

# 12. RARE CLASS DEFINITION

For each client, rare classes are defined from TRAINING DATA.

Among attack classes satisfying:

```text
N_ij > 0
```

select the two attack classes with the smallest training support.

If only one attack class is present:

* evaluate that one class

If no attack class is present:

* exclude that client from rare-class aggregation

Tie behavior must eventually be deterministic and documented.

---

# 13. UNSEEN CLASS DEFINITION

Attack classes satisfying:

```text
N_ij = 0
```

during client training are unseen classes.

Rare classes and unseen classes are different evaluation categories.

Never merge them into one metric.

---

# 14. DATA PARTITIONING

Target Dirichlet non-IID settings are:

```text
alpha = 0.75
alpha = 0.50
alpha = 0.25
```

Multiple deterministic seeds are required.

For a given:

```text
(dataset, alpha, seed)
```

all compared aggregation configurations must reuse exactly the same partition.

Persist partition manifests.

Do not regenerate a different Dirichlet split for every beta candidate.

Paired comparison requires identical:

* client partition
* train split
* validation split
* test split
* class mapping
* model initialization policy
* training hyperparameters

unless a deviation is explicitly justified and documented.

---

# 15. VALIDATION AND TEST SEPARATION

Beta selection must use validation data only.

Primary planned selection metric:

```text
rare-class macro-F1
```

The test set must not influence beta selection.

Maintain a clear separation between:

```text
candidate evaluation
beta selection
final test evaluation
```

Do not use test results to tune:

* beta
* training epochs
* thresholds
* seed choice
* class selection
* model configuration

Do not call the selected beta globally optimal.

Correct wording is:

```text
best among the tested beta candidates under the defined validation procedure
```

---

# 16. REQUIRED EVALUATION GRANULARITY

Do not report only global accuracy.

Preserve enough information for analysis at:

* dataset level
* alpha level
* seed level
* client level
* class level

Important outputs include:

* rare-class macro-F1
* rare-class recall
* overall macro-F1
* per-client metrics
* per-class precision/recall/F1 where relevant
* evaluation support
* unseen-class results
* `N_ij`
* number of clients containing a class
* number of contributing prototypes

Actual realized class imbalance must be recorded.

Dirichlet alpha alone is not sufficient to characterize a partition.

---

# 17. DIRECT PROTOTYPE ANALYSIS

The research distinguishes:

```text
direct aggregation effect
```

from:

```text
end-to-end training effect
```

Where instrumentation is implemented, preserve fixed:

```text
C_ij
N_ij
```

from the same communication round and recompute candidate global prototypes without retraining.

Useful measures include:

```text
||Cbar_j(beta) - Cbar_j(0)||
```

as well as:

* normalized client weights
* prototype displacement
* distance to local prototypes
* pairwise differences between beta settings

Do not interpret direct displacement alone as improved detection.

---

# 18. CODE QUALITY

Prefer:

* small cohesive modules
* typed interfaces where practical
* deterministic behavior
* explicit configuration
* clear validation errors
* minimal global state
* no duplicated experiment logic
* reusable dataset abstractions
* reproducible random-state handling

Avoid:

* magic constants
* hardcoded absolute user paths inside research logic
* giant scripts containing preprocessing, training, evaluation, and plotting together
* silent fallback behavior
* hidden data mutation
* unnecessary dependencies
* unrelated refactoring during methodological changes

Every important research parameter should be visible in configuration or experiment metadata.

---

# 19. TESTING EXPECTATIONS

Before large experiments, establish tests for critical research invariants.

Expected tests eventually include:

* beta=0 arithmetic-mean property
* beta=1 support-weighted property
* single-contributor invariance
* equal-support invariance
* missing-class exclusion
* normalized weight sum
* finite numerical output
* deterministic partitions
* partition save/reload identity
* baseline behavior preservation
* validation/test separation

Prefer focused tests before expensive training runs.

---

# 20. COMPUTE DISCIPLINE

Do not launch a full experimental sweep merely because the runner exists.

Before expensive experiments:

1. unit tests must pass
2. data pipeline must be validated
3. one small smoke run must succeed
4. baseline behavior must be checked
5. output artifacts must be inspected
6. experiment metadata must be reproducible

Do not invent runtime estimates.

Measure first.

---

# 21. ARTIFACTS AND RESULTS

Generated outputs should be structured and machine-readable.

Prefer:

* CSV
* JSON
* JSONL
* Parquet where appropriate

over terminal-only text.

Do not overwrite previous experiment results silently.

Each experiment should be identifiable from metadata such as:

```text
dataset
alpha
seed
aggregation_mode
beta
training configuration
code revision
```

Checkpoint/model files should not be committed unless explicitly required.

---

# 22. DOCUMENTATION

Research decisions that affect interpretation must be documented.

Expected documentation areas:

```text
docs/research/
docs/baseline/
docs/handoff/
```

Prefer updating an existing authoritative document over creating multiple overlapping reports.

Do not create `final_v2_revised_latest.md` style document proliferation.

---

# 23. GIT DISCIPLINE

The intended remote repository is:

```text
git@github.com:willtanoe/prototype-agregation.git
```

Before checkpointing:

* inspect `git status`
* inspect staged diff
* verify no secrets
* verify no raw datasets
* verify no large accidental artifacts
* verify `planningawal/` remains unchanged

Do not force push.

Do not rewrite shared history unless explicitly instructed.

Use checkpoint commits at meaningful research/development boundaries.

---

# 24. SECURITY AND SECRETS

Never commit:

* API keys
* access tokens
* passwords
* SSH private keys
* `.env` secrets
* service credentials

Do not print discovered secrets into reports.

Use environment variables or ignored local configuration for secret material.

---

# 25. SOURCE OF TRUTH HIERARCHY

When deciding what to trust, use approximately this order:

1. immutable research specification for intended thesis design
2. verified original PROTEAN paper/code for baseline behavior
3. actual repository implementation
4. documented project decisions
5. inference

Do not present inference as source-confirmed fact.

When sources conflict, document the conflict rather than hiding it.

---

# 26. AGENT BEHAVIOR

Before modifying code:

* inspect relevant files
* understand current architecture
* identify the smallest required change
* preserve established behavior outside the target scope

When uncertain:

* investigate evidence first
* write down unresolved ambiguity if it cannot be resolved
* avoid silent assumptions

Do not optimize for producing the largest amount of code.

Optimize for correctness, reproducibility, and traceability.

---

# 27. USE OF SUBAGENTS

Subagents may be used for parallel READ-ONLY investigation where tasks are independent.

Good subagent tasks:

* inspect original PROTEAN source
* inspect dataset schema
* review experimental methodology
* review code/tests
* inspect reproducibility issues

Avoid multiple agents concurrently editing the same files.

A single primary agent should own integration and final edits.

Subagent findings are advisory until verified by the primary agent.

Do not merge contradictory findings without resolving their evidence.

---

# 28. BEFORE DECLARING A PHASE COMPLETE

Verify the phase against its actual acceptance criteria.

Do not declare completion because code merely runs.

For implementation phases, check:

* relevant tests
* expected artifacts
* configuration reproducibility
* no unintended baseline changes
* no data leakage
* no source-folder modifications

For research/planning phases, check:

* evidence is documented
* unknowns remain marked as unknown
* the handoff is current

---

# 29. TERMINAL OUTPUT

Keep terminal summaries concise.

Long reports belong in files.

At the end of substantial work, terminal output should usually summarize:

* what changed
* what was verified
* tests/checks performed
* known blockers
* handoff location
* exact next task

Do not rely on terminal scrollback as project documentation.