docs: add project agent instructions

2026-09-24 17:41:09 +00:00
1 parent 0aa8e2ba79
commit 374651a0fe
@@ -0,0 +1,831 @@
+# PROTEAN Prototype Aggregation Research
+
+## Project Purpose
+
+This repository contains a master's research implementation based on:
+
+`planningawal/PROTEAN_Revisi_Lurus.md`
+
+Research focus:
+
+**Server-side prototype aggregation in PROTEAN using class-specific sample-support weighting for rare attack detection under non-IID federated data.**
+
+The proposed aggregation is:
+
+$$
+\bar{C}_j(\beta)
+=
+\frac{
+\sum_{i \in S_j} N_{ij}^{\beta} C_{ij}
+}{
+\sum_{i \in S_j} N_{ij}^{\beta}
+}
+$$
+
+with planned candidate values:
+
+```text
+beta ∈ {0.00, 0.25, 0.50, 0.75, 1.00}
+```
+
+where:
+
+* `C_ij` is client `i`'s local prototype for class `j`
+* `N_ij` is the number of local training samples of class `j`
+* `S_j` is the set of clients contributing a valid prototype for class `j`
+
+This repository is a research codebase. Scientific traceability and experimental isolation are more important than implementation convenience.
+
+---
+
+# 1. AUTHORITATIVE RESEARCH SOURCE
+
+The primary research specification is:
+
+```text
+planningawal/PROTEAN_Revisi_Lurus.md
+```
+
+Read this file before making methodological decisions.
+
+Do not silently reinterpret or broaden the research objective.
+
+If implementation evidence conflicts with an assumption in the planning document:
+
+1. preserve the source document unchanged
+2. document the discrepancy
+3. distinguish paper behavior, code behavior, inference, and project decision
+4. do not silently choose one interpretation
+
+---
+
+# 2. `planningawal/` IS IMMUTABLE
+
+The entire directory:
+
+```text
+planningawal/
+```
+
+is read-only research source material.
+
+Agents MAY:
+
+* read files
+* cite them internally
+* use them to guide implementation
+
+Agents MUST NOT:
+
+* edit files inside `planningawal/`
+* create files inside `planningawal/`
+* rename files inside `planningawal/`
+* move files into or out of it
+* delete files
+* autoformat its contents
+* rewrite Markdown formatting
+* place reports, logs, code, or handoffs there
+
+All generated project material must live elsewhere.
+
+Before major checkpoints, verify that `planningawal/` remains unchanged.
+
+---
+
+# 3. RAW DATASETS ARE EXTERNAL AND IMMUTABLE
+
+Raw datasets exist outside the repository:
+
+```text
+~/datasets/5G-NIDD/5G-NIDD.csv
+~/datasets/X-IIoTID/X-IIoTID-dataset.csv
+```
+
+These are source data.
+
+Do NOT:
+
+* modify them
+* overwrite them
+* rename them
+* move them
+* copy full raw datasets into the repository
+* commit them to Git
+
+Dataset locations must be supplied through configuration, environment variables, or CLI arguments rather than duplicated throughout source code.
+
+Generated derivatives may be stored inside the repository under appropriate ignored artifact directories.
+
+Raw data and generated data must always remain distinguishable.
+
+---
+
+# 4. PROJECT STRUCTURE
+
+Prefer the following separation of concerns:
+
+```text
+perdalam-protean/
+├── AGENTS.md
+├── README.md
+├── planningawal/                # immutable research source
+├── docs/
+│   ├── research/
+│   ├── baseline/
+│   └── handoff/
+├── configs/
+│   ├── datasets/
+│   ├── experiments/
+│   └── models/
+├── src/
+│   └── protean_research/
+│       ├── data/
+│       ├── federated/
+│       ├── aggregation/
+│       ├── models/
+│       ├── evaluation/
+│       └── utils/
+├── scripts/
+├── tests/
+├── artifacts/
+│   ├── partitions/
+│   ├── processed/
+│   ├── checkpoints/
+│   └── analysis/
+└── results/
+```
+
+This is a design guideline, not an instruction to create empty directories without purpose.
+
+Do not introduce new top-level structures without a clear reason.
+
+---
+
+# 5. SESSION CONTINUITY
+
+The canonical session continuation file is:
+
+```text
+docs/handoff/SESSION_HANDOFF.md
+```
+
+At the beginning of a new substantial session:
+
+1. read `AGENTS.md`
+2. read `docs/handoff/SESSION_HANDOFF.md` if it exists
+3. read relevant project documentation
+4. consult `planningawal/PROTEAN_Revisi_Lurus.md` when research semantics matter
+5. inspect actual repository state before making assumptions
+
+Do not reconstruct state from chat history when repository documentation is available.
+
+The handoff must reflect actual repository state.
+
+Do not copy stale plans forward as if they were completed work.
+
+---
+
+# 6. HANDOFF REQUIREMENT
+
+At the end of any substantial development/research session, update:
+
+```text
+docs/handoff/SESSION_HANDOFF.md
+```
+
+The handoff should contain:
+
+* current objective
+* completed work
+* important findings
+* files created
+* files modified
+* commands/tests performed
+* current architecture
+* methodological decisions
+* unresolved questions
+* known failures or incomplete work
+* Git state where relevant
+* exact next task
+* a concise ready-to-paste next-session prompt
+
+Do not write huge terminal-only reports that future sessions cannot recover.
+
+Long analysis should be written into project Markdown files.
+
+---
+
+# 7. BASELINE FIRST
+
+Do not implement the proposed beta weighting as a replacement for PROTEAN before baseline semantics are understood.
+
+The project must preserve a faithful original baseline.
+
+Important unresolved or verifiable baseline properties include:
+
+* local prototype construction
+* missing-class handling
+* server aggregation participant set
+* aggregation denominator
+* class support/count availability
+* global prototype redistribution
+* prototype alignment behavior
+* inference behavior
+
+Never assume:
+
+```text
+beta = 0 == original PROTEAN
+```
+
+This equivalence is only valid if aggregation participants and missing-class handling are also identical.
+
+If needed, distinguish:
+
+```text
+original_protean
+equal_available
+support_weighted
+```
+
+`equal_available` should exist only if methodological evidence shows it is required as a control.
+
+---
+
+# 8. RESEARCH SCOPE
+
+The intended modification is narrowly scoped to:
+
+```text
+server-side prototype aggregation
+```
+
+Do NOT independently add research dimensions such as:
+
+* client rejection
+* Byzantine or poisoning defense
+* adaptive loss functions
+* prototype clustering
+* alternative distance metrics
+* cosine similarity experiments
+* dynamic architectures
+* personalized FL
+* attention modules
+* client scoring
+* prototype-quality scoring
+* additional aggregation algorithms
+
+unless required to faithfully reproduce the verified baseline.
+
+A technically interesting addition is not automatically relevant to this thesis.
+
+---
+
+# 9. SCIENTIFIC INTERPRETATION
+
+`N_ij` represents:
+
+```text
+class-specific sample support
+```
+
+It does NOT represent:
+
+```text
+ground-truth prototype quality
+```
+
+Do not state or encode assumptions that a larger `N_ij` automatically means a better prototype.
+
+Low-support clients may contain useful variation unavailable at higher-support clients.
+
+Use neutral terminology such as:
+
+```text
+support_weighted
+sample_support_weighting
+beta_weighted
+```
+
+Avoid names implying proven superiority, such as:
+
+```text
+improved_aggregation
+optimal_aggregation
+better_prototype
+```
+
+until supported by evidence.
+
+---
+
+# 10. BETA SEMANTICS
+
+The proposed weight is:
+
+```text
+w_ij = N_ij ** beta
+```
+
+with:
+
+```text
+0 <= beta <= 1
+```
+
+Expected properties:
+
+## beta = 0
+
+For an identical contributor set `S_j`:
+
+```text
+N_ij ** 0 = 1
+```
+
+therefore aggregation becomes equal weighting.
+
+This must not automatically be labeled the original PROTEAN baseline.
+
+## beta = 1
+
+Weight is directly proportional to class-specific support.
+
+Because each local prototype is a local class embedding mean, this should correspond to the combined mean of included embeddings when all assumptions match.
+
+## One contributor
+
+If `|S_j| = 1`, aggregation is unchanged for every beta.
+
+## Equal support
+
+If all `N_ij` values are equal for fixed local prototypes, changing beta must not change the aggregate.
+
+These invariants should eventually have tests.
+
+---
+
+# 11. MISSING CLASS SEMANTICS
+
+For the proposed formulation:
+
+```text
+S_j = clients with N_ij > 0 and a valid local prototype C_ij
+```
+
+Do not fabricate a prototype for a missing class.
+
+Do not use a zero vector unless this is specifically required to reproduce the original baseline in `original_protean` mode.
+
+Missing-class handling must remain explicit in code.
+
+Do not hide it inside generic tensor padding.
+
+---
+
+# 12. RARE CLASS DEFINITION
+
+For each client, rare classes are defined from TRAINING DATA.
+
+Among attack classes satisfying:
+
+```text
+N_ij > 0
+```
+
+select the two attack classes with the smallest training support.
+
+If only one attack class is present:
+
+* evaluate that one class
+
+If no attack class is present:
+
+* exclude that client from rare-class aggregation
+
+Tie behavior must eventually be deterministic and documented.
+
+---
+
+# 13. UNSEEN CLASS DEFINITION
+
+Attack classes satisfying:
+
+```text
+N_ij = 0
+```
+
+during client training are unseen classes.
+
+Rare classes and unseen classes are different evaluation categories.
+
+Never merge them into one metric.
+
+---
+
+# 14. DATA PARTITIONING
+
+Target Dirichlet non-IID settings are:
+
+```text
+alpha = 0.75
+alpha = 0.50
+alpha = 0.25
+```
+
+Multiple deterministic seeds are required.
+
+For a given:
+
+```text
+(dataset, alpha, seed)
+```
+
+all compared aggregation configurations must reuse exactly the same partition.
+
+Persist partition manifests.
+
+Do not regenerate a different Dirichlet split for every beta candidate.
+
+Paired comparison requires identical:
+
+* client partition
+* train split
+* validation split
+* test split
+* class mapping
+* model initialization policy
+* training hyperparameters
+
+unless a deviation is explicitly justified and documented.
+
+---
+
+# 15. VALIDATION AND TEST SEPARATION
+
+Beta selection must use validation data only.
+
+Primary planned selection metric:
+
+```text
+rare-class macro-F1
+```
+
+The test set must not influence beta selection.
+
+Maintain a clear separation between:
+
+```text
+candidate evaluation
+beta selection
+final test evaluation
+```
+
+Do not use test results to tune:
+
+* beta
+* training epochs
+* thresholds
+* seed choice
+* class selection
+* model configuration
+
+Do not call the selected beta globally optimal.
+
+Correct wording is:
+
+```text
+best among the tested beta candidates under the defined validation procedure
+```
+
+---
+
+# 16. REQUIRED EVALUATION GRANULARITY
+
+Do not report only global accuracy.
+
+Preserve enough information for analysis at:
+
+* dataset level
+* alpha level
+* seed level
+* client level
+* class level
+
+Important outputs include:
+
+* rare-class macro-F1
+* rare-class recall
+* overall macro-F1
+* per-client metrics
+* per-class precision/recall/F1 where relevant
+* evaluation support
+* unseen-class results
+* `N_ij`
+* number of clients containing a class
+* number of contributing prototypes
+
+Actual realized class imbalance must be recorded.
+
+Dirichlet alpha alone is not sufficient to characterize a partition.
+
+---
+
+# 17. DIRECT PROTOTYPE ANALYSIS
+
+The research distinguishes:
+
+```text
+direct aggregation effect
+```
+
+from:
+
+```text
+end-to-end training effect
+```
+
+Where instrumentation is implemented, preserve fixed:
+
+```text
+C_ij
+N_ij
+```
+
+from the same communication round and recompute candidate global prototypes without retraining.
+
+Useful measures include:
+
+```text
+||Cbar_j(beta) - Cbar_j(0)||
+```
+
+as well as:
+
+* normalized client weights
+* prototype displacement
+* distance to local prototypes
+* pairwise differences between beta settings
+
+Do not interpret direct displacement alone as improved detection.
+
+---
+
+# 18. CODE QUALITY
+
+Prefer:
+
+* small cohesive modules
+* typed interfaces where practical
+* deterministic behavior
+* explicit configuration
+* clear validation errors
+* minimal global state
+* no duplicated experiment logic
+* reusable dataset abstractions
+* reproducible random-state handling
+
+Avoid:
+
+* magic constants
+* hardcoded absolute user paths inside research logic
+* giant scripts containing preprocessing, training, evaluation, and plotting together
+* silent fallback behavior
+* hidden data mutation
+* unnecessary dependencies
+* unrelated refactoring during methodological changes
+
+Every important research parameter should be visible in configuration or experiment metadata.
+
+---
+
+# 19. TESTING EXPECTATIONS
+
+Before large experiments, establish tests for critical research invariants.
+
+Expected tests eventually include:
+
+* beta=0 arithmetic-mean property
+* beta=1 support-weighted property
+* single-contributor invariance
+* equal-support invariance
+* missing-class exclusion
+* normalized weight sum
+* finite numerical output
+* deterministic partitions
+* partition save/reload identity
+* baseline behavior preservation
+* validation/test separation
+
+Prefer focused tests before expensive training runs.
+
+---
+
+# 20. COMPUTE DISCIPLINE
+
+Do not launch a full experimental sweep merely because the runner exists.
+
+Before expensive experiments:
+
+1. unit tests must pass
+2. data pipeline must be validated
+3. one small smoke run must succeed
+4. baseline behavior must be checked
+5. output artifacts must be inspected
+6. experiment metadata must be reproducible
+
+Do not invent runtime estimates.
+
+Measure first.
+
+---
+
+# 21. ARTIFACTS AND RESULTS
+
+Generated outputs should be structured and machine-readable.
+
+Prefer:
+
+* CSV
+* JSON
+* JSONL
+* Parquet where appropriate
+
+over terminal-only text.
+
+Do not overwrite previous experiment results silently.
+
+Each experiment should be identifiable from metadata such as:
+
+```text
+dataset
+alpha
+seed
+aggregation_mode
+beta
+training configuration
+code revision
+```
+
+Checkpoint/model files should not be committed unless explicitly required.
+
+---
+
+# 22. DOCUMENTATION
+
+Research decisions that affect interpretation must be documented.
+
+Expected documentation areas:
+
+```text
+docs/research/
+docs/baseline/
+docs/handoff/
+```
+
+Prefer updating an existing authoritative document over creating multiple overlapping reports.
+
+Do not create `final_v2_revised_latest.md` style document proliferation.
+
+---
+
+# 23. GIT DISCIPLINE
+
+The intended remote repository is:
+
+```text
+git@github.com:willtanoe/prototype-agregation.git
+```
+
+Before checkpointing:
+
+* inspect `git status`
+* inspect staged diff
+* verify no secrets
+* verify no raw datasets
+* verify no large accidental artifacts
+* verify `planningawal/` remains unchanged
+
+Do not force push.
+
+Do not rewrite shared history unless explicitly instructed.
+
+Use checkpoint commits at meaningful research/development boundaries.
+
+---
+
+# 24. SECURITY AND SECRETS
+
+Never commit:
+
+* API keys
+* access tokens
+* passwords
+* SSH private keys
+* `.env` secrets
+* service credentials
+
+Do not print discovered secrets into reports.
+
+Use environment variables or ignored local configuration for secret material.
+
+---
+
+# 25. SOURCE OF TRUTH HIERARCHY
+
+When deciding what to trust, use approximately this order:
+
+1. immutable research specification for intended thesis design
+2. verified original PROTEAN paper/code for baseline behavior
+3. actual repository implementation
+4. documented project decisions
+5. inference
+
+Do not present inference as source-confirmed fact.
+
+When sources conflict, document the conflict rather than hiding it.
+
+---
+
+# 26. AGENT BEHAVIOR
+
+Before modifying code:
+
+* inspect relevant files
+* understand current architecture
+* identify the smallest required change
+* preserve established behavior outside the target scope
+
+When uncertain:
+
+* investigate evidence first
+* write down unresolved ambiguity if it cannot be resolved
+* avoid silent assumptions
+
+Do not optimize for producing the largest amount of code.
+
+Optimize for correctness, reproducibility, and traceability.
+
+---
+
+# 27. USE OF SUBAGENTS
+
+Subagents may be used for parallel READ-ONLY investigation where tasks are independent.
+
+Good subagent tasks:
+
+* inspect original PROTEAN source
+* inspect dataset schema
+* review experimental methodology
+* review code/tests
+* inspect reproducibility issues
+
+Avoid multiple agents concurrently editing the same files.
+
+A single primary agent should own integration and final edits.
+
+Subagent findings are advisory until verified by the primary agent.
+
+Do not merge contradictory findings without resolving their evidence.
+
+---
+
+# 28. BEFORE DECLARING A PHASE COMPLETE
+
+Verify the phase against its actual acceptance criteria.
+
+Do not declare completion because code merely runs.
+
+For implementation phases, check:
+
+* relevant tests
+* expected artifacts
+* configuration reproducibility
+* no unintended baseline changes
+* no data leakage
+* no source-folder modifications
+
+For research/planning phases, check:
+
+* evidence is documented
+* unknowns remain marked as unknown
+* the handoff is current
+
+---
+
+# 29. TERMINAL OUTPUT
+
+Keep terminal summaries concise.
+
+Long reports belong in files.
+
+At the end of substantial work, terminal output should usually summarize:
+
+* what changed
+* what was verified
+* tests/checks performed
+* known blockers
+* handoff location
+* exact next task
+
+Do not rely on terminal scrollback as project documentation.