willtanoe/thesis-prototype-agregation

0 Fork 0

Files

T

willtanoe 374651a0fe docs: add project agent instructions

2026-09-24 17:41:09 +00:00

17 KiB

Executable File

Raw Permalink Blame History

PROTEAN Prototype Aggregation Research

Project Purpose

This repository contains a master's research implementation based on:

planningawal/PROTEAN_Revisi_Lurus.md

Research focus:

Server-side prototype aggregation in PROTEAN using class-specific sample-support weighting for rare attack detection under non-IID federated data.

The proposed aggregation is:


\bar{C}_j(\beta)
=
\frac{
\sum_{i \in S_j} N_{ij}^{\beta} C_{ij}
}{
\sum_{i \in S_j} N_{ij}^{\beta}
}

with planned candidate values:

beta ∈ {0.00, 0.25, 0.50, 0.75, 1.00}

where:

C_ij is client i's local prototype for class j
N_ij is the number of local training samples of class j
S_j is the set of clients contributing a valid prototype for class j

This repository is a research codebase. Scientific traceability and experimental isolation are more important than implementation convenience.

1. AUTHORITATIVE RESEARCH SOURCE

The primary research specification is:

planningawal/PROTEAN_Revisi_Lurus.md

Read this file before making methodological decisions.

Do not silently reinterpret or broaden the research objective.

If implementation evidence conflicts with an assumption in the planning document:

preserve the source document unchanged
document the discrepancy
distinguish paper behavior, code behavior, inference, and project decision
do not silently choose one interpretation

2. `planningawal/` IS IMMUTABLE

The entire directory:

planningawal/

is read-only research source material.

Agents MAY:

read files
cite them internally
use them to guide implementation

Agents MUST NOT:

edit files inside planningawal/
create files inside planningawal/
rename files inside planningawal/
move files into or out of it
delete files
autoformat its contents
rewrite Markdown formatting
place reports, logs, code, or handoffs there

All generated project material must live elsewhere.

Before major checkpoints, verify that planningawal/ remains unchanged.

3. RAW DATASETS ARE EXTERNAL AND IMMUTABLE

Raw datasets exist outside the repository:

~/datasets/5G-NIDD/5G-NIDD.csv
~/datasets/X-IIoTID/X-IIoTID-dataset.csv

These are source data.

Do NOT:

modify them
overwrite them
rename them
move them
copy full raw datasets into the repository
commit them to Git

Dataset locations must be supplied through configuration, environment variables, or CLI arguments rather than duplicated throughout source code.

Generated derivatives may be stored inside the repository under appropriate ignored artifact directories.

Raw data and generated data must always remain distinguishable.

4. PROJECT STRUCTURE

Prefer the following separation of concerns:

perdalam-protean/
├── AGENTS.md
├── README.md
├── planningawal/                # immutable research source
├── docs/
│   ├── research/
│   ├── baseline/
│   └── handoff/
├── configs/
│   ├── datasets/
│   ├── experiments/
│   └── models/
├── src/
│   └── protean_research/
│       ├── data/
│       ├── federated/
│       ├── aggregation/
│       ├── models/
│       ├── evaluation/
│       └── utils/
├── scripts/
├── tests/
├── artifacts/
│   ├── partitions/
│   ├── processed/
│   ├── checkpoints/
│   └── analysis/
└── results/

This is a design guideline, not an instruction to create empty directories without purpose.

Do not introduce new top-level structures without a clear reason.

5. SESSION CONTINUITY

The canonical session continuation file is:

docs/handoff/SESSION_HANDOFF.md

At the beginning of a new substantial session:

read AGENTS.md
read docs/handoff/SESSION_HANDOFF.md if it exists
read relevant project documentation
consult planningawal/PROTEAN_Revisi_Lurus.md when research semantics matter
inspect actual repository state before making assumptions

Do not reconstruct state from chat history when repository documentation is available.

The handoff must reflect actual repository state.

Do not copy stale plans forward as if they were completed work.

6. HANDOFF REQUIREMENT

At the end of any substantial development/research session, update:

docs/handoff/SESSION_HANDOFF.md

The handoff should contain:

current objective
completed work
important findings
files created
files modified
commands/tests performed
current architecture
methodological decisions
unresolved questions
known failures or incomplete work
Git state where relevant
exact next task
a concise ready-to-paste next-session prompt

Do not write huge terminal-only reports that future sessions cannot recover.

Long analysis should be written into project Markdown files.

7. BASELINE FIRST

Do not implement the proposed beta weighting as a replacement for PROTEAN before baseline semantics are understood.

The project must preserve a faithful original baseline.

Important unresolved or verifiable baseline properties include:

local prototype construction
missing-class handling
server aggregation participant set
aggregation denominator
class support/count availability
global prototype redistribution
prototype alignment behavior
inference behavior

Never assume:

beta = 0 == original PROTEAN

This equivalence is only valid if aggregation participants and missing-class handling are also identical.

If needed, distinguish:

original_protean
equal_available
support_weighted

equal_available should exist only if methodological evidence shows it is required as a control.

8. RESEARCH SCOPE

The intended modification is narrowly scoped to:

server-side prototype aggregation

Do NOT independently add research dimensions such as:

client rejection
Byzantine or poisoning defense
adaptive loss functions
prototype clustering
alternative distance metrics
cosine similarity experiments
dynamic architectures
personalized FL
attention modules
client scoring
prototype-quality scoring
additional aggregation algorithms

unless required to faithfully reproduce the verified baseline.

A technically interesting addition is not automatically relevant to this thesis.

9. SCIENTIFIC INTERPRETATION

N_ij represents:

class-specific sample support

It does NOT represent:

ground-truth prototype quality

Do not state or encode assumptions that a larger N_ij automatically means a better prototype.

Low-support clients may contain useful variation unavailable at higher-support clients.

Use neutral terminology such as:

support_weighted
sample_support_weighting
beta_weighted

Avoid names implying proven superiority, such as:

improved_aggregation
optimal_aggregation
better_prototype

until supported by evidence.

10. BETA SEMANTICS

The proposed weight is:

w_ij = N_ij ** beta

with:

0 <= beta <= 1

Expected properties:

beta = 0

For an identical contributor set S_j:

N_ij ** 0 = 1

therefore aggregation becomes equal weighting.

This must not automatically be labeled the original PROTEAN baseline.

beta = 1

Weight is directly proportional to class-specific support.

Because each local prototype is a local class embedding mean, this should correspond to the combined mean of included embeddings when all assumptions match.

One contributor

If |S_j| = 1, aggregation is unchanged for every beta.

Equal support

If all N_ij values are equal for fixed local prototypes, changing beta must not change the aggregate.

These invariants should eventually have tests.

11. MISSING CLASS SEMANTICS

For the proposed formulation:

S_j = clients with N_ij > 0 and a valid local prototype C_ij

Do not fabricate a prototype for a missing class.

Do not use a zero vector unless this is specifically required to reproduce the original baseline in original_protean mode.

Missing-class handling must remain explicit in code.

Do not hide it inside generic tensor padding.

12. RARE CLASS DEFINITION

For each client, rare classes are defined from TRAINING DATA.

Among attack classes satisfying:

N_ij > 0

select the two attack classes with the smallest training support.

If only one attack class is present:

evaluate that one class

If no attack class is present:

exclude that client from rare-class aggregation

Tie behavior must eventually be deterministic and documented.

13. UNSEEN CLASS DEFINITION

Attack classes satisfying:

N_ij = 0

during client training are unseen classes.

Rare classes and unseen classes are different evaluation categories.

Never merge them into one metric.

14. DATA PARTITIONING

Target Dirichlet non-IID settings are:

alpha = 0.75
alpha = 0.50
alpha = 0.25

Multiple deterministic seeds are required.

For a given:

(dataset, alpha, seed)

all compared aggregation configurations must reuse exactly the same partition.

Persist partition manifests.

Do not regenerate a different Dirichlet split for every beta candidate.

Paired comparison requires identical:

client partition
train split
validation split
test split
class mapping
model initialization policy
training hyperparameters

unless a deviation is explicitly justified and documented.

15. VALIDATION AND TEST SEPARATION

Beta selection must use validation data only.

Primary planned selection metric:

rare-class macro-F1

The test set must not influence beta selection.

Maintain a clear separation between:

candidate evaluation
beta selection
final test evaluation

Do not use test results to tune:

beta
training epochs
thresholds
seed choice
class selection
model configuration

Do not call the selected beta globally optimal.

Correct wording is:

best among the tested beta candidates under the defined validation procedure

16. REQUIRED EVALUATION GRANULARITY

Do not report only global accuracy.

Preserve enough information for analysis at:

dataset level
alpha level
seed level
client level
class level

Important outputs include:

rare-class macro-F1
rare-class recall
overall macro-F1
per-client metrics
per-class precision/recall/F1 where relevant
evaluation support
unseen-class results
N_ij
number of clients containing a class
number of contributing prototypes

Actual realized class imbalance must be recorded.

Dirichlet alpha alone is not sufficient to characterize a partition.

17. DIRECT PROTOTYPE ANALYSIS

The research distinguishes:

direct aggregation effect

from:

end-to-end training effect

Where instrumentation is implemented, preserve fixed:

C_ij
N_ij

from the same communication round and recompute candidate global prototypes without retraining.

Useful measures include:

||Cbar_j(beta) - Cbar_j(0)||

as well as:

normalized client weights
prototype displacement
distance to local prototypes
pairwise differences between beta settings

Do not interpret direct displacement alone as improved detection.

18. CODE QUALITY

Prefer:

small cohesive modules
typed interfaces where practical
deterministic behavior
explicit configuration
clear validation errors
minimal global state
no duplicated experiment logic
reusable dataset abstractions
reproducible random-state handling

Avoid:

magic constants
hardcoded absolute user paths inside research logic
giant scripts containing preprocessing, training, evaluation, and plotting together
silent fallback behavior
hidden data mutation
unnecessary dependencies
unrelated refactoring during methodological changes

Every important research parameter should be visible in configuration or experiment metadata.

19. TESTING EXPECTATIONS

Before large experiments, establish tests for critical research invariants.

Expected tests eventually include:

beta=0 arithmetic-mean property
beta=1 support-weighted property
single-contributor invariance
equal-support invariance
missing-class exclusion
normalized weight sum
finite numerical output
deterministic partitions
partition save/reload identity
baseline behavior preservation
validation/test separation

Prefer focused tests before expensive training runs.

20. COMPUTE DISCIPLINE

Do not launch a full experimental sweep merely because the runner exists.

Before expensive experiments:

unit tests must pass
data pipeline must be validated
one small smoke run must succeed
baseline behavior must be checked
output artifacts must be inspected
experiment metadata must be reproducible

Do not invent runtime estimates.

Measure first.

21. ARTIFACTS AND RESULTS

Generated outputs should be structured and machine-readable.

Prefer:

CSV
JSON
JSONL
Parquet where appropriate

over terminal-only text.

Do not overwrite previous experiment results silently.

Each experiment should be identifiable from metadata such as:

dataset
alpha
seed
aggregation_mode
beta
training configuration
code revision

Checkpoint/model files should not be committed unless explicitly required.

22. DOCUMENTATION

Research decisions that affect interpretation must be documented.

Expected documentation areas:

docs/research/
docs/baseline/
docs/handoff/

Prefer updating an existing authoritative document over creating multiple overlapping reports.

Do not create final_v2_revised_latest.md style document proliferation.

23. GIT DISCIPLINE

The intended remote repository is:

git@github.com:willtanoe/prototype-agregation.git

Before checkpointing:

inspect git status
inspect staged diff
verify no secrets
verify no raw datasets
verify no large accidental artifacts
verify planningawal/ remains unchanged

Do not force push.

Do not rewrite shared history unless explicitly instructed.

Use checkpoint commits at meaningful research/development boundaries.

24. SECURITY AND SECRETS

Never commit:

API keys
access tokens
passwords
SSH private keys
.env secrets
service credentials

Do not print discovered secrets into reports.

Use environment variables or ignored local configuration for secret material.

25. SOURCE OF TRUTH HIERARCHY

When deciding what to trust, use approximately this order:

immutable research specification for intended thesis design
verified original PROTEAN paper/code for baseline behavior
actual repository implementation
documented project decisions
inference

Do not present inference as source-confirmed fact.

When sources conflict, document the conflict rather than hiding it.

26. AGENT BEHAVIOR

Before modifying code:

inspect relevant files
understand current architecture
identify the smallest required change
preserve established behavior outside the target scope

When uncertain:

investigate evidence first
write down unresolved ambiguity if it cannot be resolved
avoid silent assumptions

Do not optimize for producing the largest amount of code.

Optimize for correctness, reproducibility, and traceability.

27. USE OF SUBAGENTS

Subagents may be used for parallel READ-ONLY investigation where tasks are independent.

Good subagent tasks:

inspect original PROTEAN source
inspect dataset schema
review experimental methodology
review code/tests
inspect reproducibility issues

Avoid multiple agents concurrently editing the same files.

A single primary agent should own integration and final edits.

Subagent findings are advisory until verified by the primary agent.

Do not merge contradictory findings without resolving their evidence.

28. BEFORE DECLARING A PHASE COMPLETE

Verify the phase against its actual acceptance criteria.

Do not declare completion because code merely runs.

For implementation phases, check:

relevant tests
expected artifacts
configuration reproducibility
no unintended baseline changes
no data leakage
no source-folder modifications

For research/planning phases, check:

evidence is documented
unknowns remain marked as unknown
the handoff is current

29. TERMINAL OUTPUT

Keep terminal summaries concise.

Long reports belong in files.

At the end of substantial work, terminal output should usually summarize:

what changed
what was verified
tests/checks performed
known blockers
handoff location
exact next task

Do not rely on terminal scrollback as project documentation.

17 KiB Executable File Raw Permalink Blame History