docs: refresh README and session handoff for current state

- README: add deep-comparison, post-hoc audit, and defense-preparation phases;
  correct the not-computed list (exploratory post-hoc tests now exist, with
  explicit small-n limits); add kisikisithesis/ and POSTHOC_STATISTICAL_AUDIT.md
  to the structure, artifact index, key documents, and reproduction commands.
- SESSION_HANDOFF.md: reflect HEAD 34bab45 and the post-16 sessions, document
  the prototype-snapshot tracking rule, refresh the next-session prompt.

README.md

@@ -170,7 +170,10 @@ Documented phases, in order. Session numbers are used only where the repository
 | Session 10 — 5G-NIDD a0.75 | second 5G-NIDD condition | `docs/research/5GNIDD_ALPHA075_BETA_VALIDATION.md`, `SESSION10_AUDIT.md` | audit PASS |
 | Session 11 — 5G-NIDD a0.25 | final condition; completes the grid | `docs/research/5GNIDD_ALPHA025_BETA_VALIDATION.md`, `SESSION11_AUDIT.md` | audit PASS |
 | Session 12 — synthesis | consolidate all six conditions | `docs/research/FINAL_EXPERIMENTAL_SYNTHESIS.md`, `CROSS_DATASET_BETA_ANALYSIS.md`, `docs/thesis/*`, `artifacts/analysis/final__*`, `artifacts/figures/final/*` | independent recomputation, zero mismatches |
+| Deep paired comparison | original vs proposed per condition | `docs/research/ORIGINAL_VS_PROPOSED_DEEP_COMPARISON.md`, `artifacts/analysis/deepcmp__*.csv`, `scripts/deep_comparison.py`, `scripts/make_comparison_figures.py` | manifest-hash pairing checks in code |
+| Post-hoc statistical audit | exploratory paired tests, selection stability | `docs/research/POSTHOC_STATISTICAL_AUDIT.md`, `artifacts/analysis/posthoc__*.csv`, `scripts/posthoc_statistical_audit.py` | manifest-hash pairing re-verified in code |
 | Proposal phase | write the thesis proposal and audit aids | `docs/proposal/*` | committed `e27c5a8` |
+| Defense preparation | robustness audit and examiner Q&A aids | `kisikisithesis/ROBUSTNESS_AUDIT_DAN_PERSIAPAN_PENGUJI.md`, `kisikisithesis/robustness_checks.py` | read-only checks over the actual implementation and persisted snapshots |
 | Handoff / README | keep repository state current | `docs/handoff/SESSION_HANDOFF.md`, `README.md` | this document |
 
 ## 7. Experiment Coverage and Reproducibility
@@ -234,9 +237,16 @@ run. No scikit-learn `classification_report` is used; metrics are pure NumPy.
 - **Per-seed** values and **pooled** values are different quantities; the pooled score averages
   per-seed per-client-mean scores.
 
-**Not computed by this repository:** micro-F1, ROC/AUC, statistical significance tests, and
-persisted per-class unseen-class recall (unseen-class evaluation is defined in code but only the
-unseen flags and union counts are saved).
+**Not computed by this repository:** micro-F1, ROC/AUC, and persisted per-class unseen-class
+recall (unseen-class evaluation is defined in code but only the unseen flags and union counts
+are saved).
+
+**Exploratory statistical audit (no inferential claims).** `scripts/posthoc_statistical_audit.py`
+(read-only) computes paired per-seed deltas for the selected beta and exploratory exact tests
+(two-sided sign-flip permutation, exact Wilcoxon signed-rank) plus paired-t 95% CIs — all with
+explicit small-n limitations: three seeds give a minimum attainable sign-flip p of 0.25, so no
+statistical significance is claimed anywhere. Outputs: `artifacts/analysis/posthoc__*.csv`;
+full discussion in `docs/research/POSTHOC_STATISTICAL_AUDIT.md`.
 
 ## 9. Quantitative Experimental Results
 
@@ -759,10 +769,11 @@ scripts/                training runner, partition generation, and read-only ana
 configs/                dataset, experiment protocol, and model configuration (YAML)
 docs/proposal/          thesis proposal and internal audit aids
 docs/research/          frozen protocol, per-condition reports, cross-alpha/cross-dataset synthesis,
-                        session audits
+                        post-hoc statistical audit, session audits
 docs/baseline/          PROTEAN baseline investigation, implementation, and validation
 docs/thesis/            Chapter 4 blueprint and claims register
 docs/handoff/           session handoff (start here for a new session)
+kisikisithesis/         robustness audit and examiner-preparation aids (read-only checks)
 planningawal/           immutable research specification (do not modify)
 artifacts/partitions/   deterministic frozen Dirichlet partition manifests (18)
 artifacts/analysis/     committed machine-readable analysis outputs
@@ -787,6 +798,12 @@ results/                local run outputs (git-ignored; 108 non-smoke run dirs +
 
 # beta=0 vs original equivalence audit (read-only)
 .venv/bin/python scripts/verify_beta0_equivalence.py --dataset 5gnidd --alpha 0.25 --seeds 0 1 2
+
+# post-hoc statistical audit (read-only; writes only posthoc__*.csv)
+.venv/bin/python scripts/posthoc_statistical_audit.py
+
+# deterministic robustness checks over the actual implementation (prints only)
+.venv/bin/python kisikisithesis/robustness_checks.py
 ```
 
 A full training run (not needed; the grid is complete) is:
@@ -842,6 +859,7 @@ Analysis outputs (`artifacts/analysis/`), by function:
 | Per-condition test | `*__multiseed__test_descriptive.csv` |
 | Cross-alpha | `{xiiotid,5gnidd}__cross_alpha__*.csv/.json` |
 | Cross-dataset | `cross_dataset__matched_alpha_summary.csv/.json`, `cross_dataset__beta_curve.csv` |
+| Post-hoc audit | `posthoc__paired_global_per_seed.csv`, `posthoc__paired_global_by_condition.csv`, `posthoc__paired_test_results.csv`, `posthoc__selection_stability.csv`, `posthoc__per_class_summary.csv`, `posthoc__per_client_summary.csv` |
 | beta=0 equivalence | `*__multiseed__beta0_equivalence.csv` |
 | Runtime | `*__multiseed__runtime.csv` |
 | Prototype snapshots & weights | `*__prototype_snapshot.json`, `*__prototype_weights.csv`, `*__prototype_effect*.csv` |
@@ -859,6 +877,7 @@ Key documents:
 | `docs/research/FINAL_EXPERIMENTAL_SYNTHESIS.md` | authoritative six-condition synthesis |
 | `docs/research/ORIGINAL_VS_PROPOSED_DEEP_COMPARISON.md` | paired original-vs-proposed deep comparison |
 | `docs/research/CROSS_DATASET_BETA_ANALYSIS.md` | cross-dataset comparison |
+| `docs/research/POSTHOC_STATISTICAL_AUDIT.md` | exploratory paired statistical audit of the completed grid |
 | `docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md` | X-IIoTID cross-alpha analysis |
 | `docs/research/5GNIDD_CROSS_ALPHA_ANALYSIS.md` | 5G-NIDD cross-alpha analysis |
 | `docs/thesis/CHAPTER4_RESULTS_OUTLINE.md` | Chapter 4 blueprint |

docs/handoff/SESSION_HANDOFF.md

@@ -3,12 +3,17 @@
 This is the canonical continuation point for the next agent session. It is written to be
 readable without prior chat history, terminal scrollback, or git history.
 
-Last updated: 2026-10-02 (Session 16). Session 16 type: **handoff refresh** — no training, no
-experiments, no source/model/aggregation change, no proposal content change.
+Last updated: 2026-10-17 (Session 18). Session 18 type: **repository hygiene and documentation
+refresh** — no training, no experiments, no source/model/aggregation change, no proposal content
+change. It committed the previously untracked post-hoc statistical audit and defense-preparation
+work, codified the prototype-snapshot tracking rule in `.gitignore`, refreshed this handoff and
+`README.md`, declared `matplotlib` in `pyproject.toml`, and applied three behavior-preserving code
+nits (docstring fix, device-safe embedding extraction, empty-table guard).
 
 > **Current state in one line.** The 108-run experimental grid is complete and synthesized; the
-> thesis proposal Markdown has been finalized and committed. Remaining work is academic writing
-> and, later, formatting into the university template — not new experiments.
+> thesis proposal Markdown has been finalized and committed; the post-hoc statistical audit and
+> robustness/defense-preparation aids are committed. Remaining work is academic writing and, later,
+> formatting into the university template — not new experiments.
 
 ---
 
@@ -80,6 +85,15 @@ It is **frozen** unless academic/supervisor feedback requires revision. It is wr
 
 - `docs/research/FINAL_EXPERIMENTAL_SYNTHESIS.md` — authoritative six-condition synthesis
   (RQ1/RQ2/RQ3 answers, threats to validity, contribution, unsupported claims, gap assessment).
+- `docs/research/ORIGINAL_VS_PROPOSED_DEEP_COMPARISON.md` — paired original-vs-proposed deep
+  comparison (`artifacts/analysis/deepcmp__*.csv`).
+- `docs/research/POSTHOC_STATISTICAL_AUDIT.md` — exploratory paired statistical audit of the
+  completed grid (`scripts/posthoc_statistical_audit.py`, `artifacts/analysis/posthoc__*.csv`);
+  explicit small-n limits (minimum attainable sign-flip p = 0.25 with three seeds); no
+  significance claims.
+- `kisikisithesis/ROBUSTNESS_AUDIT_DAN_PERSIAPAN_PENGUJI.md` — robustness audit and examiner
+  Q&A preparation; `kisikisithesis/robustness_checks.py` runs deterministic read-only checks
+  over the actual implementation and persisted snapshots (prints only, writes nothing).
 - `docs/research/CROSS_DATASET_BETA_ANALYSIS.md` — cross-dataset comparison.
 - `docs/research/XIIOTID_CROSS_ALPHA_ANALYSIS.md` — X-IIoTID cross-alpha analysis.
 - `docs/research/5GNIDD_CROSS_ALPHA_ANALYSIS.md` — 5G-NIDD cross-alpha analysis.
@@ -143,7 +157,10 @@ These constraints must be respected in every future text (proposal, chapters, sl
 3. **Later, draft thesis chapters** (starting with Chapter 4) from the final synthesis documents and
    `artifacts/analysis/final__*` + `artifacts/figures/final/`, obeying `CLAIMS_REGISTER.md` and
    `CHAPTER4_RESULTS_OUTLINE.md`.
-4. **No more experiments** unless required by new academic feedback.
+4. **For defense preparation**, use `kisikisithesis/ROBUSTNESS_AUDIT_DAN_PERSIAPAN_PENGUJI.md`
+   (robustness audit, examiner Q&A, oral cheat sheet) and
+   `docs/research/POSTHOC_STATISTICAL_AUDIT.md`; keep the exploratory-test caveats explicit.
+5. **No more experiments** unless required by new academic feedback.
 
 ---
 
@@ -174,11 +191,20 @@ These constraints must be respected in every future text (proposal, chapters, sl
 
 ## 12. Git state
 
-- Branch `main`, `HEAD = e27c5a83abcb84ab31c667b4d39d580012fa6f26`, in sync with `origin/main`.
-- Session 15 commit: `docs: finalize thesis proposal markdown` (four proposal files).
-- No tracked working-tree changes expected between sessions; only pre-existing untracked
-  `artifacts/analysis/*prototype_snapshot.json` files (94) remain untracked and must not be staged
-  accidentally.
+- Branch `main`, `HEAD = 34bab45` (`docs: improve readability of PROTEAN comparison figures`);
+  sessions after 16 added the deep paired comparison, the redesigned final figures, and the
+  experiments documentation before this branch.
+- Session 18 work lives on branch **`chore/posthoc-audit-and-docs-refresh`** (not yet merged or
+  pushed):
+  - `111adb2` `research: add post-hoc statistical audit and defense preparation aids`
+    (posthoc script + doc + 6 CSVs, `kisikisithesis/`, 4 remaining beta0-snapshot effect/weight
+    CSVs, `.gitignore` snapshot rule)
+  - documentation refresh (README, this handoff, `pyproject.toml`, three code nits) — commits
+    after `111adb2` on the same branch.
+- Prototype-snapshot tracking rule (now explicit in `.gitignore`): the **18 beta0-round-0
+  snapshots and their effect/weight CSVs are tracked**; the remaining ~76 non-beta0 snapshot
+  JSONs (~28 MB, inputs to already-committed displacement tables) stay **untracked** — cited by
+  path in the audit documents, like `results/`. Do not stage them.
 - `results/` is git-ignored. Do not commit raw datasets, large artifacts, or `results/`.
 
 ## 13. Integrity / important commands
@@ -196,12 +222,18 @@ git diff --stat
 .venv/bin/python scripts/analyze_partition_heterogeneity.py --dataset 5gnidd --alphas 0.25 0.5 0.75
 .venv/bin/python scripts/analyze_cross_alpha.py --dataset 5gnidd --alphas 0.25 0.5 0.75
 .venv/bin/python scripts/verify_beta0_equivalence.py --dataset 5gnidd --alpha 0.25 --seeds 0 1 2
+
+# Post-hoc statistical audit (read-only; writes only posthoc__*.csv)
+.venv/bin/python scripts/posthoc_statistical_audit.py
+
+# Deterministic robustness checks (prints only, writes nothing)
+.venv/bin/python kisikisithesis/robustness_checks.py
 ```
 
 ## 14. Recommended next-session prompt
 
 ```
-You are continuing the perdalam-protean research repository (Session 17).
+You are continuing the perdalam-protean research repository (Session 19).
 
 FIRST, read completely:
   1. docs/handoff/SESSION_HANDOFF.md
@@ -218,11 +250,18 @@ Critical rules:
   - Beta selection uses validation only; PRIMARY is pooled across clients and seeds.
   - The 108-run 2-dataset x 3-alpha x 3-seed grid is COMPLETE. Do NOT run new training.
   - The proposal Markdown is finalized and frozen unless academic feedback requires revision.
+  - Non-beta0 prototype snapshots stay untracked (see .gitignore); do not stage them.
 
-Context: Session 15 finalized and committed the proposal Markdown
-(docs/proposal/THESIS_PROPOSAL_DRAFT.md) at HEAD e27c5a8. Session 16 refreshed this handoff.
-"Optimasi" in the title means empirical beta selection among predefined candidates, not a global
-optimum. All six conditions are synthesized; no material scientific gap remains (classification A).
+Context: Session 18 (branch chore/posthoc-audit-and-docs-refresh) committed the previously
+untracked post-hoc statistical audit (scripts/posthoc_statistical_audit.py,
+docs/research/POSTHOC_STATISTICAL_AUDIT.md, artifacts/analysis/posthoc__*.csv) and the
+defense-preparation aids (kisikisithesis/), refreshed the handoff/README, and added matplotlib
+to pyproject.toml. Sessions after 16 added the deep paired comparison
+(docs/research/ORIGINAL_VS_PROPOSED_DEEP_COMPARISON.md, deepcmp__*.csv) and the redesigned
+final figures. "Optimasi" in the title means empirical beta selection among predefined
+candidates, not a global optimum. All six conditions are synthesized; no material scientific
+gap remains (classification A). The post-hoc tests are EXPLORATORY ONLY (n=3 seeds; minimum
+attainable sign-flip p = 0.25); never present them as significance.
 
 Task: perform the requested writing/formatting or review work using the final synthesis documents
 and artifacts/analysis/final__*. Obey CLAIMS_REGISTER.md wording, keep all limitations explicit,