experiment: validate beta selection at alpha 0.25
This commit is contained in:
willtanoe committed
1 parent
6b97a0d38b
commit
ff68d380e2
22 files changed
+33471
-209
No files matched your search
@@ -0,0 +1,4 @@
|
||||
seed,same_round0_snapshot,same_support,same_characterization,same_rare_unseen,independent_agg_maxdiff,history_rare_macro_f1_round0_diff,history_rare_macro_f1_maxdiff,test_metric_maxdiff,classification
|
||||
0,True,True,True,True,4.441e-16,0.0,0.006573313419992788,0.0015518657162341132,B
|
||||
1,True,True,True,True,4.441e-16,0.0,0.003579790124616733,0.0038506181255412164,B
|
||||
2,True,True,True,True,2.22e-16,0.0,0.0037078048655888596,0.0022243579866212393,B
|
||||
|
@@ -0,0 +1,151 @@
|
||||
seed,beta,client_id,n_samples,val_rare_macro_f1,val_rare_recall
|
||||
0,0.0,0,10551,0.8767130558393128,0.8088886018302521
|
||||
0,0.0,1,7254,0.4780361757105943,0.7391304347826086
|
||||
0,0.0,2,7159,0.8863076087410688,0.9892532741867344
|
||||
0,0.0,3,19990,0.36382113821138207,0.6711853720050441
|
||||
0,0.0,4,4082,0.30612244897959184,1.0
|
||||
0,0.0,5,2275,1.0,1.0
|
||||
0,0.0,6,8928,0.0966183574879227,1.0
|
||||
0,0.0,7,535,0.9590412621359223,0.980952380952381
|
||||
0,0.0,8,821,0.5175438596491229,0.7702702702702703
|
||||
0,0.0,9,19819,0.6795977011494253,0.532058492688414
|
||||
0,0.25,0,10551,0.8598921240050272,0.7636084790366988
|
||||
0,0.25,1,7254,0.4780361757105943,0.7391304347826086
|
||||
0,0.25,2,7159,0.8806063207817163,0.9927254964089565
|
||||
0,0.25,3,19990,0.32088694223925546,0.6519546027742749
|
||||
0,0.25,4,4082,0.30927835051546393,1.0
|
||||
0,0.25,5,2275,1.0,1.0
|
||||
0,0.25,6,8928,0.09216589861751152,1.0
|
||||
0,0.25,7,535,0.9423503325942351,0.9761904761904762
|
||||
0,0.25,8,821,0.5175438596491229,0.7702702702702703
|
||||
0,0.25,9,19819,0.6983676366217175,0.5534308211473565
|
||||
0,0.5,0,10551,0.8109255462281098,0.6833513167126383
|
||||
0,0.5,1,7254,0.5354545454545454,0.7826086956521738
|
||||
0,0.5,2,7159,0.8482759582399807,0.9961977186311788
|
||||
0,0.5,3,19990,0.33019773810370584,0.7096469104665826
|
||||
0,0.5,4,4082,0.28169014084507044,1.0
|
||||
0,0.5,5,2275,0.923076923076923,0.8571428571428571
|
||||
0,0.5,6,8928,0.08032128514056225,1.0
|
||||
0,0.5,7,535,0.8955765047135605,0.9380952380952381
|
||||
0,0.5,8,821,0.5175438596491229,0.7702702702702703
|
||||
0,0.5,9,19819,0.7294117647058823,0.592800899887514
|
||||
0,0.75,0,10551,0.7650514921373703,0.6196521609183174
|
||||
0,0.75,1,7254,0.5257936507936508,0.8695652173913043
|
||||
0,0.75,2,7159,0.8430833872010343,0.9049429657794676
|
||||
0,0.75,3,19990,0.33459165343261044,0.7481084489281211
|
||||
0,0.75,4,4082,0.3076923076923077,1.0
|
||||
0,0.75,5,2275,1.0,1.0
|
||||
0,0.75,6,8928,0.0980392156862745,1.0
|
||||
0,0.75,7,535,0.9373040752351097,0.9666666666666667
|
||||
0,0.75,8,821,0.8934426229508197,0.8243243243243243
|
||||
0,0.75,9,19819,0.8353312302839117,0.7446569178852643
|
||||
0,1.0,0,10551,0.7307935634713245,0.5777679840125225
|
||||
0,1.0,1,7254,0.4320987654320988,0.8695652173913043
|
||||
0,1.0,2,7159,0.6293270562171144,0.602991656104774
|
||||
0,1.0,3,19990,0.31020073825225003,0.7563051702395964
|
||||
0,1.0,4,4082,0.3468208092485549,1.0
|
||||
0,1.0,5,2275,1.0,1.0
|
||||
0,1.0,6,8928,0.1492537313432836,1.0
|
||||
0,1.0,7,535,0.9776119402985075,0.9571428571428571
|
||||
0,1.0,8,821,0.3870967741935484,0.32432432432432434
|
||||
0,1.0,9,19819,0.8206419131529264,0.7334083239595051
|
||||
1,0.0,0,14655,0.37485215299039587,0.8175675675675675
|
||||
1,0.0,1,5155,0.25082508250825086,0.987012987012987
|
||||
1,0.0,2,8151,0.45087719298245615,0.7747252747252747
|
||||
1,0.0,3,2774,0.553901654144097,0.7633711507293355
|
||||
1,0.0,4,2964,0.0,0.0
|
||||
1,0.0,5,7424,1.0,1.0
|
||||
1,0.0,6,14368,0.6703499079189686,0.5723270440251572
|
||||
1,0.0,7,3502,0.5696692470091484,1.0
|
||||
1,0.0,8,5412,0.9829059829059829,1.0
|
||||
1,0.0,9,17009,0.07086614173228346,0.6
|
||||
1,0.25,0,14655,0.3927903482953152,0.8310810810810811
|
||||
1,0.25,1,5155,0.2778793418647167,0.987012987012987
|
||||
1,0.25,2,8151,0.5226369617614487,0.8186813186813187
|
||||
1,0.25,3,2774,0.5843055014401075,0.7811993517017828
|
||||
1,0.25,4,2964,0.0,0.0
|
||||
1,0.25,5,7424,1.0,1.0
|
||||
1,0.25,6,14368,0.7,0.5723270440251572
|
||||
1,0.25,7,3502,0.6131182795698924,1.0
|
||||
1,0.25,8,5412,0.9871244635193133,1.0
|
||||
1,0.25,9,17009,0.06451612903225806,0.6
|
||||
1,0.5,0,14655,0.4251666666666667,0.7901969766376546
|
||||
1,0.5,1,5155,0.2639175257731959,0.8311688311688312
|
||||
1,0.5,2,8151,0.6865038413026029,0.8791208791208791
|
||||
1,0.5,3,2774,0.6338951535029966,0.7853176395568191
|
||||
1,0.5,4,2964,0.0,0.0
|
||||
1,0.5,5,7424,0.9990689013035382,0.998139534883721
|
||||
1,0.5,6,14368,0.7145708582834331,0.5628930817610063
|
||||
1,0.5,7,3502,0.5987373737373738,0.9431818181818181
|
||||
1,0.5,8,5412,0.9934924078091107,0.9956521739130435
|
||||
1,0.5,9,17009,0.05504587155963303,0.6
|
||||
1,0.75,0,14655,0.42544992973389545,0.7867613376087952
|
||||
1,0.75,1,5155,0.23255813953488372,0.7792207792207793
|
||||
1,0.75,2,8151,0.7675840978593271,0.8901098901098901
|
||||
1,0.75,3,2774,0.5923052828189471,0.7031644392486117
|
||||
1,0.75,4,2964,0.0,0.0
|
||||
1,0.75,5,7424,0.9887111947318908,0.9776744186046512
|
||||
1,0.75,6,14368,0.7421874999999999,0.5974842767295597
|
||||
1,0.75,7,3502,0.5603810591202016,0.875
|
||||
1,0.75,8,5412,0.9934924078091107,0.9956521739130435
|
||||
1,0.75,9,17009,0.05405405405405406,0.6
|
||||
1,1.0,0,14655,0.4215705629546387,0.7443884562528631
|
||||
1,1.0,1,5155,0.2141491395793499,0.7272727272727273
|
||||
1,1.0,2,8151,0.8282828282828283,0.8983516483516483
|
||||
1,1.0,3,2774,0.5832886200673137,0.6916332332545102
|
||||
1,1.0,4,2964,0.0,0.0
|
||||
1,1.0,5,7424,0.9727663640707119,0.9469767441860465
|
||||
1,1.0,6,14368,0.7120000000000001,0.559748427672956
|
||||
1,1.0,7,3502,0.5309265010351967,0.8522727272727273
|
||||
1,1.0,8,5412,0.9934924078091107,0.9956521739130435
|
||||
1,1.0,9,17009,0.052941176470588235,0.6
|
||||
2,0.0,0,22704,0.7881474513028079,0.7838358826479439
|
||||
2,0.0,1,1652,0.8051948051948051,0.7209302325581395
|
||||
2,0.0,2,6208,0.0,0.0
|
||||
2,0.0,3,16604,0.7633121431907208,0.8361356312175985
|
||||
2,0.0,4,5221,0.39530227185213707,0.7651515151515151
|
||||
2,0.0,5,5376,0.8669438669438669,0.8973214285714286
|
||||
2,0.0,6,1900,0.4444444444444444,0.4444444444444444
|
||||
2,0.0,7,506,0.9285714285714286,0.875
|
||||
2,0.0,8,6726,0.8520578420467185,0.7422480620155039
|
||||
2,0.0,9,14517,0.8190912162607596,0.7126536874624204
|
||||
2,0.25,0,22704,0.8699976526715136,0.9599266549648947
|
||||
2,0.25,1,1652,0.6461538461538462,0.4883720930232558
|
||||
2,0.25,2,6208,0.0,0.0
|
||||
2,0.25,3,16604,0.8316192298493184,0.9754098360655737
|
||||
2,0.25,4,5221,0.4228395061728395,0.7878787878787878
|
||||
2,0.25,5,5376,0.8148148148148149,0.8080357142857143
|
||||
2,0.25,6,1900,0.14159292035398233,0.8888888888888888
|
||||
2,0.25,7,506,0.9285714285714286,0.875
|
||||
2,0.25,8,6726,0.9009584664536741,0.8197674418604651
|
||||
2,0.25,9,14517,0.8626722379928422,0.7822473394408458
|
||||
2,0.5,0,22704,0.8236881827717655,0.9117352056168506
|
||||
2,0.5,1,1652,0.5333333333333333,0.37209302325581395
|
||||
2,0.5,2,6208,0.0,0.0
|
||||
2,0.5,3,16604,0.7464203337027828,0.9143897996357013
|
||||
2,0.5,4,5221,0.474621054095397,0.8333333333333333
|
||||
2,0.5,5,5376,0.78125,0.78125
|
||||
2,0.5,6,1900,0.10465116279069768,1.0
|
||||
2,0.5,7,506,0.9285714285714286,0.875
|
||||
2,0.5,8,6726,0.970573566084788,0.9428294573643411
|
||||
2,0.5,9,14517,0.8895624774115674,0.8296507581075597
|
||||
2,0.75,0,22704,0.7122906952706892,0.8397693079237714
|
||||
2,0.75,1,1652,0.20408163265306123,0.11627906976744186
|
||||
2,0.75,2,6208,0.0,0.0
|
||||
2,0.75,3,16604,0.6229273147445304,0.8570127504553734
|
||||
2,0.75,4,5221,0.4472202166064982,0.8333333333333333
|
||||
2,0.75,5,5376,0.6481481481481481,0.625
|
||||
2,0.75,6,1900,0.05607476635514019,1.0
|
||||
2,0.75,7,506,0.9285714285714286,0.875
|
||||
2,0.75,8,6726,0.9761904761904763,0.9534883720930233
|
||||
2,0.75,9,14517,0.8913439824328309,0.8408708684693009
|
||||
2,1.0,0,22704,0.6385139585209478,0.7903711133400201
|
||||
2,1.0,1,1652,0.1276595744680851,0.06976744186046512
|
||||
2,1.0,2,6208,0.0,0.0
|
||||
2,1.0,3,16604,0.5555306618870823,0.7978142076502732
|
||||
2,1.0,4,5221,0.4468259385665529,0.8333333333333333
|
||||
2,1.0,5,5376,0.568438003220612,0.5267857142857143
|
||||
2,1.0,6,1900,0.04433497536945812,1.0
|
||||
2,1.0,7,506,0.7,0.625
|
||||
2,1.0,8,6726,0.8938906752411575,0.8081395348837209
|
||||
2,1.0,9,14517,0.8945043401170115,0.8482283178868363
|
||||
|
@@ -0,0 +1,151 @@
|
||||
seed,beta,client_id,min_positive_attack_Nij,val_rare_macro_f1,val_rare_recall,delta_vs_beta0
|
||||
0,0.0,0,43,0.8767130558393128,0.8088886018302521,0.0
|
||||
0,0.0,1,21,0.4780361757105943,0.7391304347826086,0.0
|
||||
0,0.0,2,9,0.8863076087410688,0.9892532741867344,0.0
|
||||
0,0.0,3,2,0.36382113821138207,0.6711853720050441,0.0
|
||||
0,0.0,4,2,0.30612244897959184,1.0,0.0
|
||||
0,0.0,5,4,1.0,1.0,0.0
|
||||
0,0.0,6,57,0.0966183574879227,1.0,0.0
|
||||
0,0.0,7,9,0.9590412621359223,0.980952380952381,0.0
|
||||
0,0.0,8,7,0.5175438596491229,0.7702702702702703,0.0
|
||||
0,0.0,9,18,0.6795977011494253,0.532058492688414,0.0
|
||||
0,0.25,0,43,0.8598921240050272,0.7636084790366988,-0.01682093183428557
|
||||
0,0.25,1,21,0.4780361757105943,0.7391304347826086,0.0
|
||||
0,0.25,2,9,0.8806063207817163,0.9927254964089565,-0.005701287959352541
|
||||
0,0.25,3,2,0.32088694223925546,0.6519546027742749,-0.04293419597212661
|
||||
0,0.25,4,2,0.30927835051546393,1.0,0.0031559015358720854
|
||||
0,0.25,5,4,1.0,1.0,0.0
|
||||
0,0.25,6,57,0.09216589861751152,1.0,-0.00445245887041118
|
||||
0,0.25,7,9,0.9423503325942351,0.9761904761904762,-0.01669092954168727
|
||||
0,0.25,8,7,0.5175438596491229,0.7702702702702703,0.0
|
||||
0,0.25,9,18,0.6983676366217175,0.5534308211473565,0.01876993547229222
|
||||
0,0.5,0,43,0.8109255462281098,0.6833513167126383,-0.06578750961120305
|
||||
0,0.5,1,21,0.5354545454545454,0.7826086956521738,0.05741836974395109
|
||||
0,0.5,2,9,0.8482759582399807,0.9961977186311788,-0.03803165050108814
|
||||
0,0.5,3,2,0.33019773810370584,0.7096469104665826,-0.03362340010767623
|
||||
0,0.5,4,2,0.28169014084507044,1.0,-0.024432308134521408
|
||||
0,0.5,5,4,0.923076923076923,0.8571428571428571,-0.07692307692307698
|
||||
0,0.5,6,57,0.08032128514056225,1.0,-0.016297072347360456
|
||||
0,0.5,7,9,0.8955765047135605,0.9380952380952381,-0.0634647574223618
|
||||
0,0.5,8,7,0.5175438596491229,0.7702702702702703,0.0
|
||||
0,0.5,9,18,0.7294117647058823,0.592800899887514,0.04981406355645701
|
||||
0,0.75,0,43,0.7650514921373703,0.6196521609183174,-0.11166156370194247
|
||||
0,0.75,1,21,0.5257936507936508,0.8695652173913043,0.0477574750830565
|
||||
0,0.75,2,9,0.8430833872010343,0.9049429657794676,-0.04322422154003458
|
||||
0,0.75,3,2,0.33459165343261044,0.7481084489281211,-0.02922948477877163
|
||||
0,0.75,4,2,0.3076923076923077,1.0,0.0015698587127158659
|
||||
0,0.75,5,4,1.0,1.0,0.0
|
||||
0,0.75,6,57,0.0980392156862745,1.0,0.0014208581983517904
|
||||
0,0.75,7,9,0.9373040752351097,0.9666666666666667,-0.02173718690081261
|
||||
0,0.75,8,7,0.8934426229508197,0.8243243243243243,0.3758987633016968
|
||||
0,0.75,9,18,0.8353312302839117,0.7446569178852643,0.15573352913448635
|
||||
0,1.0,0,43,0.7307935634713245,0.5777679840125225,-0.1459194923679883
|
||||
0,1.0,1,21,0.4320987654320988,0.8695652173913043,-0.045937410278495516
|
||||
0,1.0,2,9,0.6293270562171144,0.602991656104774,-0.25698055252395446
|
||||
0,1.0,3,2,0.31020073825225003,0.7563051702395964,-0.05362039995913204
|
||||
0,1.0,4,2,0.3468208092485549,1.0,0.04069836026896306
|
||||
0,1.0,5,4,1.0,1.0,0.0
|
||||
0,1.0,6,57,0.1492537313432836,1.0,0.05263537385536089
|
||||
0,1.0,7,9,0.9776119402985075,0.9571428571428571,0.018570678162585175
|
||||
0,1.0,8,7,0.3870967741935484,0.32432432432432434,-0.13044708545557449
|
||||
0,1.0,9,18,0.8206419131529264,0.7334083239595051,0.14104421200350115
|
||||
1,0.0,0,1,0.37485215299039587,0.8175675675675675,0.0
|
||||
1,0.0,1,16,0.25082508250825086,0.987012987012987,0.0
|
||||
1,0.0,2,2,0.45087719298245615,0.7747252747252747,0.0
|
||||
1,0.0,3,21,0.553901654144097,0.7633711507293355,0.0
|
||||
1,0.0,4,22,0.0,0.0,0.0
|
||||
1,0.0,5,1,1.0,1.0,0.0
|
||||
1,0.0,6,138,0.6703499079189686,0.5723270440251572,0.0
|
||||
1,0.0,7,1,0.5696692470091484,1.0,0.0
|
||||
1,0.0,8,1,0.9829059829059829,1.0,0.0
|
||||
1,0.0,9,5,0.07086614173228346,0.6,0.0
|
||||
1,0.25,0,1,0.3927903482953152,0.8310810810810811,0.017938195304919302
|
||||
1,0.25,1,16,0.2778793418647167,0.987012987012987,0.027054259356465826
|
||||
1,0.25,2,2,0.5226369617614487,0.8186813186813187,0.07175976877899254
|
||||
1,0.25,3,21,0.5843055014401075,0.7811993517017828,0.030403847296010533
|
||||
1,0.25,4,22,0.0,0.0,0.0
|
||||
1,0.25,5,1,1.0,1.0,0.0
|
||||
1,0.25,6,138,0.7,0.5723270440251572,0.02965009208103131
|
||||
1,0.25,7,1,0.6131182795698924,1.0,0.04344903256074395
|
||||
1,0.25,8,1,0.9871244635193133,1.0,0.004218480613330411
|
||||
1,0.25,9,5,0.06451612903225806,0.6,-0.006350012700025401
|
||||
1,0.5,0,1,0.4251666666666667,0.7901969766376546,0.05031451367627082
|
||||
1,0.5,1,16,0.2639175257731959,0.8311688311688312,0.01309244326494502
|
||||
1,0.5,2,2,0.6865038413026029,0.8791208791208791,0.23562664832014674
|
||||
1,0.5,3,21,0.6338951535029966,0.7853176395568191,0.07999349935889966
|
||||
1,0.5,4,22,0.0,0.0,0.0
|
||||
1,0.5,5,1,0.9990689013035382,0.998139534883721,-0.0009310986964617962
|
||||
1,0.5,6,138,0.7145708582834331,0.5628930817610063,0.044220950364464495
|
||||
1,0.5,7,1,0.5987373737373738,0.9431818181818181,0.02906812672822534
|
||||
1,0.5,8,1,0.9934924078091107,0.9956521739130435,0.010586424903127889
|
||||
1,0.5,9,5,0.05504587155963303,0.6,-0.015820270172650433
|
||||
1,0.75,0,1,0.42544992973389545,0.7867613376087952,0.05059777674349958
|
||||
1,0.75,1,16,0.23255813953488372,0.7792207792207793,-0.018266942973367145
|
||||
1,0.75,2,2,0.7675840978593271,0.8901098901098901,0.316706904876871
|
||||
1,0.75,3,21,0.5923052828189471,0.7031644392486117,0.03840362867485014
|
||||
1,0.75,4,22,0.0,0.0,0.0
|
||||
1,0.75,5,1,0.9887111947318908,0.9776744186046512,-0.011288805268109159
|
||||
1,0.75,6,138,0.7421874999999999,0.5974842767295597,0.07183759208103124
|
||||
1,0.75,7,1,0.5603810591202016,0.875,-0.009288187888946808
|
||||
1,0.75,8,1,0.9934924078091107,0.9956521739130435,0.010586424903127889
|
||||
1,0.75,9,5,0.05405405405405406,0.6,-0.016812087678229407
|
||||
1,1.0,0,1,0.4215705629546387,0.7443884562528631,0.04671840996424281
|
||||
1,1.0,1,16,0.2141491395793499,0.7272727272727273,-0.03667594292890097
|
||||
1,1.0,2,2,0.8282828282828283,0.8983516483516483,0.37740563530037213
|
||||
1,1.0,3,21,0.5832886200673137,0.6916332332545102,0.029386965923216768
|
||||
1,1.0,4,22,0.0,0.0,0.0
|
||||
1,1.0,5,1,0.9727663640707119,0.9469767441860465,-0.02723363592928807
|
||||
1,1.0,6,138,0.7120000000000001,0.559748427672956,0.04165009208103143
|
||||
1,1.0,7,1,0.5309265010351967,0.8522727272727273,-0.038742745973951775
|
||||
1,1.0,8,1,0.9934924078091107,0.9956521739130435,0.010586424903127889
|
||||
1,1.0,9,5,0.052941176470588235,0.6,-0.01792496526169523
|
||||
2,0.0,0,16,0.7881474513028079,0.7838358826479439,0.0
|
||||
2,0.0,1,1,0.8051948051948051,0.7209302325581395,0.0
|
||||
2,0.0,2,1,0.0,0.0,0.0
|
||||
2,0.0,3,16,0.7633121431907208,0.8361356312175985,0.0
|
||||
2,0.0,4,11,0.39530227185213707,0.7651515151515151,0.0
|
||||
2,0.0,5,4,0.8669438669438669,0.8973214285714286,0.0
|
||||
2,0.0,6,1,0.4444444444444444,0.4444444444444444,0.0
|
||||
2,0.0,7,2,0.9285714285714286,0.875,0.0
|
||||
2,0.0,8,4,0.8520578420467185,0.7422480620155039,0.0
|
||||
2,0.0,9,1,0.8190912162607596,0.7126536874624204,0.0
|
||||
2,0.25,0,16,0.8699976526715136,0.9599266549648947,0.08185020136870569
|
||||
2,0.25,1,1,0.6461538461538462,0.4883720930232558,-0.15904095904095894
|
||||
2,0.25,2,1,0.0,0.0,0.0
|
||||
2,0.25,3,16,0.8316192298493184,0.9754098360655737,0.06830708665859753
|
||||
2,0.25,4,11,0.4228395061728395,0.7878787878787878,0.02753723432070243
|
||||
2,0.25,5,4,0.8148148148148149,0.8080357142857143,-0.052129052129052056
|
||||
2,0.25,6,1,0.14159292035398233,0.8888888888888888,-0.3028515240904621
|
||||
2,0.25,7,2,0.9285714285714286,0.875,0.0
|
||||
2,0.25,8,4,0.9009584664536741,0.8197674418604651,0.04890062440695553
|
||||
2,0.25,9,1,0.8626722379928422,0.7822473394408458,0.04358102173208256
|
||||
2,0.5,0,16,0.8236881827717655,0.9117352056168506,0.03554073146895753
|
||||
2,0.5,1,1,0.5333333333333333,0.37209302325581395,-0.2718614718614718
|
||||
2,0.5,2,1,0.0,0.0,0.0
|
||||
2,0.5,3,16,0.7464203337027828,0.9143897996357013,-0.016891809487938003
|
||||
2,0.5,4,11,0.474621054095397,0.8333333333333333,0.07931878224325994
|
||||
2,0.5,5,4,0.78125,0.78125,-0.08569386694386694
|
||||
2,0.5,6,1,0.10465116279069768,1.0,-0.3397932816537467
|
||||
2,0.5,7,2,0.9285714285714286,0.875,0.0
|
||||
2,0.5,8,4,0.970573566084788,0.9428294573643411,0.11851572403806943
|
||||
2,0.5,9,1,0.8895624774115674,0.8296507581075597,0.0704712611508077
|
||||
2,0.75,0,16,0.7122906952706892,0.8397693079237714,-0.07585675603211872
|
||||
2,0.75,1,1,0.20408163265306123,0.11627906976744186,-0.6011131725417439
|
||||
2,0.75,2,1,0.0,0.0,0.0
|
||||
2,0.75,3,16,0.6229273147445304,0.8570127504553734,-0.14038482844619038
|
||||
2,0.75,4,11,0.4472202166064982,0.8333333333333333,0.05191794475436112
|
||||
2,0.75,5,4,0.6481481481481481,0.625,-0.2187957187957188
|
||||
2,0.75,6,1,0.05607476635514019,1.0,-0.3883696780893042
|
||||
2,0.75,7,2,0.9285714285714286,0.875,0.0
|
||||
2,0.75,8,4,0.9761904761904763,0.9534883720930233,0.12413263414375775
|
||||
2,0.75,9,1,0.8913439824328309,0.8408708684693009,0.07225276617207121
|
||||
2,1.0,0,16,0.6385139585209478,0.7903711133400201,-0.1496334927818601
|
||||
2,1.0,1,1,0.1276595744680851,0.06976744186046512,-0.6775352307267201
|
||||
2,1.0,2,1,0.0,0.0,0.0
|
||||
2,1.0,3,16,0.5555306618870823,0.7978142076502732,-0.2077814813036385
|
||||
2,1.0,4,11,0.4468259385665529,0.8333333333333333,0.05152366671441583
|
||||
2,1.0,5,4,0.568438003220612,0.5267857142857143,-0.29850586372325494
|
||||
2,1.0,6,1,0.04433497536945812,1.0,-0.4001094690749863
|
||||
2,1.0,7,2,0.7,0.625,-0.22857142857142865
|
||||
2,1.0,8,4,0.8938906752411575,0.8081395348837209,0.04183283319443898
|
||||
2,1.0,9,1,0.8945043401170115,0.8482283178868363,0.07541312385625187
|
||||
|
@@ -0,0 +1,16 @@
|
||||
seed,beta,n_eligible,n_improve_vs_beta0,n_degrade_vs_beta0,n_unchanged,mean_delta_vs_beta0,median_delta_vs_beta0,min_delta_vs_beta0,max_delta_vs_beta0
|
||||
0,0.0,10,0,0,10,0.0,0.0,0.0,0.0
|
||||
0,0.25,10,2,5,3,-0.0064673967169698865,-0.00222622943520559,-0.04293419597212661,0.01876993547229222
|
||||
0,0.5,10,2,7,1,-0.021132734174687993,-0.029027854121098817,-0.07692307692307698,0.05741836974395109
|
||||
0,0.75,10,5,4,1,0.037652802750874605,0.0007104290991758952,-0.11166156370194247,0.3758987633016968
|
||||
0,1.0,10,4,5,1,-0.037995631629473454,-0.022968705139247758,-0.25698055252395446,0.14104421200350115
|
||||
1,0.0,10,0,0,10,0.0,0.0,0.0,0.0
|
||||
1,0.25,10,7,1,2,0.02181236632914685,0.022496227330692564,-0.006350012700025401,0.07175976877899254
|
||||
1,0.5,10,7,2,1,0.04461512377469677,0.02108028499658518,-0.015820270172650433,0.23562664832014674
|
||||
1,0.75,10,5,4,1,0.04324763034707273,0.0052932124515639445,-0.018266942973367145,0.316706904876871
|
||||
1,1.0,10,5,4,1,0.0385170238078155,0.0052932124515639445,-0.038742745973951775,0.37740563530037213
|
||||
2,0.0,10,0,0,10,0.0,0.0,0.0,0.0
|
||||
2,0.25,10,5,3,2,-0.024384536677342937,0.013768617160351215,-0.3028515240904621,0.08185020136870569
|
||||
2,0.5,10,4,4,2,-0.04103939310459288,0.0,-0.3397932816537467,0.11851572403806943
|
||||
2,0.75,10,3,5,2,-0.1176216808834886,-0.03792837801605936,-0.6011131725417439,0.12413263414375775
|
||||
2,1.0,10,3,6,1,-0.1793367342416782,-0.1787074870427493,-0.6775352307267201,0.07541312385625187
|
||||
|
@@ -0,0 +1,46 @@
|
||||
class_id,class_name,beta,n_seeds,n_contributors_seed0,mean_displacement,std_sample_displacement,min_displacement,max_displacement,mean_relative_displacement
|
||||
0,C&C,0.0,3,10,0.0,0.0,0.0,0.0,0.0
|
||||
0,C&C,0.25,3,10,1.27837054967781,0.41822833159420114,0.8053802494515935,1.5992758251729138,0.27003430350349095
|
||||
0,C&C,0.5,3,10,2.3990142795607627,0.581298481867443,1.7282910993003162,2.7568644582945026,0.5059337917781374
|
||||
0,C&C,0.75,3,10,3.306991238919704,0.508900823524494,2.719365234189696,3.6021485283426884,0.6970245995294005
|
||||
0,C&C,1.0,3,10,4.01952572799183,0.29030488023104745,3.691238890234156,4.24238658552617,0.8472326753379348
|
||||
1,Exfiltration,0.0,3,8,0.0,0.0,0.0,0.0,0.0
|
||||
1,Exfiltration,0.25,3,8,2.1505748980950266,0.3101835560753807,1.7945831890795807,2.362721386473373,0.33309243067039757
|
||||
1,Exfiltration,0.5,3,8,3.3139866073431565,0.5408774954238195,2.7215672864242446,3.7814503953657908,0.5118900118897919
|
||||
1,Exfiltration,0.75,3,8,3.939594030444926,0.7312554933468559,3.239979172736248,4.6988387374077725,0.6069298415028075
|
||||
1,Exfiltration,1.0,3,8,4.316178984369498,0.8975835034935007,3.588238413562595,5.319080439597726,0.6638254826227857
|
||||
2,Exploitation,0.0,3,6,0.0,0.0,0.0,0.0,0.0
|
||||
2,Exploitation,0.25,3,6,1.1587284531130924,0.36096288681414823,0.7459613372186213,1.4152287416432763,0.23470674175846215
|
||||
2,Exploitation,0.5,3,6,2.1283145398223593,0.7573863420735953,1.2547532262512742,2.6011790535745782,0.43060284027932577
|
||||
2,Exploitation,0.75,3,6,2.8881691399860485,1.0052398500157818,1.7282726897405847,3.5066832125742606,0.5846207929781134
|
||||
2,Exploitation,1.0,3,6,3.464004570499769,1.112779630373774,2.1801801381263144,4.152010451170094,0.7020410983796918
|
||||
3,Lateral_movement,0.0,3,9,0.0,0.0,0.0,0.0,0.0
|
||||
3,Lateral_movement,0.25,3,9,1.983346175953378,0.5729109230595639,1.451269830260352,2.5898222983977996,0.4196845501731568
|
||||
3,Lateral_movement,0.5,3,9,3.0206065448113826,1.0150094706368122,2.0165054024590443,4.046183521651553,0.639663941527728
|
||||
3,Lateral_movement,0.75,3,9,3.500429531100328,1.2632988067241848,2.238889947127781,4.765480241626273,0.7415506396377006
|
||||
3,Lateral_movement,1.0,3,9,3.7263813611395125,1.3833916615643342,2.3423286413016733,5.109111015402566,0.7895377723300525
|
||||
4,Normal,0.0,3,10,0.0,0.0,0.0,0.0,0.0
|
||||
4,Normal,0.25,3,10,1.0864756872230459,0.496567699923766,0.7950378669816794,1.6598363613323508,0.20451956788647774
|
||||
4,Normal,0.5,3,10,1.5767289610073465,0.5644465343832925,1.2491208619495557,2.228492630024489,0.2945513132600518
|
||||
4,Normal,0.75,3,10,1.8318768258260405,0.552047828490767,1.506814977752638,2.4692844719040514,0.34065176127320745
|
||||
4,Normal,1.0,3,10,1.9966341269597232,0.5327206047251788,1.6673282103552307,2.611242030412833,0.3701992118201333
|
||||
5,RDOS,0.0,3,10,0.0,0.0,0.0,0.0,0.0
|
||||
5,RDOS,0.25,3,10,1.0860124145813117,0.4602780439364506,0.8192120495667494,1.6174943118797114,0.13866684735468646
|
||||
5,RDOS,0.5,3,10,1.7190552688418272,0.6887072700340696,1.321273511038344,2.514305903819429,0.21934364560707523
|
||||
5,RDOS,0.75,3,10,2.1822469246461806,0.9474271568645856,1.6261816527161852,3.2761911842442637,0.2787148221815406
|
||||
5,RDOS,1.0,3,10,2.5658707936642617,1.2193483512552363,1.8476687608995317,3.9737567173599997,0.3281078404629119
|
||||
6,Reconnaissance,0.0,3,8,0.0,0.0,0.0,0.0,0.0
|
||||
6,Reconnaissance,0.25,3,8,1.4788712344122157,0.594533938907045,0.9730263451403185,2.1337405853808944,0.28537388942486464
|
||||
6,Reconnaissance,0.5,3,8,2.350247764054481,0.7603933259238349,1.5662204391401529,3.084568124445345,0.4522079760615029
|
||||
6,Reconnaissance,0.75,3,8,2.9233293995154472,0.8953049366648174,1.8921171248359299,3.502353696944525,0.5612721092899503
|
||||
6,Reconnaissance,1.0,3,8,3.3391889586431325,1.1073644700689869,2.088616712497195,4.1953600367224775,0.6402396428113026
|
||||
7,Tampering,0.0,3,8,0.0,0.0,0.0,0.0,0.0
|
||||
7,Tampering,0.25,3,8,1.3505936491446324,0.2586312123810999,1.0546580118479258,1.5332987381385625,0.30342797209829775
|
||||
7,Tampering,0.5,3,8,2.5524751358791735,0.4082046541057479,2.1814748752660527,2.989769119570483,0.5767593796858125
|
||||
7,Tampering,0.75,3,8,3.4991562847024547,0.3575352999845257,3.2805423027173237,3.9117575814369485,0.7935853172069489
|
||||
7,Tampering,1.0,3,8,4.247813103636777,0.24113229030491104,3.9775464519022146,4.440926893005426,0.9653304277937593
|
||||
8,Weaponization,0.0,3,8,0.0,0.0,0.0,0.0,0.0
|
||||
8,Weaponization,0.25,3,8,1.7105264729761451,0.22286866444144524,1.4995962499113429,1.9436699786803406,0.30652098641043396
|
||||
8,Weaponization,0.5,3,8,2.5685181426368175,0.35839871727693023,2.232233440488681,2.945548238376779,0.4619957398263043
|
||||
8,Weaponization,0.75,3,8,2.951764426805454,0.4937625539064774,2.42251119168137,3.4000203434274914,0.5317633959257942
|
||||
8,Weaponization,1.0,3,8,3.1235303582992926,0.585141395295389,2.485749772741962,3.6355977615306365,0.5634271266802224
|
||||
|
@@ -0,0 +1,19 @@
|
||||
seed,tag,wall_seconds,cpu_percent,peak_rss_kb,source
|
||||
0,original,2410.80,746,8790948,/tmp/opencode/s7_logs/seed0__original_protean_betaNA.time
|
||||
0,beta0,2457.24,744,8791100,/tmp/opencode/s7_logs/seed0__support_weighted_beta0.0.time
|
||||
0,beta0.25,2427.96,745,8790844,/tmp/opencode/s7_logs/seed0__support_weighted_beta0.25.time
|
||||
0,beta0.5,2421.77,743,8791344,/tmp/opencode/s7_logs/seed0__support_weighted_beta0.5.time
|
||||
0,beta0.75,2389.26,746,8790992,/tmp/opencode/s7_logs/seed0__support_weighted_beta0.75.time
|
||||
0,beta1,2425.86,745,8791192,/tmp/opencode/s7_logs/seed0__support_weighted_beta1.0.time
|
||||
1,original,2429.86,745,11323828,/tmp/opencode/s7_logs/seed1__original_protean_betaNA.time
|
||||
1,beta0,2392.79,745,11323896,/tmp/opencode/s7_logs/seed1__support_weighted_beta0.0.time
|
||||
1,beta0.25,2445.79,745,11324076,/tmp/opencode/s7_logs/seed1__support_weighted_beta0.25.time
|
||||
1,beta0.5,2418.93,745,11324196,/tmp/opencode/s7_logs/seed1__support_weighted_beta0.5.time
|
||||
1,beta0.75,2398.73,744,11323956,/tmp/opencode/s7_logs/seed1__support_weighted_beta0.75.time
|
||||
1,beta1,2447.41,745,11324036,/tmp/opencode/s7_logs/seed1__support_weighted_beta1.0.time
|
||||
2,original,2413.48,745,7875900,/tmp/opencode/s7_logs/seed2__original_protean_betaNA.time
|
||||
2,beta0,2424.04,746,7875560,/tmp/opencode/s7_logs/seed2__support_weighted_beta0.0.time
|
||||
2,beta0.25,2505.12,743,7875828,/tmp/opencode/s7_logs/seed2__support_weighted_beta0.25.time
|
||||
2,beta0.5,2422.35,744,7875624,/tmp/opencode/s7_logs/seed2__support_weighted_beta0.5.time
|
||||
2,beta0.75,2531.66,741,7875584,/tmp/opencode/s7_logs/seed2__support_weighted_beta0.75.time
|
||||
2,beta1,2435.98,744,7875704,/tmp/opencode/s7_logs/seed2__support_weighted_beta1.0.time
|
||||
|
@@ -0,0 +1,111 @@
|
||||
{
|
||||
"alpha": 0.25,
|
||||
"betas": [
|
||||
0.0,
|
||||
0.25,
|
||||
0.5,
|
||||
0.75,
|
||||
1.0
|
||||
],
|
||||
"dataset": "xiiotid",
|
||||
"missing_artifacts": [],
|
||||
"n_missing_artifacts": 0,
|
||||
"outputs": {
|
||||
"client_deltas": "/home/willtanoe/projects/perdalam-protean/artifacts/analysis/xiiotid__alpha0.25__multiseed__client_validation_deltas.csv",
|
||||
"client_validation_detail": "/home/willtanoe/projects/perdalam-protean/artifacts/analysis/xiiotid__alpha0.25__multiseed__client_validation_detail.csv",
|
||||
"client_validation_summary": "/home/willtanoe/projects/perdalam-protean/artifacts/analysis/xiiotid__alpha0.25__multiseed__client_validation_summary.csv",
|
||||
"per_seed": "/home/willtanoe/projects/perdalam-protean/artifacts/analysis/xiiotid__alpha0.25__multiseed__validation_per_seed.csv",
|
||||
"pooled": "/home/willtanoe/projects/perdalam-protean/artifacts/analysis/xiiotid__alpha0.25__multiseed__validation_pooled.csv",
|
||||
"prototype_effect_pooled": "/home/willtanoe/projects/perdalam-protean/artifacts/analysis/xiiotid__alpha0.25__multiseed__prototype_effect_pooled.csv",
|
||||
"test_descriptive": "/home/willtanoe/projects/perdalam-protean/artifacts/analysis/xiiotid__alpha0.25__multiseed__test_descriptive.csv"
|
||||
},
|
||||
"per_seed_diagnostic": [
|
||||
{
|
||||
"seed": 0,
|
||||
"selected_beta": 0.75,
|
||||
"test_used_for_selection": false,
|
||||
"val_rare_macro_f1_per_client_mean": 0.6540329635413089
|
||||
},
|
||||
{
|
||||
"seed": 1,
|
||||
"selected_beta": 0.5,
|
||||
"test_used_for_selection": false,
|
||||
"val_rare_macro_f1_per_client_mean": 0.5370398599938551
|
||||
},
|
||||
{
|
||||
"seed": 2,
|
||||
"selected_beta": 0.0,
|
||||
"test_used_for_selection": false,
|
||||
"val_rare_macro_f1_per_client_mean": 0.666306546980769
|
||||
}
|
||||
],
|
||||
"pooled_table": [
|
||||
{
|
||||
"beta": 0.0,
|
||||
"max_val_rare_macro_f1": 0.666306546980769,
|
||||
"min_val_rare_macro_f1": 0.4924247362191584,
|
||||
"n_seeds": 3,
|
||||
"pooled_val_rare_macro_f1_per_client_mean": 0.5917038146634539,
|
||||
"pooled_val_rare_recall_global": 0.8285135973975263,
|
||||
"pooled_val_rare_recall_per_client_mean": 0.7594821244948339,
|
||||
"std_pop_val_rare_macro_f1": 0.07309998892871802,
|
||||
"std_sample_val_rare_macro_f1": 0.08952883653922933
|
||||
},
|
||||
{
|
||||
"beta": 0.25,
|
||||
"max_val_rare_macro_f1": 0.641922010303426,
|
||||
"min_val_rare_macro_f1": 0.5142371025483052,
|
||||
"n_seeds": 3,
|
||||
"pooled_val_rare_macro_f1_per_client_mean": 0.5886906256417319,
|
||||
"pooled_val_rare_recall_global": 0.8310900379118298,
|
||||
"pooled_val_rare_recall_per_client_mean": 0.7807713039840465,
|
||||
"std_pop_val_rare_macro_f1": 0.05424415966608841,
|
||||
"std_sample_val_rare_macro_f1": 0.06643525635398827
|
||||
},
|
||||
{
|
||||
"beta": 0.5,
|
||||
"max_val_rare_macro_f1": 0.625267153876176,
|
||||
"min_val_rare_macro_f1": 0.5370398599938551,
|
||||
"n_seeds": 3,
|
||||
"pooled_val_rare_macro_f1_per_client_mean": 0.5858514801619258,
|
||||
"pooled_val_rare_recall_global": 0.814237235145609,
|
||||
"pooled_val_rare_recall_per_client_mean": 0.7725355473131942,
|
||||
"std_pop_val_rare_macro_f1": 0.03662628112682344,
|
||||
"std_sample_val_rare_macro_f1": 0.044857849968223555
|
||||
},
|
||||
{
|
||||
"beta": 0.75,
|
||||
"max_val_rare_macro_f1": 0.6540329635413089,
|
||||
"min_val_rare_macro_f1": 0.5356723665662311,
|
||||
"n_seeds": 3,
|
||||
"pooled_val_rare_macro_f1_per_client_mean": 0.5794633987349401,
|
||||
"pooled_val_rare_recall_global": 0.7756026677004408,
|
||||
"pooled_val_rare_recall_per_client_mean": 0.7607912573123681,
|
||||
"std_pop_val_rare_macro_f1": 0.05299557391939959,
|
||||
"std_sample_val_rare_macro_f1": 0.0649060573642385
|
||||
},
|
||||
{
|
||||
"beta": 1.0,
|
||||
"max_val_rare_macro_f1": 0.5783845291609608,
|
||||
"min_val_rare_macro_f1": 0.48696981273909073,
|
||||
"n_seeds": 3,
|
||||
"pooled_val_rare_macro_f1_per_client_mean": 0.5320987006423418,
|
||||
"pooled_val_rare_recall_global": 0.7173552521012324,
|
||||
"pooled_val_rare_recall_per_client_mean": 0.704574711153059,
|
||||
"std_pop_val_rare_macro_f1": 0.037328867098742294,
|
||||
"std_sample_val_rare_macro_f1": 0.045718338534042856
|
||||
}
|
||||
],
|
||||
"primary_selection": {
|
||||
"rule": "max pooled validation per-client-mean rare-class macro-F1 (tie-break: recall, then lower std, then smaller beta)",
|
||||
"selected_beta": 0.0,
|
||||
"selection_split": "validation",
|
||||
"test_used_for_selection": false,
|
||||
"tie_break_used": false
|
||||
},
|
||||
"seeds": [
|
||||
0,
|
||||
1,
|
||||
2
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
seed,aggregation,beta,test_rare_macro_f1,test_rare_recall,test_macro_f1,test_accuracy,test_weighted_f1,test_rare_support
|
||||
0,support_weighted_beta0,0.0,0.6413622282749385,0.830700667573069,0.6745583013411162,0.8745509147521049,0.9061557688476052,79180
|
||||
0,support_weighted_beta0.25,0.25,0.6450645576749079,0.8225881730832947,0.6777953236518771,0.8666347317157053,0.9069221037691833,79180
|
||||
0,support_weighted_beta0.5,0.5,0.6317424352754177,0.7942361392087552,0.6659654615240669,0.8532159109751829,0.8989312745528685,79180
|
||||
0,support_weighted_beta0.75,0.75,0.6661769665289847,0.7860825017014961,0.6970424816039874,0.8546468424317237,0.9015580009841035,79180
|
||||
0,support_weighted_beta1,1.0,0.5841314629029717,0.6978253919369697,0.6244888388326827,0.8470929982620017,0.8921046450361301,79180
|
||||
0,original_protean,,0.6398103625587044,0.8296012943628512,0.6731025782780014,0.8737218342944525,0.9054655196913645,79180
|
||||
1,support_weighted_beta0,0.0,0.5805553678969027,0.8256994230374527,0.6650385577782836,0.8634903673133494,0.899863925003673,50928
|
||||
1,support_weighted_beta0.25,0.25,0.6058924033633463,0.8348945476495885,0.6853679819763017,0.8716460624819599,0.9074814344488223,50928
|
||||
1,support_weighted_beta0.5,0.5,0.6486662065718658,0.8267491767388827,0.7192562168644683,0.8823934017478244,0.9165028567563237,50928
|
||||
1,support_weighted_beta0.75,0.75,0.6563526907348872,0.8114174349128286,0.7248191993206956,0.8810116009850704,0.9165944358099051,50928
|
||||
1,support_weighted_beta1,1.0,0.6533153255816548,0.8002798521046107,0.7223464690799543,0.8767617959725114,0.9134156189816298,50928
|
||||
1,original_protean,,0.5825215614975117,0.8266995877156649,0.6669813754005415,0.8673409854388906,0.9020747305215218,50928
|
||||
2,support_weighted_beta0,0.0,0.6744891962601014,0.807370314447251,0.7044332575239617,0.8840024319693425,0.9146272396181951,79180
|
||||
2,support_weighted_beta0.25,0.25,0.6593623777922051,0.8144273519980725,0.690739333811702,0.8737341169678993,0.9107388028746422,79180
|
||||
2,support_weighted_beta0.5,0.5,0.641339655463725,0.800380991610641,0.6752636268058488,0.8713205716356222,0.9106725545555584,79180
|
||||
2,support_weighted_beta0.75,0.75,0.5589726904308177,0.6983357629940257,0.6021421914368235,0.8532773243424163,0.8909131680484705,79180
|
||||
2,support_weighted_beta1,1.0,0.5010089560026679,0.6246979149481214,0.5507942039876121,0.8401962771216783,0.8724513135414319,79180
|
||||
2,original_protean,,0.6760389917828752,0.8095946724338723,0.7059686496708458,0.8852306993140127,0.9154163421325315,79180
|
||||
|
@@ -0,0 +1,16 @@
|
||||
seed,beta,eligible_clients,val_rare_macro_f1_per_client_mean,val_rare_macro_f1_global,val_rare_recall_per_client_mean,val_rare_recall_global,val_macro_f1_global,val_accuracy_global,val_weighted_f1_global,manifest_content_hash,manifest_reload_ok
|
||||
0,0.0,10,0.6163801607904343,0.6429394238517592,0.8491738826715703,0.8418861263689219,0.6760317563154139,0.8747758370796177,0.9063720888612694,839ba18e2d87245d198be8f5f89d4a24f13cdb6fc64c004d406c6d4f8246f3da,True
|
||||
0,0.25,10,0.6099127640734644,0.64642287320039,0.8447310580610642,0.8345804291098624,0.6790888091486504,0.8670376102390253,0.9071365824193555,839ba18e2d87245d198be8f5f89d4a24f13cdb6fc64c004d406c6d4f8246f3da,True
|
||||
0,0.5,10,0.5952474266157463,0.6336768366893184,0.8330113906858452,0.8071722252443003,0.6677548068733682,0.8532561967229223,0.899144941308395,839ba18e2d87245d198be8f5f89d4a24f13cdb6fc64c004d406c6d4f8246f3da,True
|
||||
0,0.75,10,0.6540329635413089,0.6664773722358116,0.8677916701893466,0.7974003562167593,0.6973534348602826,0.8543125261011619,0.9012559311699491,839ba18e2d87245d198be8f5f89d4a24f13cdb6fc64c004d406c6d4f8246f3da,True
|
||||
0,1.0,10,0.5783845291609608,0.5857548026131679,0.7821505533174884,0.7103049547244267,0.6260176153436492,0.8471884442479181,0.8919882287550097,839ba18e2d87245d198be8f5f89d4a24f13cdb6fc64c004d406c6d4f8246f3da,True
|
||||
1,0.0,10,0.4924247362191584,0.5809756273782132,0.7515004024060322,0.8242919155450245,0.6650214090100871,0.8617068317488392,0.8981287468366383,84c20b4bd6cd20010fa35d139a0416cb06e79c82fbd8e1d57441cafdcdfbf15d,True
|
||||
1,0.25,10,0.5142371025483052,0.6055575436981985,0.7590301782502327,0.8331880351078561,0.6848471122978553,0.8703908418699486,0.9057332472478276,84c20b4bd6cd20010fa35d139a0416cb06e79c82fbd8e1d57441cafdcdfbf15d,True
|
||||
1,0.5,10,0.5370398599938551,0.6475675504916489,0.7385670935223774,0.8293216255316027,0.7181550199522855,0.8809172869531039,0.9145666731523477,84c20b4bd6cd20010fa35d139a0416cb06e79c82fbd8e1d57441cafdcdfbf15d,True
|
||||
1,0.75,10,0.5356723665662311,0.6555535221155093,0.7205067315435331,0.8170169039121253,0.723896208561121,0.8792099638882748,0.9146947997656324,84c20b4bd6cd20010fa35d139a0416cb06e79c82fbd8e1d57441cafdcdfbf15d,True
|
||||
1,1.0,10,0.5309417600269739,0.6522566530641632,0.7016296138176522,0.8039397728109426,0.7211637751072608,0.8743459355884737,0.9110267072936014,84c20b4bd6cd20010fa35d139a0416cb06e79c82fbd8e1d57441cafdcdfbf15d,True
|
||||
2,0.0,10,0.666306546980769,0.6739452923674923,0.6777720884068994,0.8193627502786327,0.7039106193272061,0.8828948338123664,0.9139200328543172,d6e68ee13cb11f98cfb4aea668f3e905d3ebc94884632352c0045fc5f86df28c,True
|
||||
2,0.25,10,0.641922010303426,0.6554024698010497,0.7385526756408426,0.8255016495177709,0.6871335591276826,0.8720244675362959,0.9093649291176834,d6e68ee13cb11f98cfb4aea668f3e905d3ebc94884632352c0045fc5f86df28c,True
|
||||
2,0.5,10,0.625267153876176,0.6362812290985806,0.74602815773136,0.8062178546609238,0.6706513066568411,0.868462426609674,0.908575732835077,d6e68ee13cb11f98cfb4aea668f3e905d3ebc94884632352c0045fc5f86df28c,True
|
||||
2,0.75,10,0.5486848660972803,0.5580441540296612,0.6940753702042244,0.7123907429724381,0.6012106363245412,0.8510575576682143,0.8894144424905409,d6e68ee13cb11f98cfb4aea668f3e905d3ebc94884632352c0045fc5f86df28c,True
|
||||
2,1.0,10,0.48696981273909073,0.49971103336651723,0.6299439663240364,0.6378210287683279,0.5495578946710995,0.8385167170265556,0.8710862911673571,d6e68ee13cb11f98cfb4aea668f3e905d3ebc94884632352c0045fc5f86df28c,True
|
||||
|
@@ -0,0 +1,6 @@
|
||||
beta,n_seeds,pooled_val_rare_macro_f1_per_client_mean,std_sample_val_rare_macro_f1,std_pop_val_rare_macro_f1,min_val_rare_macro_f1,max_val_rare_macro_f1,pooled_val_rare_recall_per_client_mean,pooled_val_rare_recall_global
|
||||
0.0,3,0.5917038146634539,0.08952883653922933,0.07309998892871802,0.4924247362191584,0.666306546980769,0.7594821244948339,0.8285135973975263
|
||||
0.25,3,0.5886906256417319,0.06643525635398827,0.05424415966608841,0.5142371025483052,0.641922010303426,0.7807713039840465,0.8310900379118298
|
||||
0.5,3,0.5858514801619258,0.044857849968223555,0.03662628112682344,0.5370398599938551,0.625267153876176,0.7725355473131942,0.814237235145609
|
||||
0.75,3,0.5794633987349401,0.0649060573642385,0.05299557391939959,0.5356723665662311,0.6540329635413089,0.7607912573123681,0.7756026677004408
|
||||
1.0,3,0.5320987006423418,0.045718338534042856,0.037328867098742294,0.48696981273909073,0.5783845291609608,0.704574711153059,0.7173552521012324
|
||||
|
+10134
File diff suppressed because it is too large.
Load diff
+46
@@ -0,0 +1,46 @@
|
||||
class_id,class_name,beta,n_contributors,support_min,support_max,support_mean,prototype_norm,reference_norm_beta0,displacement_from_beta0,relative_displacement
|
||||
0,C&C,0.0,10,21,1065,200.4,4.779006743952039,4.779006743952039,0.0,0.0
|
||||
0,C&C,0.25,10,21,1065,200.4,4.614577391616592,4.779006743952039,0.8053802494515935,0.1685246103640309
|
||||
0,C&C,0.5,10,21,1065,200.4,4.685268392014225,4.779006743952039,1.7282910993003162,0.36164232274572844
|
||||
0,C&C,0.75,10,21,1065,200.4,5.044029006129829,4.779006743952039,2.719365234189696,0.5690231003819204
|
||||
0,C&C,1.0,10,21,1065,200.4,5.616978848302162,4.779006743952039,3.691238890234156,0.772386206591049
|
||||
1,Exfiltration,0.0,8,4,7950,1936.75,6.314177427425566,6.314177427425566,0.0,0.0
|
||||
1,Exfiltration,0.25,8,4,7950,1936.75,7.4029341676828455,6.314177427425566,2.2944201187321265,0.3633759337782204
|
||||
1,Exfiltration,0.5,8,4,7950,1936.75,8.213834927949703,6.314177427425566,3.4389421402394342,0.5446381860133394
|
||||
1,Exfiltration,0.75,8,4,7950,1936.75,8.57974765740683,6.314177427425566,3.8799641811907577,0.6144845034506272
|
||||
1,Exfiltration,1.0,8,4,7950,1936.75,8.7179035285881,6.314177427425566,4.041218099948173,0.64002289235573
|
||||
2,Exploitation,0.0,6,2,595,132.16666666666666,4.509776364275994,4.509776364275994,0.0,0.0
|
||||
2,Exploitation,0.25,6,2,595,132.16666666666666,4.248170040665384,4.509776364275994,0.7459613372186213,0.16540982899456458
|
||||
2,Exploitation,0.5,6,2,595,132.16666666666666,4.410032263950292,4.509776364275994,1.2547532262512742,0.27822958942947806
|
||||
2,Exploitation,0.75,6,2,595,132.16666666666666,4.79779903763718,4.509776364275994,1.7282726897405847,0.38322802510364484
|
||||
2,Exploitation,1.0,6,2,595,132.16666666666666,5.22118256993748,4.509776364275994,2.1801801381263144,0.483434202058559
|
||||
3,Lateral_movement,0.0,9,2,13857,2457.4444444444443,4.726865250941442,4.726865250941442,0.0,0.0
|
||||
3,Lateral_movement,0.25,9,2,13857,2457.4444444444443,5.188159217707745,4.726865250941442,1.908946399201982,0.40385039510525084
|
||||
3,Lateral_movement,0.5,9,2,13857,2457.4444444444443,5.735292868968967,4.726865250941442,2.999130710323551,0.6344861871673239
|
||||
3,Lateral_movement,0.75,9,2,13857,2457.4444444444443,6.104720435805063,4.726865250941442,3.4969184045469306,0.7397965076009846
|
||||
3,Lateral_movement,1.0,9,2,13857,2457.4444444444443,6.372689060687179,4.726865250941442,3.727704426714298,0.7886208361814115
|
||||
4,Normal,0.0,10,1,122800,29277.7,4.8309698078336565,4.8309698078336565,0.0,0.0
|
||||
4,Normal,0.25,10,1,122800,29277.7,6.155985925751386,4.8309698078336565,1.6598363613323508,0.3435824332085131
|
||||
4,Normal,0.5,10,1,122800,29277.7,6.687542914070098,4.8309698078336565,2.228492630024489,0.4612930154129463
|
||||
4,Normal,0.75,10,1,122800,29277.7,6.936086518528269,4.8309698078336565,2.4692844719040514,0.5111363908546861
|
||||
4,Normal,1.0,10,1,122800,29277.7,7.08021017092975,4.8309698078336565,2.611242030412833,0.5405212895718319
|
||||
5,RDOS,0.0,10,166,31089,9888.3,8.229149264804645,8.229149264804645,0.0,0.0
|
||||
5,RDOS,0.25,10,166,31089,9888.3,8.685177781543024,8.229149264804645,0.8192120495667494,0.09955002919566036
|
||||
5,RDOS,0.5,10,166,31089,9888.3,8.900437840007243,8.229149264804645,1.3215863916677093,0.1605981796101353
|
||||
5,RDOS,0.75,10,166,31089,9888.3,8.965437965289066,8.229149264804645,1.6443679369780935,0.19982234907451635
|
||||
5,RDOS,1.0,10,166,31089,9888.3,8.960933599821217,8.229149264804645,1.8761869027332534,0.22799281461056264
|
||||
6,Reconnaissance,0.0,8,46,65307,11145.125,5.2904394168358095,5.2904394168358095,0.0,0.0
|
||||
6,Reconnaissance,0.25,8,46,65307,11145.125,6.212406287806259,5.2904394168358095,1.3298467727154344,0.2513679238974842
|
||||
6,Reconnaissance,0.5,8,46,65307,11145.125,6.770340724052806,5.2904394168358095,2.399954728577945,0.4536399605939251
|
||||
6,Reconnaissance,0.75,8,46,65307,11145.125,7.2060433894741625,5.2904394168358095,3.3755173767658864,0.6380410228352581
|
||||
6,Reconnaissance,1.0,8,46,65307,11145.125,7.591981109259281,5.2904394168358095,4.1953600367224775,0.7930078593040019
|
||||
7,Tampering,0.0,8,9,2272,448.25,4.054842335154068,4.054842335154068,0.0,0.0
|
||||
7,Tampering,0.25,8,9,2272,448.25,4.115353650774137,4.054842335154068,1.0546580118479258,0.260098402027721
|
||||
7,Tampering,0.5,8,9,2272,448.25,4.421404905372332,4.054842335154068,2.1814748752660527,0.5379925271948127
|
||||
7,Tampering,0.75,8,9,2272,448.25,4.927762657409038,4.054842335154068,3.3051689699530913,0.815116519154994
|
||||
7,Tampering,1.0,8,9,2272,448.25,5.51904946428857,4.054842335154068,4.32496596600269,1.0666175423164408
|
||||
8,Weaponization,0.0,8,43,23381,5636.75,5.753540808058593,5.753540808058593,0.0,0.0
|
||||
8,Weaponization,0.25,8,43,23381,5636.75,6.540973973907011,5.753540808058593,1.4995962499113429,0.2606388483090205
|
||||
8,Weaponization,0.5,8,43,23381,5636.75,7.135707609860154,5.753540808058593,2.5277727490449937,0.4393421083421386
|
||||
8,Weaponization,0.75,8,43,23381,5636.75,7.429254230877788,5.753540808058593,3.032761745307501,0.5271122334023803
|
||||
8,Weaponization,1.0,8,43,23381,5636.75,7.546828961184063,5.753540808058593,3.2492435406252778,0.5647380715670398
|
||||
|
+386
@@ -0,0 +1,386 @@
|
||||
class_id,class_name,beta,client_id,N_ij,weight
|
||||
0,C&C,0.0,0,193.0,0.1
|
||||
0,C&C,0.0,1,247.0,0.1
|
||||
0,C&C,0.0,2,131.0,0.1
|
||||
0,C&C,0.0,3,29.0,0.1
|
||||
0,C&C,0.0,4,21.0,0.1
|
||||
0,C&C,0.0,5,51.0,0.1
|
||||
0,C&C,0.0,6,87.0,0.1
|
||||
0,C&C,0.0,7,116.0,0.1
|
||||
0,C&C,0.0,8,1065.0,0.1
|
||||
0,C&C,0.0,9,64.0,0.1
|
||||
0,C&C,0.25,0,193.0,0.11265579318819667
|
||||
0,C&C,0.25,1,247.0,0.11982251771647057
|
||||
0,C&C,0.25,2,131.0,0.10225440509080942
|
||||
0,C&C,0.25,3,29.0,0.07013966736548138
|
||||
0,C&C,0.25,4,21.0,0.06470219677642326
|
||||
0,C&C,0.25,5,51.0,0.08077123930353605
|
||||
0,C&C,0.25,6,87.0,0.09230899349684206
|
||||
0,C&C,0.25,7,116.0,0.09919246884860135
|
||||
0,C&C,0.25,8,1065.0,0.17266392313804343
|
||||
0,C&C,0.25,9,64.0,0.0854887950755957
|
||||
0,C&C,0.5,0,193.0,0.11684591755415244
|
||||
0,C&C,0.5,1,247.0,0.13218536221550875
|
||||
0,C&C,0.5,2,131.0,0.09626547024124973
|
||||
0,C&C,0.5,3,29.0,0.045293292062060106
|
||||
0,C&C,0.5,4,21.0,0.03854291312925992
|
||||
0,C&C,0.5,5,51.0,0.06006479191235852
|
||||
0,C&C,0.5,6,87.0,0.07845028309354424
|
||||
0,C&C,0.5,7,116.0,0.09058658412412021
|
||||
0,C&C,0.5,8,1065.0,0.2744793601748162
|
||||
0,C&C,0.5,9,64.0,0.06728602549292985
|
||||
0,C&C,0.75,0,193.0,0.11085880590986728
|
||||
0,C&C,0.75,1,247.0,0.13339050893971305
|
||||
0,C&C,0.75,2,131.0,0.08290022062708707
|
||||
0,C&C,0.75,3,29.0,0.026754738653823083
|
||||
0,C&C,0.75,4,21.0,0.021002291680106373
|
||||
0,C&C,0.75,5,51.0,0.040858258022302904
|
||||
0,C&C,0.75,6,87.0,0.06098761556837048
|
||||
0,C&C,0.75,7,116.0,0.07567382852396858
|
||||
0,C&C,0.75,8,1065.0,0.39913004444251027
|
||||
0,C&C,0.75,9,64.0,0.04844368763225094
|
||||
0,C&C,1.0,0,193.0,0.09630738522954092
|
||||
0,C&C,1.0,1,247.0,0.12325349301397205
|
||||
0,C&C,1.0,2,131.0,0.06536926147704591
|
||||
0,C&C,1.0,3,29.0,0.014471057884231538
|
||||
0,C&C,1.0,4,21.0,0.010479041916167664
|
||||
0,C&C,1.0,5,51.0,0.025449101796407185
|
||||
0,C&C,1.0,6,87.0,0.04341317365269461
|
||||
0,C&C,1.0,7,116.0,0.05788423153692615
|
||||
0,C&C,1.0,8,1065.0,0.531437125748503
|
||||
0,C&C,1.0,9,64.0,0.031936127744510975
|
||||
1,Exfiltration,0.0,0,364.0,0.125
|
||||
1,Exfiltration,0.0,2,187.0,0.125
|
||||
1,Exfiltration,0.0,3,45.0,0.125
|
||||
1,Exfiltration,0.0,5,4.0,0.125
|
||||
1,Exfiltration,0.0,6,7950.0,0.125
|
||||
1,Exfiltration,0.0,7,9.0,0.125
|
||||
1,Exfiltration,0.0,8,2523.0,0.125
|
||||
1,Exfiltration,0.0,9,4412.0,0.125
|
||||
1,Exfiltration,0.25,0,364.0,0.11350553582881193
|
||||
1,Exfiltration,0.25,2,187.0,0.09609528937825479
|
||||
1,Exfiltration,0.25,3,45.0,0.06730461212183213
|
||||
1,Exfiltration,0.25,5,4.0,0.0367499451420711
|
||||
1,Exfiltration,0.25,6,7950.0,0.24537678927132367
|
||||
1,Exfiltration,0.25,7,9.0,0.045009306836673806
|
||||
1,Exfiltration,0.25,8,2523.0,0.18417089980115256
|
||||
1,Exfiltration,0.25,9,4412.0,0.21178762161988002
|
||||
1,Exfiltration,0.5,0,364.0,0.0762306981639795
|
||||
1,Exfiltration,0.5,2,187.0,0.05463865608867304
|
||||
1,Exfiltration,0.5,3,45.0,0.026803126889072607
|
||||
1,Exfiltration,0.5,5,4.0,0.007991148497802508
|
||||
1,Exfiltration,0.5,6,7950.0,0.35625647776417824
|
||||
1,Exfiltration,0.5,7,9.0,0.011986722746703761
|
||||
1,Exfiltration,0.5,8,2523.0,0.20069559053081398
|
||||
1,Exfiltration,0.5,9,4412.0,0.2653975793187764
|
||||
1,Exfiltration,0.75,0,364.0,0.04389343611377405
|
||||
1,Exfiltration,0.75,2,187.0,0.026635125488382226
|
||||
1,Exfiltration,0.75,3,45.0,0.00915130284381765
|
||||
1,Exfiltration,0.75,5,4.0,0.0014897676377329024
|
||||
1,Exfiltration,0.75,6,7950.0,0.44345431879826414
|
||||
1,Exfiltration,0.75,7,9.0,0.0027368779108178027
|
||||
1,Exfiltration,0.75,8,2523.0,0.18750440727611864
|
||||
1,Exfiltration,0.75,9,4412.0,0.28513476393109255
|
||||
1,Exfiltration,1.0,0,364.0,0.02349296501871692
|
||||
1,Exfiltration,1.0,2,187.0,0.012069188072802375
|
||||
1,Exfiltration,1.0,3,45.0,0.002904350070995224
|
||||
1,Exfiltration,1.0,5,4.0,0.000258164450755131
|
||||
1,Exfiltration,1.0,6,7950.0,0.5131018458758229
|
||||
1,Exfiltration,1.0,7,9.0,0.0005808700141990448
|
||||
1,Exfiltration,1.0,8,2523.0,0.16283722731379888
|
||||
1,Exfiltration,1.0,9,4412.0,0.2847553891829095
|
||||
2,Exploitation,0.0,0,73.0,0.16666666666666666
|
||||
2,Exploitation,0.0,3,3.0,0.16666666666666666
|
||||
2,Exploitation,0.0,4,2.0,0.16666666666666666
|
||||
2,Exploitation,0.0,6,57.0,0.16666666666666666
|
||||
2,Exploitation,0.0,7,595.0,0.16666666666666666
|
||||
2,Exploitation,0.0,8,63.0,0.16666666666666666
|
||||
2,Exploitation,0.25,0,73.0,0.18346582623734933
|
||||
2,Exploitation,0.25,3,3.0,0.0826047040771633
|
||||
2,Exploitation,0.25,4,2.0,0.07464177611172308
|
||||
2,Exploitation,0.25,6,57.0,0.17246190539335765
|
||||
2,Exploitation,0.25,7,595.0,0.30999429937941236
|
||||
2,Exploitation,0.25,8,63.0,0.17683148880099422
|
||||
2,Exploitation,0.5,0,73.0,0.16567786188433428
|
||||
2,Exploitation,0.5,3,3.0,0.03358641721456565
|
||||
2,Exploitation,0.5,4,2.0,0.027423194821304973
|
||||
2,Exploitation,0.5,6,57.0,0.14639979851388651
|
||||
2,Exploitation,0.5,7,595.0,0.4730004283577936
|
||||
2,Exploitation,0.5,8,63.0,0.15391229920811497
|
||||
2,Exploitation,0.75,0,73.0,0.12972659444587858
|
||||
2,Exploitation,0.75,3,3.0,0.011840711869793978
|
||||
2,Exploitation,0.75,4,2.0,0.008735934495121323
|
||||
2,Exploitation,0.75,6,57.0,0.10775638515005181
|
||||
2,Exploitation,0.75,7,595.0,0.6257842038870794
|
||||
2,Exploitation,0.75,8,63.0,0.11615617015207494
|
||||
2,Exploitation,1.0,0,73.0,0.09205548549810845
|
||||
2,Exploitation,1.0,3,3.0,0.0037831021437578815
|
||||
2,Exploitation,1.0,4,2.0,0.0025220680958385876
|
||||
2,Exploitation,1.0,6,57.0,0.07187894073139975
|
||||
2,Exploitation,1.0,7,595.0,0.7503152585119798
|
||||
2,Exploitation,1.0,8,63.0,0.07944514501891552
|
||||
3,Lateral_movement,0.0,0,6253.0,0.1111111111111111
|
||||
3,Lateral_movement,0.0,1,21.0,0.1111111111111111
|
||||
3,Lateral_movement,0.0,2,578.0,0.1111111111111111
|
||||
3,Lateral_movement,0.0,3,2.0,0.1111111111111111
|
||||
3,Lateral_movement,0.0,4,13857.0,0.1111111111111111
|
||||
3,Lateral_movement,0.0,5,808.0,0.1111111111111111
|
||||
3,Lateral_movement,0.0,7,573.0,0.1111111111111111
|
||||
3,Lateral_movement,0.0,8,7.0,0.1111111111111111
|
||||
3,Lateral_movement,0.0,9,18.0,0.1111111111111111
|
||||
3,Lateral_movement,0.25,0,6253.0,0.21230281367724058
|
||||
3,Lateral_movement,0.25,1,21.0,0.05110795061636257
|
||||
3,Lateral_movement,0.25,2,578.0,0.11706190907012302
|
||||
3,Lateral_movement,0.25,3,2.0,0.02839168328426866
|
||||
3,Lateral_movement,0.25,4,13857.0,0.25903052575487395
|
||||
3,Lateral_movement,0.25,5,808.0,0.12728771284649473
|
||||
3,Lateral_movement,0.25,7,573.0,0.11680792209104751
|
||||
3,Lateral_movement,0.25,8,7.0,0.03883364469883158
|
||||
3,Lateral_movement,0.25,9,18.0,0.049175837960757335
|
||||
3,Lateral_movement,0.5,0,6253.0,0.2764106866571873
|
||||
3,Lateral_movement,0.5,1,21.0,0.016018441623319713
|
||||
3,Lateral_movement,0.5,2,578.0,0.08403777292411502
|
||||
3,Lateral_movement,0.5,3,2.0,0.004943398407300884
|
||||
3,Lateral_movement,0.5,4,13857.0,0.4114766770601744
|
||||
3,Lateral_movement,0.5,5,808.0,0.09936107827804197
|
||||
3,Lateral_movement,0.5,7,573.0,0.0836734982447361
|
||||
3,Lateral_movement,0.5,8,7.0,0.009248251583221944
|
||||
3,Lateral_movement,0.5,9,18.0,0.014830195221902648
|
||||
3,Lateral_movement,0.75,0,6253.0,0.2940400905653756
|
||||
3,Lateral_movement,0.75,1,21.0,0.004102085398278852
|
||||
3,Lateral_movement,0.75,2,578.0,0.049293097018599547
|
||||
3,Lateral_movement,0.75,3,2.0,0.0007032548291401554
|
||||
3,Lateral_movement,0.75,4,13857.0,0.5340625672107457
|
||||
3,Lateral_movement,0.75,5,808.0,0.06337219456577235
|
||||
3,Lateral_movement,0.75,7,573.0,0.04897294179734221
|
||||
3,Lateral_movement,0.75,8,7.0,0.0017995493305288906
|
||||
3,Lateral_movement,0.75,9,18.0,0.0036542192842167567
|
||||
3,Lateral_movement,1.0,0,6253.0,0.2827236967038929
|
||||
3,Lateral_movement,1.0,1,21.0,0.000949495862910883
|
||||
3,Lateral_movement,1.0,2,578.0,0.026133743274404306
|
||||
3,Lateral_movement,1.0,3,2.0,9.04281774200841e-05
|
||||
3,Lateral_movement,1.0,4,13857.0,0.6265316272550526
|
||||
3,Lateral_movement,1.0,5,808.0,0.03653298367771397
|
||||
3,Lateral_movement,1.0,7,573.0,0.025907672830854094
|
||||
3,Lateral_movement,1.0,8,7.0,0.00031649862097029435
|
||||
3,Lateral_movement,1.0,9,18.0,0.0008138535967807569
|
||||
4,Normal,0.0,0,1.0,0.1
|
||||
4,Normal,0.0,1,102.0,0.1
|
||||
4,Normal,0.0,2,1048.0,0.1
|
||||
4,Normal,0.0,3,38918.0,0.1
|
||||
4,Normal,0.0,4,122800.0,0.1
|
||||
4,Normal,0.0,5,633.0,0.1
|
||||
4,Normal,0.0,6,6773.0,0.1
|
||||
4,Normal,0.0,7,71934.0,0.1
|
||||
4,Normal,0.0,8,2928.0,0.1
|
||||
4,Normal,0.0,9,47640.0,0.1
|
||||
4,Normal,0.25,0,1.0,0.010501168010189358
|
||||
4,Normal,0.25,1,102.0,0.03337241609549466
|
||||
4,Normal,0.25,2,1048.0,0.05974862668898884
|
||||
4,Normal,0.25,3,38918.0,0.14749429940363237
|
||||
4,Normal,0.25,4,122800.0,0.1965790349081601
|
||||
4,Normal,0.25,5,633.0,0.05267305821870829
|
||||
4,Normal,0.25,6,6773.0,0.09526489120011604
|
||||
4,Normal,0.25,7,71934.0,0.1719773131419695
|
||||
4,Normal,0.25,8,2928.0,0.0772467782041376
|
||||
4,Normal,0.25,9,47640.0,0.15514241412860327
|
||||
4,Normal,0.5,0,1.0,0.0008069612591442831
|
||||
4,Normal,0.5,1,102.0,0.008149909221794567
|
||||
4,Normal,0.5,2,1048.0,0.02612361813570612
|
||||
4,Normal,0.5,3,38918.0,0.15919445714528208
|
||||
4,Normal,0.5,4,122800.0,0.2827820697599026
|
||||
4,Normal,0.5,5,633.0,0.02030273473918987
|
||||
4,Normal,0.5,6,6773.0,0.06641148987016199
|
||||
4,Normal,0.5,7,71934.0,0.21643116181232358
|
||||
4,Normal,0.5,8,2928.0,0.04366547830000812
|
||||
4,Normal,0.5,9,47640.0,0.17613211975648674
|
||||
4,Normal,0.75,0,1.0,5.424197722390008e-05
|
||||
4,Normal,0.75,1,102.0,0.001740947364108095
|
||||
4,Normal,0.75,2,1048.0,0.00999094283304485
|
||||
4,Normal,0.75,3,38918.0,0.15029633740019577
|
||||
4,Normal,0.75,4,122800.0,0.3558232213396307
|
||||
4,Normal,0.75,5,633.0,0.0068452339325315885
|
||||
4,Normal,0.75,6,6773.0,0.040496857019315796
|
||||
4,Normal,0.75,7,71934.0,0.23825178876295247
|
||||
4,Normal,0.75,8,2928.0,0.021590555496532098
|
||||
4,Normal,0.75,9,47640.0,0.17490987387446486
|
||||
4,Normal,1.0,0,1.0,3.415568845913443e-06
|
||||
4,Normal,1.0,1,102.0,0.00034838802228317117
|
||||
4,Normal,1.0,2,1048.0,0.003579516150517288
|
||||
4,Normal,1.0,3,38918.0,0.13292710834525936
|
||||
4,Normal,1.0,4,122800.0,0.41943185427817076
|
||||
4,Normal,1.0,5,633.0,0.0021620550794632094
|
||||
4,Normal,1.0,6,6773.0,0.023133647793371747
|
||||
4,Normal,1.0,7,71934.0,0.2456955293619376
|
||||
4,Normal,1.0,8,2928.0,0.010000785580834561
|
||||
4,Normal,1.0,9,47640.0,0.1627176998193164
|
||||
5,RDOS,0.0,0,26440.0,0.1
|
||||
5,RDOS,0.0,1,166.0,0.1
|
||||
5,RDOS,0.0,2,3516.0,0.1
|
||||
5,RDOS,0.0,3,31089.0,0.1
|
||||
5,RDOS,0.0,4,727.0,0.1
|
||||
5,RDOS,0.0,5,14233.0,0.1
|
||||
5,RDOS,0.0,6,1637.0,0.1
|
||||
5,RDOS,0.0,7,4941.0,0.1
|
||||
5,RDOS,0.0,8,1704.0,0.1
|
||||
5,RDOS,0.0,9,14430.0,0.1
|
||||
5,RDOS,0.25,0,26440.0,0.14902831385030976
|
||||
5,RDOS,0.25,1,166.0,0.04194985336060633
|
||||
5,RDOS,0.25,2,3516.0,0.08999443228743546
|
||||
5,RDOS,0.25,3,31089.0,0.1551869198228594
|
||||
5,RDOS,0.25,4,727.0,0.06068579249468527
|
||||
5,RDOS,0.25,5,14233.0,0.12765197467430314
|
||||
5,RDOS,0.25,6,1637.0,0.07433880831824737
|
||||
5,RDOS,0.25,7,4941.0,0.09798444533032141
|
||||
5,RDOS,0.25,8,1704.0,0.07508804941075024
|
||||
5,RDOS,0.25,9,14430.0,0.12809141045048153
|
||||
5,RDOS,0.5,0,26440.0,0.1960201148732251
|
||||
5,RDOS,0.5,1,166.0,0.01553187754922321
|
||||
5,RDOS,0.5,2,3516.0,0.07148161353606823
|
||||
5,RDOS,0.5,3,31089.0,0.21255596168760035
|
||||
5,RDOS,0.5,4,727.0,0.032504023206806905
|
||||
5,RDOS,0.5,5,14233.0,0.1438196205669684
|
||||
5,RDOS,0.5,6,1637.0,0.04877466033513249
|
||||
5,RDOS,0.5,7,4941.0,0.08473783052366345
|
||||
5,RDOS,0.5,8,1704.0,0.04976278858623039
|
||||
5,RDOS,0.5,9,14430.0,0.14481150913508145
|
||||
5,RDOS,0.75,0,26440.0,0.23591059375922666
|
||||
5,RDOS,0.75,1,166.0,0.0052617767981629725
|
||||
5,RDOS,0.75,2,3516.0,0.051950293483453786
|
||||
5,RDOS,0.75,3,31089.0,0.26638294761395714
|
||||
5,RDOS,0.75,4,727.0,0.015929500713807457
|
||||
5,RDOS,0.75,5,14233.0,0.14825989649150298
|
||||
5,RDOS,0.75,6,1637.0,0.02928113218302151
|
||||
5,RDOS,0.75,7,4941.0,0.06705211728312903
|
||||
5,RDOS,0.75,8,1704.0,0.030175435619355758
|
||||
5,RDOS,0.75,9,14430.0,0.14979630605438274
|
||||
5,RDOS,1.0,0,26440.0,0.2673867095456246
|
||||
5,RDOS,1.0,1,166.0,0.001678751655997492
|
||||
5,RDOS,1.0,2,3516.0,0.03555717362944085
|
||||
5,RDOS,1.0,3,31089.0,0.3144018688753375
|
||||
5,RDOS,1.0,4,727.0,0.007352123216326365
|
||||
5,RDOS,1.0,5,14233.0,0.14393778505911026
|
||||
5,RDOS,1.0,6,1637.0,0.01655491843896322
|
||||
5,RDOS,1.0,7,4941.0,0.04996814417038318
|
||||
5,RDOS,1.0,8,1704.0,0.017232486878432086
|
||||
5,RDOS,1.0,9,14430.0,0.1459300385303844
|
||||
6,Reconnaissance,0.0,0,46.0,0.125
|
||||
6,Reconnaissance,0.0,1,65307.0,0.125
|
||||
6,Reconnaissance,0.0,2,4027.0,0.125
|
||||
6,Reconnaissance,0.0,4,3644.0,0.125
|
||||
6,Reconnaissance,0.0,5,4363.0,0.125
|
||||
6,Reconnaissance,0.0,7,1867.0,0.125
|
||||
6,Reconnaissance,0.0,8,67.0,0.125
|
||||
6,Reconnaissance,0.0,9,9840.0,0.125
|
||||
6,Reconnaissance,0.25,0,46.0,0.04210838420044249
|
||||
6,Reconnaissance,0.25,1,65307.0,0.25847530259353113
|
||||
6,Reconnaissance,0.25,2,4027.0,0.12880256931968362
|
||||
6,Reconnaissance,0.25,4,3644.0,0.12562431624687467
|
||||
6,Reconnaissance,0.25,5,4363.0,0.13140909432776285
|
||||
6,Reconnaissance,0.25,7,1867.0,0.10628334111166636
|
||||
6,Reconnaissance,0.25,8,67.0,0.04625916822663658
|
||||
6,Reconnaissance,0.25,9,9840.0,0.16103782397340236
|
||||
6,Reconnaissance,0.5,0,46.0,0.01125132451796716
|
||||
6,Reconnaissance,0.5,1,65307.0,0.42394020186162906
|
||||
6,Reconnaissance,0.5,2,4027.0,0.10527264868637515
|
||||
6,Reconnaissance,0.5,4,3644.0,0.10014146012136282
|
||||
6,Reconnaissance,0.5,5,4363.0,0.10957647912598294
|
||||
6,Reconnaissance,0.5,7,1867.0,0.07167981804128705
|
||||
6,Reconnaissance,0.5,8,67.0,0.013578823289270495
|
||||
6,Reconnaissance,0.5,9,9840.0,0.16455924435612532
|
||||
6,Reconnaissance,0.75,0,46.0,0.002556282128257411
|
||||
6,Reconnaissance,0.75,1,65307.0,0.5912351020891183
|
||||
6,Reconnaissance,0.75,2,4027.0,0.07316049448941997
|
||||
6,Reconnaissance,0.75,4,3644.0,0.06787724177216226
|
||||
6,Reconnaissance,0.75,5,4363.0,0.07769253994165495
|
||||
6,Reconnaissance,0.75,7,1867.0,0.04110537820583696
|
||||
6,Reconnaissance,0.75,8,67.0,0.0033891946487139184
|
||||
6,Reconnaissance,0.75,9,9840.0,0.14298376672483615
|
||||
6,Reconnaissance,1.0,0,46.0,0.0005159206379470845
|
||||
6,Reconnaissance,1.0,1,65307.0,0.7324615022263097
|
||||
6,Reconnaissance,1.0,2,4027.0,0.045165487152454546
|
||||
6,Reconnaissance,1.0,4,3644.0,0.04086988705824295
|
||||
6,Reconnaissance,1.0,5,4363.0,0.04893395094267673
|
||||
6,Reconnaissance,1.0,7,1867.0,0.020939648501026233
|
||||
6,Reconnaissance,1.0,8,67.0,0.0007514496248359709
|
||||
6,Reconnaissance,1.0,9,9840.0,0.11036215385650677
|
||||
7,Tampering,0.0,0,159.0,0.125
|
||||
7,Tampering,0.0,1,2272.0,0.125
|
||||
7,Tampering,0.0,2,9.0,0.125
|
||||
7,Tampering,0.0,4,307.0,0.125
|
||||
7,Tampering,0.0,5,453.0,0.125
|
||||
7,Tampering,0.0,6,241.0,0.125
|
||||
7,Tampering,0.0,7,131.0,0.125
|
||||
7,Tampering,0.0,8,14.0,0.125
|
||||
7,Tampering,0.25,0,159.0,0.11741177578576531
|
||||
7,Tampering,0.25,1,2272.0,0.2282782093603549
|
||||
7,Tampering,0.25,2,9.0,0.05726944875348431
|
||||
7,Tampering,0.25,4,307.0,0.13840346701063316
|
||||
7,Tampering,0.25,5,453.0,0.1525411183148868
|
||||
7,Tampering,0.25,6,241.0,0.13027666612436903
|
||||
7,Tampering,0.25,7,131.0,0.1118613868527606
|
||||
7,Tampering,0.25,8,14.0,0.06395792779774581
|
||||
7,Tampering,0.5,0,159.0,0.09495722082475343
|
||||
7,Tampering,0.5,1,2272.0,0.35894970166310375
|
||||
7,Tampering,0.5,2,9.0,0.022591792365137604
|
||||
7,Tampering,0.5,4,307.0,0.13194672673163435
|
||||
7,Tampering,0.5,5,453.0,0.16027970491476554
|
||||
7,Tampering,0.5,6,241.0,0.1169063104593432
|
||||
7,Tampering,0.5,7,131.0,0.08619162744676871
|
||||
7,Tampering,0.5,8,14.0,0.028176915594493413
|
||||
7,Tampering,0.75,0,159.0,0.06807849170641107
|
||||
7,Tampering,0.75,1,2272.0,0.500343575034468
|
||||
7,Tampering,0.75,2,9.0,0.007900306930126435
|
||||
7,Tampering,0.75,4,307.0,0.11151052580924481
|
||||
7,Tampering,0.75,5,453.0,0.1492917241120361
|
||||
7,Tampering,0.75,6,241.0,0.09299826878350163
|
||||
7,Tampering,0.75,7,131.0,0.05887291690365253
|
||||
7,Tampering,0.75,8,14.0,0.011004190720559325
|
||||
7,Tampering,1.0,0,159.0,0.04433909648633575
|
||||
7,Tampering,1.0,1,2272.0,0.6335750139431121
|
||||
7,Tampering,1.0,2,9.0,0.0025097601784718347
|
||||
7,Tampering,1.0,4,307.0,0.08561070831009482
|
||||
7,Tampering,1.0,5,453.0,0.12632459564974902
|
||||
7,Tampering,1.0,6,241.0,0.06720580033463469
|
||||
7,Tampering,1.0,7,131.0,0.03653095370886782
|
||||
7,Tampering,1.0,8,14.0,0.0039040713887339654
|
||||
8,Weaponization,0.0,0,43.0,0.125
|
||||
8,Weaponization,0.0,2,168.0,0.125
|
||||
8,Weaponization,0.0,3,3342.0,0.125
|
||||
8,Weaponization,0.0,5,347.0,0.125
|
||||
8,Weaponization,0.0,6,23381.0,0.125
|
||||
8,Weaponization,0.0,7,16998.0,0.125
|
||||
8,Weaponization,0.0,8,704.0,0.125
|
||||
8,Weaponization,0.0,9,111.0,0.125
|
||||
8,Weaponization,0.25,0,43.0,0.05094903203624749
|
||||
8,Weaponization,0.25,2,168.0,0.07163019673078404
|
||||
8,Weaponization,0.25,3,3342.0,0.15127614016973676
|
||||
8,Weaponization,0.25,5,347.0,0.08587192372937374
|
||||
8,Weaponization,0.25,6,23381.0,0.24602803274435572
|
||||
8,Weaponization,0.25,7,16998.0,0.22717905032575356
|
||||
8,Weaponization,0.25,8,704.0,0.10248539711898164
|
||||
8,Weaponization,0.25,9,111.0,0.06458022714476704
|
||||
8,Weaponization,0.5,0,43.0,0.015751320871454397
|
||||
8,Weaponization,0.5,2,168.0,0.031134177117178998
|
||||
8,Weaponization,0.5,3,3342.0,0.13886281777781842
|
||||
8,Weaponization,0.5,5,347.0,0.04474530659832112
|
||||
8,Weaponization,0.5,6,23381.0,0.3672943872553869
|
||||
8,Weaponization,0.5,7,16998.0,0.3131710895840922
|
||||
8,Weaponization,0.5,8,704.0,0.06373369246482577
|
||||
8,Weaponization,0.5,9,111.0,0.025307208330922263
|
||||
8,Weaponization,0.75,0,43.0,0.004062163596284168
|
||||
8,Weaponization,0.75,2,168.0,0.011288546582599692
|
||||
8,Weaponization,0.75,3,3342.0,0.10633124583252436
|
||||
8,Weaponization,0.75,5,347.0,0.01944926412869786
|
||||
8,Weaponization,0.75,6,23381.0,0.4574076034613948
|
||||
8,Weaponization,0.75,7,16998.0,0.36012596649674583
|
||||
8,Weaponization,0.75,8,704.0,0.033062490342064633
|
||||
8,Weaponization,0.75,9,111.0,0.008272719559688738
|
||||
8,Weaponization,1.0,0,43.0,0.0009535636670066971
|
||||
8,Weaponization,1.0,2,168.0,0.003725551071095933
|
||||
8,Weaponization,1.0,3,3342.0,0.07411185523572981
|
||||
8,Weaponization,1.0,5,347.0,0.0076950370337517185
|
||||
8,Weaponization,1.0,6,23381.0,0.5184946999600833
|
||||
8,Weaponization,1.0,7,16998.0,0.37694593515767066
|
||||
8,Weaponization,1.0,8,704.0,0.015611833059830577
|
||||
8,Weaponization,1.0,9,111.0,0.002461524814831241
|
||||
|
+10527
File diff suppressed because it is too large.
Load diff
+46
@@ -0,0 +1,46 @@
|
||||
class_id,class_name,beta,n_contributors,support_min,support_max,support_mean,prototype_norm,reference_norm_beta0,displacement_from_beta0,relative_displacement
|
||||
0,C&C,0.0,9,1,986,222.66666666666666,4.901829612186144,4.901829612186144,0.0,0.0
|
||||
0,C&C,0.25,9,1,986,222.66666666666666,4.72079899427204,4.901829612186144,1.430455574408923,0.291820746044039
|
||||
0,C&C,0.5,9,1,986,222.66666666666666,4.890895542865988,4.901829612186144,2.71188728108747,0.5532398095489918
|
||||
0,C&C,0.75,9,1,986,222.66666666666666,5.181441550458014,4.901829612186144,3.6021485283426884,0.7348579639299586
|
||||
0,C&C,1.0,9,1,986,222.66666666666666,5.414782806142079,4.901829612186144,4.124951708215163,0.841512666609294
|
||||
1,Exfiltration,0.0,10,1,6204,1549.4,7.437633790484835,7.437633790484835,0.0,0.0
|
||||
1,Exfiltration,0.25,10,1,6204,1549.4,8.408034452936382,7.437633790484835,2.362721386473373,0.31767111060187797
|
||||
1,Exfiltration,0.5,10,1,6204,1549.4,9.237181140659196,7.437633790484835,3.7814503953657908,0.5084211594557804
|
||||
1,Exfiltration,0.75,10,1,6204,1549.4,9.832122971059055,7.437633790484835,4.6988387374077725,0.6317652723664781
|
||||
1,Exfiltration,1.0,10,1,6204,1549.4,10.2450652291743,7.437633790484835,5.319080439597726,0.7151576145631919
|
||||
2,Exploitation,0.0,8,1,321,99.125,5.36630251548257,5.36630251548257,0.0,0.0
|
||||
2,Exploitation,0.25,8,1,321,99.125,5.326047698023617,5.36630251548257,1.4152287416432763,0.2637251138116299
|
||||
2,Exploitation,0.5,8,1,321,99.125,5.528750275499038,5.36630251548257,2.6011790535745782,0.48472463974399416
|
||||
2,Exploitation,0.75,8,1,321,99.125,5.828170677453147,5.36630251548257,3.5066832125742606,0.6534635724424714
|
||||
2,Exploitation,1.0,8,1,321,99.125,6.111354491590084,5.36630251548257,4.152010451170094,0.7737190438278377
|
||||
3,Lateral_movement,0.0,10,1,10094,2211.7,4.860651208940663,4.860651208940663,0.0,0.0
|
||||
3,Lateral_movement,0.25,10,1,10094,2211.7,5.708485760872119,4.860651208940663,1.451269830260352,0.29857518424504353
|
||||
3,Lateral_movement,0.5,10,1,10094,2211.7,6.141702476957467,4.860651208940663,2.0165054024590443,0.414863218070429
|
||||
3,Lateral_movement,0.75,10,1,10094,2211.7,6.337154533359781,4.860651208940663,2.238889947127781,0.46061522435709346
|
||||
3,Lateral_movement,1.0,10,1,10094,2211.7,6.436373994217236,4.860651208940663,2.3423286413016733,0.48189605479060155
|
||||
4,Normal,0.0,8,198,129678,36597.125,6.181311232696605,6.181311232696605,0.0,0.0
|
||||
4,Normal,0.25,8,198,129678,36597.125,6.69600219159878,6.181311232696605,0.8045528333551073,0.1301589263293122
|
||||
4,Normal,0.5,8,198,129678,36597.125,6.992707199581582,6.181311232696605,1.2491208619495557,0.2020802407331017
|
||||
4,Normal,0.75,8,198,129678,36597.125,7.173688481685205,6.181311232696605,1.5195310278214316,0.245826649171738
|
||||
4,Normal,1.0,8,198,129678,36597.125,7.299358046725321,6.181311232696605,1.7113321401111063,0.27685584428411564
|
||||
5,RDOS,0.0,9,168,53482,10987.0,8.060975920281644,8.060975920281644,0.0,0.0
|
||||
5,RDOS,0.25,9,168,53482,10987.0,8.680890966835888,8.060975920281644,0.8213308822974745,0.10188975757029403
|
||||
5,RDOS,0.5,9,168,53482,10987.0,9.030393126079064,8.060975920281644,1.321273511038344,0.16390986948788452
|
||||
5,RDOS,0.75,9,168,53482,10987.0,9.177061156711039,8.060975920281644,1.6261816527161852,0.20173508379111593
|
||||
5,RDOS,1.0,9,168,53482,10987.0,9.219541428348034,8.060975920281644,1.8476687608995317,0.22921154698536497
|
||||
6,Reconnaissance,0.0,8,138,65101,11145.125,5.552670925770398,5.552670925770398,0.0,0.0
|
||||
6,Reconnaissance,0.25,8,138,65101,11145.125,6.202648567112416,5.552670925770398,0.9730263451403185,0.1752357303625583
|
||||
6,Reconnaissance,0.5,8,138,65101,11145.125,6.6819021889605885,5.552670925770398,1.5662204391401529,0.2820661371937595
|
||||
6,Reconnaissance,0.75,8,138,65101,11145.125,6.959736251937528,5.552670925770398,1.8921171248359299,0.3407580154002032
|
||||
6,Reconnaissance,1.0,8,138,65101,11145.125,7.1120599245635825,5.552670925770398,2.088616712497195,0.37614631596548515
|
||||
7,Tampering,0.0,9,2,1804,398.44444444444446,4.875316439435264,4.875316439435264,0.0,0.0
|
||||
7,Tampering,0.25,9,2,1804,398.44444444444446,4.996674834681458,4.875316439435264,1.4638241974474089,0.3002521406829896
|
||||
7,Tampering,0.5,9,2,1804,398.44444444444446,5.362799949043623,4.875316439435264,2.4861814128009847,0.5099528294596142
|
||||
7,Tampering,0.75,9,2,1804,398.44444444444446,5.776682191332628,4.875316439435264,3.2805423027173237,0.6728880767988319
|
||||
7,Tampering,1.0,9,2,1804,398.44444444444446,6.203076443682819,4.875316439435264,3.9775464519022146,0.8158540068761069
|
||||
8,Weaponization,0.0,9,1,12024,5010.444444444444,6.284565801694992,6.284565801694992,0.0,0.0
|
||||
8,Weaponization,0.25,9,1,12024,5010.444444444444,7.306633664874003,6.284565801694992,1.6883131903367523,0.2686443651972587
|
||||
8,Weaponization,0.5,9,1,12024,5010.444444444444,7.676025651231469,6.284565801694992,2.232233440488681,0.3551929458494383
|
||||
8,Weaponization,0.75,9,1,12024,5010.444444444444,7.80599547081691,6.284565801694992,2.42251119168137,0.3854699382776773
|
||||
8,Weaponization,1.0,9,1,12024,5010.444444444444,7.844912999516274,6.284565801694992,2.485749772741962,0.3955324601854814
|
||||
|
+401
@@ -0,0 +1,401 @@
|
||||
class_id,class_name,beta,client_id,N_ij,weight
|
||||
0,C&C,0.0,0,1.0,0.1111111111111111
|
||||
0,C&C,0.0,1,21.0,0.1111111111111111
|
||||
0,C&C,0.0,3,21.0,0.1111111111111111
|
||||
0,C&C,0.0,4,46.0,0.1111111111111111
|
||||
0,C&C,0.0,5,181.0,0.1111111111111111
|
||||
0,C&C,0.0,6,723.0,0.1111111111111111
|
||||
0,C&C,0.0,7,20.0,0.1111111111111111
|
||||
0,C&C,0.0,8,986.0,0.1111111111111111
|
||||
0,C&C,0.0,9,5.0,0.1111111111111111
|
||||
0,C&C,0.25,0,1.0,0.03853156927135251
|
||||
0,C&C,0.25,1,21.0,0.0824843431885809
|
||||
0,C&C,0.25,3,21.0,0.0824843431885809
|
||||
0,C&C,0.25,4,46.0,0.10034740701428155
|
||||
0,C&C,0.25,5,181.0,0.14133064486247707
|
||||
0,C&C,0.25,6,723.0,0.19980266249105139
|
||||
0,C&C,0.25,7,20.0,0.08148434816559524
|
||||
0,C&C,0.25,8,986.0,0.21591654666962237
|
||||
0,C&C,0.25,9,5.0,0.05761813514845802
|
||||
0,C&C,0.5,0,1.0,0.010482333506064985
|
||||
0,C&C,0.5,1,21.0,0.048036086751314634
|
||||
0,C&C,0.5,3,21.0,0.048036086751314634
|
||||
0,C&C,0.5,4,46.0,0.07109464483130316
|
||||
0,C&C,0.5,5,181.0,0.14102537412664234
|
||||
0,C&C,0.5,6,723.0,0.2818558945179352
|
||||
0,C&C,0.5,7,20.0,0.046878420564770026
|
||||
0,C&C,0.5,8,986.0,0.32915194866826997
|
||||
0,C&C,0.5,9,5.0,0.023439210282385013
|
||||
0,C&C,0.75,0,1.0,0.0024049010671160875
|
||||
0,C&C,0.75,1,21.0,0.02359183304370899
|
||||
0,C&C,0.75,3,21.0,0.02359183304370899
|
||||
0,C&C,0.75,4,46.0,0.04247814947601655
|
||||
0,C&C,0.75,5,181.0,0.11867415448983984
|
||||
0,C&C,0.75,6,723.0,0.3353134221681126
|
||||
0,C&C,0.75,7,20.0,0.02274415004707823
|
||||
0,C&C,0.75,8,986.0,0.42316028529911204
|
||||
0,C&C,0.75,9,5.0,0.008041271365306676
|
||||
0,C&C,1.0,0,1.0,0.000499001996007984
|
||||
0,C&C,1.0,1,21.0,0.010479041916167664
|
||||
0,C&C,1.0,3,21.0,0.010479041916167664
|
||||
0,C&C,1.0,4,46.0,0.022954091816367265
|
||||
0,C&C,1.0,5,181.0,0.09031936127744511
|
||||
0,C&C,1.0,6,723.0,0.36077844311377244
|
||||
0,C&C,1.0,7,20.0,0.00998003992015968
|
||||
0,C&C,1.0,8,986.0,0.49201596806387227
|
||||
0,C&C,1.0,9,5.0,0.00249500998003992
|
||||
1,Exfiltration,0.0,0,48.0,0.1
|
||||
1,Exfiltration,0.0,1,129.0,0.1
|
||||
1,Exfiltration,0.0,2,3823.0,0.1
|
||||
1,Exfiltration,0.0,3,449.0,0.1
|
||||
1,Exfiltration,0.0,4,262.0,0.1
|
||||
1,Exfiltration,0.0,5,1.0,0.1
|
||||
1,Exfiltration,0.0,6,1643.0,0.1
|
||||
1,Exfiltration,0.0,7,6204.0,0.1
|
||||
1,Exfiltration,0.0,8,1919.0,0.1
|
||||
1,Exfiltration,0.0,9,1016.0,0.1
|
||||
1,Exfiltration,0.25,0,48.0,0.05161276990706587
|
||||
1,Exfiltration,0.25,1,129.0,0.06608368999961864
|
||||
1,Exfiltration,0.25,2,3823.0,0.154187061660416
|
||||
1,Exfiltration,0.25,3,449.0,0.09026272548684558
|
||||
1,Exfiltration,0.25,4,262.0,0.07889004076997556
|
||||
1,Exfiltration,0.25,5,1.0,0.01960861220535664
|
||||
1,Exfiltration,0.25,6,1643.0,0.12484071534380817
|
||||
1,Exfiltration,0.25,7,6204.0,0.1740262672513917
|
||||
1,Exfiltration,0.25,8,1919.0,0.12978234031408534
|
||||
1,Exfiltration,0.25,9,1016.0,0.11070577706143654
|
||||
1,Exfiltration,0.5,0,48.0,0.022101433509696778
|
||||
1,Exfiltration,0.5,1,129.0,0.03623219788469621
|
||||
1,Exfiltration,0.5,2,3823.0,0.1972431698992126
|
||||
1,Exfiltration,0.5,3,449.0,0.06759631093074608
|
||||
1,Exfiltration,0.5,4,262.0,0.0516357477016669
|
||||
1,Exfiltration,0.5,5,1.0,0.003190067146575013
|
||||
1,Exfiltration,0.5,6,1643.0,0.12930597882788272
|
||||
1,Exfiltration,0.5,7,6204.0,0.25126715302934755
|
||||
1,Exfiltration,0.5,8,1919.0,0.13974533265530642
|
||||
1,Exfiltration,0.5,9,1016.0,0.10168260841486985
|
||||
1,Exfiltration,0.75,0,48.0,0.008548019660347309
|
||||
1,Exfiltration,0.75,1,129.0,0.017942248487679474
|
||||
1,Exfiltration,0.75,2,3823.0,0.22789657999596144
|
||||
1,Exfiltration,0.75,3,449.0,0.04572138104591042
|
||||
1,Exfiltration,0.75,4,262.0,0.030525343014136976
|
||||
1,Exfiltration,0.75,5,1.0,0.00046874277238292023
|
||||
1,Exfiltration,0.75,6,1643.0,0.12096584315794441
|
||||
1,Exfiltration,0.75,7,6204.0,0.3276713166207476
|
||||
1,Exfiltration,0.75,8,1919.0,0.13590668374209638
|
||||
1,Exfiltration,0.75,9,1016.0,0.08435384150279308
|
||||
1,Exfiltration,1.0,0,48.0,0.0030979734090615723
|
||||
1,Exfiltration,1.0,1,129.0,0.008325803536852975
|
||||
1,Exfiltration,1.0,2,3823.0,0.24674067380921647
|
||||
1,Exfiltration,1.0,3,449.0,0.028978959597263456
|
||||
1,Exfiltration,1.0,4,262.0,0.016909771524461083
|
||||
1,Exfiltration,1.0,5,1.0,6.454111268878275e-05
|
||||
1,Exfiltration,1.0,6,1643.0,0.10604104814767007
|
||||
1,Exfiltration,1.0,7,6204.0,0.4004130631212082
|
||||
1,Exfiltration,1.0,8,1919.0,0.12385439524977411
|
||||
1,Exfiltration,1.0,9,1016.0,0.06557377049180328
|
||||
2,Exploitation,0.0,0,78.0,0.125
|
||||
2,Exploitation,0.0,1,16.0,0.125
|
||||
2,Exploitation,0.0,3,315.0,0.125
|
||||
2,Exploitation,0.0,4,22.0,0.125
|
||||
2,Exploitation,0.0,5,3.0,0.125
|
||||
2,Exploitation,0.0,7,321.0,0.125
|
||||
2,Exploitation,0.0,8,1.0,0.125
|
||||
2,Exploitation,0.0,9,37.0,0.125
|
||||
2,Exploitation,0.25,0,78.0,0.14592379135207148
|
||||
2,Exploitation,0.25,1,16.0,0.09820473992568025
|
||||
2,Exploitation,0.25,3,315.0,0.20686170152868258
|
||||
2,Exploitation,0.25,4,22.0,0.10634280815546648
|
||||
2,Exploitation,0.25,5,3.0,0.06462235308247294
|
||||
2,Exploitation,0.25,7,321.0,0.2078397983076116
|
||||
2,Exploitation,0.25,8,1.0,0.04910236996284013
|
||||
2,Exploitation,0.25,9,37.0,0.1211024376851745
|
||||
2,Exploitation,0.5,0,78.0,0.14244384518055178
|
||||
2,Exploitation,0.5,1,16.0,0.06451435782127485
|
||||
2,Exploitation,0.5,3,315.0,0.2862540660195741
|
||||
2,Exploitation,0.5,4,22.0,0.07564979016494887
|
||||
2,Exploitation,0.5,5,3.0,0.027935536391031652
|
||||
2,Exploitation,0.5,7,321.0,0.28896743536192443
|
||||
2,Exploitation,0.5,8,1.0,0.016128589455318712
|
||||
2,Exploitation,0.5,9,37.0,0.09810637960537563
|
||||
2,Exploitation,0.75,0,78.0,0.12305314651142159
|
||||
2,Exploitation,0.75,1,16.0,0.03750695075141924
|
||||
2,Exploitation,0.75,3,315.0,0.3505538059273375
|
||||
2,Exploitation,0.75,4,22.0,0.04762541596160341
|
||||
2,Exploitation,0.75,5,3.0,0.01068716986533943
|
||||
2,Exploitation,0.75,7,321.0,0.35554988747195565
|
||||
2,Exploitation,0.75,8,1.0,0.004688368843927405
|
||||
2,Exploitation,0.75,9,37.0,0.07033525466699575
|
||||
2,Exploitation,1.0,0,78.0,0.09836065573770492
|
||||
2,Exploitation,1.0,1,16.0,0.0201765447667087
|
||||
2,Exploitation,1.0,3,315.0,0.39722572509457754
|
||||
2,Exploitation,1.0,4,22.0,0.027742749054224466
|
||||
2,Exploitation,1.0,5,3.0,0.0037831021437578815
|
||||
2,Exploitation,1.0,7,321.0,0.4047919293820933
|
||||
2,Exploitation,1.0,8,1.0,0.0012610340479192938
|
||||
2,Exploitation,1.0,9,37.0,0.04665825977301387
|
||||
3,Lateral_movement,0.0,0,1.0,0.1
|
||||
3,Lateral_movement,0.0,1,10094.0,0.1
|
||||
3,Lateral_movement,0.0,2,2.0,0.1
|
||||
3,Lateral_movement,0.0,3,82.0,0.1
|
||||
3,Lateral_movement,0.0,4,793.0,0.1
|
||||
3,Lateral_movement,0.0,5,6163.0,0.1
|
||||
3,Lateral_movement,0.0,6,830.0,0.1
|
||||
3,Lateral_movement,0.0,7,83.0,0.1
|
||||
3,Lateral_movement,0.0,8,2228.0,0.1
|
||||
3,Lateral_movement,0.0,9,1841.0,0.1
|
||||
3,Lateral_movement,0.25,0,1.0,0.019533058893322856
|
||||
3,Lateral_movement,0.25,1,10094.0,0.19578800656293474
|
||||
3,Lateral_movement,0.25,2,2.0,0.02322885261370672
|
||||
3,Lateral_movement,0.25,3,82.0,0.05877920699329291
|
||||
3,Lateral_movement,0.25,4,793.0,0.1036545943376126
|
||||
3,Lateral_movement,0.25,5,6163.0,0.17306860831336734
|
||||
3,Lateral_movement,0.25,6,830.0,0.10484308283708428
|
||||
3,Lateral_movement,0.25,7,83.0,0.05895759813964752
|
||||
3,Lateral_movement,0.25,8,2228.0,0.13419893092820492
|
||||
3,Lateral_movement,0.25,9,1841.0,0.12794806038082612
|
||||
3,Lateral_movement,0.5,0,1.0,0.0028849017367578752
|
||||
3,Lateral_movement,0.5,1,10094.0,0.28984290600664425
|
||||
3,Lateral_movement,0.5,2,2.0,0.004079867162236684
|
||||
3,Lateral_movement,0.5,3,82.0,0.026123896312024088
|
||||
3,Lateral_movement,0.5,4,793.0,0.08123957052067449
|
||||
3,Lateral_movement,0.5,5,6163.0,0.22647856729391336
|
||||
3,Lateral_movement,0.5,6,830.0,0.0831132129418743
|
||||
3,Lateral_movement,0.5,7,83.0,0.026282705655090655
|
||||
3,Lateral_movement,0.5,8,2228.0,0.13617225155114346
|
||||
3,Lateral_movement,0.5,9,1841.0,0.1237821208196409
|
||||
3,Lateral_movement,0.75,0,1.0,0.00037460696291537047
|
||||
3,Lateral_movement,0.75,1,10094.0,0.37724485094362636
|
||||
3,Lateral_movement,0.75,2,2.0,0.0006300113044892323
|
||||
3,Lateral_movement,0.75,3,82.0,0.010207895953468616
|
||||
3,Lateral_movement,0.75,4,793.0,0.05597972181340088
|
||||
3,Lateral_movement,0.75,5,6163.0,0.2605673350522155
|
||||
3,Lateral_movement,0.75,6,830.0,0.05792745049543543
|
||||
3,Lateral_movement,0.75,7,83.0,0.010301119249208183
|
||||
3,Lateral_movement,0.75,8,2228.0,0.12148208450098266
|
||||
3,Lateral_movement,0.75,9,1841.0,0.10528492372425773
|
||||
3,Lateral_movement,1.0,0,1.0,4.521408871004205e-05
|
||||
3,Lateral_movement,1.0,1,10094.0,0.45639101143916444
|
||||
3,Lateral_movement,1.0,2,2.0,9.04281774200841e-05
|
||||
3,Lateral_movement,1.0,3,82.0,0.003707555274223448
|
||||
3,Lateral_movement,1.0,4,793.0,0.03585477234706334
|
||||
3,Lateral_movement,1.0,5,6163.0,0.27865442871998913
|
||||
3,Lateral_movement,1.0,6,830.0,0.0375276936293349
|
||||
3,Lateral_movement,1.0,7,83.0,0.00375276936293349
|
||||
3,Lateral_movement,1.0,8,2228.0,0.10073698964597369
|
||||
3,Lateral_movement,1.0,9,1841.0,0.08323913731518741
|
||||
4,Normal,0.0,0,45538.0,0.125
|
||||
4,Normal,0.0,1,75982.0,0.125
|
||||
4,Normal,0.0,3,2706.0,0.125
|
||||
4,Normal,0.0,4,16624.0,0.125
|
||||
4,Normal,0.0,5,129678.0,0.125
|
||||
4,Normal,0.0,7,198.0,0.125
|
||||
4,Normal,0.0,8,14813.0,0.125
|
||||
4,Normal,0.0,9,7238.0,0.125
|
||||
4,Normal,0.25,0,45538.0,0.15747987340249417
|
||||
4,Normal,0.25,1,75982.0,0.1789819556174204
|
||||
4,Normal,0.25,3,2706.0,0.07775234870733912
|
||||
4,Normal,0.25,4,16624.0,0.12240959226793337
|
||||
4,Normal,0.25,5,129678.0,0.2045729202785085
|
||||
4,Normal,0.25,7,198.0,0.04043876337973332
|
||||
4,Normal,0.25,8,14813.0,0.11893024800276067
|
||||
4,Normal,0.25,9,7238.0,0.09943429834381041
|
||||
4,Normal,0.5,0,45538.0,0.17058570312644797
|
||||
4,Normal,0.5,1,75982.0,0.22034896451728794
|
||||
4,Normal,0.5,3,2706.0,0.041583357279496164
|
||||
4,Normal,0.5,4,16624.0,0.10306789791686834
|
||||
4,Normal,0.5,5,129678.0,0.28786496087203295
|
||||
4,Normal,0.5,7,198.0,0.011248335169934585
|
||||
4,Normal,0.5,8,14813.0,0.09729200757664001
|
||||
4,Normal,0.5,9,7238.0,0.06800877354129198
|
||||
4,Normal,0.75,0,45538.0,0.1680778269911699
|
||||
4,Normal,0.75,1,75982.0,0.2467533177339209
|
||||
4,Normal,0.75,3,2706.0,0.020229064737902296
|
||||
4,Normal,0.75,4,16624.0,0.07893718006799944
|
||||
4,Normal,0.75,5,129678.0,0.3684509561992108
|
||||
4,Normal,0.75,7,198.0,0.002845960399531247
|
||||
4,Normal,0.75,8,14813.0,0.07239560922985498
|
||||
4,Normal,0.75,9,7238.0,0.042310084640410396
|
||||
4,Normal,1.0,0,45538.0,0.15553817410520634
|
||||
4,Normal,1.0,1,75982.0,0.2595217520501952
|
||||
4,Normal,1.0,3,2706.0,0.009242529297041776
|
||||
4,Normal,1.0,4,16624.0,0.05678041649446507
|
||||
4,Normal,1.0,5,129678.0,0.4429241368003634
|
||||
4,Normal,1.0,7,198.0,0.0006762826314908616
|
||||
4,Normal,1.0,8,14813.0,0.05059482131451583
|
||||
4,Normal,1.0,9,7238.0,0.024721887306721497
|
||||
5,RDOS,0.0,0,168.0,0.1111111111111111
|
||||
5,RDOS,0.0,1,5627.0,0.1111111111111111
|
||||
5,RDOS,0.0,3,2835.0,0.1111111111111111
|
||||
5,RDOS,0.0,4,12854.0,0.1111111111111111
|
||||
5,RDOS,0.0,5,53482.0,0.1111111111111111
|
||||
5,RDOS,0.0,6,998.0,0.1111111111111111
|
||||
5,RDOS,0.0,7,232.0,0.1111111111111111
|
||||
5,RDOS,0.0,8,4108.0,0.1111111111111111
|
||||
5,RDOS,0.0,9,18579.0,0.1111111111111111
|
||||
5,RDOS,0.25,0,168.0,0.04824888254260608
|
||||
5,RDOS,0.25,1,5627.0,0.11607245437311266
|
||||
5,RDOS,0.25,3,2835.0,0.09779084634125362
|
||||
5,RDOS,0.25,4,12854.0,0.14269853446814731
|
||||
5,RDOS,0.25,5,53482.0,0.20380359894764252
|
||||
5,RDOS,0.25,6,998.0,0.07532559559628273
|
||||
5,RDOS,0.25,7,232.0,0.05230364254846632
|
||||
5,RDOS,0.25,8,4108.0,0.10729205108690332
|
||||
5,RDOS,0.25,9,18579.0,0.15646439409558546
|
||||
5,RDOS,0.5,0,168.0,0.017680904938266448
|
||||
5,RDOS,0.5,1,5627.0,0.10232654387357101
|
||||
5,RDOS,0.5,3,2835.0,0.07263172853844765
|
||||
5,RDOS,0.5,4,12854.0,0.1546567914992692
|
||||
5,RDOS,0.5,5,53482.0,0.3154668055762096
|
||||
5,RDOS,0.5,6,998.0,0.043093832466487125
|
||||
5,RDOS,0.5,7,232.0,0.020777526301780088
|
||||
5,RDOS,0.5,8,4108.0,0.08743092351227466
|
||||
5,RDOS,0.5,9,18579.0,0.18593494329369423
|
||||
5,RDOS,0.75,0,168.0,0.005725344876556219
|
||||
5,RDOS,0.75,1,5627.0,0.0797126738893055
|
||||
5,RDOS,0.75,3,2835.0,0.04766882954543171
|
||||
5,RDOS,0.75,4,12854.0,0.1481147850714375
|
||||
5,RDOS,0.75,5,53482.0,0.431494655367228
|
||||
5,RDOS,0.75,6,998.0,0.02178550326511013
|
||||
5,RDOS,0.75,7,232.0,0.00729349483420234
|
||||
5,RDOS,0.75,8,4108.0,0.06295678142208486
|
||||
5,RDOS,0.75,9,18579.0,0.1952479317286438
|
||||
5,RDOS,1.0,0,168.0,0.0016989775795637268
|
||||
5,RDOS,1.0,1,5627.0,0.05690563595360173
|
||||
5,RDOS,1.0,3,2835.0,0.028670246655137892
|
||||
5,RDOS,1.0,4,12854.0,0.12999201076019135
|
||||
5,RDOS,1.0,5,53482.0,0.5408614220846859
|
||||
5,RDOS,1.0,6,998.0,0.010092735859551187
|
||||
5,RDOS,1.0,7,232.0,0.002346207133683242
|
||||
5,RDOS,1.0,8,4108.0,0.04154404700504637
|
||||
5,RDOS,1.0,9,18579.0,0.18788871696853857
|
||||
6,Reconnaissance,0.0,0,4841.0,0.125
|
||||
6,Reconnaissance,0.0,1,65101.0,0.125
|
||||
6,Reconnaissance,0.0,3,6322.0,0.125
|
||||
6,Reconnaissance,0.0,4,202.0,0.125
|
||||
6,Reconnaissance,0.0,5,439.0,0.125
|
||||
6,Reconnaissance,0.0,6,138.0,0.125
|
||||
6,Reconnaissance,0.0,7,2503.0,0.125
|
||||
6,Reconnaissance,0.0,9,9615.0,0.125
|
||||
6,Reconnaissance,0.25,0,4841.0,0.13457644815917696
|
||||
6,Reconnaissance,0.25,1,65101.0,0.25771055496222744
|
||||
6,Reconnaissance,0.25,3,6322.0,0.1438629342907076
|
||||
6,Reconnaissance,0.25,4,202.0,0.06082374176073279
|
||||
6,Reconnaissance,0.25,5,439.0,0.07385015672299558
|
||||
6,Reconnaissance,0.25,6,138.0,0.05529744468184099
|
||||
6,Reconnaissance,0.25,7,2503.0,0.11411704140876835
|
||||
6,Reconnaissance,0.25,9,9615.0,0.15976167801355035
|
||||
6,Reconnaissance,0.5,0,4841.0,0.1161100612144034
|
||||
6,Reconnaissance,0.5,1,65101.0,0.42579067165486595
|
||||
6,Reconnaissance,0.5,3,6322.0,0.13268736158095895
|
||||
6,Reconnaissance,0.5,4,202.0,0.023717996324596407
|
||||
6,Reconnaissance,0.5,5,439.0,0.034965083750405274
|
||||
6,Reconnaissance,0.5,6,138.0,0.01960387189632919
|
||||
6,Reconnaissance,0.5,7,2503.0,0.08348966866135346
|
||||
6,Reconnaissance,0.5,9,9615.0,0.16363528491708731
|
||||
6,Reconnaissance,0.75,0,4841.0,0.08436084995680497
|
||||
6,Reconnaissance,0.75,1,65101.0,0.5924209108219368
|
||||
6,Reconnaissance,0.75,3,6322.0,0.10305771365368427
|
||||
6,Reconnaissance,0.75,4,202.0,0.00778848812040265
|
||||
6,Reconnaissance,0.75,5,439.0,0.013940810536040266
|
||||
6,Reconnaissance,0.75,6,138.0,0.00585260143607737
|
||||
6,Reconnaissance,0.75,7,2503.0,0.051438141871729785
|
||||
6,Reconnaissance,0.75,9,9615.0,0.14114048360332393
|
||||
6,Reconnaissance,1.0,0,4841.0,0.054295039310909475
|
||||
6,Reconnaissance,1.0,1,65101.0,0.7301510750215902
|
||||
6,Reconnaissance,1.0,3,6322.0,0.07090544071959713
|
||||
6,Reconnaissance,1.0,4,202.0,0.0022655645405502407
|
||||
6,Reconnaissance,1.0,5,439.0,0.004923677392581959
|
||||
6,Reconnaissance,1.0,6,138.0,0.0015477619138412535
|
||||
6,Reconnaissance,1.0,7,2503.0,0.02807281210394679
|
||||
6,Reconnaissance,1.0,9,9615.0,0.10783862899698299
|
||||
7,Tampering,0.0,0,2.0,0.1111111111111111
|
||||
7,Tampering,0.0,1,577.0,0.1111111111111111
|
||||
7,Tampering,0.0,2,151.0,0.1111111111111111
|
||||
7,Tampering,0.0,3,170.0,0.1111111111111111
|
||||
7,Tampering,0.0,5,18.0,0.1111111111111111
|
||||
7,Tampering,0.0,6,1804.0,0.1111111111111111
|
||||
7,Tampering,0.0,7,314.0,0.1111111111111111
|
||||
7,Tampering,0.0,8,16.0,0.1111111111111111
|
||||
7,Tampering,0.0,9,534.0,0.1111111111111111
|
||||
7,Tampering,0.25,0,2.0,0.03625607236333655
|
||||
7,Tampering,0.25,1,577.0,0.14942291665944094
|
||||
7,Tampering,0.25,2,151.0,0.10687287508428511
|
||||
7,Tampering,0.25,3,170.0,0.11008686047431138
|
||||
7,Tampering,0.25,5,18.0,0.06279735941619272
|
||||
7,Tampering,0.25,6,1804.0,0.19869291869684522
|
||||
7,Tampering,0.25,7,314.0,0.12833811851832422
|
||||
7,Tampering,0.25,8,16.0,0.060975202563017944
|
||||
7,Tampering,0.25,9,534.0,0.146557676224246
|
||||
7,Tampering,0.5,0,2.0,0.009937814474557329
|
||||
7,Tampering,0.5,1,577.0,0.1687966384710106
|
||||
7,Tampering,0.5,2,151.0,0.0863504013775816
|
||||
7,Tampering,0.5,3,170.0,0.09162212235651006
|
||||
7,Tampering,0.5,5,18.0,0.02981344342367198
|
||||
7,Tampering,0.5,6,1804.0,0.29846551088980194
|
||||
7,Tampering,0.5,7,314.0,0.12452045846094215
|
||||
7,Tampering,0.5,8,16.0,0.028108384020533256
|
||||
7,Tampering,0.5,9,534.0,0.162385226525391
|
||||
7,Tampering,0.75,0,2.0,0.002441662047936183
|
||||
7,Tampering,0.75,1,577.0,0.1709207996149052
|
||||
7,Tampering,0.75,2,151.0,0.06253825608168917
|
||||
7,Tampering,0.75,3,170.0,0.06835176713012665
|
||||
7,Tampering,0.75,5,18.0,0.012687248165814433
|
||||
7,Tampering,0.75,6,1804.0,0.40187450555664667
|
||||
7,Tampering,0.75,7,314.0,0.10829547040469581
|
||||
7,Tampering,0.75,8,16.0,0.011614567519351296
|
||||
7,Tampering,0.75,9,534.0,0.16127572347883457
|
||||
7,Tampering,1.0,0,2.0,0.0005577244841048522
|
||||
7,Tampering,1.0,1,577.0,0.16090351366424985
|
||||
7,Tampering,1.0,2,151.0,0.04210819854991634
|
||||
7,Tampering,1.0,3,170.0,0.047406581148912434
|
||||
7,Tampering,1.0,5,18.0,0.0050195203569436695
|
||||
7,Tampering,1.0,6,1804.0,0.5030674846625767
|
||||
7,Tampering,1.0,7,314.0,0.08756274400446179
|
||||
7,Tampering,1.0,8,16.0,0.004461795872838818
|
||||
7,Tampering,1.0,9,534.0,0.14891243725599554
|
||||
8,Weaponization,0.0,0,8960.0,0.1111111111111111
|
||||
8,Weaponization,0.0,1,245.0,0.1111111111111111
|
||||
8,Weaponization,0.0,2,5302.0,0.1111111111111111
|
||||
8,Weaponization,0.0,4,9118.0,0.1111111111111111
|
||||
8,Weaponization,0.0,5,3361.0,0.1111111111111111
|
||||
8,Weaponization,0.0,6,5936.0,0.1111111111111111
|
||||
8,Weaponization,0.0,7,1.0,0.1111111111111111
|
||||
8,Weaponization,0.0,8,147.0,0.1111111111111111
|
||||
8,Weaponization,0.0,9,12024.0,0.1111111111111111
|
||||
8,Weaponization,0.25,0,8960.0,0.15361316310274253
|
||||
8,Weaponization,0.25,1,245.0,0.06246589266779502
|
||||
8,Weaponization,0.25,2,5302.0,0.1347290850216984
|
||||
8,Weaponization,0.25,4,9118.0,0.15428593136022012
|
||||
8,Weaponization,0.25,5,3361.0,0.12021773954301719
|
||||
8,Weaponization,0.25,6,5936.0,0.13858776871398634
|
||||
8,Weaponization,0.25,7,1.0,0.01578888383749
|
||||
8,Weaponization,0.25,8,147.0,0.05497696528620278
|
||||
8,Weaponization,0.25,9,12024.0,0.16533457046684766
|
||||
8,Weaponization,0.5,0,8960.0,0.17646417246220708
|
||||
8,Weaponization,0.5,1,245.0,0.029180019727988275
|
||||
8,Weaponization,0.5,2,5302.0,0.1357445495448714
|
||||
8,Weaponization,0.5,4,9118.0,0.17801325152448746
|
||||
8,Weaponization,0.5,5,3361.0,0.10807787453169888
|
||||
8,Weaponization,0.5,6,5936.0,0.14363143025757336
|
||||
8,Weaponization,0.5,7,1.0,0.001864243077044763
|
||||
8,Weaponization,0.5,8,147.0,0.022602746089700494
|
||||
8,Weaponization,0.5,9,12024.0,0.2044217127844284
|
||||
8,Weaponization,0.75,0,8960.0,0.1900232344782966
|
||||
8,Weaponization,0.75,1,245.0,0.012777628646861857
|
||||
8,Weaponization,0.75,2,5302.0,0.12820515599812551
|
||||
8,Weaponization,0.75,4,9118.0,0.19253087731769986
|
||||
8,Weaponization,0.75,5,3361.0,0.09108085174165452
|
||||
8,Weaponization,0.75,6,5936.0,0.13953916444528988
|
||||
8,Weaponization,0.75,7,1.0,0.00020633632867562082
|
||||
8,Weaponization,0.75,8,147.0,0.00871091347565661
|
||||
8,Weaponization,0.75,9,12024.0,0.23692583756773947
|
||||
8,Weaponization,1.0,0,8960.0,0.19869605712511643
|
||||
8,Weaponization,1.0,1,245.0,0.005433095312014902
|
||||
8,Weaponization,1.0,2,5302.0,0.11757661773184902
|
||||
8,Weaponization,1.0,4,9118.0,0.20219984920388523
|
||||
8,Weaponization,1.0,5,3361.0,0.07453319732115138
|
||||
8,Weaponization,1.0,6,5936.0,0.13163613784538963
|
||||
8,Weaponization,1.0,7,1.0,2.2175899232713886e-05
|
||||
8,Weaponization,1.0,8,147.0,0.003259857187208941
|
||||
8,Weaponization,1.0,9,12024.0,0.2666430123741518
|
||||
|
+9872
File diff suppressed because it is too large.
Load diff
+46
@@ -0,0 +1,46 @@
|
||||
class_id,class_name,beta,n_contributors,support_min,support_max,support_mean,prototype_norm,reference_norm_beta0,displacement_from_beta0,relative_displacement
|
||||
0,C&C,0.0,6,2,1265,334.0,4.572526901605314,4.572526901605314,0.0,0.0
|
||||
0,C&C,0.25,6,2,1265,334.0,4.488323623313573,4.572526901605314,1.5992758251729138,0.349757554102403
|
||||
0,C&C,0.5,6,2,1265,334.0,4.64238349428513,4.572526901605314,2.7568644582945026,0.6029192430396917
|
||||
0,C&C,0.75,6,2,1265,334.0,4.823784216346094,4.572526901605314,3.5994599542267287,0.7871927342763226
|
||||
0,C&C,1.0,6,2,1265,334.0,4.9943459759985185,4.572526901605314,4.24238658552617,0.9277991528134611
|
||||
1,Exfiltration,0.0,8,4,8269,1936.75,5.639260260262674,5.639260260262674,0.0,0.0
|
||||
1,Exfiltration,0.25,8,4,8269,1936.75,6.682525777414029,5.639260260262674,1.7945831890795807,0.3182302476310944
|
||||
1,Exfiltration,0.5,8,4,8269,1936.75,7.330510507543627,5.639260260262674,2.7215672864242446,0.48261069020025604
|
||||
1,Exfiltration,0.75,8,4,8269,1936.75,7.691760111791469,5.639260260262674,3.239979172736248,0.5745397486913171
|
||||
1,Exfiltration,1.0,8,4,8269,1936.75,7.919045745714998,5.639260260262674,3.588238413562595,0.6362959409494352
|
||||
2,Exploitation,0.0,6,1,485,132.16666666666666,4.7820569474466526,4.7820569474466526,0.0,0.0
|
||||
2,Exploitation,0.25,6,1,485,132.16666666666666,5.058528386379323,4.7820569474466526,1.3149952804773795,0.2749852824691919
|
||||
2,Exploitation,0.5,6,1,485,132.16666666666666,5.631252401560569,4.7820569474466526,2.529011339641225,0.5288542916645051
|
||||
2,Exploitation,0.75,6,1,485,132.16666666666666,6.164859782840714,4.7820569474466526,3.4295515176433,0.7171707813882238
|
||||
2,Exploitation,1.0,6,1,485,132.16666666666666,6.563880442917527,4.7820569474466526,4.059823122202898,0.8489700492526785
|
||||
3,Lateral_movement,0.0,7,1,12496,3159.5714285714284,4.652697972917494,4.652697972917494,0.0,0.0
|
||||
3,Lateral_movement,0.25,7,1,12496,3159.5714285714284,5.500950148060144,4.652697972917494,2.5898222983977996,0.556628071169176
|
||||
3,Lateral_movement,0.5,7,1,12496,3159.5714285714284,6.56604801731326,4.652697972917494,4.046183521651553,0.8696424193454312
|
||||
3,Lateral_movement,0.75,7,1,12496,3159.5714285714284,7.202542104608667,4.652697972917494,4.765480241626273,1.0242401869550235
|
||||
3,Lateral_movement,1.0,7,1,12496,3159.5714285714284,7.5314095779928145,4.652697972917494,5.109111015402566,1.0980964260181445
|
||||
4,Normal,0.0,9,452,118937,32530.777777777777,5.686260685156379,5.686260685156379,0.0,0.0
|
||||
4,Normal,0.25,9,452,118937,32530.777777777777,6.206393909319786,5.686260685156379,0.7950378669816794,0.13981734412160793
|
||||
4,Normal,0.5,9,452,118937,32530.777777777777,6.529912771816577,5.686260685156379,1.2525733910479948,0.22028068363410735
|
||||
4,Normal,0.75,9,452,118937,32530.777777777777,6.71811801619301,5.686260685156379,1.506814977752638,0.26499224379319836
|
||||
4,Normal,1.0,9,452,118937,32530.777777777777,6.8391544140403875,5.686260685156379,1.6673282103552307,0.29322050160445245
|
||||
5,RDOS,0.0,10,2,48839,9888.3,7.538630769790161,7.538630769790161,0.0,0.0
|
||||
5,RDOS,0.25,10,2,48839,9888.3,8.012251426204648,7.538630769790161,1.6174943118797114,0.21456075529810498
|
||||
5,RDOS,0.5,10,2,48839,9888.3,8.179789715169823,7.538630769790161,2.514305903819429,0.3335228877232059
|
||||
5,RDOS,0.75,10,2,48839,9888.3,8.273460981732642,7.538630769790161,3.2761911842442637,0.43458703367898954
|
||||
5,RDOS,1.0,10,2,48839,9888.3,8.365259941161074,7.538630769790161,3.9737567173599997,0.5271191597928081
|
||||
6,Reconnaissance,0.0,10,1,46541,8916.1,4.967755753565685,4.967755753565685,0.0,0.0
|
||||
6,Reconnaissance,0.25,10,1,46541,8916.1,6.179109954356463,4.967755753565685,2.1337405853808944,0.4295180140145514
|
||||
6,Reconnaissance,0.5,10,1,46541,8916.1,6.859920022054287,4.967755753565685,3.084568124445345,0.6209178303968241
|
||||
6,Reconnaissance,0.75,10,1,46541,8916.1,7.154968003286312,4.967755753565685,3.502353696944525,0.7050172896343898
|
||||
6,Reconnaissance,1.0,10,1,46541,8916.1,7.302076678416271,4.967755753565685,3.7335901267097253,0.7515647531644208
|
||||
7,Tampering,0.0,10,1,2584,358.6,4.381687640803716,4.381687640803716,0.0,0.0
|
||||
7,Tampering,0.25,10,1,2584,358.6,4.739508161488117,4.381687640803716,1.5332987381385625,0.3499333735841826
|
||||
7,Tampering,0.5,10,1,2584,358.6,5.399681843080317,4.381687640803716,2.989769119570483,0.6823327824030107
|
||||
7,Tampering,0.75,10,1,2584,358.6,5.979612116483903,4.381687640803716,3.9117575814369485,0.892751355667021
|
||||
7,Tampering,1.0,10,1,2584,358.6,6.373713249735831,4.381687640803716,4.440926893005426,1.01351973418873
|
||||
8,Weaponization,0.0,9,4,22853,5010.444444444444,4.980196897150235,4.980196897150235,0.0,0.0
|
||||
8,Weaponization,0.25,9,4,22853,5010.444444444444,5.907611317342421,4.980196897150235,1.9436699786803406,0.39027974572502266
|
||||
8,Weaponization,0.5,9,4,22853,5010.444444444444,6.535688601153058,4.980196897150235,2.945548238376779,0.591452165287336
|
||||
8,Weaponization,0.75,9,4,22853,5010.444444444444,6.866686063025316,4.980196897150235,3.4000203434274914,0.6827080160973251
|
||||
8,Weaponization,1.0,9,4,22853,5010.444444444444,7.064053114587049,4.980196897150235,3.6355977615306365,0.7300108482881461
|
||||
|
+376
@@ -0,0 +1,376 @@
|
||||
class_id,class_name,beta,client_id,N_ij,weight
|
||||
0,C&C,0.0,1,2.0,0.16666666666666666
|
||||
0,C&C,0.0,2,1265.0,0.16666666666666666
|
||||
0,C&C,0.0,4,428.0,0.16666666666666666
|
||||
0,C&C,0.0,7,17.0,0.16666666666666666
|
||||
0,C&C,0.0,8,128.0,0.16666666666666666
|
||||
0,C&C,0.0,9,164.0,0.16666666666666666
|
||||
0,C&C,0.25,1,2.0,0.05752149083785082
|
||||
0,C&C,0.25,2,1265.0,0.28846642080786783
|
||||
0,C&C,0.25,4,428.0,0.22000551966676293
|
||||
0,C&C,0.25,7,17.0,0.09821659299779027
|
||||
0,C&C,0.25,8,128.0,0.16269534494161672
|
||||
0,C&C,0.25,9,164.0,0.17309463074811146
|
||||
0,C&C,0.5,1,2.0,0.01646113496466026
|
||||
0,C&C,0.5,2,1265.0,0.41399018096497286
|
||||
0,C&C,0.5,4,428.0,0.24080564438048116
|
||||
0,C&C,0.5,7,17.0,0.04799204305674498
|
||||
0,C&C,0.5,8,128.0,0.13168907971728208
|
||||
0,C&C,0.5,9,164.0,0.14906191691585868
|
||||
0,C&C,0.75,1,2.0,0.004202920714147631
|
||||
0,C&C,0.75,2,1265.0,0.5300863178677904
|
||||
0,C&C,0.75,4,428.0,0.23515895811836796
|
||||
0,C&C,0.75,7,17.0,0.020922587444366075
|
||||
0,C&C,0.75,8,128.0,0.09510123960842232
|
||||
0,C&C,0.75,9,164.0,0.11452797624690572
|
||||
0,C&C,1.0,1,2.0,0.000998003992015968
|
||||
0,C&C,1.0,2,1265.0,0.6312375249500998
|
||||
0,C&C,1.0,4,428.0,0.21357285429141717
|
||||
0,C&C,1.0,7,17.0,0.008483033932135729
|
||||
0,C&C,1.0,8,128.0,0.06387225548902195
|
||||
0,C&C,1.0,9,164.0,0.08183632734530938
|
||||
1,Exfiltration,0.0,0,430.0,0.125
|
||||
1,Exfiltration,0.0,1,2477.0,0.125
|
||||
1,Exfiltration,0.0,2,388.0,0.125
|
||||
1,Exfiltration,0.0,3,8269.0,0.125
|
||||
1,Exfiltration,0.0,5,4.0,0.125
|
||||
1,Exfiltration,0.0,6,1201.0,0.125
|
||||
1,Exfiltration,0.0,7,2713.0,0.125
|
||||
1,Exfiltration,0.0,8,12.0,0.125
|
||||
1,Exfiltration,0.25,0,430.0,0.10852021538194323
|
||||
1,Exfiltration,0.25,1,2477.0,0.1681221885286965
|
||||
1,Exfiltration,0.25,2,388.0,0.10576731135060888
|
||||
1,Exfiltration,0.25,3,8269.0,0.22725151183360365
|
||||
1,Exfiltration,0.25,5,4.0,0.033702221687081625
|
||||
1,Exfiltration,0.25,6,1201.0,0.14029083011625348
|
||||
1,Exfiltration,0.25,7,2713.0,0.17199110296068054
|
||||
1,Exfiltration,0.25,8,12.0,0.04435461814113203
|
||||
1,Exfiltration,0.5,0,430.0,0.07586215695137205
|
||||
1,Exfiltration,0.5,1,2477.0,0.18207652751910283
|
||||
1,Exfiltration,0.5,2,388.0,0.07206208467700269
|
||||
1,Exfiltration,0.5,3,8269.0,0.33267292353571715
|
||||
1,Exfiltration,0.5,5,4.0,0.007316796132832881
|
||||
1,Exfiltration,0.5,6,1201.0,0.12678341993827041
|
||||
1,Exfiltration,0.5,7,2713.0,0.190553028595012
|
||||
1,Exfiltration,0.5,8,12.0,0.01267306265069003
|
||||
1,Exfiltration,0.75,0,430.0,0.04746790851406514
|
||||
1,Exfiltration,0.75,1,2477.0,0.17649941079295953
|
||||
1,Exfiltration,0.75,2,388.0,0.04394632358480145
|
||||
1,Exfiltration,0.75,3,8269.0,0.43590163594210013
|
||||
1,Exfiltration,0.75,5,4.0,0.0014218171502729262
|
||||
1,Exfiltration,0.75,6,1201.0,0.10255480437584362
|
||||
1,Exfiltration,0.75,7,2713.0,0.1889670574122107
|
||||
1,Exfiltration,0.75,8,12.0,0.0032410422277464666
|
||||
1,Exfiltration,1.0,0,430.0,0.027752678456176585
|
||||
1,Exfiltration,1.0,1,2477.0,0.1598683361301149
|
||||
1,Exfiltration,1.0,2,388.0,0.025041951723247708
|
||||
1,Exfiltration,1.0,3,8269.0,0.5336904608235445
|
||||
1,Exfiltration,1.0,5,4.0,0.000258164450755131
|
||||
1,Exfiltration,1.0,6,1201.0,0.07751387633922809
|
||||
1,Exfiltration,1.0,7,2713.0,0.17510003872466762
|
||||
1,Exfiltration,1.0,8,12.0,0.0007744933522653931
|
||||
2,Exploitation,0.0,1,225.0,0.16666666666666666
|
||||
2,Exploitation,0.0,2,11.0,0.16666666666666666
|
||||
2,Exploitation,0.0,4,11.0,0.16666666666666666
|
||||
2,Exploitation,0.0,6,1.0,0.16666666666666666
|
||||
2,Exploitation,0.0,7,60.0,0.16666666666666666
|
||||
2,Exploitation,0.0,9,485.0,0.16666666666666666
|
||||
2,Exploitation,0.25,1,225.0,0.24219317856265488
|
||||
2,Exploitation,0.25,2,11.0,0.11388445524126942
|
||||
2,Exploitation,0.25,4,11.0,0.11388445524126942
|
||||
2,Exploitation,0.25,6,1.0,0.06253400980921343
|
||||
2,Exploitation,0.25,7,60.0,0.17404200989394278
|
||||
2,Exploitation,0.25,9,485.0,0.2934618912516501
|
||||
2,Exploitation,0.5,1,225.0,0.2862489889095173
|
||||
2,Exploitation,0.5,2,11.0,0.06329203285543153
|
||||
2,Exploitation,0.5,4,11.0,0.06329203285543153
|
||||
2,Exploitation,0.5,6,1.0,0.01908326592730115
|
||||
2,Exploitation,0.5,7,60.0,0.1478183422553696
|
||||
2,Exploitation,0.5,9,485.0,0.42026533719694886
|
||||
2,Exploitation,0.75,1,225.0,0.2962775109628085
|
||||
2,Exploitation,0.75,2,11.0,0.03080394311786928
|
||||
2,Exploitation,0.75,4,11.0,0.03080394311786928
|
||||
2,Exploitation,0.75,6,1.0,0.005099901625843301
|
||||
2,Exploitation,0.75,7,60.0,0.109944937486363
|
||||
2,Exploitation,0.75,9,485.0,0.5270697636892466
|
||||
2,Exploitation,1.0,1,225.0,0.2837326607818411
|
||||
2,Exploitation,1.0,2,11.0,0.013871374527112233
|
||||
2,Exploitation,1.0,4,11.0,0.013871374527112233
|
||||
2,Exploitation,1.0,6,1.0,0.0012610340479192938
|
||||
2,Exploitation,1.0,7,60.0,0.07566204287515763
|
||||
2,Exploitation,1.0,9,485.0,0.6116015132408575
|
||||
3,Lateral_movement,0.0,1,1.0,0.14285714285714285
|
||||
3,Lateral_movement,0.0,2,839.0,0.14285714285714285
|
||||
3,Lateral_movement,0.0,3,631.0,0.14285714285714285
|
||||
3,Lateral_movement,0.0,4,8040.0,0.14285714285714285
|
||||
3,Lateral_movement,0.0,5,29.0,0.14285714285714285
|
||||
3,Lateral_movement,0.0,7,81.0,0.14285714285714285
|
||||
3,Lateral_movement,0.0,8,12496.0,0.14285714285714285
|
||||
3,Lateral_movement,0.25,1,1.0,0.027206001689836622
|
||||
3,Lateral_movement,0.25,2,839.0,0.14642162113053686
|
||||
3,Lateral_movement,0.25,3,631.0,0.1363553117087354
|
||||
3,Lateral_movement,0.25,4,8040.0,0.25761949993409533
|
||||
3,Lateral_movement,0.25,5,29.0,0.06313413290543586
|
||||
3,Lateral_movement,0.25,7,81.0,0.08161800506950986
|
||||
3,Lateral_movement,0.25,8,12496.0,0.2876454275618501
|
||||
3,Lateral_movement,0.5,1,1.0,0.003691100224854745
|
||||
3,Lateral_movement,0.5,2,839.0,0.10691455144116346
|
||||
3,Lateral_movement,0.5,3,631.0,0.09271937968365221
|
||||
3,Lateral_movement,0.5,4,8040.0,0.33096636656037864
|
||||
3,Lateral_movement,0.5,5,29.0,0.019877183030494027
|
||||
3,Lateral_movement,0.5,7,81.0,0.03321990202369271
|
||||
3,Lateral_movement,0.5,8,12496.0,0.41261151703576415
|
||||
3,Lateral_movement,0.75,1,1.0,0.000424944715058991
|
||||
3,Lateral_movement,0.75,2,839.0,0.06624512181122151
|
||||
3,Lateral_movement,0.75,3,631.0,0.0535000825115442
|
||||
3,Lateral_movement,0.75,4,8040.0,0.3608065964613491
|
||||
3,Lateral_movement,0.75,5,29.0,0.00531044519049081
|
||||
3,Lateral_movement,0.75,7,81.0,0.011473507306592757
|
||||
3,Lateral_movement,0.75,8,12496.0,0.5022393020037427
|
||||
3,Lateral_movement,1.0,1,1.0,4.521408871004205e-05
|
||||
3,Lateral_movement,1.0,2,839.0,0.03793462042772528
|
||||
3,Lateral_movement,1.0,3,631.0,0.028530089976036534
|
||||
3,Lateral_movement,1.0,4,8040.0,0.36352127322873806
|
||||
3,Lateral_movement,1.0,5,29.0,0.0013112085725912194
|
||||
3,Lateral_movement,1.0,7,81.0,0.003662341185513406
|
||||
3,Lateral_movement,1.0,8,12496.0,0.5649952525206855
|
||||
4,Normal,0.0,0,118937.0,0.1111111111111111
|
||||
4,Normal,0.0,1,10987.0,0.1111111111111111
|
||||
4,Normal,0.0,3,452.0,0.1111111111111111
|
||||
4,Normal,0.0,4,5015.0,0.1111111111111111
|
||||
4,Normal,0.0,5,61418.0,0.1111111111111111
|
||||
4,Normal,0.0,6,35251.0,0.1111111111111111
|
||||
4,Normal,0.0,7,28172.0,0.1111111111111111
|
||||
4,Normal,0.0,8,576.0,0.1111111111111111
|
||||
4,Normal,0.0,9,31969.0,0.1111111111111111
|
||||
4,Normal,0.25,0,118937.0,0.18116763610564726
|
||||
4,Normal,0.25,1,10987.0,0.09987830277319146
|
||||
4,Normal,0.25,3,452.0,0.04498171207751687
|
||||
4,Normal,0.25,4,5015.0,0.08209545363915403
|
||||
4,Normal,0.25,5,61418.0,0.15357668543007644
|
||||
4,Normal,0.25,6,35251.0,0.1336731656247811
|
||||
4,Normal,0.25,7,28172.0,0.12638798576895244
|
||||
4,Normal,0.25,8,576.0,0.04779219679031755
|
||||
4,Normal,0.25,9,31969.0,0.1304468617903629
|
||||
4,Normal,0.5,0,118937.0,0.25584174545614013
|
||||
4,Normal,0.5,1,10987.0,0.07775932494054551
|
||||
4,Normal,0.5,3,452.0,0.01577183179795273
|
||||
4,Normal,0.5,4,5015.0,0.05253495784074218
|
||||
4,Normal,0.5,5,61418.0,0.18384878091075488
|
||||
4,Normal,0.5,6,35251.0,0.13928316925498305
|
||||
4,Normal,0.5,7,28172.0,0.12451502475376588
|
||||
4,Normal,0.5,8,576.0,0.017804269565964148
|
||||
4,Normal,0.5,9,31969.0,0.1326408954791515
|
||||
4,Normal,0.75,0,118937.0,0.3313499822870636
|
||||
4,Normal,0.75,1,10987.0,0.055521163700217264
|
||||
4,Normal,0.75,3,452.0,0.005071694003860001
|
||||
4,Normal,0.75,4,5015.0,0.030832051292052083
|
||||
4,Normal,0.75,5,61418.0,0.2018463820243199
|
||||
4,Normal,0.75,6,35251.0,0.13309992107767998
|
||||
4,Normal,0.75,7,28172.0,0.11250257855498146
|
||||
4,Normal,0.75,8,576.0,0.006082975668990935
|
||||
4,Normal,0.75,9,31969.0,0.12369325139083488
|
||||
4,Normal,1.0,0,118937.0,0.4062375118264071
|
||||
4,Normal,1.0,1,10987.0,0.03752685491005099
|
||||
4,Normal,1.0,3,452.0,0.001543837118352876
|
||||
4,Normal,1.0,4,5015.0,0.017129077762255914
|
||||
4,Normal,1.0,5,61418.0,0.20977740737831183
|
||||
4,Normal,1.0,6,35251.0,0.12040221738729477
|
||||
4,Normal,1.0,7,28172.0,0.09622340552707351
|
||||
4,Normal,1.0,8,576.0,0.001967367655246143
|
||||
4,Normal,1.0,9,31969.0,0.10919232043500685
|
||||
5,RDOS,0.0,0,2912.0,0.1
|
||||
5,RDOS,0.0,1,8514.0,0.1
|
||||
5,RDOS,0.0,2,48839.0,0.1
|
||||
5,RDOS,0.0,3,17375.0,0.1
|
||||
5,RDOS,0.0,4,11529.0,0.1
|
||||
5,RDOS,0.0,5,1181.0,0.1
|
||||
5,RDOS,0.0,6,1382.0,0.1
|
||||
5,RDOS,0.0,7,2.0,0.1
|
||||
5,RDOS,0.0,8,281.0,0.1
|
||||
5,RDOS,0.0,9,6868.0,0.1
|
||||
5,RDOS,0.25,0,2912.0,0.09181615218461747
|
||||
5,RDOS,0.25,1,8514.0,0.12006174668361975
|
||||
5,RDOS,0.25,2,48839.0,0.1858073701655904
|
||||
5,RDOS,0.25,3,17375.0,0.1435001810902317
|
||||
5,RDOS,0.25,4,11529.0,0.12951472734583627
|
||||
5,RDOS,0.25,5,1181.0,0.0732712701773013
|
||||
5,RDOS,0.25,6,1382.0,0.07620759507496339
|
||||
5,RDOS,0.25,7,2.0,0.014863762068088645
|
||||
5,RDOS,0.25,8,281.0,0.05117381865562517
|
||||
5,RDOS,0.25,9,6868.0,0.11378337655412586
|
||||
5,RDOS,0.5,0,2912.0,0.06927129602501403
|
||||
5,RDOS,0.5,1,8514.0,0.11844710506402363
|
||||
5,RDOS,0.5,2,48839.0,0.2836880285786474
|
||||
5,RDOS,0.5,3,17375.0,0.16920766578044144
|
||||
5,RDOS,0.5,4,11529.0,0.13783307569291034
|
||||
5,RDOS,0.5,5,1181.0,0.04411462345294822
|
||||
5,RDOS,0.5,6,1382.0,0.04772123223671048
|
||||
5,RDOS,0.5,7,2.0,0.0018154012310774377
|
||||
5,RDOS,0.5,8,281.0,0.02151844020802961
|
||||
5,RDOS,0.5,9,6868.0,0.10638313173019738
|
||||
5,RDOS,0.75,0,2912.0,0.046929531453136754
|
||||
5,RDOS,0.75,1,8514.0,0.10493078565985209
|
||||
5,RDOS,0.75,2,48839.0,0.3889356588650953
|
||||
5,RDOS,0.75,3,17375.0,0.17916216432012008
|
||||
5,RDOS,0.75,4,11529.0,0.13171839180525952
|
||||
5,RDOS,0.75,5,1181.0,0.023850095009784134
|
||||
5,RDOS,0.75,6,1382.0,0.026833895398442848
|
||||
5,RDOS,0.75,7,2.0,0.0001991018003000225
|
||||
5,RDOS,0.75,8,281.0,0.008125169510406613
|
||||
5,RDOS,0.75,9,6868.0,0.08931520617760255
|
||||
5,RDOS,1.0,0,2912.0,0.029448944712437933
|
||||
5,RDOS,1.0,1,8514.0,0.08610175662146173
|
||||
5,RDOS,1.0,2,48839.0,0.49390694052567174
|
||||
5,RDOS,1.0,3,17375.0,0.1757127109816652
|
||||
5,RDOS,1.0,4,11529.0,0.11659233639756075
|
||||
5,RDOS,1.0,5,1181.0,0.011943407865861675
|
||||
5,RDOS,1.0,6,1382.0,0.013976113184268276
|
||||
5,RDOS,1.0,7,2.0,2.0225923566234842e-05
|
||||
5,RDOS,1.0,8,281.0,0.0028417422610559954
|
||||
5,RDOS,1.0,9,6868.0,0.06945582152645045
|
||||
6,Reconnaissance,0.0,0,118.0,0.1
|
||||
6,Reconnaissance,0.0,1,6193.0,0.1
|
||||
6,Reconnaissance,0.0,2,121.0,0.1
|
||||
6,Reconnaissance,0.0,3,14322.0,0.1
|
||||
6,Reconnaissance,0.0,4,34.0,0.1
|
||||
6,Reconnaissance,0.0,5,7835.0,0.1
|
||||
6,Reconnaissance,0.0,6,12411.0,0.1
|
||||
6,Reconnaissance,0.0,7,46541.0,0.1
|
||||
6,Reconnaissance,0.0,8,1585.0,0.1
|
||||
6,Reconnaissance,0.0,9,1.0,0.1
|
||||
6,Reconnaissance,0.25,0,118.0,0.04655284204474535
|
||||
6,Reconnaissance,0.25,1,6193.0,0.1252999702623376
|
||||
6,Reconnaissance,0.25,2,121.0,0.04684594891747439
|
||||
6,Reconnaissance,0.25,3,14322.0,0.1545171237033241
|
||||
6,Reconnaissance,0.25,4,34.0,0.03410714968485772
|
||||
6,Reconnaissance,0.25,5,7835.0,0.13288790297113542
|
||||
6,Reconnaissance,0.25,6,12411.0,0.14908274883798045
|
||||
6,Reconnaissance,0.25,7,46541.0,0.20746012037075498
|
||||
6,Reconnaissance,0.25,8,1585.0,0.08912160816126116
|
||||
6,Reconnaissance,0.25,9,1.0,0.014124585046128934
|
||||
6,Reconnaissance,0.5,0,118.0,0.015915466954160424
|
||||
6,Reconnaissance,0.5,1,6193.0,0.1152998975872745
|
||||
6,Reconnaissance,0.5,2,121.0,0.016116512400999587
|
||||
6,Reconnaissance,0.5,3,14322.0,0.17533968267000244
|
||||
6,Reconnaissance,0.5,4,34.0,0.008543146229355121
|
||||
6,Reconnaissance,0.5,5,7835.0,0.12968742939131098
|
||||
6,Reconnaissance,0.5,6,12411.0,0.16322315541601418
|
||||
6,Reconnaissance,0.5,7,46541.0,0.316079432394587
|
||||
6,Reconnaissance,0.5,8,1585.0,0.058330139465295805
|
||||
6,Reconnaissance,0.5,9,1.0,0.0014651374909999625
|
||||
6,Reconnaissance,0.75,0,118.0,0.004758941214920019
|
||||
6,Reconnaissance,0.75,1,6193.0,0.0927950146602146
|
||||
6,Reconnaissance,0.75,2,121.0,0.00484939837328321
|
||||
6,Reconnaissance,0.75,3,14322.0,0.17402097065998273
|
||||
6,Reconnaissance,0.75,4,34.0,0.001871578392060792
|
||||
6,Reconnaissance,0.75,5,7835.0,0.11069502057024065
|
||||
6,Reconnaissance,0.75,6,12411.0,0.1562981783702804
|
||||
6,Reconnaissance,0.75,7,46541.0,0.4211876395158828
|
||||
6,Reconnaissance,0.75,8,1585.0,0.03339033564096019
|
||||
6,Reconnaissance,0.75,9,1.0,0.00013292260217471276
|
||||
6,Reconnaissance,1.0,0,118.0,0.001323448592994695
|
||||
6,Reconnaissance,1.0,1,6193.0,0.06945861980013683
|
||||
6,Reconnaissance,1.0,2,121.0,0.0013570955911216787
|
||||
6,Reconnaissance,1.0,3,14322.0,0.16063076905822052
|
||||
6,Reconnaissance,1.0,4,34.0,0.0003813326454391494
|
||||
6,Reconnaissance,1.0,5,7835.0,0.08787474344163929
|
||||
6,Reconnaissance,1.0,6,12411.0,0.13919763125133186
|
||||
6,Reconnaissance,1.0,7,46541.0,0.5219883132759839
|
||||
6,Reconnaissance,1.0,8,1585.0,0.01777683067708976
|
||||
6,Reconnaissance,1.0,9,1.0,1.1215666042327924e-05
|
||||
7,Tampering,0.0,0,16.0,0.1
|
||||
7,Tampering,0.0,1,5.0,0.1
|
||||
7,Tampering,0.0,2,1.0,0.1
|
||||
7,Tampering,0.0,3,16.0,0.1
|
||||
7,Tampering,0.0,4,18.0,0.1
|
||||
7,Tampering,0.0,5,2584.0,0.1
|
||||
7,Tampering,0.0,6,2.0,0.1
|
||||
7,Tampering,0.0,7,355.0,0.1
|
||||
7,Tampering,0.0,8,582.0,0.1
|
||||
7,Tampering,0.0,9,7.0,0.1
|
||||
7,Tampering,0.25,0,16.0,0.07206430415572625
|
||||
7,Tampering,0.25,1,5.0,0.05388063469441028
|
||||
7,Tampering,0.25,2,1.0,0.036032152077863124
|
||||
7,Tampering,0.25,3,16.0,0.07206430415572625
|
||||
7,Tampering,0.25,4,18.0,0.07421784297424704
|
||||
7,Tampering,0.25,5,2584.0,0.2568995414985236
|
||||
7,Tampering,0.25,6,2.0,0.0428496916198549
|
||||
7,Tampering,0.25,7,355.0,0.15640379624933395
|
||||
7,Tampering,0.25,8,582.0,0.1769786785369322
|
||||
7,Tampering,0.25,9,7.0,0.05860905403738232
|
||||
7,Tampering,0.5,0,16.0,0.03529271004143725
|
||||
7,Tampering,0.5,1,5.0,0.019729224690710777
|
||||
7,Tampering,0.5,2,1.0,0.008823177510359312
|
||||
7,Tampering,0.5,3,16.0,0.03529271004143725
|
||||
7,Tampering,0.5,4,18.0,0.03743357189512626
|
||||
7,Tampering,0.5,5,2584.0,0.44850911273254856
|
||||
7,Tampering,0.5,6,2.0,0.01247785729837542
|
||||
7,Tampering,0.5,7,355.0,0.16624140215257804
|
||||
7,Tampering,0.5,8,582.0,0.2128563001716384
|
||||
7,Tampering,0.5,9,7.0,0.023343933465788762
|
||||
7,Tampering,0.75,0,16.0,0.013383044245276
|
||||
7,Tampering,0.75,1,5.0,0.005593613181311763
|
||||
7,Tampering,0.75,2,1.0,0.0016728805306595
|
||||
7,Tampering,0.75,3,16.0,0.013383044245276
|
||||
7,Tampering,0.75,4,18.0,0.014619055188322223
|
||||
7,Tampering,0.75,5,2584.0,0.6062954496195728
|
||||
7,Tampering,0.75,6,2.0,0.00281343848275861
|
||||
7,Tampering,0.75,7,355.0,0.1368157802603339
|
||||
7,Tampering,0.75,8,582.0,0.1982244243256233
|
||||
7,Tampering,0.75,9,7.0,0.007199269920865995
|
||||
7,Tampering,1.0,0,16.0,0.004461795872838818
|
||||
7,Tampering,1.0,1,5.0,0.0013943112102621305
|
||||
7,Tampering,1.0,2,1.0,0.0002788622420524261
|
||||
7,Tampering,1.0,3,16.0,0.004461795872838818
|
||||
7,Tampering,1.0,4,18.0,0.0050195203569436695
|
||||
7,Tampering,1.0,5,2584.0,0.720580033463469
|
||||
7,Tampering,1.0,6,2.0,0.0005577244841048522
|
||||
7,Tampering,1.0,7,355.0,0.09899609592861126
|
||||
7,Tampering,1.0,8,582.0,0.162297824874512
|
||||
7,Tampering,1.0,9,7.0,0.0019520356943669827
|
||||
8,Weaponization,0.0,0,113.0,0.1111111111111111
|
||||
8,Weaponization,0.0,1,11858.0,0.1111111111111111
|
||||
8,Weaponization,0.0,2,6425.0,0.1111111111111111
|
||||
8,Weaponization,0.0,3,329.0,0.1111111111111111
|
||||
8,Weaponization,0.0,5,3257.0,0.1111111111111111
|
||||
8,Weaponization,0.0,6,240.0,0.1111111111111111
|
||||
8,Weaponization,0.0,7,22853.0,0.1111111111111111
|
||||
8,Weaponization,0.0,8,4.0,0.1111111111111111
|
||||
8,Weaponization,0.0,9,15.0,0.1111111111111111
|
||||
8,Weaponization,0.25,0,113.0,0.06029339607125806
|
||||
8,Weaponization,0.25,1,11858.0,0.19297586537881017
|
||||
8,Weaponization,0.25,2,6425.0,0.16556495623335654
|
||||
8,Weaponization,0.25,3,329.0,0.07875880357288718
|
||||
8,Weaponization,0.25,5,3257.0,0.13970262781302026
|
||||
8,Weaponization,0.25,6,240.0,0.07278685356545217
|
||||
8,Weaponization,0.25,7,22853.0,0.22737145479956772
|
||||
8,Weaponization,0.25,8,4.0,0.026152615782921846
|
||||
8,Weaponization,0.25,9,15.0,0.036393426782726085
|
||||
8,Weaponization,0.5,0,113.0,0.023758434922716884
|
||||
8,Weaponization,0.5,1,11858.0,0.24337968772850077
|
||||
8,Weaponization,0.5,2,6425.0,0.17914932350283266
|
||||
8,Weaponization,0.5,3,329.0,0.040539328949834245
|
||||
8,Weaponization,0.5,5,3257.0,0.1275520625974983
|
||||
8,Weaponization,0.5,6,240.0,0.03462455714479562
|
||||
8,Weaponization,0.5,7,22853.0,0.3378704547612359
|
||||
8,Weaponization,0.5,8,4.0,0.004470011106386682
|
||||
8,Weaponization,0.5,9,15.0,0.008656139286198905
|
||||
8,Weaponization,0.75,0,113.0,0.008009547089511748
|
||||
8,Weaponization,0.75,1,11858.0,0.262607854128171
|
||||
8,Weaponization,0.75,2,6425.0,0.16584560822489527
|
||||
8,Weaponization,0.75,3,329.0,0.017852377013625905
|
||||
8,Weaponization,0.75,5,3257.0,0.09963511934166894
|
||||
8,Weaponization,0.75,6,240.0,0.014091510795449966
|
||||
8,Weaponization,0.75,7,22853.0,0.4295428962814878
|
||||
8,Weaponization,0.75,8,4.0,0.000653648275758195
|
||||
8,Weaponization,0.75,9,15.0,0.0017614388494312458
|
||||
8,Weaponization,1.0,0,113.0,0.0025058766132966693
|
||||
8,Weaponization,1.0,1,11858.0,0.2629618131015213
|
||||
8,Weaponization,1.0,2,6425.0,0.14248015257018673
|
||||
8,Weaponization,1.0,3,329.0,0.007295870847562868
|
||||
8,Weaponization,1.0,5,3257.0,0.07222690380094914
|
||||
8,Weaponization,1.0,6,240.0,0.005322215815851333
|
||||
8,Weaponization,1.0,7,22853.0,0.5067858251652104
|
||||
8,Weaponization,1.0,8,4.0,8.870359693085554e-05
|
||||
8,Weaponization,1.0,9,15.0,0.0003326384884907083
|
||||
|
+253
-209
@@ -3,25 +3,27 @@
|
||||
This is the canonical continuation point for the next agent session. It is written to be
|
||||
readable without prior chat history, terminal scrollback, or git history.
|
||||
|
||||
Last updated: 2026-09-25 (Session 6 + post-experiment audit). Session 6 type:
|
||||
**protocol-scaling / stability experiment**. The single Session-5 condition was extended
|
||||
from seed 0 to seeds `{0,1,2}`; beta was selected by the **pooled-across-seed validation**
|
||||
rule and the selected beta evaluated on test. **No full grid was launched; no
|
||||
alpha/dataset expansion was run.** A strict independent audit
|
||||
(`docs/research/SESSION6_AUDIT.md`) returned **PASS WITH MINOR CORRECTIONS**; `beta_star =
|
||||
0.75` was independently reproduced from validation artifacts.
|
||||
Last updated: 2026-09-26 (Session 7, post-experiment audit complete). Session 7 type:
|
||||
**protocol-scaling / non-IID-severity experiment**. Session 6 completed X-IIoTID `alpha=0.50`
|
||||
for seeds `{0,1,2}`. Session 7 completed X-IIoTID **`alpha=0.25`** for seeds `{0,1,2}` across
|
||||
the five frozen beta candidates plus `original_protean`, using the **same pooled-across-seed
|
||||
validation-only** selection rule. **alpha=0.75 and 5G-NIDD were not run.** No commit or push
|
||||
was made.
|
||||
|
||||
> **Audit summary (Session 6).** Verdict: **PASS WITH MINOR CORRECTIONS**. Independently
|
||||
> recomputed `beta_star = 0.75` from `val_per_client.csv`/`val_selection.json`; all 15
|
||||
> seed×beta seed-level scores and the pooled mean/sample-std match the report exactly.
|
||||
> Validation-only selection confirmed in code (`analyze_multiseed_beta.py` reads only
|
||||
> `val_selection.json`; test read in a later block); `test_used_for_selection=false`
|
||||
> upheld. Rare-class flags match the training-derived definition for all 30 cells;
|
||||
> manifests reproduce from raw data with disjoint splits. beta=0 vs `original_protean` =
|
||||
> semantically equivalent + floating-point divergence (class B). Minor corrections:
|
||||
> persisted `artifacts/analysis/xiiotid__alpha0.5__multiseed__runtime.csv` (runtime
|
||||
> provenance) and clarified the client improve/degrade counts (unchanged/tie clients).
|
||||
> **Session 6 is safe to checkpoint.**
|
||||
> **Audit status (Session 7).** Independent audit verdict: **PASS WITH MINOR CORRECTIONS**
|
||||
> (`docs/research/SESSION7_AUDIT.md`). `beta_star(alpha=0.25) = 0.00` confirmed from validation
|
||||
> only; leakage audit PASS; `beta=0` class **B**; source/data integrity PASS; pytest 75 passed,
|
||||
> 0 failed, 0 skipped, 0 warnings. Corrections were documentation-only (cross-alpha label
|
||||
> disambiguation + one wording nuance); no `beta_star` or scientific interpretation changed.
|
||||
> **Session 7 is safe to checkpoint.**
|
||||
|
||||
> **Headline (Session 7).** PRIMARY pooled-validation selected beta for X-IIoTID
|
||||
> `alpha=0.25` = **0.00** (validation only). The pooled validation curve is monotonically
|
||||
> **decreasing** in beta at alpha=0.25, whereas at alpha=0.50 it peaked at 0.75. Per-seed
|
||||
> diagnostics were 0.75 / 0.50 / 0.00. beta=0 is class **B** (semantically equivalent +
|
||||
> floating-point divergence) for all three seeds; no semantic discrepancy. The realized
|
||||
> alpha=0.25 partitions are materially more heterogeneous (19/27 class-condition cells with
|
||||
> `|S_j| < M` vs 7/27 at alpha=0.50). Effect remains small relative to between-seed variance.
|
||||
|
||||
---
|
||||
|
||||
@@ -29,8 +31,8 @@ alpha/dataset expansion was run.** A strict independent audit
|
||||
|
||||
Build the code and experiment infrastructure for a thesis study that modifies **only the
|
||||
server-side prototype aggregation** of PROTEAN (Chennoufi et al., 2025), a federated
|
||||
intrusion-detection framework. The planned mechanism weights each contributing client's
|
||||
class prototype by its class sample support:
|
||||
intrusion-detection framework. The mechanism weights each contributing client's class
|
||||
prototype by its class sample support:
|
||||
|
||||
```
|
||||
Cbar_j(beta) = ( sum_{i in S_j} N_ij^beta * C_ij ) / ( sum_{i in S_j} N_ij^beta )
|
||||
@@ -55,303 +57,345 @@ bitwise identical to the baseline (see §7).
|
||||
- Frozen baseline (`original_protean`) source unchanged; no `equal_available` mode added;
|
||||
beta candidate set unchanged (`{0, .25, .5, .75, 1}`).
|
||||
|
||||
## 3. Session 6 objective and outcome
|
||||
## 3. Session 7 objective and outcome
|
||||
|
||||
Objective: extend the validated paired protocol from one seed to seeds `{0,1,2}` for
|
||||
X-IIoTID alpha=0.50, reuse the seed-0 Session-5 results, run the two new seeds across the
|
||||
five beta candidates plus `original_protean`, select beta with **pooled validation only**,
|
||||
evaluate the selected beta on test, and produce cross-seed direct-effect / client-level /
|
||||
stability analyses + docs. **All acceptance items were met.** No commit or push was made.
|
||||
Objective: evaluate one additional non-IID severity (X-IIoTID `alpha=0.25`, seeds `{0,1,2}`)
|
||||
across the five beta candidates using the exact Session-6 three-seed pooled-validation
|
||||
protocol, and test whether the pooled-selected beta changes under a more heterogeneous
|
||||
condition. All 18 runs (15 `support_weighted` + 3 `original_protean`) completed with rc=0.
|
||||
Report: `docs/research/ALPHA025_BETA_VALIDATION.md`.
|
||||
|
||||
Outcome: **PRIMARY pooled-validation selected beta = 0.75** (validation only). Per-seed
|
||||
diagnostic selections varied (0.50 / 1.00 / 0.75), and between-seed variance is comparable
|
||||
to the between-beta spread, so the selection is arithmetically clear but statistically
|
||||
weak. The protocol ran cleanly end to end.
|
||||
Outcome: **PRIMARY pooled-validation selected beta = 0.00** (validation only);
|
||||
`test_used_for_selection = false` for all 15 support_weighted runs. Pooled means: 0.00 →
|
||||
0.591704, 0.25 → 0.588691, 0.50 → 0.585851, 0.75 → 0.579463, 1.00 → 0.532099 (monotone
|
||||
decreasing). Between-seed std 0.045–0.090; the effect is small relative to seed variance.
|
||||
|
||||
## 4. Condition and pairing
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| dataset / alpha / seeds | `xiiotid` / 0.50 / `{0,1,2}` |
|
||||
| runs | 15 `support_weighted` + 3 `original_protean` (seed 0 reused; 12 run this session) |
|
||||
| dataset / alpha / seeds | `xiiotid` / 0.25 / `{0,1,2}` |
|
||||
| runs | 15 `support_weighted` + 3 `original_protean`, all new this session |
|
||||
| training | 10 clients, 10 rounds, 3 local epochs, batch 256, lr 1e-3, mu 0.1, lambda 1.0, Adam |
|
||||
| code revision | unchanged from HEAD `50f2da72166e4f609077d65b4916f745da5e8e3b`; only read-only analysis code added |
|
||||
| code revision | unchanged from HEAD `6b97a0d38b839c737c49f41006817503d0672871`; only docs/analysis artifacts added |
|
||||
|
||||
Manifests (reused, never regenerated; one content hash per seed across all six runs):
|
||||
Manifests (reused, never regenerated; one content hash per seed across all six runs; each was
|
||||
reproduced from the raw dataset with split/`N_ij`/partition hashes verified):
|
||||
|
||||
| seed | manifest | content hash | partition hash | split hash | rng seed |
|
||||
|---|---|---|---|---|---|
|
||||
| 0 | `artifacts/partitions/xiiotid__alpha0.5__seed0.json` | `e81a6538…` | `2465485539…` | `d0acc0c1…` | 2009062324 |
|
||||
| 1 | `artifacts/partitions/xiiotid__alpha0.5__seed1.json` | `54ea1df1…` | `4061c5a1…` | `880a9f82…` | 621679668 |
|
||||
| 2 | `artifacts/partitions/xiiotid__alpha0.5__seed2.json` | `be881ac6…` | `1118e7fa…` | `8e0db0be…` | 1438483982 |
|
||||
| 0 | `artifacts/partitions/xiiotid__alpha0.25__seed0.json` | `839ba18e…` | `60faf299…` | `d0acc0c1…` | 1617371214 |
|
||||
| 1 | `artifacts/partitions/xiiotid__alpha0.25__seed1.json` | `84c20b4b…` | `6ffb10ae…` | `880a9f82…` | 1670706564 |
|
||||
| 2 | `artifacts/partitions/xiiotid__alpha0.25__seed2.json` | `d6e68ee1…` | `c8e1ad31…` | `8e0db0be…` | 795318573 |
|
||||
|
||||
Pairing proof: round-0 snapshot byte-identical across the five betas **and**
|
||||
`original_protean` within each seed — seed 0 `e27f41b3a62eb83c…`, seed 1
|
||||
`b7f61c995e97474c…`, seed 2 `5b749f4185e11fc7…`. `train_support_Nij.csv`,
|
||||
`characterization.csv`, `rare_unseen.csv` identical across each seed's runs.
|
||||
`manifest_reload_ok = true` everywhere.
|
||||
`original_protean` within each seed — seed 0 `cd821beb…`, seed 1 `59d2d2bc…`, seed 2
|
||||
`47befeff…`. `train_support_Nij.csv`, `characterization.csv`, `rare_unseen.csv` identical
|
||||
across each seed's runs. `manifest_reload_ok = true`, `smoke = false` everywhere.
|
||||
|
||||
## 5. Validation results (USED FOR SELECTION) and pooled selection
|
||||
|
||||
Primary metric: validation per-client-mean rare-class macro-F1. All 10 clients eligible
|
||||
in every seed. Full per-seed table:
|
||||
`artifacts/analysis/xiiotid__alpha0.5__multiseed__validation_per_seed.csv`.
|
||||
Primary metric: validation per-client-mean rare-class macro-F1. All 10 clients eligible in
|
||||
every seed. Full tables: `artifacts/analysis/xiiotid__alpha0.25__multiseed__validation_per_seed.csv`.
|
||||
|
||||
Per-seed validation per-client-mean rare-class macro-F1:
|
||||
|
||||
| beta | seed 0 | seed 1 | seed 2 |
|
||||
|---|---|---|---|
|
||||
| 0.00 | 0.557340 | 0.628624 | 0.447118 |
|
||||
| 0.25 | 0.571787 | 0.636205 | 0.458310 |
|
||||
| 0.50 | 0.585762 | 0.645189 | 0.471015 |
|
||||
| 0.75 | 0.578003 | 0.655160 | 0.504937 |
|
||||
| 1.00 | 0.551443 | 0.683117 | 0.485877 |
|
||||
| 0.00 | 0.616380 | 0.492425 | 0.666307 |
|
||||
| 0.25 | 0.609913 | 0.514237 | 0.641922 |
|
||||
| 0.50 | 0.595247 | 0.537040 | 0.625267 |
|
||||
| 0.75 | 0.654033 | 0.535672 | 0.548685 |
|
||||
| 1.00 | 0.578385 | 0.530942 | 0.486970 |
|
||||
|
||||
Pooled across seeds (`...__validation_pooled.csv`):
|
||||
|
||||
| beta | pooled mean | std (sample) | min | max | selected |
|
||||
|---|---|---|---|---|---|
|
||||
| 0.00 | 0.544361 | 0.091447 | 0.447118 | 0.628624 | |
|
||||
| 0.25 | 0.555434 | 0.090068 | 0.458310 | 0.636205 | |
|
||||
| 0.50 | 0.567322 | 0.088539 | 0.471015 | 0.645189 | |
|
||||
| **0.75** | **0.579367** | **0.075121** | 0.504937 | 0.655160 | **yes** |
|
||||
| 1.00 | 0.573479 | 0.100450 | 0.485877 | 0.683117 | |
|
||||
| **0.00** | **0.591704** | 0.089529 | 0.492425 | 0.666307 | **yes** |
|
||||
| 0.25 | 0.588691 | 0.066435 | 0.514237 | 0.641922 | |
|
||||
| 0.50 | 0.585851 | 0.044858 | 0.537040 | 0.625267 | |
|
||||
| 0.75 | 0.579463 | 0.064906 | 0.535672 | 0.654033 | |
|
||||
| 1.00 | 0.532099 | 0.045718 | 0.486970 | 0.578385 | |
|
||||
|
||||
- **PRIMARY selected beta (`beta_star`) = 0.75**, max pooled validation per-client-mean
|
||||
rare-class macro-F1; **no tie-break used**; `test_used_for_selection = false` for all 15
|
||||
runs. The secondary pooled recall is highest at beta=0.25 (recall is only a tie-break).
|
||||
- **SECONDARY per-seed diagnostics only:** seed 0 → 0.50, seed 1 → 1.00, seed 2 → 0.75.
|
||||
These are stability diagnostics, not the condition's selection rule.
|
||||
- **PRIMARY selected beta (`beta_star_alpha025`) = 0.00**, max pooled validation
|
||||
per-client-mean rare-class macro-F1; **no tie-break used**; selection is validation-only.
|
||||
- **SECONDARY per-seed diagnostics only:** seed 0 → 0.75, seed 1 → 0.50, seed 2 → 0.00.
|
||||
|
||||
## 6. Final test evaluation of the pooled-selected beta
|
||||
|
||||
`beta_star = 0.75` was fixed from pooled validation before test was read.
|
||||
`beta_star = 0.00` was fixed from pooled validation before test was read.
|
||||
|
||||
| metric (test) | seed 0 | seed 1 | seed 2 | mean ± std |
|
||||
| metric (test) | seed 0 | seed 1 | seed 2 | mean ± std (sample) |
|
||||
|---|---|---|---|---|
|
||||
| rare-class macro-F1 | 0.636189 | 0.671937 | 0.688381 | 0.665503 ± 0.026685 |
|
||||
| rare-class recall | 0.857543 | 0.851883 | 0.878321 | 0.862583 ± 0.013921 |
|
||||
| macro-F1 | 0.726298 | 0.729015 | 0.738194 | 0.731169 ± 0.006234 |
|
||||
| accuracy | 0.888670 | 0.925493 | 0.905012 | 0.906392 ± 0.018450 |
|
||||
| weighted-F1 | 0.922278 | 0.944734 | 0.933884 | 0.933632 ± 0.011230 |
|
||||
| rare-class macro-F1 | 0.641362 | 0.580555 | 0.674489 | 0.632136 ± 0.047642 |
|
||||
| rare-class recall | 0.830701 | 0.825699 | 0.807370 | 0.821257 ± 0.012283 |
|
||||
| macro-F1 | 0.674558 | 0.665039 | 0.704433 | 0.681343 ± 0.020555 |
|
||||
| accuracy | 0.874551 | 0.863490 | 0.884002 | 0.874015 ± 0.010267 |
|
||||
| weighted-F1 | 0.906156 | 0.899864 | 0.914627 | 0.906882 ± 0.007408 |
|
||||
|
||||
`original_protean` for the same seeds/condition:
|
||||
|
||||
| metric (test) | seed 0 | seed 1 | seed 2 | mean ± std |
|
||||
| metric (test) | seed 0 | seed 1 | seed 2 | mean ± std (sample) |
|
||||
|---|---|---|---|---|
|
||||
| rare-class macro-F1 | 0.609620 | 0.655665 | 0.662275 | 0.642520 ± 0.028683 |
|
||||
| rare-class recall | 0.871487 | 0.889365 | 0.870018 | 0.876957 ± 0.010771 |
|
||||
| macro-F1 | 0.707701 | 0.709176 | 0.710017 | 0.708965 ± 0.001172 |
|
||||
| accuracy | 0.890432 | 0.900885 | 0.907045 | 0.899454 ± 0.008398 |
|
||||
| weighted-F1 | 0.917585 | 0.929132 | 0.927481 | 0.924733 ± 0.006245 |
|
||||
| rare-class macro-F1 | 0.639810 | 0.582522 | 0.676039 | 0.632790 ± 0.047152 |
|
||||
| rare-class recall | 0.829601 | 0.826700 | 0.809595 | 0.821965 ± 0.010811 |
|
||||
| macro-F1 | 0.673103 | 0.666981 | 0.705969 | 0.682018 ± 0.020967 |
|
||||
| accuracy | 0.873722 | 0.867341 | 0.885231 | 0.875431 ± 0.009067 |
|
||||
| weighted-F1 | 0.905466 | 0.902075 | 0.915416 | 0.907652 ± 0.006934 |
|
||||
|
||||
Test columns are **descriptive only** and played no part in selection. No superiority or
|
||||
generalisation is claimed: single dataset, single alpha, three project-defined seeds, and
|
||||
between-seed spread ~ between-method difference.
|
||||
Descriptive difference for test rare-class macro-F1 (`beta_star` − `original_protean`) is
|
||||
+0.001552 / −0.001967 / −0.001550 (mean −0.000655) — i.e. essentially identical, as expected
|
||||
because `beta = 0` is mathematically the equal mean over `S_j`. Test is descriptive only; no
|
||||
superiority or generalisation is claimed.
|
||||
|
||||
## 7. `beta = 0` equivalence result (seeds 1 and 2)
|
||||
## 7. `beta = 0` equivalence result (all three seeds)
|
||||
|
||||
For seeds 1 and 2, `support_weighted beta=0` vs frozen `original_protean`:
|
||||
For seeds 0, 1, 2, `support_weighted beta=0` vs frozen `original_protean`:
|
||||
|
||||
- Same manifest/split/hashes: **yes**.
|
||||
- Round-0 snapshot byte-identical: **yes** (seed 1 `b7f61c99…`, seed 2 `5b749f41…`).
|
||||
- Round-0 snapshot byte-identical: **yes** (`cd821beb…`, `59d2d2bc…`, `47befeff…`).
|
||||
- `train_support_Nij.csv`, `characterization.csv`, `rare_unseen.csv`: **identical**.
|
||||
- Aggregation: **mathematically equivalent, not bitwise** (`Σ(C/count)` vs
|
||||
`Σ((1/count)·C)`, ~1 ULP ordering difference that feeds back through round≥1).
|
||||
- Test/history/per-client/per-class: **differ slightly**.
|
||||
- Classification: **mathematically equivalent + floating-point numerical divergence;
|
||||
no semantic discrepancy** in either seed. Same result as Session 5 seed 0.
|
||||
- Independent aggregation (`ΣC/n` vs `(1/n)ΣC`): ≤ 4.4e-16 (1 ULP ordering).
|
||||
- Test/history/per-client/per-class: **differ slightly** (history rare-macro-F1 max diff
|
||||
3.6e-3–6.6e-3; test metric max diff 1.6e-3–3.9e-3).
|
||||
- Classification: **B — mathematically equivalent + floating-point numerical divergence; no
|
||||
semantic discrepancy** in any seed. Machine-readable:
|
||||
`artifacts/analysis/xiiotid__alpha0.25__multiseed__beta0_equivalence.csv`.
|
||||
|
||||
Conditional on the shared `S_j` reproduction assumption; not proof `beta=0` reproduces
|
||||
original PROTEAN.
|
||||
|
||||
## 8. Direct prototype-effect findings (fixed round-0 snapshots, no retraining)
|
||||
## 8. Partition characterization (realized)
|
||||
|
||||
`||Cbar_j(1) - Cbar_j(0)||`, cross-seed mean (std):
|
||||
`|S_j| < M` class-condition cells and missing client-class cells (training split):
|
||||
|
||||
- Largest means: `Lateral_movement` 2.724 (1.035), `RDOS` 2.631 (1.165),
|
||||
`Weaponization` 2.561 (1.228), `Tampering` 2.354 (1.165), `Exploitation` 2.344 (0.412).
|
||||
- Smallest: `Reconnaissance` 1.002 (0.261), `Normal` 1.557 (0.548), `C&C` 1.694 (0.505).
|
||||
- Every class moves for beta>0 (all classes have unequal support); the **rank order is
|
||||
seed-sensitive** (seed 0 top = Weaponization, seed 1 top = RDOS, seed 2 top =
|
||||
Lateral_movement). Displacement is a direct-aggregation measure only, not a detection
|
||||
result. Outputs: `...__prototype_effect_pooled.csv` and the per-seed
|
||||
`...__prototype_snapshot__prototype_effect.csv` / `..._weights.csv`.
|
||||
| condition | `|S_j| < M` cells | missing client-class cells | client-size max/min (seeds) |
|
||||
|---|---|---|---|
|
||||
| alpha=0.25 | **19 / 27** | **38** | 15.6, 20.8, 7.8 |
|
||||
| alpha=0.50 | 7 / 27 | 9 | 5.1, 7.2, 9.9 |
|
||||
|
||||
## 9. Client-level findings (validation, vs beta = 0)
|
||||
Union unseen classes per seed: alpha=0.25 = 6/6/5 (max 4 on a client); alpha=0.50 = 3/3/1
|
||||
(max 2). Union rare classes per seed: alpha=0.25 = 8/7/8; alpha=0.50 = 6/7/7. All clients have
|
||||
exactly two rare classes at both alphas. Full per-class `N_ij`, CV and max/min in
|
||||
`docs/research/ALPHA025_BETA_VALIDATION.md` §B. The lower alpha does **not** uniformly increase
|
||||
every realized statistic (e.g. alpha=0.25 seed 2 client-size ratio 7.8 < alpha=0.50 seed 2's
|
||||
9.9), so interpretation uses the realized partitions, not alpha alone.
|
||||
|
||||
Full detail: `...__multiseed__client_validation_detail.csv`; summary:
|
||||
## 9. Direct prototype-effect findings (fixed round-0 snapshots, no retraining)
|
||||
|
||||
`||Cbar_j(1) - Cbar_j(0)||`, cross-seed mean (std), alpha=0.25 vs alpha=0.50:
|
||||
|
||||
| class | a25 mean (std) | a50 mean (std) |
|
||||
|---|---|---|
|
||||
| Exfiltration | 4.316 (0.898) | 2.020 (0.502) |
|
||||
| Tampering | 4.248 (0.241) | 2.354 (1.165) |
|
||||
| C&C | 4.020 (0.290) | 1.694 (0.505) |
|
||||
| Lateral_movement | 3.726 (1.383) | 2.724 (1.035) |
|
||||
| Exploitation | 3.464 (1.113) | 2.344 (0.412) |
|
||||
| Reconnaissance | 3.339 (1.107) | 1.002 (0.261) |
|
||||
| Weaponization | 3.124 (0.585) | 2.561 (1.228) |
|
||||
| RDOS | 2.566 (1.219) | 2.631 (1.165) |
|
||||
| Normal | 1.997 (0.533) | 1.557 (0.548) |
|
||||
|
||||
Mean displacement across classes is larger at alpha=0.25 (3.42) than alpha=0.50 (2.10), because
|
||||
support is more imbalanced and more classes are missing contributors. Displacement is a
|
||||
direct-aggregation measure only, not a detection result. Outputs:
|
||||
`artifacts/analysis/xiiotid__alpha0.25__multiseed__prototype_effect_pooled.csv` and per-seed
|
||||
`...__seed{0,1,2}__support_weighted__beta0__prototype_snapshot__prototype_effect.csv`.
|
||||
|
||||
## 10. Client-level findings (validation, vs beta = 0)
|
||||
|
||||
Detail: `...__multiseed__client_validation_detail.csv`; summary:
|
||||
`...__client_validation_summary.csv`.
|
||||
|
||||
- At `beta = 0.75`, improve/degrade/**unchanged** counts are 7/3/0 (seed 0), 8/1/1
|
||||
(seed 1, tie = client 7), 8/1/1 (seed 2, tie = client 4); mean Δ is positive in all
|
||||
three (+0.021 / +0.027 / +0.058).
|
||||
- At `beta = 1.00`, seed 0 flips to 4 improve / 6 degrade / 0 unchanged, while seed 1
|
||||
improves 9/10 with 1 unchanged (client 7) — the high-beta response is seed-dependent.
|
||||
- `min positive attack N_ij` vs ΔF1 is **non-monotonic**: N=1 clients range −0.021…+0.041
|
||||
at beta=0.75; the largest gain (+0.177) is an N=10 client; low support neither
|
||||
systematically benefits nor degrades. Client ids are different realisations across
|
||||
seeds and are not treated as one population. Descriptive only.
|
||||
- At alpha=0.25 the higher-beta response is **seed-dependent and often negative**:
|
||||
improve/degrade/unchanged at beta=0.75 = 5/4/1 (seed 0, mean Δ +0.038), 5/4/1 (seed 1,
|
||||
+0.043), 3/5/2 (seed 2, −0.118). At beta=1.00 seed 2 degrades 6/10 clients with a worst
|
||||
client Δ = −0.678, while seed 1 still gains on average.
|
||||
- This **contrasts with alpha=0.50**, where beta=0.75 improved the majority of clients in every
|
||||
seed. Client ids are per-seed realisations and are not one population; the analysis is
|
||||
descriptive and non-causal, and `N_ij` vs ΔF1 remains non-monotonic.
|
||||
|
||||
## 10. Runtime / resources
|
||||
## 11. Comparison with alpha=0.50 (RQ3)
|
||||
|
||||
CPU only, no GPU, no swap, exit 0 on all 18 runs (15 new-session + reused). Per-run
|
||||
wall-clock 40:23–42:21; CPU 733–745 %; peak RSS 6.21 GiB (seed 0), 7.48 GiB (seed 1),
|
||||
6.45 GiB (seed 2). New compute this session: 12 runs, ≈ 8 h 11 m wall-clock, run
|
||||
sequentially (8 cores; concurrent runs would not reduce total CPU work).
|
||||
Seed-0 numbers were reused and not re-run. Machine-readable per-run values:
|
||||
`artifacts/analysis/xiiotid__alpha0.5__multiseed__runtime.csv` (added by the audit;
|
||||
seed 0 sourced from Session-5 docs, seeds 1–2 from `/usr/bin/time -v` logs).
|
||||
| | alpha=0.50 | alpha=0.25 |
|
||||
|---|---|---|
|
||||
| pooled-selected beta | 0.75 | **0.00** |
|
||||
| pooled validation curve | rises to 0.75 | monotone decreasing |
|
||||
| pooled mean range | 0.544–0.579 | 0.532–0.592 |
|
||||
| pooled std range | 0.075–0.100 | 0.045–0.090 |
|
||||
| `|S_j| < M` cells | 7/27 | 19/27 |
|
||||
| mean `||Cbar(1)−Cbar(0)||` | 2.10 | 3.42 |
|
||||
|
||||
## 11. Methodological decisions / assumptions
|
||||
The selection **changes** with non-IID severity. This is documented as a descriptive
|
||||
association accompanied by the realized partition characteristics; **no causal claim is made
|
||||
that alpha by itself caused the difference**. The effect is small relative to between-seed
|
||||
variance at both settings.
|
||||
|
||||
> **Audited label (do not confuse).** Condition-level **pooled** selections are alpha=0.50 →
|
||||
> **0.75** and alpha=0.25 → **0.00**. The value **0.50** is only the Session-5 **seed-0
|
||||
> single-condition** choice and the Session-6 alpha=0.50 **per-seed diagnostic** for seed 0;
|
||||
> it is not the alpha=0.50 pooled selection. Any "0.50 → 0.00" phrasing for condition-level
|
||||
> pooled selection is wrong. Verified in `docs/research/SESSION7_AUDIT.md` §M.
|
||||
|
||||
## 12. Runtime / resources
|
||||
|
||||
CPU only, no GPU, no swap, exit 0 on all 18 runs. Per-run wall-clock 39:49–42:12; CPU
|
||||
741–746 %; peak RSS 8.38 GiB (seed 0), 10.80 GiB (seed 1), 7.51 GiB (seed 2). New compute:
|
||||
18 runs, ≈ 43,799 s ≈ **12.17 h** wall-clock, sequential (8 cores). Machine-readable:
|
||||
`artifacts/analysis/xiiotid__alpha0.25__multiseed__runtime.csv`.
|
||||
|
||||
## 13. Methodological decisions / assumptions
|
||||
|
||||
1. **`S_j` participant/denominator rule** (`S_j = {i | N_ij > 0 and valid C_ij}`, equal
|
||||
weights in the baseline) is a **REPRODUCTION ASSUMPTION**; material in 53/162
|
||||
class-condition cells (32.7 %), 15/18 conditions (`BASELINE_VALIDATION.md` §D).
|
||||
Carry this caveat in all reporting.
|
||||
weights in the baseline) is a **REPRODUCTION ASSUMPTION**; material in 19/27
|
||||
class-condition cells at alpha=0.25 (7/27 at alpha=0.50). Carry this caveat in all reporting.
|
||||
2. `N_ij` is class-specific sample support, not prototype quality.
|
||||
3. Beta is one value for all classes/clients; candidates frozen at `{0,.25,.5,.75,1}`.
|
||||
4. **PRIMARY selection = pooled validation across clients and seeds**; per-seed selection
|
||||
is secondary stability diagnostics; test used only after the pooled beta is fixed.
|
||||
Documented in `EXPERIMENT_PROTOCOL.md` §8 (updated this session).
|
||||
5. The union of rare classes (and global rare support) differs by seed because partitions
|
||||
differ; only the per-client-mean metric is directly comparable across seeds.
|
||||
6. `lambda=1.0`, Adam, lr 1e-3, batch 256 remain PROJECT DECISIONS. No `equal_available`
|
||||
mode was added.
|
||||
4. **PRIMARY selection = pooled validation across clients and seeds**; per-seed selection is
|
||||
secondary stability diagnostics; test used only after the pooled beta is fixed.
|
||||
5. The `scripts/analyze_multiseed_beta.py` already supported `--alpha` (default 0.5) and
|
||||
already includes alpha in output names; **no code change was needed**. alpha=0.50 outputs
|
||||
were re-run and are byte-identical to the Session-6 artifacts (reproducibility verified).
|
||||
6. `lambda=1.0`, Adam, lr 1e-3, batch 256 remain PROJECT DECISIONS. No `equal_available` mode.
|
||||
|
||||
## 12. Unresolved questions / blockers
|
||||
## 14. Unresolved questions / blockers
|
||||
|
||||
1. `S_j` reproduction ambiguity — unresolved; no official PROTEAN code exists.
|
||||
2. **Pooled beta selection is statistically weak here:** pooled means span only
|
||||
0.544→0.579 while within-beta across-seed std is 0.075–0.100; three seeds give a
|
||||
standard error of the same order as the between-beta spread. The beta effect is small
|
||||
relative to seed variance.
|
||||
3. `beta = 0` is not bitwise identical to `original_protean`; documentation decision on how
|
||||
to report this remains open.
|
||||
2. **Pooled beta selection remains statistically weak at both alphas (descriptive only):**
|
||||
the across-seed sample std (0.045–0.090) is comparable to, or larger than, the total
|
||||
between-beta pooled-mean spread (0.035 at alpha=0.50, 0.060 at alpha=0.25). No inferential
|
||||
test was performed, so this is a descriptive comparison, not evidence of insignificance.
|
||||
3. `beta = 0` is not bitwise identical to `original_protean`; documentation decision on how to
|
||||
report this remains open.
|
||||
4. Session-2 manifests still store `preprocessing_hash = null` / `config_hash = null`.
|
||||
5. Only alpha=0.50, X-IIoTID, three seeds measured.
|
||||
5. Only alpha=0.25 and 0.50, X-IIoTID, three seeds measured. No 5G-NIDD beta result.
|
||||
6. The alpha=0.25 result (pooled-selected beta = 0) is **one additional condition, not yet a
|
||||
trend across alpha**. Whether any relationship between non-IID severity and the selected beta
|
||||
is monotone would require alpha=0.75 (and is still not implied by two conditions).
|
||||
|
||||
**Audit status (see `docs/research/SESSION6_AUDIT.md`).** No unresolved issue that could
|
||||
change `beta_star` or the scientific interpretation. One inherent audit limitation: the
|
||||
trained model is not persisted, so per-client prediction metrics cannot be recomputed
|
||||
without retraining (forbidden); they were verified via defining source code, exact
|
||||
agreement of their mean with `val_selection.json`, and exact agreement of the global
|
||||
union-rare metric recomputed from `val_confusion_matrix.json`. Pooled std uses the
|
||||
**sample** convention (ddof=1), labelled "(sample)" in the report.
|
||||
## 15. Git status
|
||||
|
||||
## 13. Git status
|
||||
|
||||
No commit or push was made. `git status --short` (results/ is ignored):
|
||||
No commit or push was made. `git status --short` shows only untracked docs/analysis/run
|
||||
artifacts (`results/` is gitignored; `artifacts/analysis/` is intentionally tracked):
|
||||
|
||||
```
|
||||
M docs/handoff/SESSION_HANDOFF.md
|
||||
M docs/research/EXPERIMENT_PROTOCOL.md
|
||||
?? docs/research/MULTISEED_BETA_VALIDATION.md
|
||||
?? docs/research/SESSION6_AUDIT.md
|
||||
?? scripts/analyze_multiseed_beta.py
|
||||
?? artifacts/analysis/xiiotid__alpha0.5__multiseed__*.csv / __selection_summary.json
|
||||
?? artifacts/analysis/xiiotid__alpha0.5__seed{1,2}__*__prototype_snapshot.json
|
||||
?? artifacts/analysis/xiiotid__alpha0.5__seed{1,2}__support_weighted__beta0__prototype_snapshot__{prototype_effect,prototype_weights}.csv
|
||||
?? artifacts/analysis/xiiotid__alpha0.5__seed0__support_weighted__beta{0.25,0.5,0.75,1}__prototype_snapshot.json (pre-existing from Session 5)
|
||||
?? docs/research/ALPHA025_BETA_VALIDATION.md
|
||||
?? docs/research/SESSION7_AUDIT.md
|
||||
?? artifacts/analysis/xiiotid__alpha0.25__* (runs' snapshots, effect CSVs, multiseed tables)
|
||||
?? (pre-existing Session-5/6 untracked alpha=0.5 views, unchanged)
|
||||
```
|
||||
|
||||
`planningawal/` and both raw datasets are unchanged; `results/` is gitignored.
|
||||
`HEAD = 50f2da72166e4f609077d65b4916f745da5e8e3b`. No commit or push was made.
|
||||
`planningawal/` and both raw datasets are unchanged (audit re-verified SHA-256:
|
||||
X-IIoTID `7b929005…`, 5G-NIDD `fa36f808…`). No model/training/aggregation source changed
|
||||
(working-tree blobs equal HEAD). `artifacts/partitions/` unmodified. No alpha=0.75 run and no
|
||||
5G-NIDD `support_weighted` run exists. `equal_available` absent.
|
||||
`HEAD = 6b97a0d38b839c737c49f41006817503d0672871`. No commit or push was made.
|
||||
|
||||
## 14. Files created / modified this session
|
||||
## 15b. Session 7 audit result (independent)
|
||||
|
||||
- Verdict: **PASS WITH MINOR CORRECTIONS** (`docs/research/SESSION7_AUDIT.md`).
|
||||
- Run inventory 18/18, pairing hash-identical, manifests reproduced from raw data.
|
||||
- Validation recomputed exactly; **pooled `beta_star(alpha=0.25) = 0.00`** (validation only,
|
||||
monotone decreasing in beta). `beta_star` test and `original_protean` test means/stds exact.
|
||||
- Leakage audit: selection reads validation only; `test_used_for_selection=false` upheld.
|
||||
- `beta=0` equivalence: class **B** for all three seeds (no semantic discrepancy).
|
||||
- Source/data integrity: PASS. pytest: **75 passed, 0 failed, 0 skipped, 0 warnings**.
|
||||
- Corrections: cross-alpha label disambiguation (documentation-only) and one wording nuance.
|
||||
No arithmetic, metric-label, std-convention, or provenance defect was found.
|
||||
|
||||
## 16. Files created / modified this session
|
||||
|
||||
Created:
|
||||
|
||||
- `docs/research/MULTISEED_BETA_VALIDATION.md`
|
||||
- `docs/research/SESSION6_AUDIT.md` (post-experiment audit)
|
||||
- `scripts/analyze_multiseed_beta.py`
|
||||
- `artifacts/analysis/xiiotid__alpha0.5__multiseed__*` (validation per-seed/pooled,
|
||||
client summary/detail/deltas, prototype-effect pooled, test descriptive, selection
|
||||
summary, and `...__runtime.csv` added by the audit)
|
||||
- seed-1/seed-2 `results/...` run dirs (gitignored) and their snapshots / effect CSVs
|
||||
- `docs/research/ALPHA025_BETA_VALIDATION.md`
|
||||
- `docs/research/SESSION7_AUDIT.md` (independent post-experiment audit)
|
||||
- `artifacts/analysis/xiiotid__alpha0.25__seed{0,1,2}__*__prototype_snapshot.json`
|
||||
- `artifacts/analysis/xiiotid__alpha0.25__seed{0,1,2}__support_weighted__beta0__prototype_snapshot__{prototype_effect,prototype_weights}.csv`
|
||||
- `artifacts/analysis/xiiotid__alpha0.25__multiseed__{validation_per_seed,validation_pooled,client_validation_deltas,client_validation_detail,client_validation_summary,prototype_effect_pooled,test_descriptive,runtime,beta0_equivalence}.csv` and `...__selection_summary.json`
|
||||
- `results/xiiotid__alpha0.25__seed{0,1,2}__*` (18 run dirs, gitignored)
|
||||
|
||||
Modified:
|
||||
|
||||
- `docs/research/EXPERIMENT_PROTOCOL.md` (§8 selection hierarchy clarified)
|
||||
- `docs/handoff/SESSION_HANDOFF.md` (this file)
|
||||
|
||||
Reused (not regenerated): all seed-0 Session-5 runs and artifacts.
|
||||
No source or test code was changed. `scripts/analyze_multiseed_beta.py` was re-run for
|
||||
alpha=0.50 and produced byte-identical outputs.
|
||||
|
||||
## 15. Exact recommended next phase
|
||||
## 17. Exact recommended next phase
|
||||
|
||||
**Session 6 is safe to checkpoint** (audit PASS WITH MINOR CORRECTIONS; no issue affects
|
||||
`beta_star`). The protocol is **mechanically** ready to scale, but Session 6 shows the beta
|
||||
effect at alpha=0.50 is small relative to seed-to-seed variance. Recommended, in order:
|
||||
1. **Session 7 is audited and safe to checkpoint** (verdict PASS WITH MINOR CORRECTIONS;
|
||||
see §15b and `docs/research/SESSION7_AUDIT.md`). alpha=0.25 behaved cleanly: pairing valid,
|
||||
beta=0 class B, pooled selection clean, artifacts complete, no test leakage, runtime
|
||||
manageable (≈12 h per alpha). Before checkpointing, stage only the intended docs/analysis
|
||||
artifacts; do not commit `results/` or the raw datasets.
|
||||
2. **Next experiment:** run X-IIoTID `alpha=0.75` under the same protocol (seeds `{0,1,2}`,
|
||||
five betas + `original_protean`, sequential, reusing frozen manifests, pooled
|
||||
validation-only selection), then test evaluation, direct-effect, client-level, and
|
||||
cross-alpha comparison. This would complete the three planned non-IID severities. Do **not**
|
||||
infer a monotone alpha→beta relation from two conditions.
|
||||
3. Do **not** treat `beta = 0.00` (alpha=0.25) or `0.75` (alpha=0.50) as optima. Report each as
|
||||
"best among tested candidates under pooled validation for that condition".
|
||||
4. **Do not launch 5G-NIDD or the full grid** until alpha=0.75 is inspected.
|
||||
5. Use `.venv/bin/python`. Run one alpha at a time; measure runtime as before.
|
||||
6. Update `docs/research/ALPHA025_BETA_VALIDATION.md` (or add `ALPHA075_BETA_VALIDATION.md`),
|
||||
`EXPERIMENT_PROTOCOL.md` only if a real ambiguity appears, and this handoff. Do not commit or
|
||||
push unless asked.
|
||||
|
||||
1. **Do not treat beta=0.75 as a settled optimum.** Report it as "best among tested
|
||||
candidates under pooled validation for X-IIoTID alpha=0.50".
|
||||
2. Before expanding alpha, decide the interpretation question: is the goal to characterise
|
||||
small beta effects against seed noise (requires more seeds), or to run alpha=0.25/0.75
|
||||
as a paired characterisation (accepting the same weak-effect caveat)?
|
||||
3. If expanding, run alpha=0.25 then alpha=0.75 for seeds `{0,1,2}`, the five betas each,
|
||||
reusing frozen manifests, sequentially (~8 h per alpha), with the same pooled
|
||||
validation-only selection. Extend `scripts/analyze_multiseed_beta.py` with `--alpha`.
|
||||
4. Do **not** launch 5G-NIDD or the full grid until the alpha results are inspected.
|
||||
|
||||
## 16. Important commands
|
||||
## 18. Important commands
|
||||
|
||||
```bash
|
||||
# Tests (75 expected)
|
||||
.venv/bin/python -m pytest
|
||||
|
||||
# One full paired beta run (CPU, ~41 min, ~6.5-7.5 GiB)
|
||||
.venv/bin/python scripts/run_baseline.py --dataset xiiotid --alpha 0.5 --seed 1 \
|
||||
# One full paired beta run (CPU, ~40 min, ~7.5-11.3 GiB)
|
||||
.venv/bin/python scripts/run_baseline.py --dataset xiiotid --alpha 0.25 --seed 1 \
|
||||
--clients 10 --rounds 10 --local-epochs 3 \
|
||||
--aggregation-mode support_weighted --beta 0.5
|
||||
|
||||
# Pooled multi-seed validation analysis (read-only, no retraining)
|
||||
.venv/bin/python scripts/analyze_multiseed_beta.py --dataset xiiotid --alpha 0.5 \
|
||||
--seeds 0 1 2
|
||||
# Pooled multi-seed validation analysis (read-only, no retraining; --alpha already supported)
|
||||
.venv/bin/python scripts/analyze_multiseed_beta.py --dataset xiiotid --alpha 0.25 --seeds 0 1 2
|
||||
|
||||
# Direct prototype-effect analysis (no retraining)
|
||||
.venv/bin/python scripts/analyze_prototype_effect.py \
|
||||
--snapshot artifacts/analysis/xiiotid__alpha0.5__seed1__support_weighted__beta0__prototype_snapshot.json \
|
||||
--snapshot artifacts/analysis/xiiotid__alpha0.25__seed0__support_weighted__beta0__prototype_snapshot.json \
|
||||
--class-names "C&C,Exfiltration,Exploitation,Lateral_movement,Normal,RDOS,Reconnaissance,Tampering,Weaponization" \
|
||||
--output-dir artifacts/analysis
|
||||
```
|
||||
|
||||
## 17. Recommended next-session prompt
|
||||
## 19. Recommended next-session prompt
|
||||
|
||||
```
|
||||
You are continuing the perdalam-protean research repository (Session 7).
|
||||
You are continuing the perdalam-protean research repository (Session 8).
|
||||
|
||||
FIRST, read completely:
|
||||
1. docs/handoff/SESSION_HANDOFF.md
|
||||
2. docs/research/MULTISEED_BETA_VALIDATION.md
|
||||
3. docs/research/SESSION6_AUDIT.md
|
||||
2. docs/research/ALPHA025_BETA_VALIDATION.md
|
||||
3. docs/research/MULTISEED_BETA_VALIDATION.md
|
||||
4. docs/research/EXPERIMENT_PROTOCOL.md
|
||||
5. docs/baseline/BASELINE_VALIDATION.md
|
||||
|
||||
Critical rules:
|
||||
- planningawal/ is immutable. Do not edit/create/move/rename/delete anything inside it.
|
||||
- ~/datasets/X-IIoTID/X-IIoTID-dataset.csv and ~/datasets/5G-NIDD/5G-NIDD.csv are
|
||||
read-only external source data. Do not modify or copy them into the repo.
|
||||
- The S_j participant rule is a REPRODUCTION ASSUMPTION (material in 32.7% of
|
||||
class-condition cells). Carry this caveat in all reporting.
|
||||
- planningawal/ is immutable.
|
||||
- ~/datasets/X-IIoTID/X-IIoTID-dataset.csv and ~/datasets/5G-NIDD/5G-NIDD.csv are read-only.
|
||||
- The S_j participant rule is a REPRODUCTION ASSUMPTION (material in 19/27 cells at alpha=0.25).
|
||||
- Do NOT add an executable equal_available mode.
|
||||
- Do NOT launch the full alpha x seed x dataset x beta grid.
|
||||
- Beta selection uses validation metrics only; test never influences selection.
|
||||
- PRIMARY selection is pooled across clients and seeds; per-seed selection is
|
||||
diagnostic only.
|
||||
- Beta selection uses validation only; PRIMARY is pooled across clients and seeds.
|
||||
- Do NOT launch 5G-NIDD or the full grid.
|
||||
|
||||
Context: Session 6 completed X-IIoTID alpha=0.50 for seeds {0,1,2} across beta in
|
||||
{0,0.25,0.5,0.75,1}. Seed 0 was reused from Session 5. Pooled validation selected
|
||||
beta=0.75 (val per-client-mean rare macro-F1 0.579367); per-seed diagnostics were
|
||||
0.50/1.00/0.75. Pooled-selected test rare macro-F1 0.665503 +/- 0.026685 vs
|
||||
original_protean 0.642520 +/- 0.028683. beta=0 is mathematically but not bitwise equal
|
||||
to original_protean (float ordering), no semantic discrepancy. Between-seed variance is
|
||||
comparable to the between-beta spread, so the selection is weak. A strict independent
|
||||
audit (docs/research/SESSION6_AUDIT.md) returned PASS WITH MINOR CORRECTIONS and
|
||||
independently reproduced beta_star=0.75 from validation artifacts alone.
|
||||
Context: Session 7 completed X-IIoTID alpha=0.25 for seeds {0,1,2} (18 runs, ~12.17 h) and was
|
||||
independently audited (PASS WITH MINOR CORRECTIONS; docs/research/SESSION7_AUDIT.md). Pooled
|
||||
validation selected beta=0.00 (pooled mean 0.591704; monotone decreasing in beta), unlike
|
||||
alpha=0.50 which selected 0.75. Per-seed diagnostics 0.75/0.50/0.00. beta=0 equivalence class B
|
||||
(no semantic discrepancy). Realized partitions more heterogeneous (19/27 |S_j|<M vs 7/27).
|
||||
Condition-level pooled labels: alpha=0.50 -> 0.75, alpha=0.25 -> 0.00 (the value 0.50 is only the
|
||||
Session-5 seed-0 choice, not a condition-level selection).
|
||||
|
||||
Task: decide and document the alpha-expansion plan (recommended: alpha=0.25 then 0.75,
|
||||
seeds {0,1,2}, five betas each, sequential, reusing frozen manifests, pooled
|
||||
validation-only selection) OR argue for more seeds at alpha=0.50 first. Do not launch
|
||||
5G-NIDD or the full grid. Update docs and docs/handoff/SESSION_HANDOFF.md. Use
|
||||
.venv/bin/python. Do not commit or push unless asked.
|
||||
Task: run X-IIoTID alpha=0.75 for seeds {0,1,2} across the five frozen betas plus
|
||||
original_protean, sequential, reusing frozen manifests, pooled validation-only selection, then
|
||||
test evaluation, direct-effect, client-level, and cross-alpha comparison. Produce
|
||||
docs/research/ALPHA075_BETA_VALIDATION.md and update the handoff. Use .venv/bin/python. Do not
|
||||
commit or push unless asked.
|
||||
```
|
||||
@@ -0,0 +1,467 @@
|
||||
# Alpha-0.25 Beta Validation (Session 7)
|
||||
|
||||
Status: **three-seed paired protocol completed for a second non-IID condition**
|
||||
(X-IIoTID `alpha = 0.25`, seeds `{0,1,2}`) with pooled-across-seed validation-only beta
|
||||
selection, followed by a single final test evaluation of the pooled-selected beta. This is a
|
||||
**paired characterisation of a more heterogeneous non-IID setting**, not a sweep, not a
|
||||
hyper-parameter search over seeds, and not evidence of a global optimum.
|
||||
|
||||
Companion documents:
|
||||
|
||||
- `docs/research/EXPERIMENT_PROTOCOL.md` — frozen protocol and selection hierarchy.
|
||||
- `docs/research/MULTISEED_BETA_VALIDATION.md` — Session-6 alpha=0.50 three-seed condition.
|
||||
- `docs/research/PAIRED_BETA_VALIDATION.md` — Session-5 single-seed condition.
|
||||
- `docs/research/SESSION6_AUDIT.md` — independent audit of the alpha=0.50 protocol.
|
||||
- `docs/research/BETA_AGGREGATION_IMPLEMENTATION.md` — beta mode implementation.
|
||||
- `docs/baseline/BASELINE_VALIDATION.md` — frozen baseline evidence.
|
||||
- `docs/handoff/SESSION_HANDOFF.md` — continuation point.
|
||||
|
||||
---
|
||||
|
||||
## A. Scope
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| dataset | `xiiotid` (X-IIoTID) |
|
||||
| alpha | 0.25 |
|
||||
| seeds | 0, 1, 2 |
|
||||
| aggregation | `original_protean` (per seed) + `support_weighted` for `beta ∈ {0, 0.25, 0.5, 0.75, 1}` |
|
||||
| runs | 18: 15 `support_weighted` + 3 `original_protean` |
|
||||
| new this session | all 18 runs (no alpha=0.25 run pre-existed) |
|
||||
| compute | sequential on CPU, one run at a time, ≈ 12.17 h wall-clock total |
|
||||
|
||||
No other alpha, dataset, seed, mode or beta was run. `equal_available` was not added.
|
||||
alpha=0.75 and 5G-NIDD were deliberately **not** run.
|
||||
|
||||
---
|
||||
|
||||
## B. Partition characterization (realized, before interpretation)
|
||||
|
||||
Manifests are reused, never regenerated. Each manifest was reloaded and independently
|
||||
reproduced from the raw dataset (`reproduce_train_assignments`: split hash, `N_ij` and
|
||||
partition hash all verified). All three seeds share train/validation/test sizes
|
||||
569,909 / 81,414 / 162,831 (10 clients, 9 classes, `Normal` benign).
|
||||
|
||||
| seed | manifest | content hash | partition hash | split hash | rng seed |
|
||||
|---|---|---|---|---|---|
|
||||
| 0 | `artifacts/partitions/xiiotid__alpha0.25__seed0.json` | `839ba18e…` | `60faf299…` | `d0acc0c1…` | 1617371214 |
|
||||
| 1 | `artifacts/partitions/xiiotid__alpha0.25__seed1.json` | `84c20b4b…` | `6ffb10ae…` | `880a9f82…` | 1670706564 |
|
||||
| 2 | `artifacts/partitions/xiiotid__alpha0.25__seed2.json` | `d6e68ee1…` | `c8e1ad31…` | `8e0db0be…` | 795318573 |
|
||||
|
||||
Client-sample totals and class-support characterization (`N_ij`, training split). `CV` is the
|
||||
population coefficient of variation of the positive supports within a class; `max/min` is over
|
||||
contributing clients only.
|
||||
|
||||
### Seed 0 (`|S_j| < 10` in 6/9 classes)
|
||||
|
||||
| class | total | #clients | min+ | max | mean+ | CV | max/min |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| C&C | 2,004 | 10 | 21 | 1,065 | 200.4 | 1.478 | 50.7 |
|
||||
| Exfiltration | 15,494 | 8 | 4 | 7,950 | 1,936.8 | 1.405 | 1987.5 |
|
||||
| Exploitation | 793 | 6 | 2 | 595 | 132.2 | 1.580 | 297.5 |
|
||||
| Lateral_movement | 22,117 | 9 | 2 | 13,857 | 2,457.4 | 1.810 | 6928.5 |
|
||||
| Normal | 292,777 | 10 | 1 | 122,800 | 29,277.7 | 1.346 | 122800.0 |
|
||||
| RDOS | 98,883 | 10 | 166 | 31,089 | 9,888.3 | 1.080 | 187.3 |
|
||||
| Reconnaissance | 89,161 | 8 | 46 | 65,307 | 11,145.1 | 1.855 | 1419.7 |
|
||||
| Tampering | 3,586 | 8 | 9 | 2,272 | 448.2 | 1.569 | 252.4 |
|
||||
| Weaponization | 45,094 | 8 | 43 | 23,381 | 5,636.8 | 1.528 | 543.7 |
|
||||
|
||||
client-sample totals min 9,075 / max 141,358 (ratio 15.6).
|
||||
|
||||
### Seed 1 (`|S_j| < 10` in 7/9 classes)
|
||||
|
||||
| class | total | #clients | min+ | max | mean+ | CV | max/min |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| C&C | 2,004 | 9 | 1 | 986 | 222.7 | 1.559 | 986.0 |
|
||||
| Exfiltration | 15,494 | 10 | 1 | 6,204 | 1,549.4 | 1.238 | 6204.0 |
|
||||
| Exploitation | 793 | 8 | 1 | 321 | 99.1 | 1.295 | 321.0 |
|
||||
| Lateral_movement | 22,117 | 10 | 1 | 10,094 | 2,211.7 | 1.434 | 10094.0 |
|
||||
| Normal | 292,777 | 8 | 198 | 129,678 | 36,597.1 | 1.163 | 654.9 |
|
||||
| RDOS | 98,883 | 9 | 168 | 53,482 | 10,987.0 | 1.468 | 318.4 |
|
||||
| Reconnaissance | 89,161 | 8 | 138 | 65,101 | 11,145.1 | 1.852 | 471.8 |
|
||||
| Tampering | 3,586 | 9 | 2 | 1,804 | 398.4 | 1.348 | 902.0 |
|
||||
| Weaponization | 45,094 | 9 | 1 | 12,024 | 5,010.4 | 0.833 | 12024.0 |
|
||||
|
||||
client-sample totals min 9,278 / max 193,326 (ratio 20.8).
|
||||
|
||||
### Seed 2 (`|S_j| < 10` in 6/9 classes)
|
||||
|
||||
| class | total | #clients | min+ | max | mean+ | CV | max/min |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| C&C | 2,004 | 6 | 2 | 1,265 | 334.0 | 1.315 | 632.5 |
|
||||
| Exfiltration | 15,494 | 8 | 4 | 8,269 | 1,936.8 | 1.337 | 2067.3 |
|
||||
| Exploitation | 793 | 6 | 1 | 485 | 132.2 | 1.328 | 485.0 |
|
||||
| Lateral_movement | 22,117 | 7 | 1 | 12,496 | 3,159.6 | 1.475 | 12496.0 |
|
||||
| Normal | 292,777 | 9 | 452 | 118,937 | 32,530.8 | 1.104 | 263.1 |
|
||||
| RDOS | 98,883 | 10 | 2 | 48,839 | 9,888.3 | 1.421 | 24419.5 |
|
||||
| Reconnaissance | 89,161 | 10 | 1 | 46,541 | 8,916.1 | 1.518 | 46541.0 |
|
||||
| Tampering | 3,586 | 10 | 1 | 2,584 | 358.6 | 2.134 | 2584.0 |
|
||||
| Weaponization | 45,094 | 9 | 4 | 22,853 | 5,010.4 | 1.471 | 5713.3 |
|
||||
|
||||
client-sample totals min 15,664 / max 122,526 (ratio 7.8).
|
||||
|
||||
### Rare / unseen (training-derived, per client)
|
||||
|
||||
Every client has exactly two rare classes in every seed. Unseen classes are much more
|
||||
frequent than at alpha=0.50:
|
||||
|
||||
| seed | rare per client | union rare | unseen per client | union unseen |
|
||||
|---|---|---|---|---|
|
||||
| 0 | 2, 2, 2, 2, 2, 2, 2, 2, 2, 2 | 8 classes | 0, 3, 1, 2, 2, 1, 2, 0, 0, 2 | 6 classes |
|
||||
| 1 | 2 × 10 | 7 classes | 0, 0, 4, 1, 1, 0, 1, 0, 1, 0 | 6 classes |
|
||||
| 2 | 2 × 10 | 8 classes | 3, 0, 0, 2, 2, 2, 2, 0, 1, 2 | 5 classes |
|
||||
|
||||
### Comparison of realized heterogeneity (alpha=0.25 vs alpha=0.50)
|
||||
|
||||
| statistic | alpha=0.25 | alpha=0.50 |
|
||||
|---|---|---|
|
||||
| class-condition cells with `|S_j| < M` | **19 / 27** | 7 / 27 |
|
||||
| missing client-class cells | **38** | 9 |
|
||||
| client-sample-total max/min (per seed) | 15.6, 20.8, 7.8 | 5.1, 7.2, 9.9 |
|
||||
| union rare classes per seed | 8, 7, 8 | 6, 7, 7 |
|
||||
| union unseen classes per seed | 6, 6, 5 | 3, 3, 1 |
|
||||
| max unseen classes on a client | 4 | 2 |
|
||||
|
||||
The lower Dirichlet parameter does propagate to more missing classes, more unseen classes, and
|
||||
a larger client-size spread — but **not uniformly**: seed 2 at alpha=0.25 has a smaller
|
||||
client-size ratio (7.8) than alpha=0.50 seed 2 (9.9), and its `|S_j| < M` count is 6/9 versus
|
||||
1/9 at alpha=0.50. Interpretation uses the realized partitions, not alpha alone.
|
||||
|
||||
---
|
||||
|
||||
## C. Frozen protocol
|
||||
|
||||
Identical to `EXPERIMENT_PROTOCOL.md` §14 and to Session 6: 10 clients, 10 rounds, 3 local
|
||||
epochs, batch 256, Adam `lr=1e-3`, `mu=0.1`, `lambda=1.0`; unchanged architecture,
|
||||
preprocessing, train/validation/test split recipe, partition recipe, loss terms and inference.
|
||||
All 18 runs were performed this session with `scripts/run_baseline.py` at HEAD `6b97a0d`
|
||||
(no training/aggregation source changed).
|
||||
|
||||
### C.1 Manifest and build-level pairing
|
||||
|
||||
Within each seed all six runs share one manifest content hash and byte-identical deterministic
|
||||
artifacts:
|
||||
|
||||
| seed | manifest content hash | round-0 snapshot SHA-256 (all 6 runs) | support / characterization / rare_unseen |
|
||||
|---|---|---|---|
|
||||
| 0 | `839ba18e…` | `cd821beb…` | identical |
|
||||
| 1 | `84c20b4b…` | `59d2d2bc…` | identical |
|
||||
| 2 | `d6e68ee1…` | `47befeff…` | identical |
|
||||
|
||||
`manifest_reload_ok = true` and `smoke = false` in all 18 runs; metadata confirms
|
||||
`dataset=xiiotid`, `alpha=0.25`, seed, mode, beta, clients=10, rounds=10, local_epochs=3,
|
||||
batch=256, lr=1e-3, mu=0.1, lambda=1.0. Pairing is proven by content hashes, not filenames.
|
||||
|
||||
### C.2 Selection rule (unchanged)
|
||||
|
||||
- **PRIMARY:** pooled-across-seed validation per-client-mean rare-class macro-F1 (average the
|
||||
seed-level per-client mean over seeds `{0,1,2}`), selected with validation only.
|
||||
- **SECONDARY (diagnostic):** per-seed validation selection, for stability only.
|
||||
- **TEST:** read only after the pooled beta is fixed.
|
||||
|
||||
---
|
||||
|
||||
## D. Per-seed validation results
|
||||
|
||||
`val rareMacroF1 (pc mean)` is the primary metric; all 10 clients are eligible in every seed.
|
||||
Validation only.
|
||||
|
||||
### Seed 0
|
||||
|
||||
| beta | eligible | val rareMacroF1 (pc mean) | val rareMacroF1 (global) | val rareRecall (global) | val macroF1 | val acc |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 0.00 | 10 | 0.616380 | 0.642939 | 0.841886 | 0.676032 | 0.874776 |
|
||||
| 0.25 | 10 | 0.609913 | 0.646423 | 0.834580 | 0.679089 | 0.867038 |
|
||||
| 0.50 | 10 | 0.595247 | 0.633677 | 0.807172 | 0.667755 | 0.853256 |
|
||||
| **0.75** | 10 | **0.654033** | **0.666477** | 0.797400 | **0.697353** | 0.854313 |
|
||||
| 1.00 | 10 | 0.578385 | 0.585755 | 0.710305 | 0.626018 | 0.847188 |
|
||||
|
||||
### Seed 1
|
||||
|
||||
| beta | eligible | val rareMacroF1 (pc mean) | val rareMacroF1 (global) | val rareRecall (global) | val macroF1 | val acc |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 0.00 | 10 | 0.492425 | 0.580976 | 0.824292 | 0.665021 | 0.861707 |
|
||||
| 0.25 | 10 | 0.514237 | 0.605558 | 0.833188 | 0.684847 | 0.870391 |
|
||||
| **0.50** | 10 | **0.537040** | 0.647568 | 0.829322 | 0.718155 | 0.880917 |
|
||||
| 0.75 | 10 | 0.535672 | **0.655554** | 0.817017 | **0.723896** | 0.879210 |
|
||||
| 1.00 | 10 | 0.530942 | 0.652257 | 0.803940 | 0.721164 | 0.874346 |
|
||||
|
||||
### Seed 2
|
||||
|
||||
| beta | eligible | val rareMacroF1 (pc mean) | val rareMacroF1 (global) | val rareRecall (global) | val macroF1 | val acc |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **0.00** | 10 | **0.666307** | **0.673945** | 0.819363 | **0.703911** | **0.882895** |
|
||||
| 0.25 | 10 | 0.641922 | 0.655402 | 0.825502 | 0.687134 | 0.872024 |
|
||||
| 0.50 | 10 | 0.625267 | 0.636281 | 0.806218 | 0.670651 | 0.868462 |
|
||||
| 0.75 | 10 | 0.548685 | 0.558044 | 0.712391 | 0.601211 | 0.851058 |
|
||||
| 1.00 | 10 | 0.486970 | 0.499711 | 0.637821 | 0.549558 | 0.838517 |
|
||||
|
||||
Machine-readable: `artifacts/analysis/xiiotid__alpha0.25__multiseed__validation_per_seed.csv`.
|
||||
|
||||
---
|
||||
|
||||
## E. Per-seed diagnostic beta choices (SECONDARY, stability only)
|
||||
|
||||
| seed | selected beta | val rareMacroF1 (pc mean) | tie-break used | test_used_for_selection |
|
||||
|---|---|---|---|---|
|
||||
| 0 | 0.75 | 0.654033 | no | false |
|
||||
| 1 | 0.50 | 0.537040 | no | false |
|
||||
| 2 | 0.00 | 0.666307 | no | false |
|
||||
|
||||
The preferred beta is seed-dependent (0.75 / 0.50 / 0.00). These per-seed choices are
|
||||
**diagnostic only** and are not the condition-level selection rule.
|
||||
`val_selection.json` records `"selection_split": "validation"` and
|
||||
`"test_used_for_selection": false` for all 15 `support_weighted` runs.
|
||||
|
||||
---
|
||||
|
||||
## F. PRIMARY pooled-across-seed validation selection
|
||||
|
||||
| beta | pooled mean | std (sample, ddof=1) | std (pop) | min | max | pooled val rareRecall (pc mean) | selected |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| **0.00** | **0.591704** | 0.089529 | 0.073100 | 0.492425 | 0.666307 | 0.759482 | **yes** |
|
||||
| 0.25 | 0.588691 | 0.066435 | 0.054244 | 0.514237 | 0.641922 | 0.780771 | |
|
||||
| 0.50 | 0.585851 | 0.044858 | 0.036626 | 0.537040 | 0.625267 | 0.772536 | |
|
||||
| 0.75 | 0.579463 | 0.064906 | 0.052996 | 0.535672 | 0.654033 | 0.760791 | |
|
||||
| 1.00 | 0.532099 | 0.045718 | 0.037329 | 0.486970 | 0.578385 | 0.704575 | |
|
||||
|
||||
- **PRIMARY selected beta (`beta_star_alpha025`) = 0.00.**
|
||||
- **Criterion:** maximum pooled validation per-client-mean rare-class macro-F1.
|
||||
- **Tie-break used:** no (no exact tie; the pooled means separate by > 1e-3).
|
||||
- **`test_used_for_selection = false`.** Test metrics were not inspected until the pooled beta
|
||||
was fixed.
|
||||
- **Interpretation:** `beta = 0.00` is the *best among the tested candidates under pooled
|
||||
validation across the three project-defined seeds for X-IIoTID alpha=0.25.* It is **not** a
|
||||
global optimum. The pooled validation ordering is **monotonically decreasing in beta**.
|
||||
|
||||
Machine-readable: `artifacts/analysis/xiiotid__alpha0.25__multiseed__validation_pooled.csv`
|
||||
and `...__selection_summary.json`.
|
||||
|
||||
---
|
||||
|
||||
## G. Final selected-beta test metrics (descriptive only)
|
||||
|
||||
`beta_star = 0.00` was fixed from pooled validation; only then were test metrics read.
|
||||
|
||||
| metric (test) | seed 0 | seed 1 | seed 2 | mean ± std (sample) |
|
||||
|---|---|---|---|---|
|
||||
| rare-class macro-F1 | 0.641362 | 0.580555 | 0.674489 | 0.632136 ± 0.047642 |
|
||||
| rare-class recall | 0.830701 | 0.825699 | 0.807370 | 0.821257 ± 0.012283 |
|
||||
| macro-F1 | 0.674558 | 0.665039 | 0.704433 | 0.681343 ± 0.020555 |
|
||||
| accuracy | 0.874551 | 0.863490 | 0.884002 | 0.874015 ± 0.010267 |
|
||||
| weighted-F1 | 0.906156 | 0.899864 | 0.914627 | 0.906882 ± 0.007408 |
|
||||
|
||||
---
|
||||
|
||||
## H. `original_protean` comparison
|
||||
|
||||
| metric (test) | seed 0 | seed 1 | seed 2 | mean ± std (sample) |
|
||||
|---|---|---|---|---|
|
||||
| rare-class macro-F1 | 0.639810 | 0.582522 | 0.676039 | 0.632790 ± 0.047152 |
|
||||
| rare-class recall | 0.829601 | 0.826700 | 0.809595 | 0.821965 ± 0.010811 |
|
||||
| macro-F1 | 0.673103 | 0.666981 | 0.705969 | 0.682018 ± 0.020967 |
|
||||
| accuracy | 0.873722 | 0.867341 | 0.885231 | 0.875431 ± 0.009067 |
|
||||
| weighted-F1 | 0.905466 | 0.902075 | 0.915416 | 0.907652 ± 0.006934 |
|
||||
|
||||
Descriptive difference (`beta_star` − `original_protean`) for test rare-class macro-F1 is
|
||||
+0.001552 (seed 0), −0.001967 (seed 1), −0.001550 (seed 2); mean −0.000655. This is expected:
|
||||
`beta = 0` is mathematically the equal mean over `S_j` (see §I), so the end-to-end difference is
|
||||
floating-point trajectory noise, not a method effect. No superiority or generalisation is
|
||||
claimed; this is one dataset, one alpha, three project-defined seeds.
|
||||
|
||||
Test columns are descriptive only and are recorded in
|
||||
`artifacts/analysis/xiiotid__alpha0.25__multiseed__test_descriptive.csv`.
|
||||
|
||||
---
|
||||
|
||||
## I. `beta = 0` equivalence vs `original_protean`
|
||||
|
||||
For all three seeds, `support_weighted beta=0` vs frozen `original_protean`:
|
||||
|
||||
| property | seed 0 | seed 1 | seed 2 |
|
||||
|---|---|---|---|
|
||||
| same manifest / split / hashes | yes | yes | yes |
|
||||
| round-0 snapshot byte-identical | yes (`cd821beb…`) | yes (`59d2d2bc…`) | yes (`47befeff…`) |
|
||||
| `train_support_Nij.csv`, `characterization.csv`, `rare_unseen.csv` identical | yes | yes | yes |
|
||||
| independent aggregation diff (`ΣC/n` vs `(1/n)ΣC`) | 4.4e-16 | 4.4e-16 | 2.2e-16 |
|
||||
| history rare-macro-F1 round-0 diff | 0.0 | 0.0 | 0.0 |
|
||||
| history rare-macro-F1 max diff (rounds ≥ 1) | 6.6e-3 | 3.6e-3 | 3.7e-3 |
|
||||
| test metric max diff | 1.6e-3 | 3.9e-3 | 2.2e-3 |
|
||||
| **classification** | **B** | **B** | **B** |
|
||||
|
||||
Class **B**: semantic equivalence + floating-point numerical divergence; **no semantic
|
||||
discrepancy** in any seed. Round 0 is byte-identical; the ~1 ULP ordering difference enters the
|
||||
round-1 prototype-alignment training and the trajectories diverge slowly. Equivalence remains
|
||||
conditional on the shared `S_j` reproduction assumption and is **not** proof that `beta = 0`
|
||||
reproduces original PROTEAN.
|
||||
|
||||
---
|
||||
|
||||
## J. Direct prototype-effect findings (fixed round-0 snapshots, no retraining)
|
||||
|
||||
Computed with the production aggregation from each seed's fixed `{C_ij, N_ij}` round-0
|
||||
snapshot. `||Cbar_j(beta) - Cbar_j(0)||` at `beta = 1.0`, cross-seed mean (sample std):
|
||||
|
||||
| class | mean disp | std | min | max | mean rel disp | typical `|S_j|` |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Exfiltration | 4.316 | 0.898 | 3.588 | 5.319 | 0.664 | 8 |
|
||||
| Tampering | 4.248 | 0.241 | 3.978 | 4.441 | 0.965 | 8 |
|
||||
| C&C | 4.020 | 0.290 | 3.691 | 4.242 | 0.847 | 10 |
|
||||
| Lateral_movement | 3.726 | 1.383 | 2.342 | 5.109 | 0.790 | 9 |
|
||||
| Exploitation | 3.464 | 1.113 | 2.180 | 4.152 | 0.702 | 6 |
|
||||
| Reconnaissance | 3.339 | 1.107 | 2.089 | 4.195 | 0.640 | 8 |
|
||||
| Weaponization | 3.124 | 0.585 | 2.486 | 3.636 | 0.563 | 8 |
|
||||
| RDOS | 2.566 | 1.219 | 1.848 | 3.974 | 0.328 | 10 |
|
||||
| Normal | 1.997 | 0.533 | 1.667 | 2.611 | 0.370 | 10 |
|
||||
|
||||
Findings (descriptive):
|
||||
|
||||
- Every class moves for `beta > 0` in every seed, because every class has unequal support across
|
||||
its contributors.
|
||||
- Classes with the fewest contributors / strongest support imbalance (`Tampering`, `C&C`,
|
||||
`Exfiltration`, `Exploitation`) have the largest relative displacement; `Tampering` at
|
||||
`rel = 0.965` moves almost one full prototype norm.
|
||||
- `RDOS` and `Normal` move least in relative terms despite large absolute class sizes, because
|
||||
their support is spread more evenly across all 10 clients (`RDOS` CV ≈ 1.08–1.47).
|
||||
- Displacement is a **direct-aggregation measure only** and is not interpreted as improved or
|
||||
degraded detection. The end-to-end effect (§F/§G) is reported separately.
|
||||
|
||||
Outputs: `artifacts/analysis/xiiotid__alpha0.25__seed{0,1,2}__support_weighted__beta0__prototype_snapshot__prototype_effect.csv`
|
||||
and `...__prototype_weights.csv`; cross-seed aggregate
|
||||
`artifacts/analysis/xiiotid__alpha0.25__multiseed__prototype_effect_pooled.csv`.
|
||||
|
||||
---
|
||||
|
||||
## K. Client-level effect (validation, relative to beta = 0)
|
||||
|
||||
Full per-client table:
|
||||
`artifacts/analysis/xiiotid__alpha0.25__multiseed__client_validation_detail.csv`; summary:
|
||||
`...__client_validation_summary.csv`.
|
||||
|
||||
| seed | beta | improve | degrade | unchanged | mean Δ | min Δ | max Δ |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| 0 | 0.25 | 2 | 5 | 3 | −0.0065 | −0.0429 | +0.0188 |
|
||||
| 0 | 0.50 | 2 | 7 | 1 | −0.0211 | −0.0769 | +0.0574 |
|
||||
| 0 | 0.75 | 5 | 4 | 1 | +0.0377 | −0.1117 | +0.3759 |
|
||||
| 0 | 1.00 | 4 | 5 | 1 | −0.0380 | −0.2570 | +0.1410 |
|
||||
| 1 | 0.25 | 7 | 1 | 2 | +0.0218 | −0.0064 | +0.0718 |
|
||||
| 1 | 0.50 | 7 | 2 | 1 | +0.0446 | −0.0158 | +0.2356 |
|
||||
| 1 | 0.75 | 5 | 4 | 1 | +0.0432 | −0.0183 | +0.3167 |
|
||||
| 1 | 1.00 | 5 | 4 | 1 | +0.0385 | −0.0387 | +0.3774 |
|
||||
| 2 | 0.25 | 5 | 3 | 2 | −0.0244 | −0.3029 | +0.0819 |
|
||||
| 2 | 0.50 | 4 | 4 | 2 | −0.0410 | −0.3398 | +0.1185 |
|
||||
| 2 | 0.75 | 3 | 5 | 2 | −0.1176 | −0.6011 | +0.1241 |
|
||||
| 2 | 1.00 | 3 | 6 | 1 | −0.1793 | −0.6775 | +0.0754 |
|
||||
|
||||
Descriptive observations:
|
||||
|
||||
- Unlike alpha=0.50 (where `beta = 0.75` improved the majority of clients in every seed), at
|
||||
alpha=0.25 the higher-beta response is **seed-dependent and often negative**: seeds 0 and 2
|
||||
degrade on average for `beta ≥ 0.25`, while seed 1 improves for `0.25–1.00`.
|
||||
- The magnitude of losses grows with beta: seed 2 loses up to −0.677 on one client at
|
||||
`beta = 1.00`. The largest single gains also grow with beta (seed 1, +0.377), i.e. beta
|
||||
redistributes sharply rather than uniformly.
|
||||
- Client ids are per-seed realisations and are not treated as one population. This analysis is
|
||||
descriptive and non-causal; `N_ij` vs ΔF1 remains non-monotonic (some low-support clients
|
||||
gain, some lose).
|
||||
|
||||
---
|
||||
|
||||
## L. Comparison with alpha=0.50
|
||||
|
||||
Validation-side evidence (selection basis); test is not used to explain the selection.
|
||||
|
||||
| beta | alpha=0.50 pooled mean (std) | alpha=0.25 pooled mean (std) |
|
||||
|---|---|---|
|
||||
| 0.00 | 0.544361 (0.091447) | **0.591704 (0.089529)** |
|
||||
| 0.25 | 0.555434 (0.090068) | 0.588691 (0.066435) |
|
||||
| 0.50 | 0.567322 (0.088539) | 0.585851 (0.044858) |
|
||||
| 0.75 | **0.579367 (0.075121)** | 0.579463 (0.064906) |
|
||||
| 1.00 | 0.573479 (0.100450) | 0.532099 (0.045718) |
|
||||
| selected | **0.75** | **0.00** |
|
||||
| pooled mean range | 0.544–0.579 (0.035) | 0.532–0.592 (0.060) |
|
||||
|
||||
> **Disambiguation (audited, Session 7).** The condition-level pooled selections are
|
||||
> alpha=0.50 → **0.75** (Session 6) and alpha=0.25 → **0.00** (Session 7). The value **0.50**
|
||||
> is *not* the alpha=0.50 condition-level selection: it is only the Session-5 **seed-0
|
||||
> single-condition** choice and the Session-6 alpha=0.50 **per-seed diagnostic** for seed 0.
|
||||
> Any phrasing of the form "0.50 → 0.00" that refers to condition-level pooled selection is
|
||||
> incorrect. See `docs/research/SESSION7_AUDIT.md` §M.
|
||||
|
||||
Observations:
|
||||
|
||||
- **The pooled validation-selected beta changes with non-IID severity**: 0.75 at alpha=0.50 vs
|
||||
**0.00** at alpha=0.25. At alpha=0.25 the pooled validation curve is monotonically decreasing
|
||||
in beta; at alpha=0.50 it rises to a peak at 0.75.
|
||||
- **Across-seed std does not shrink at the more heterogeneous setting**: 0.045–0.090
|
||||
(alpha=0.25) vs 0.075–0.100 (alpha=0.50). The selected beta has a lower across-seed std at
|
||||
alpha=0.25 (0.0895) but this is not the selection criterion (no tie).
|
||||
- **Realized partition differences accompany the beta change** (§B): alpha=0.25 has 19/27
|
||||
`|S_j| < M` cell conditions vs 7/27, more unseen classes, and larger per-class support
|
||||
imbalance, so up-weighting high-support clients redistributes the global prototypes further
|
||||
(mean `||Cbar(1)−Cbar(0)||` 3.42 vs 2.10; §M/§J) and is more often harmful on validation.
|
||||
- **Client-level pattern flips**: at alpha=0.50 higher beta helped most clients in every seed;
|
||||
at alpha=0.25 it helps mainly in seed 1 and hurts on average in seeds 0 and 2.
|
||||
|
||||
These are descriptive associations. The realized partition characteristics (not alpha alone)
|
||||
are reported alongside, and **no causal claim is made that alpha by itself caused the
|
||||
difference**. The beta effect remains small relative to between-seed variance at both settings.
|
||||
|
||||
---
|
||||
|
||||
## M. Runtime and resources
|
||||
|
||||
CPU only, no GPU, no swap, exit status 0 for every run (`/usr/bin/time -v`). Per-run
|
||||
wall-clock 39:49–42:12.
|
||||
|
||||
| seed | tag | wall | CPU % | peak RSS |
|
||||
|---|---|---|---|---|
|
||||
| 0 | `original_protean` | 40:10.80 | 746 | 8.38 GiB |
|
||||
| 0 | beta 0.00 | 40:57.24 | 744 | 8.38 GiB |
|
||||
| 0 | beta 0.25 | 40:27.96 | 745 | 8.38 GiB |
|
||||
| 0 | beta 0.50 | 40:21.77 | 743 | 8.39 GiB |
|
||||
| 0 | beta 0.75 | 39:49.26 | 746 | 8.38 GiB |
|
||||
| 0 | beta 1.00 | 40:25.86 | 745 | 8.38 GiB |
|
||||
| 1 | `original_protean` | 40:29.86 | 745 | 10.80 GiB |
|
||||
| 1 | beta 0.00 | 39:52.79 | 745 | 10.80 GiB |
|
||||
| 1 | beta 0.25 | 40:45.79 | 745 | 10.80 GiB |
|
||||
| 1 | beta 0.50 | 40:18.93 | 745 | 10.80 GiB |
|
||||
| 1 | beta 0.75 | 39:58.73 | 744 | 10.80 GiB |
|
||||
| 1 | beta 1.00 | 40:47.41 | 745 | 10.80 GiB |
|
||||
| 2 | `original_protean` | 40:13.48 | 745 | 7.51 GiB |
|
||||
| 2 | beta 0.00 | 40:24.04 | 746 | 7.51 GiB |
|
||||
| 2 | beta 0.25 | 41:45.12 | 743 | 7.51 GiB |
|
||||
| 2 | beta 0.50 | 40:22.35 | 744 | 7.51 GiB |
|
||||
| 2 | beta 0.75 | 42:11.66 | 741 | 7.51 GiB |
|
||||
| 2 | beta 1.00 | 40:35.98 | 744 | 7.51 GiB |
|
||||
|
||||
- New compute this session: **18 runs, ≈ 43,799 s ≈ 12.17 h wall-clock**, sequential on an
|
||||
8-core host (seed chains ≈ 4.04 / 4.04 / 4.09 h).
|
||||
- Peak RSS is partition-dependent, not beta-dependent: 8.38 GiB (seed 0), 10.80 GiB (seed 1),
|
||||
7.51 GiB (seed 2); all well within the 29 GiB host.
|
||||
- No warnings, no client failures, 9/9 global prototypes in every round, no retries.
|
||||
- Machine-readable: `artifacts/analysis/xiiotid__alpha0.25__multiseed__runtime.csv`.
|
||||
|
||||
---
|
||||
|
||||
## N. Limitations
|
||||
|
||||
- **One dataset** (X-IIoTID). No cross-dataset conclusion.
|
||||
- **Only alpha 0.25 and 0.50 evaluated so far.** alpha=0.75 remains unmeasured.
|
||||
- **Three project-defined seeds** (`{0,1,2}`); the pooled mean’s standard error is of the same
|
||||
order as the between-beta spread, so the selected beta is statistically weak.
|
||||
- The `S_j` participant/denominator rule
|
||||
(`S_j = {i | N_ij > 0 and valid C_ij}`, equal weights in the baseline) remains a
|
||||
**REPRODUCTION ASSUMPTION**. At alpha=0.25 it is material in **19/27 class-condition cells**.
|
||||
The `beta = 0` comparison is conditional on both modes sharing this rule and does **not**
|
||||
prove `beta = 0` reproduces original PROTEAN.
|
||||
- Train/validation/test adaptation (70/10/20 stratified) differs from the paper’s literal
|
||||
80/20; the validation eval partition is a project decision.
|
||||
- **No 5G-NIDD beta result.**
|
||||
- **No global beta optimum claim.** `beta = 0.00` is only the best among the five tested
|
||||
candidates under pooled validation for X-IIoTID alpha=0.25 across seeds `{0,1,2}`.
|
||||
- The union of rare classes (and hence global rare support) differs by seed and by alpha;
|
||||
only the per-client-mean metric is directly comparable within a condition. Cross-alpha
|
||||
comparisons in §L are descriptive, not controlled.
|
||||
- Test metrics of the selected beta are single-condition and must not be read as
|
||||
generalisation; no test metric influenced selection.
|
||||
@@ -0,0 +1,378 @@
|
||||
# Session 7 Post-Experiment Audit (X-IIoTID alpha=0.25)
|
||||
|
||||
Status: **PASS WITH MINOR CORRECTIONS.** Independent recomputation from machine-readable
|
||||
artifacts confirms every Session 7 claim. No discovered issue changes `beta_star` or the
|
||||
scientific interpretation. Corrections are documentation-only (cross-alpha disambiguation and
|
||||
one wording clarification). **No training was run, no aggregation/model source changed, no
|
||||
test-based selection occurred.**
|
||||
|
||||
Audit performed: 2026-09-26. Audited HEAD: `6b97a0d38b839c737c49f41006817503d0672871`.
|
||||
|
||||
Companion documents: `docs/research/ALPHA025_BETA_VALIDATION.md`,
|
||||
`docs/research/MULTISEED_BETA_VALIDATION.md`, `docs/research/SESSION6_AUDIT.md`,
|
||||
`docs/research/EXPERIMENT_PROTOCOL.md`, `docs/research/PAIRED_BETA_VALIDATION.md`,
|
||||
`docs/handoff/SESSION_HANDOFF.md`.
|
||||
|
||||
---
|
||||
|
||||
## A. Scope
|
||||
|
||||
Independently verify the Session 7 claim for:
|
||||
|
||||
- dataset `xiiotid`, alpha `0.25`, seeds `{0,1,2}`, beta `{0, 0.25, 0.5, 0.75, 1.0}` plus
|
||||
`original_protean` (18 runs);
|
||||
- PRIMARY pooled-validation selection `beta_star = 0.00`, using validation per-client
|
||||
rare-class macro-F1 averaged within seed then across seeds.
|
||||
|
||||
The audit recomputed metrics from run artifacts and re-derived the selection; it did not repeat
|
||||
the report, and it ran no training. No alpha=0.75 or 5G-NIDD run was launched.
|
||||
|
||||
---
|
||||
|
||||
## B. Run inventory
|
||||
|
||||
18 full-data runs were enumerated from `results/*/metadata.json` (15 `support_weighted` + 3
|
||||
`original_protean`). Every expected run is present, none duplicated, none partial, none
|
||||
unexpected.
|
||||
|
||||
| seed | runs present | metadata fields | manifest reload | smoke | missing artifacts |
|
||||
|---|---|---|---|---|---|
|
||||
| 0 | original + beta {0,.25,.5,.75,1} | all correct | true | false | none |
|
||||
| 1 | original + beta {0,.25,.5,.75,1} | all correct | true | false | none |
|
||||
| 2 | original + beta {0,.25,.5,.75,1} | all correct | true | false | none |
|
||||
|
||||
Verified per run: dataset=xiiotid, alpha=0.25, seed, aggregation mode, beta, clients=10,
|
||||
rounds=10, local_epochs=3, batch=256, lr=1e-3, mu=0.1, lambda=1.0, `manifest_reload_ok=true`,
|
||||
`smoke=false`; every run has all 14 expected artifacts. No run exists for any alpha ≠ 0.25
|
||||
(alpha=0.50 runs are the Session-6 condition) and no `5gnidd...support_weighted` run exists
|
||||
(only `5gnidd__alpha0.5__seed0__original_protean`).
|
||||
|
||||
---
|
||||
|
||||
## C. Pairing / manifest integrity
|
||||
|
||||
Within each seed, all six runs share one manifest content hash and the deterministic artifacts
|
||||
are byte-identical (SHA-256 computed by this audit):
|
||||
|
||||
| seed | manifest content hash | round-0 snapshot SHA-256 (all 6 runs) | support / characterization / rare_unseen |
|
||||
|---|---|---|---|
|
||||
| 0 | `839ba18e2d87245d198be8f5f89d4a24f13cdb6fc64c004d406c6d4f8246f3da` | `cd821beb8684ed762934a00b9f541bdfae5a4498f061ffe07132ad9d00ad300c` | identical |
|
||||
| 1 | `84c20b4bd6cd20010fa35d139a0416cb06e79c82fbd8e1d57441cafdcdfbf15d` | `59d2d2bc1a86a858d061d08219ee84669177ce8346e388fc8eada299e6fb5cf5` | identical |
|
||||
| 2 | `d6e68ee13cb11f98cfb4aea668f3e905d3ebc94884632352c0045fc5f86df28c` | `47befefff4cfa84250d1b6783fe00ce98dcea83b0649efe9745fa70bd4f3fa33` | identical |
|
||||
|
||||
The stored `train_support_Nij.csv` equals the manifest support matrix, and each manifest was
|
||||
independently reproduced from the raw dataset (`prepare_dataset` →
|
||||
`reproduce_train_assignments`): all three seeds return `reproduce_ok=True`, `support_match=True`,
|
||||
`split_hash_ok=True`, `partition_hash_ok=True`, `nij_sum = 569,909`, 10 clients, split sizes
|
||||
569,909 / 81,414 / 162,831 for both alpha=0.25 and alpha=0.50. `history.json` and
|
||||
`test_global_metrics.json` differ across runs of a seed, exactly as expected because beta
|
||||
changes training. Pairing is established by content hashes, not filenames.
|
||||
|
||||
---
|
||||
|
||||
## D. Partition characterization audit (realized)
|
||||
|
||||
Independently recomputed from `artifacts/partitions/xiiotid__alpha{0.25,0.5}__seed{0,1,2}.json`
|
||||
(`support` matrix, training split). `M = 10` clients; `|S_j| < M` means fewer than 10 clients
|
||||
contribute a class prototype; "missing cells" counts `(client, class)` pairs with `N_ij = 0`
|
||||
over attack+benign classes.
|
||||
|
||||
| statistic | alpha=0.25 seed0 / seed1 / seed2 | alpha=0.50 seed0 / seed1 / seed2 |
|
||||
|---|---|---|
|
||||
| class-condition cells with `|S_j| < M` | 6 / 7 / 6 (sum **19/27**) | 3 / 3 / 1 (sum **7/27**) |
|
||||
| missing client-class cells | 13 / 10 / 15 (sum **38**) | 4 / 4 / 1 (sum **9**) |
|
||||
| client-size max/min | 15.58 / 20.84 / 7.82 | 5.11 / 7.22 / 9.89 |
|
||||
| union unseen classes | 6 / 6 / 5 | 3 / 3 / 1 |
|
||||
| max unseen on one client | 3 / 4 / 3 | 2 / 2 / 1 |
|
||||
| union rare classes | 8 / 7 / 8 | 6 / 7 / 7 |
|
||||
| rare classes per client | 2 × 10 | 2 × 10 |
|
||||
|
||||
These match the report claims exactly: alpha=0.25 = **19/27 cells**, **38 missing cells**,
|
||||
client-size ratios **15.6 / 20.8 / 7.8**, unseen union/count **6/6/5** (max 4). The comparison
|
||||
against alpha=0.50 is confirmed (7/27, 9, 5.1/7.2/9.9, 3/3/1, max 2). Per-class `N_ij`, CV and
|
||||
max/min recomputed from the manifests equal `characterization.csv` for every seed and class.
|
||||
Interpretation uses these realized statistics, not alpha alone; note the non-uniform case
|
||||
(alpha=0.25 seed 2 ratio 7.82 < alpha=0.50 seed 2 ratio 9.89).
|
||||
|
||||
---
|
||||
|
||||
## E. Validation metric recomputation
|
||||
|
||||
Recomputed from raw artifacts (no report numbers copied):
|
||||
|
||||
- seed-level primary score = mean over eligible clients of `val_per_client.csv`
|
||||
`rare_class_macro_f1`; **equals `val_selection.json` per-client-mean to ≤ 1.1e-16**;
|
||||
- global validation accuracy / macro-F1 / weighted-F1 / rare macro-F1 / rare recall
|
||||
recomputed **independently from `val_confusion_matrix.json`** using the production formulas;
|
||||
**equal to `val_global_metrics.json` exactly (diff 0.0)**.
|
||||
|
||||
| seed | beta | eligible | recomputed seed mean | report claim |
|
||||
|---|---|---|---|---|
|
||||
| 0 | 0.00 / 0.25 / 0.50 / 0.75 / 1.00 | 10 each | 0.616380 / 0.609913 / 0.595247 / 0.654033 / 0.578385 | same |
|
||||
| 1 | 0.00 / 0.25 / 0.50 / 0.75 / 1.00 | 10 each | 0.492425 / 0.514237 / 0.537040 / 0.535672 / 0.530942 | same |
|
||||
| 2 | 0.00 / 0.25 / 0.50 / 0.75 / 1.00 | 10 each | 0.666307 / 0.641922 / 0.625267 / 0.548685 / 0.486970 | same |
|
||||
|
||||
All recomputed values agree with the report to six-decimal formatting precision; no mismatch
|
||||
beyond precision was found. Per-seed diagnostic argmax reproduces **0.75 / 0.50 / 0.00**.
|
||||
|
||||
Limitation (unavoidable, documented): the *per-client predictions* cannot be recomputed because
|
||||
the trained model is not persisted and retraining is forbidden by the audit. The per-client
|
||||
metric is verified via (a) its defining source code
|
||||
(`evaluation/metrics.py::rare_class_metrics` over each client's training-derived rare ids),
|
||||
(b) exact equality of its mean with `val_selection.json`, and (c) exact equality of the global
|
||||
union-rare metric recomputed from the confusion matrix. This does not affect `beta_star`, which
|
||||
depends only on the seed-level means.
|
||||
|
||||
---
|
||||
|
||||
## F. Rare / unseen integrity
|
||||
|
||||
Per-client rare classes recomputed from training `N_ij` (two smallest positive-support attack
|
||||
classes, tie by class id) and compared to `rare_unseen.csv` flags across **all 18 runs**:
|
||||
|
||||
- **0 mismatches** (flag mismatches counted over every `(seed, mode, beta, client, class)` cell).
|
||||
- Benign class (`Normal`) never flagged rare/unseen; rare and unseen never overlap.
|
||||
- Unseen = attack class with `N_ij = 0`; union of training-derived rare classes equals
|
||||
`val_global_metrics.evaluated_rare_classes` **and**
|
||||
`test_global_metrics.evaluated_rare_classes` for every run — i.e. validation/test labels did
|
||||
**not** influence rare-class identity.
|
||||
- Eligible clients = **10 in every seed and run**; every client has exactly 2 rare classes; no
|
||||
client has only 1 attack class.
|
||||
- Union rare classes: seed 0 = `[0,1,2,3,5,6,7,8]`, seed 1 = `[0,1,2,3,6,7,8]`,
|
||||
seed 2 = `[0,1,2,3,5,6,7,8]` (class ids; `Normal` is benign).
|
||||
|
||||
---
|
||||
|
||||
## G. Pooled-selection recomputation
|
||||
|
||||
Computed from the independently recomputed seed-level scores (not from the report). Pooled
|
||||
score = mean of the three seed-level per-client means; std is sample (`ddof=1`).
|
||||
|
||||
| beta | pooled mean | sample std | pop std | min | max |
|
||||
|---|---|---|---|---|---|
|
||||
| **0.00** | **0.591703815** | **0.089528837** | 0.073099989 | 0.492424736 | 0.666306547 |
|
||||
| 0.25 | 0.588690626 | 0.066435256 | 0.054244160 | 0.514237103 | 0.641922010 |
|
||||
| 0.50 | 0.585851480 | 0.044857850 | 0.036626281 | 0.537039860 | 0.625267154 |
|
||||
| 0.75 | 0.579463399 | 0.064906057 | 0.052995574 | 0.535672367 | 0.654032964 |
|
||||
| 1.00 | 0.532098701 | 0.045718339 | 0.037328867 | 0.486969813 | 0.578384529 |
|
||||
|
||||
- Recompute **matches the report** (mean and sample std) to ≤ 4e-7 (formatting precision).
|
||||
- `argmax` of the pooled validation score = **0.00** (no tie; `tie_break_used=false`).
|
||||
- The pooled validation curve is **numerically monotonically decreasing** in beta:
|
||||
0.591704 > 0.588691 > 0.585851 > 0.579463 > 0.532099.
|
||||
- The secondary pooled validation rare-class recall peaks at beta=0.25, but recall is only a
|
||||
tie-break and is not reached; the primary metric governs.
|
||||
- **Std convention:** pooled std is sample (`ddof=1`), explicitly labelled `std (sample)` in
|
||||
the report; `std_pop` is also emitted. Consistent with Session 6; no convention changed.
|
||||
|
||||
---
|
||||
|
||||
## H. Validation / test leakage audit
|
||||
|
||||
- `run_baseline.py`: `union_rare` is derived from the **training** support matrix only
|
||||
(lines 189–190). The `val_selection` payload (lines 297–313) is built **only** from
|
||||
`val_rare`, `val_metrics`, `val_per_client_rows`. Test variables appear in a separate block
|
||||
and feed only test artifacts. `test_used_for_selection: false` and
|
||||
`selection_split: "validation"` are recorded for all 15 `support_weighted` runs.
|
||||
- `analyze_multiseed_beta.py`: the PRIMARY selection block (lines 176–208) consumes only
|
||||
`per_seed_rows`, which are built from `val_selection.json`. Tie-break uses pooled validation
|
||||
recall then validation std then smaller beta. `test_rows` are read in a later block (line 348+)
|
||||
and are never referenced by the selection logic or tie-break.
|
||||
- No code path sorts/filters candidates by a test field.
|
||||
|
||||
Conclusion: selection is **programmatic and validation-only**; the boolean flag is corroborated
|
||||
by the independent recomputation of `beta_star` from validation artifacts alone.
|
||||
`test_used_for_selection = false` is upheld.
|
||||
|
||||
---
|
||||
|
||||
## I. Final test recomputation (selected beta and original)
|
||||
|
||||
`beta_star = 0.00` was fixed from pooled validation; only then were test metrics read.
|
||||
Test metrics recomputed independently from `test_confusion_matrix.json` (diff vs the JSON = 0.0),
|
||||
then averaged across seeds with sample std:
|
||||
|
||||
| metric | recomputed mean ± std | report claim |
|
||||
|---|---|---|
|
||||
| rare-class macro-F1 | 0.632136 ± 0.047642 | 0.632136 ± 0.047642 |
|
||||
| rare-class recall | 0.821257 ± 0.012283 | 0.821257 ± 0.012283 |
|
||||
| macro-F1 | 0.681343 ± 0.020555 | 0.681343 ± 0.020555 |
|
||||
| accuracy | 0.874015 ± 0.010267 | 0.874015 ± 0.010267 |
|
||||
| weighted-F1 | 0.906882 ± 0.007408 | 0.906882 ± 0.007408 |
|
||||
|
||||
`original_protean` recomputed identically: rare macro-F1 0.632790 ± 0.047152; rare recall
|
||||
0.821965 ± 0.010811; macro-F1 0.682018 ± 0.020967; accuracy 0.875431 ± 0.009067;
|
||||
weighted-F1 0.907652 ± 0.006934 — all match the report. No superiority claim is made.
|
||||
|
||||
---
|
||||
|
||||
## J. beta = 0 equivalence audit
|
||||
|
||||
For seeds 0, 1 and 2, `original_protean` vs `support_weighted beta=0`:
|
||||
|
||||
| property | seed 0 | seed 1 | seed 2 |
|
||||
|---|---|---|---|
|
||||
| same manifest / split / hashes | yes | yes | yes |
|
||||
| round-0 snapshot byte-identical | yes (`cd821beb…`) | yes (`59d2d2bc…`) | yes (`47befeff…`) |
|
||||
| support/characterization/rare_unseen identical | yes | yes | yes |
|
||||
| independent aggregation diff (`ΣC/n` vs `(1/n)ΣC`) | 4.4e-16 | 4.4e-16 | 2.2e-16 |
|
||||
| history rare-macro-F1 round-0 diff | 0.0 | 0.0 | 0.0 |
|
||||
| history rare-macro-F1 max diff (rounds ≥ 1) | 6.6e-3 | 3.6e-3 | 3.7e-3 |
|
||||
| test metric max diff | 1.6e-3 | 3.9e-3 | 2.2e-3 |
|
||||
| classification | **B** | **B** | **B** |
|
||||
|
||||
Class **B** (semantically equivalent + floating-point numerical divergence); **no semantic
|
||||
discrepancy** in any seed. Round 0 is byte-identical; the ~1 ULP ordering difference enters the
|
||||
round-1 prototype-alignment training and trajectories diverge slowly. Equivalence remains
|
||||
conditional on the shared `S_j` reproduction assumption and is **not** proof that `beta = 0`
|
||||
reproduces original PROTEAN. Machine-readable:
|
||||
`artifacts/analysis/xiiotid__alpha0.25__multiseed__beta0_equivalence.csv`.
|
||||
|
||||
---
|
||||
|
||||
## K. Direct-effect audit
|
||||
|
||||
Recomputed independently from the fixed round-0 snapshots using the production aggregation
|
||||
(`direct_prototype.prototype_effect_rows`). `||Cbar_j(1) − Cbar_j(0)||`, cross-seed mean:
|
||||
|
||||
| class | alpha=0.25 mean (std) | alpha=0.50 mean (std) |
|
||||
|---|---|---|
|
||||
| Exfiltration | 4.316 (0.898) | 2.020 (0.502) |
|
||||
| Tampering | 4.248 (0.241) | 2.354 (1.165) |
|
||||
| C&C | 4.020 (0.290) | 1.694 (0.505) |
|
||||
| Lateral_movement | 3.726 (1.383) | 2.724 (1.035) |
|
||||
| Exploitation | 3.464 (1.113) | 2.344 (0.412) |
|
||||
| Reconnaissance | 3.339 (1.107) | 1.002 (0.261) |
|
||||
| Weaponization | 3.124 (0.585) | 2.561 (1.228) |
|
||||
| RDOS | 2.566 (1.219) | 2.631 (1.165) |
|
||||
| Normal | 1.997 (0.533) | 1.557 (0.548) |
|
||||
|
||||
- Mean across classes: alpha=0.25 **3.422**, alpha=0.50 **2.099** (report: 3.42 / 2.10).
|
||||
- alpha=0.25 largest classes: Exfiltration 4.316, Tampering 4.248, C&C 4.020; smallest:
|
||||
Normal 1.997, RDOS 2.566 — all match the report.
|
||||
- Per-`(class, beta)` displacement and normalised weights recomputed from snapshots match the
|
||||
per-seed `...__prototype_effect.csv` and pooled `...__prototype_effect_pooled.csv` **exactly**.
|
||||
- Displacement is a **direct-aggregation measure only**; no detection causality is inferred.
|
||||
The end-to-end effect is reported separately and is not used to interpret displacement.
|
||||
|
||||
---
|
||||
|
||||
## L. Client-level audit
|
||||
|
||||
Recomputed from `val_per_client.csv` deltas vs `beta = 0` (independent of the summary CSV):
|
||||
|
||||
| seed | beta=0.75 improve/degrade/unchanged | mean Δ | min Δ | max Δ |
|
||||
|---|---|---|---|---|
|
||||
| 0 | **5 / 4 / 1** | +0.0377 | −0.1117 | +0.3759 |
|
||||
| 1 | **5 / 4 / 1** | +0.0432 | −0.0183 | +0.3167 |
|
||||
| 2 | **3 / 5 / 2** | −0.1176 | −0.6011 | +0.1241 |
|
||||
|
||||
- At beta=1.00 seed 2 degrades 6/10 clients with a worst client Δ = **−0.6775** (≈ −0.678),
|
||||
while seed 1 still gains on average — all confirmed.
|
||||
- `client_validation_detail.csv` deltas match recomputation with max abs diff **0.0**.
|
||||
- `min positive attack N_ij` vs ΔF1 is confirmed **non-monotonic** at beta=0.75 (e.g. seed 2:
|
||||
N=1 → −0.601, N=2 → 0.000, N=4 → +0.124; seed 0: N=2 → −0.029 and +0.002). Client ids are
|
||||
per-seed realisations and are not pooled across seeds.
|
||||
|
||||
---
|
||||
|
||||
## M. Cross-alpha audit (critical)
|
||||
|
||||
Use only audited condition-level pooled results.
|
||||
|
||||
| | alpha=0.50 | alpha=0.25 |
|
||||
|---|---|---|
|
||||
| **pooled-selected beta** | **0.75** | **0.00** |
|
||||
| pooled validation curve | rises to a peak at 0.75 | numerically monotone decreasing |
|
||||
| pooled mean range | 0.544361–0.579367 (0.035) | 0.532099–0.591704 (0.060) |
|
||||
| pooled sample std range | 0.075121–0.100450 | 0.044858–0.089529 |
|
||||
| `|S_j| < M` cells | 7/27 | 19/27 |
|
||||
| missing client-class cells | 9 | 38 |
|
||||
| union unseen per seed | 3/3/1 (max 2) | 6/6/5 (max 4) |
|
||||
| mean `||Cbar(1)−Cbar(0)||` | 2.099 | 3.422 |
|
||||
| beta=0.75 client pattern | majority improve every seed | seed-dependent, often negative |
|
||||
|
||||
Alpha=0.50 pooled selection was independently reproduced from Session-6 artifacts
|
||||
(`results/xiiotid__alpha0.5__seed{0,1,2}__support_weighted__beta{0,.25,.5,.75,1}`):
|
||||
pooled means 0.544361 / 0.555434 / 0.567322 / **0.579367** / 0.573479; `argmax = 0.75`.
|
||||
|
||||
**Reporting-confusion check.** The audit searched the entire repository (docs, research notes,
|
||||
handoff) for any statement of the form "selected beta changes (0.50 -> 0.00)" or equivalent
|
||||
conflating the two conditions. **No such statement exists in the repository.** Every written
|
||||
occurrence is correct: alpha=0.50 pooled `beta_star = 0.75`; alpha=0.25 pooled
|
||||
`beta_star = 0.00`. The value `0.50` appears only as (a) the Session-5 **seed-0 single-condition**
|
||||
selection for alpha=0.50 and (b) the Session-6 **per-seed diagnostic** for alpha=0.50 seed 0.
|
||||
It is **not** the condition-level pooled selection for alpha=0.50. This disambiguation has been
|
||||
added explicitly to the report (§L) and handoff.
|
||||
|
||||
These are descriptive associations; the realized partition characteristics (not alpha alone) are
|
||||
reported alongside, and **no causal claim is made that alpha by itself caused the difference**.
|
||||
The beta effect remains small relative to between-seed variance at both settings.
|
||||
|
||||
---
|
||||
|
||||
## N. Artifact provenance
|
||||
|
||||
Every reported quantity traces to a machine-readable artifact or a deterministic recomputation.
|
||||
|
||||
| Reported quantity | Source artifact(s) | Derivation | Verified |
|
||||
|---|---|---|---|
|
||||
| run existence / config | `results/<run>/metadata.json` | direct read | yes |
|
||||
| manifest content/partition/split hashes | `artifacts/partitions/xiiotid__alpha0.25__seed{0,1,2}.json` | module hash + `reproduce_train_assignments` | yes |
|
||||
| pairing proof | `artifacts/analysis/...__prototype_snapshot.json` | sha256 across 6 runs/seed | yes |
|
||||
| seed-level validation score | `results/.../val_per_client.csv` + `val_selection.json` | mean of per-client rare macro-F1 | yes |
|
||||
| validation global metrics | `results/.../val_confusion_matrix.json` | independent numpy recompute | yes |
|
||||
| pooled mean/std, `beta_star` | recomputed seed scores | mean/`stdev` over seeds, argmax | yes |
|
||||
| test metrics (beta 0, original) | `results/.../test_confusion_matrix.json` | independent recompute + mean/std | yes |
|
||||
| test descriptive (all candidates) | `artifacts/analysis/...__test_descriptive.csv` | direct read | yes |
|
||||
| beta=0 equivalence | original vs beta0 artifacts/snapshots | sha256 + independent aggregation + history diff | yes |
|
||||
| direct-effect displacement | snapshots + `...__prototype_effect_pooled.csv` | independent `direct_prototype` recompute | yes |
|
||||
| client improve/degrade | `results/.../val_per_client.csv` + `...__client_validation_{summary,detail}.csv` | deltas vs beta0 | yes |
|
||||
| rare-class identity | `artifacts/analysis/...__rare_unseen.csv` + manifest | recompute definition | yes |
|
||||
| partition characterization | manifests + `results/.../characterization.csv` | independent numpy recompute | yes |
|
||||
| runtime / RSS | `artifacts/analysis/xiiotid__alpha0.25__multiseed__runtime.csv` (source `/tmp/opencode/s7_logs/*.time`, present) | parse logs; sum = 43,799.03 s = 12.166 h | yes |
|
||||
|
||||
No scientific number was found to exist only in Markdown without a machine-readable or
|
||||
deterministic source.
|
||||
|
||||
---
|
||||
|
||||
## O. Deviations / corrections found
|
||||
|
||||
1. **Cross-alpha label disambiguation (documentation-only).** The reported external phrasing
|
||||
"Selected beta changes (0.50 -> 0.00)" is **not present anywhere in the repository**; the
|
||||
repo already states alpha=0.50 pooled = 0.75 and alpha=0.25 pooled = 0.00. An explicit
|
||||
disambiguation note was added to `docs/research/ALPHA025_BETA_VALIDATION.md` §L and
|
||||
`docs/handoff/SESSION_HANDOFF.md` §11 so the Session-5 seed-0 `0.50` and Session-6 pooled
|
||||
`0.75` cannot be conflated. No scientific value changed.
|
||||
2. **Wording nuance (clarified).** Handoff §14 item 6 ("the beta direction may reverse under
|
||||
stronger heterogeneity … monotone in heterogeneity") is a speculation, not a settled trend;
|
||||
reworded to a neutral "one more condition; not yet a trend across alpha" framing.
|
||||
3. **No arithmetic, metric-label, std-convention, or provenance defect was found** in the
|
||||
Session 7 report or analysis artifacts. No derived CSV required correction.
|
||||
|
||||
No training outputs, aggregation, model, manifests, or scientific protocol were altered.
|
||||
|
||||
---
|
||||
|
||||
## P. Final audit verdict
|
||||
|
||||
**PASS WITH MINOR CORRECTIONS.**
|
||||
|
||||
- Run inventory: **PASS** (18/18, no duplicates/partials; no alpha=0.75 or 5G-NIDD run).
|
||||
- Pairing / manifest reproduction: **PASS** (hash-identical; raw-data reproduction exact).
|
||||
- Partition characterization: **PASS** (19/27, 38, 15.6/20.8/7.8, 6/6/5; alpha=0.50 7/27, 9).
|
||||
- Validation recomputation (seed-level and global): **PASS** (exact).
|
||||
- Rare/unseen integrity: **PASS** (0 mismatches; training-derived; no val/test influence).
|
||||
- Pooled selection: **PASS**; independent `beta_star = 0.00` using validation only.
|
||||
- Validation/test leakage: **PASS**; selection reads validation only and is reproducible.
|
||||
- Test summaries (selected beta and `original_protean`): **PASS** (exact).
|
||||
- beta=0 equivalence: **PASS**, class **B** (semantically equivalent + float divergence).
|
||||
- Direct-effect and client-level analyses: **PASS** (independent recompute exact).
|
||||
- Cross-alpha comparison: **PASS**; alpha=0.50 pooled = 0.75, alpha=0.25 pooled = 0.00.
|
||||
- Source/data integrity, pytest (75 passed, 0 failed, 0 skipped, 0 warnings): **PASS**.
|
||||
|
||||
The corrections are documentation/wording only and do not affect `beta_star` or the scientific
|
||||
interpretation. Session 7 is **safe to checkpoint** with the documented limitations (single
|
||||
dataset, two alphas, three project-defined seeds, weak effect relative to seed variance, `S_j`
|
||||
reproduction assumption, no global-optimum claim).
|
||||
Reference in new issue
Block a user