加入实验输出（批次 13/14）
This commit is contained in:
@@ -0,0 +1,5 @@
|
||||
# TSFA five-fold result bundle
|
||||
|
||||
Seven frozen-alignment comparisons on 100 held-out clips from 37 video IDs. All Python runs use the project uv environment. Read `run_manifest.json` for the complete protocol and limitations. `paired_contrasts.csv` reports left minus right with video-ID-cluster bootstrap confidence intervals. The full `outputs/tsfa/` directory retains per-clip CSVs and probe checkpoints.
|
||||
|
||||
The `alignment_only_*` summaries pool identical train-fold-standardized raw source features through each method's alignment matrix. They isolate source selection from native model value/output projections. `tsfa_temporal_diagnostics_summary.csv` reports MVR, span, entropy, and source coverage.
|
||||
@@ -0,0 +1,8 @@
|
||||
method,clip_count,video_id_count,content_auc_mean_video_macro_mean,content_auc_mean_ci95_low,content_auc_mean_ci95_high,canonical_pairwise_time_mae_mean_video_macro_mean,canonical_pairwise_time_mae_mean_ci95_low,canonical_pairwise_time_mae_mean_ci95_high,temporal_quality_1_minus_mae_video_macro_mean,temporal_quality_1_minus_mae_ci95_low,temporal_quality_1_minus_mae_ci95_high
|
||||
M3_noSourceTime,100,37,0.7079247666486141,0.6981638469348909,0.7180669733515811,0.25125551939830126,0.24533393078354204,0.25730599312459385,0.7487444806016986,0.7429385193325527,0.7543707643381519
|
||||
M3_sourceTime,100,37,0.5646895233632823,0.5540910611345894,0.5751509210736825,0.09874174948688212,0.08898600889351087,0.10906350604411601,0.9012582505131179,0.8909351143570293,0.9111176736631803
|
||||
M4_sourceTime,100,37,0.5443730644271425,0.5314674094814167,0.556163600543299,0.06439514300291252,0.05872951511472842,0.069915719671434,0.9356048569970875,0.929956941099165,0.9411986290102992
|
||||
TSFA-main,100,37,0.6265827216844504,0.6161616236415937,0.6364439680609562,0.0703286687392714,0.06460036690688006,0.07572429366463695,0.9296713312607287,0.9244517958993883,0.9352479536843178
|
||||
TSFA-multiply,100,37,0.6266223446090468,0.6163471959183728,0.6367020258052505,0.06779188350752184,0.06251919458961656,0.07342671925523597,0.932208116492478,0.9266964480620523,0.9376652354766101
|
||||
TSFA-random,100,37,0.6317354999281196,0.6245134023121242,0.6386649427182931,0.25069943769545594,0.2462397150003418,0.2551459569730119,0.7493005623045442,0.7451492641632237,0.7535130139973931
|
||||
TSFA-global,100,37,0.6850667176168063,0.6715353281663122,0.6991479845315172,0.2432284738286704,0.23726117654439974,0.24953340198194693,0.7567715261713297,0.7508184770882113,0.7626622731375607
|
||||
|
@@ -0,0 +1,8 @@
|
||||
method,clip_count,video_id_count,content_auc_mean_video_macro_mean,content_auc_mean_ci95_low,content_auc_mean_ci95_high,canonical_pairwise_time_mae_mean_video_macro_mean,canonical_pairwise_time_mae_mean_ci95_low,canonical_pairwise_time_mae_mean_ci95_high,temporal_quality_1_minus_mae_video_macro_mean,temporal_quality_1_minus_mae_ci95_low,temporal_quality_1_minus_mae_ci95_high
|
||||
M3_noSourceTime,100,37,0.5915240976792394,0.5846003467230528,0.5986003527463633,0.25125551939830126,0.24526752948109784,0.2569629667175068,0.7487444806016986,0.7426625677892807,0.7546333254224017
|
||||
M3_sourceTime,100,37,0.5159307443208153,0.5093207601202393,0.5227730292887872,0.09874174948688212,0.08938651093211228,0.10879260214913458,0.9012582505131179,0.890805973592594,0.9110025699716944
|
||||
M4_sourceTime,100,37,0.5172970388067906,0.5080581473982847,0.5275570833339672,0.06439514300291252,0.0587335791039206,0.06983744635995676,0.9356048569970875,0.9296541211962468,0.9408869116126053
|
||||
TSFA-main,100,37,0.5527356591964304,0.5398139072416833,0.5666946734419639,0.0703286687392714,0.06519967303243404,0.07598739113740521,0.9296713312607287,0.9244119213095845,0.9350019389370183
|
||||
TSFA-multiply,100,37,0.5506013033677041,0.5379239892152792,0.5636824578678856,0.06779188350752184,0.06247147901094175,0.07313585563909683,0.932208116492478,0.926778081557954,0.9373936058949076
|
||||
TSFA-random,100,37,0.5711638231839362,0.562858188378229,0.5801847256875272,0.25069943769545594,0.2462381631727499,0.25519553085514896,0.7493005623045442,0.7450096911264995,0.753607390700797
|
||||
TSFA-global,100,37,0.6077635519257857,0.5944803976516411,0.6218732946519847,0.2432284738286704,0.2374573657121255,0.2489871789332721,0.7567715261713297,0.7506538401520033,0.7624564267860716
|
||||
|
@@ -0,0 +1,22 @@
|
||||
method,pair,clip_count,draw_count,same_time_similarity_video_macro_mean,same_time_similarity_ci95_low,same_time_similarity_ci95_high,video_id_count,shifted_far_similarity_video_macro_mean,shifted_far_similarity_ci95_low,shifted_far_similarity_ci95_high,same_minus_shifted_margin_video_macro_mean,same_minus_shifted_margin_ci95_low,same_minus_shifted_margin_ci95_high,matched_vs_shifted_auc_video_macro_mean,matched_vs_shifted_auc_ci95_low,matched_vs_shifted_auc_ci95_high,exact_r1_audio_to_vision_video_macro_mean,exact_r1_audio_to_vision_ci95_low,exact_r1_audio_to_vision_ci95_high,within_pm1_r1_audio_to_vision_video_macro_mean,within_pm1_r1_audio_to_vision_ci95_low,within_pm1_r1_audio_to_vision_ci95_high,mase_slots_audio_to_vision_video_macro_mean,mase_slots_audio_to_vision_ci95_low,mase_slots_audio_to_vision_ci95_high,exact_r1_vision_to_audio_video_macro_mean,exact_r1_vision_to_audio_ci95_low,exact_r1_vision_to_audio_ci95_high,within_pm1_r1_vision_to_audio_video_macro_mean,within_pm1_r1_vision_to_audio_ci95_low,within_pm1_r1_vision_to_audio_ci95_high,mase_slots_vision_to_audio_video_macro_mean,mase_slots_vision_to_audio_ci95_low,mase_slots_vision_to_audio_ci95_high,exact_r1_text_to_audio_video_macro_mean,exact_r1_text_to_audio_ci95_low,exact_r1_text_to_audio_ci95_high,within_pm1_r1_text_to_audio_video_macro_mean,within_pm1_r1_text_to_audio_ci95_low,within_pm1_r1_text_to_audio_ci95_high,mase_slots_text_to_audio_video_macro_mean,mase_slots_text_to_audio_ci95_low,mase_slots_text_to_audio_ci95_high,exact_r1_audio_to_text_video_macro_mean,exact_r1_audio_to_text_ci95_low,exact_r1_audio_to_text_ci95_high,within_pm1_r1_audio_to_text_video_macro_mean,within_pm1_r1_audio_to_text_ci95_low,within_pm1_r1_audio_to_text_ci95_high,mase_slots_audio_to_text_video_macro_mean,mase_slots_audio_to_text_ci95_low,mase_slots_audio_to_text_ci95_high,exact_r1_text_to_vision_video_macro_mean,exact_r1_text_to_vision_ci95_low,exact_r1_text_to_vision_ci95_high,within_pm1_r1_text_to_vision_video_macro_mean,within_pm1_r1_text_to_vision_ci95_low,within_pm1_r1_text_to_vision_ci95_high,mase_slots_text_to_vision_video_macro_mean,mase_slots_text_to_vision_ci95_low,mase_slots_text_to_vision_ci95_high,exact_r1_vision_to_text_video_macro_mean,exact_r1_vision_to_text_ci95_low,exact_r1_vision_to_text_ci95_high,within_pm1_r1_vision_to_text_video_macro_mean,within_pm1_r1_vision_to_text_ci95_low,within_pm1_r1_vision_to_text_ci95_high,mase_slots_vision_to_text_video_macro_mean,mase_slots_vision_to_text_ci95_low,mase_slots_vision_to_text_ci95_high
|
||||
M3_noSourceTime,audio_vision,100,1.0,0.02983147300783216,-0.00780859958706209,0.06532931063779485,37,0.026275558042943677,-0.0135765906084752,0.06442367880532844,0.0035559149673156645,0.0011573716778813968,0.0058081254094310905,0.514271623637315,0.5086281381400399,0.5205522070322601,0.02223723723723724,0.019318307593307594,0.0252463963963964,0.0629030459030459,0.056346739596739595,0.0701120549120549,16.06521836121836,15.360586422136418,16.831551512226515,0.02093822393822394,0.01916058558558559,0.022803474903474903,0.0588974688974689,0.05515479407979409,0.06256855426855427,17.3258314028314,16.454282078507074,18.303939736164736,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_noSourceTime,text_audio,100,1.0,0.20791600453897077,0.18457893455935243,0.23068674583726764,37,0.0660212989394217,0.04607872245037598,0.08693206487027433,0.14189470507476618,0.1258566735337873,0.158192415668532,0.7246991238959324,0.7032607291086682,0.7463028065299342,,,,,,,,,,,,,,,,,,,0.13955383955383954,0.12312348777348776,0.156250858000858,0.27616302016302025,0.24729871299871298,0.30469959244959244,10.11119390819391,9.083786154011154,11.220066763191763,0.07386400686400688,0.06408464178464178,0.08439125911625915,0.16222651222651222,0.1441163770913771,0.18078545688545689,12.965773916773918,12.10177589017589,13.855149796224795,,,,,,,,,,,,,,,,,,
|
||||
M3_noSourceTime,text_vision,100,1.0,0.032579638831090094,0.012203289357956712,0.053855902535509696,37,0.020862086972797865,-0.0005364448539057191,0.04502733759713527,0.01171755187282541,0.007967536814828653,0.015164253928682914,0.535601545504471,0.5297980337765578,0.5417672611678928,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.04206177606177607,0.03788120978120979,0.04638886743886744,0.10588416988416989,0.09782853925353927,0.113992277992278,14.162812097812097,13.519815894465893,14.831819369369372,0.023044616044616047,0.02111688116688117,0.025180180180180183,0.06238695838695839,0.05710733590733591,0.06806795366795368,17.50512955812956,16.450267953667954,18.578393522093524
|
||||
M3_sourceTime,audio_vision,100,1.0,0.007696060061626377,-0.029554258849169186,0.04545826630162894,37,0.008636827414143384,-0.030810690011533682,0.04782087975048893,-0.0009407673630797903,-0.003493066123958016,0.0017008874928592713,0.5026440831028598,0.49047663000196245,0.5166576717342342,0.021732303732303733,0.018566891891891892,0.02489736164736165,0.05924495924495925,0.05102570785070786,0.06822308022308023,18.418811239811237,17.145313781638777,19.668025257400256,0.01908751608751609,0.017070527670527674,0.021123584298584297,0.05492621192621192,0.05029851994851995,0.05956598026598027,18.342253110253115,17.42752445302445,19.27038470613471,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_sourceTime,text_audio,100,1.0,0.030779919639574976,0.009495005332440814,0.051172133170129974,37,0.012129069987308123,-0.008134753400298974,0.03220063209724045,0.018650849718729958,0.01209877043793774,0.025570886226139,0.5453406619005025,0.5310778723489828,0.5597143654149971,,,,,,,,,,,,,,,,,,,0.03846546546546547,0.03189687902187902,0.04500438652938652,0.0974937794937795,0.08529543114543116,0.1109557915057915,16.05845045045045,15.243514607464608,16.823768436293438,0.026703560703560704,0.022845398970398972,0.030940111540111538,0.07566495066495067,0.06590900900900902,0.08547635135135136,15.480417417417419,14.34030947018447,16.63520718575719,,,,,,,,,,,,,,,,,,
|
||||
M3_sourceTime,text_vision,100,1.0,0.0147220734507245,-0.008115254084484056,0.03707853268159794,37,0.014395209505575931,-0.008901017726582194,0.03943286969561065,0.0003268639452983987,-0.002732993650720293,0.0035072161571967925,0.4998074879590837,0.4950654246809501,0.504675911331164,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.020641784641784643,0.017091269841269845,0.024615486915486918,0.05474774774774775,0.04679173101673103,0.0628289897039897,19.39496525096525,18.298798712998718,20.349694851994855,0.020431574431574433,0.01848766623766624,0.02242575075075075,0.05971514371514372,0.05510990990990991,0.06392915057915058,16.647335478335478,15.535322372372372,17.931235296010293
|
||||
M4_sourceTime,audio_vision,100,1.0,0.017589288202760855,-0.021384537387894743,0.05702271457169135,37,0.018760167504660787,-0.01933644305751479,0.05750124747004294,-0.0011708792981242977,-0.004425662602822854,0.0016943904201328112,0.5092526072805329,0.49760200300984875,0.5208348244378497,0.018691977691977692,0.014877273702273703,0.02274350064350065,0.04963020163020163,0.041014339339339345,0.05834750107250107,18.292485628485622,16.987942664092664,19.485725182325183,0.02013041613041613,0.017474710424710424,0.022690025740025738,0.05348219648219649,0.04821073573573574,0.05895694980694981,18.10186143286143,16.83444991419991,19.40507165379665,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M4_sourceTime,text_audio,100,1.0,0.02029622882045845,-0.015886606029959244,0.052645537742046604,37,0.004637936173545868,-0.02838808700016482,0.03639667112261427,0.01565829274056424,0.009190044274264193,0.022804289586498057,0.5389472080363038,0.514210176409312,0.5634661651963513,,,,,,,,,,,,,,,,,,,0.027861861861861865,0.02250713213213213,0.033694744744744744,0.06838953238953241,0.05814036894036893,0.07883789146289145,18.20780223080223,16.355241891891897,20.29683777348777,0.030084513084513088,0.02537934362934363,0.03572420634920635,0.07645002145002144,0.06447318747318749,0.09150033247533246,17.343640926640923,16.01122001287001,18.94879967824968,,,,,,,,,,,,,,,,,,
|
||||
M4_sourceTime,text_vision,100,1.0,0.033176119103082845,-0.0041519938906857895,0.07124988025892566,37,0.030836823717937716,-0.008539991982258287,0.0710771330847214,0.0023392953867183023,-0.001506538955824979,0.006041902250845819,0.5036913011035352,0.4923280641013353,0.5162466670602972,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.018344916344916345,0.015335810810810813,0.021540604890604893,0.05032947232947234,0.042268693693693696,0.05863465250965252,18.6358845988846,17.37207688760189,20.017716023166024,0.020106392106392104,0.017946728871728875,0.022438331188331193,0.051333762333762335,0.045313084513084516,0.057213953238953244,19.317567996567995,17.859128196053195,20.795619787644785
|
||||
TSFA-global,audio_vision,100,1.0,0.06059821270500538,0.023914981549968812,0.09491150040596706,37,0.03732938954030518,0.0038612152410352285,0.07181022922532529,0.02326882317339013,0.016046146106362763,0.031464541216613816,0.5711938543786947,0.5580918275513356,0.5851307426062079,0.05056928356928358,0.04562037752037753,0.055837269412269415,0.13590690690690693,0.12514061561561562,0.1478566066066066,13.252549978549979,12.300325911625912,14.232603314028314,0.02580952380952381,0.022693564993564997,0.02993452380952381,0.07257400257400258,0.06430289575289577,0.08383320463320464,16.92865637065637,15.890422136422137,17.992037709137712,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-global,text_audio,100,1.0,0.21948498332128252,0.18447166931635653,0.25264341574445714,37,0.13784915199225622,0.10805201589136158,0.16772503723338508,0.0816358312123013,0.0694722462373281,0.09469795330548046,0.6997278072145092,0.6772065253418312,0.7246179070170095,,,,,,,,,,,,,,,,,,,0.05226340626340627,0.043850450450450454,0.061542020592020574,0.1483732303732304,0.12583120978120982,0.1716539897039897,10.44393564993565,9.248472972972973,11.684178474903476,0.050666237666237665,0.04418186400686401,0.057607464607464604,0.13168897468897467,0.11565889103389104,0.1511234341484341,13.042119691119693,11.967449710424711,14.109006949806952,,,,,,,,,,,,,,,,,,
|
||||
TSFA-global,text_vision,100,1.0,0.03882392940236514,-0.005101331978814988,0.08023915360506113,37,0.026369558634260028,-0.019257361349713748,0.0695494351361847,0.012454370779342111,0.006579723199721584,0.018851628858371938,0.5523689941841539,0.5389573468177523,0.5678277126290424,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.036459888459888457,0.032251780351780356,0.040649946374946386,0.10280694980694982,0.09098980051480052,0.11672229729729731,13.241670098670095,12.166959716859719,14.325898702273703,0.02583869583869584,0.02256068211068211,0.030118897468897468,0.06330373230373229,0.05587844272844273,0.07113738738738737,18.464465465465462,17.013464017589012,19.91242631917632
|
||||
TSFA-main,audio_vision,100,1.0,0.013561541277063027,-0.01457795284016425,0.04262594169249218,37,0.008980510418424372,-0.021758340514473864,0.04004785757980806,0.004581030828024072,-0.0002248605515836633,0.010127997422592447,0.5114767559162506,0.5016843428677538,0.5212400272042559,0.02182110682110682,0.017747586872586874,0.02623365508365508,0.061913770913770906,0.05110411840411841,0.07329221364221364,17.73442985842986,16.826004118404118,18.77729813384814,0.022406692406692408,0.019272361647361645,0.026145913770913774,0.06514157014157014,0.056082850707850705,0.07448922136422137,15.752643071643071,14.584650504075505,16.82486615186615,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-main,text_audio,100,1.0,0.11990332463810639,0.0898675046922316,0.15305510010030474,37,0.07596570038282524,0.04989764804945704,0.10443112074815332,0.04393762438637949,0.03607569399696766,0.052024387539309866,0.6394096191556298,0.612673730897501,0.667128455450131,,,,,,,,,,,,,,,,,,,0.043396825396825406,0.03658292578292579,0.05185294937794939,0.12236121836121837,0.10395112612612614,0.14359046546546542,12.342245388245388,11.084742546117546,13.683040540540542,0.039102960102960105,0.03286315958815959,0.04523687258687259,0.10152981552981553,0.08593539253539254,0.11686853281853281,14.241285285285283,13.037042556842557,15.514015894465896,,,,,,,,,,,,,,,,,,
|
||||
TSFA-main,text_vision,100,1.0,0.04098847119260702,0.005582397132704427,0.07929293901031172,37,0.037891365960320264,0.0005784316926466229,0.07758778326306248,0.0030971052297696644,-0.000814157517568749,0.0067062238179970675,0.5073206025174111,0.49017726806879075,0.5255564948211889,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.018722436722436724,0.014964897039897044,0.022390036465036463,0.05502488202488203,0.04573553196053196,0.06428104890604891,18.6562737022737,17.207105051480053,20.233926383526384,0.020561561561561564,0.01860630630630631,0.022591741741741746,0.05686486486486487,0.05081441441441443,0.06266833976833977,18.087792363792367,16.792936625911626,19.410561314886316
|
||||
TSFA-multiply,audio_vision,100,1.0,0.00978179982461232,-0.02070967490279534,0.0421130031397444,37,0.0060832866483517,-0.02193733706144022,0.03656021604936627,0.003698513197226362,-0.0011122262813486495,0.009267909918290072,0.5129411883387948,0.5027842064281226,0.523200954159897,0.022283998283998283,0.017675825825825827,0.027219133419133415,0.05865551265551267,0.047279225654225654,0.07083616473616473,17.81548777348777,16.808302638352636,18.797270334620336,0.021042900042900044,0.01781427498927499,0.024212129987129988,0.0629056199056199,0.05411741741741742,0.07141601244101244,15.856422565422568,14.7419083976834,16.96319958172458,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-multiply,text_audio,100,1.0,0.11991994317142747,0.0889362489426478,0.15110231629245585,37,0.07792934440600745,0.052977172014714184,0.10278047049072589,0.04199059878088213,0.034127040037302034,0.05013125747396997,0.6336595358656528,0.6087905367706232,0.6589064419710364,,,,,,,,,,,,,,,,,,,0.04137323037323038,0.034712870012870024,0.048116763191763195,0.11405105105105105,0.09659961389961391,0.13316510081510083,12.645648219648223,11.431381585156586,13.97510311025311,0.04315615615615616,0.03623397683397684,0.05003088803088803,0.1083843843843844,0.09482691977691977,0.12210212355212356,14.16602059202059,13.052271911196911,15.273814092664091,,,,,,,,,,,,,,,,,,
|
||||
TSFA-multiply,text_vision,100,1.0,0.03612421115527528,0.003679617927233912,0.0706390626773947,37,0.033054201762203535,0.0003160863252284142,0.06742515595895551,0.0030700093951693155,-0.0007710019667443696,0.006867931876338059,0.5052031858986646,0.4900221838004418,0.5219854911997864,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.01671685971685972,0.013541377091377095,0.019720752895752897,0.04806392106392106,0.03873852423852424,0.05764617117117116,19.210052767052765,17.75446628056628,20.751913610038613,0.021712569712569713,0.019381831831831835,0.02410126555126555,0.05798712998712999,0.05170907335907336,0.06397966537966537,18.30359202059202,16.94032631917632,19.627432818532814
|
||||
TSFA-random,audio_vision,100,20.0,0.020861894386615707,0.005693970997846327,0.037061795072022896,37,0.01680333015403859,0.0004332554102972204,0.03303836082632705,0.0040585642224212435,0.0005754381947093255,0.007718638619050609,0.5193708956724581,0.5150352463917897,0.5238890623406223,0.03257623337623338,0.030417548262548257,0.0346354971042471,0.09581681681681682,0.08960499839124837,0.10207584298584299,15.154294980694983,14.546768505469755,15.738201547619045,0.021379450879450878,0.02055578024453025,0.022163257722007723,0.06266823251823253,0.060993607357357364,0.0645759614972115,16.373181810381816,15.610742067245818,17.204955935220937,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-random,text_audio,100,20.0,0.17050077302726946,0.13941037026448067,0.1990060504478285,37,0.09672763162535135,0.07223268336231446,0.12007034012783693,0.07377314134437793,0.06320199140044352,0.08493496876685644,0.6780049539566163,0.6551398515962296,0.7010971646109445,,,,,,,,,,,,,,,,,,,0.048387923637923644,0.04305602799227799,0.05432789575289575,0.13999991419991417,0.12458882453882453,0.15700594272844276,11.147006499356502,10.136636986808236,12.213333453989703,0.04792293007293008,0.04362501340626341,0.051922379343629345,0.12110746460746463,0.11033633526383527,0.13127376447876446,13.276559244959246,12.25213515122265,14.3009197506435,,,,,,,,,,,,,,,,,,
|
||||
TSFA-random,text_vision,100,20.0,0.028480983388760747,-0.0010045256113203147,0.05799119457573753,37,0.026108679984648473,-0.004959284461738992,0.055005786615302185,0.0023723034026413474,-0.0007144136328255115,0.005453941959808597,0.5161156199227342,0.5118581409595148,0.5205601281866974,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.030713084513084528,0.0280383697983698,0.03368947286572287,0.08937998712998714,0.08109063545688547,0.09836511100386101,14.582967910767913,13.842146657014156,15.261898960210212,0.021675847275847274,0.02076378378378379,0.022689414414414417,0.05835364650364651,0.05572534534534535,0.06092968575718576,17.194992170742168,16.10918240830116,18.36624475707851
|
||||
|
@@ -0,0 +1,11 @@
|
||||
left_method,right_method,metric,clip_count,video_id_count,left_minus_right_video_macro_mean,ci95_low,ci95_high
|
||||
TSFA-main,M4_sourceTime,content_auc_mean,100,37,0.035438620389639894,0.023293646690953858,0.04698819770251506
|
||||
TSFA-main,M4_sourceTime,canonical_pairwise_time_mae_mean,100,37,0.005933525736358895,0.005296561964374039,0.006509589017113389
|
||||
TSFA-main,TSFA-random,content_auc_mean,100,37,-0.01842816398750574,-0.02718789359398819,-0.008992237898280122
|
||||
TSFA-main,TSFA-random,canonical_pairwise_time_mae_mean,100,37,-0.18037076895618454,-0.18518644583548508,-0.17535325008436822
|
||||
TSFA-main,TSFA-global,content_auc_mean,100,37,-0.055027892729355495,-0.06412401690967184,-0.04722633193128541
|
||||
TSFA-main,TSFA-global,canonical_pairwise_time_mae_mean,100,37,-0.17289980508939898,-0.17827689029341662,-0.16739238598647427
|
||||
TSFA-multiply,TSFA-main,content_auc_mean,100,37,-0.0021343558287264036,-0.003982915698119604,-0.00020908619168815494
|
||||
TSFA-multiply,TSFA-main,canonical_pairwise_time_mae_mean,100,37,-0.0025367852317495707,-0.002737516713424809,-0.002348100112530585
|
||||
TSFA-main,M3_noSourceTime,content_auc_mean,100,37,-0.038788438482809065,-0.05495972110741062,-0.02010023840091993
|
||||
TSFA-main,M3_noSourceTime,canonical_pairwise_time_mae_mean,100,37,-0.18092685065902986,-0.18653708198539262,-0.17516024678207237
|
||||
|
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"created_utc": "2026-09-23T14:32:44.179041+00:00",
|
||||
"experiment": "TSFA shared raw-content alignment-only probe",
|
||||
"sample_count": 100,
|
||||
"heldout_count": 100,
|
||||
"video_id_count": 37,
|
||||
"fold_count": 5,
|
||||
"seed": 42,
|
||||
"delta": 0.1,
|
||||
"probe_epochs": 40,
|
||||
"random_window_repeats": 20,
|
||||
"feature_protocol": "Within each fold, feature normalization is fitted on 80 training clips. For every method and modality, the same normalized raw source feature matrix is pooled using that method's A^m; native M3/M4/TSFA value and output projections are excluded from scored features.",
|
||||
"probe_protocol": "One 64-dimensional linear projector per modality, trained on 80 training clips with the same within-clip InfoNCE protocol and identical fold seed across methods. Scores use only 20 held-out clips per fold.",
|
||||
"interpretation_limit": "A same-slot positive is a timestamp/slot convention, not independently annotated semantic ground truth. Source-time attention can still encode time through selected raw values.",
|
||||
"device": "cuda",
|
||||
"elapsed_seconds": 63.806029081344604
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
method,modality,clip_count,video_id_count,candidate_count_mean_video_macro_mean,candidate_count_mean_ci95_low,candidate_count_mean_ci95_high,candidate_fraction_valid_mean_video_macro_mean,candidate_fraction_valid_mean_ci95_low,candidate_fraction_valid_mean_ci95_high,temporal_prior_mass_mean_video_macro_mean,temporal_prior_mass_mean_ci95_low,temporal_prior_mass_mean_ci95_high,candidate_mean_center_error_video_macro_mean,candidate_mean_center_error_ci95_low,candidate_mean_center_error_ci95_high
|
||||
TSFA-global,audio,100,37,859.2909266409266,725.903935006435,1010.8943355855854,0.9999999840133748,0.9999999739948848,0.9999999929042089,1.0,1.0,1.0,0.23349895292516687,0.23334358473436637,0.23365915023423298
|
||||
TSFA-global,vision,100,37,43.82829257829257,37.07407764907765,51.22568103818104,0.9999999899584969,0.9999999802928788,0.9999999976372933,1.0,1.0,1.0,0.23209808292734374,0.23138750004937297,0.23281616748889555
|
||||
TSFA-main,audio,100,37,173.57981845948993,148.11770421039185,204.1688543426484,0.2027066824698029,0.20226793367661783,0.20315699026129216,0.7372228195819668,0.7367319915047317,0.7377194487338015,0.0008611694746814316,0.0007925661735736586,0.0009352763757801223
|
||||
TSFA-main,vision,100,37,8.697594809082126,7.356396586381408,10.302393176907227,0.19806601293752601,0.19761108234273017,0.19851417924334616,0.7214869179295039,0.7187354109908368,0.7239951614850468,0.005114482842239538,0.004486957946824082,0.0057506511149799265
|
||||
TSFA-multiply,audio,100,37,173.57981845948993,147.94183613935493,206.29472111888524,0.2027066824698029,0.20227122749140025,0.20313434986607776,0.7372228195819668,0.7367260256056134,0.737726254887409,0.0008611694746814316,0.0007943149040772448,0.0009299756654477927
|
||||
TSFA-multiply,vision,100,37,8.697594809082126,7.429751027647465,10.313039470908723,0.19806601293752601,0.19755518793732132,0.19851298628820818,0.7214869179295039,0.7185624315606288,0.7239946865184925,0.005114482842239538,0.004483692884955575,0.0057248699215185035
|
||||
TSFA-random,audio,100,37,165.4303607225725,140.97407569611394,193.95208463450228,0.1933617310581552,0.19281956570094802,0.1938787973451431,0.19230555606103,0.19073511667908855,0.19371052016348292,0.005423930897291032,0.00533451006593477,0.0055040052315254925
|
||||
TSFA-random,vision,100,37,8.36343770343799,7.137826227282222,9.895227294917355,0.19097706449311183,0.1904804513221966,0.19147352101705237,0.19106015145038713,0.18928042240004053,0.1929521264620367,0.007972950431270442,0.007579205200452694,0.008422631476527104
|
||||
|
@@ -0,0 +1,22 @@
|
||||
method,pair,clip_count,draw_count,same_time_similarity_video_macro_mean,same_time_similarity_ci95_low,same_time_similarity_ci95_high,video_id_count,shifted_far_similarity_video_macro_mean,shifted_far_similarity_ci95_low,shifted_far_similarity_ci95_high,same_minus_shifted_margin_video_macro_mean,same_minus_shifted_margin_ci95_low,same_minus_shifted_margin_ci95_high,matched_vs_shifted_auc_video_macro_mean,matched_vs_shifted_auc_ci95_low,matched_vs_shifted_auc_ci95_high,exact_r1_audio_to_vision_video_macro_mean,exact_r1_audio_to_vision_ci95_low,exact_r1_audio_to_vision_ci95_high,within_pm1_r1_audio_to_vision_video_macro_mean,within_pm1_r1_audio_to_vision_ci95_low,within_pm1_r1_audio_to_vision_ci95_high,mase_slots_audio_to_vision_video_macro_mean,mase_slots_audio_to_vision_ci95_low,mase_slots_audio_to_vision_ci95_high,exact_r1_vision_to_audio_video_macro_mean,exact_r1_vision_to_audio_ci95_low,exact_r1_vision_to_audio_ci95_high,within_pm1_r1_vision_to_audio_video_macro_mean,within_pm1_r1_vision_to_audio_ci95_low,within_pm1_r1_vision_to_audio_ci95_high,mase_slots_vision_to_audio_video_macro_mean,mase_slots_vision_to_audio_ci95_low,mase_slots_vision_to_audio_ci95_high,exact_r1_text_to_audio_video_macro_mean,exact_r1_text_to_audio_ci95_low,exact_r1_text_to_audio_ci95_high,within_pm1_r1_text_to_audio_video_macro_mean,within_pm1_r1_text_to_audio_ci95_low,within_pm1_r1_text_to_audio_ci95_high,mase_slots_text_to_audio_video_macro_mean,mase_slots_text_to_audio_ci95_low,mase_slots_text_to_audio_ci95_high,exact_r1_audio_to_text_video_macro_mean,exact_r1_audio_to_text_ci95_low,exact_r1_audio_to_text_ci95_high,within_pm1_r1_audio_to_text_video_macro_mean,within_pm1_r1_audio_to_text_ci95_low,within_pm1_r1_audio_to_text_ci95_high,mase_slots_audio_to_text_video_macro_mean,mase_slots_audio_to_text_ci95_low,mase_slots_audio_to_text_ci95_high,exact_r1_text_to_vision_video_macro_mean,exact_r1_text_to_vision_ci95_low,exact_r1_text_to_vision_ci95_high,within_pm1_r1_text_to_vision_video_macro_mean,within_pm1_r1_text_to_vision_ci95_low,within_pm1_r1_text_to_vision_ci95_high,mase_slots_text_to_vision_video_macro_mean,mase_slots_text_to_vision_ci95_low,mase_slots_text_to_vision_ci95_high,exact_r1_vision_to_text_video_macro_mean,exact_r1_vision_to_text_ci95_low,exact_r1_vision_to_text_ci95_high,within_pm1_r1_vision_to_text_video_macro_mean,within_pm1_r1_vision_to_text_ci95_low,within_pm1_r1_vision_to_text_ci95_high,mase_slots_vision_to_text_video_macro_mean,mase_slots_vision_to_text_ci95_low,mase_slots_vision_to_text_ci95_high
|
||||
M3_noSourceTime,audio_vision,100,1.0,0.036963944587425426,0.012654210555109019,0.06213064156953601,37,0.01284708399055921,-0.010789174090569271,0.03801324850198797,0.02411686070606988,0.018906500978365978,0.02932537980979977,0.5615005482306014,0.5492661873214401,0.5739596572161066,0.0547975117975118,0.04492898970398971,0.06514066924066923,0.128025311025311,0.11184757614757616,0.1455527027027027,13.629726297726297,12.843761196911194,14.369294884169886,0.03019090519090519,0.026839886314886315,0.033779096954096956,0.07865679965679967,0.07125554483054485,0.08641603389103389,16.362507507507505,15.478168918918918,17.330904729729728,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_noSourceTime,text_audio,100,1.0,0.5428216995771276,0.5157750271393224,0.5659249134957856,37,0.1862772215156948,0.16884496117041078,0.2044323245350392,0.35654447643267695,0.32902113457284277,0.38482697127963067,0.9559461184208523,0.9410152465582983,0.968752133678131,,,,,,,,,,,,,,,,,,,0.36576791076791076,0.3160490561990562,0.421391108966109,0.6394950664950665,0.5802250858000858,0.6932264800514802,2.3383740883740884,1.8257137387387388,2.967019691119691,0.3325842985842986,0.2862534749034749,0.3864762226512226,0.5848228228228229,0.5289179107679108,0.6410546332046333,3.5629060489060485,2.914686679536679,4.190957496782497,,,,,,,,,,,,,,,,,,
|
||||
M3_noSourceTime,text_vision,100,1.0,0.07063237097845877,0.049190198520261096,0.09198749123186528,37,0.022404568813529854,0.0027337225595416618,0.042923773043187015,0.04822780188845877,0.039832519238572194,0.05659220698571646,0.6063276332943885,0.5904511958719739,0.6233192521834144,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.11063191763191764,0.09528938223938226,0.12811927284427282,0.22938867438867436,0.20507918275418277,0.2533813384813385,10.313163449163447,9.49544851994852,11.128362998712998,0.03617932217932218,0.030936861861861866,0.041976029601029606,0.09173659373659374,0.08058330115830119,0.10378899613899614,15.593415701415703,14.841660489060489,16.365030341055338
|
||||
M3_sourceTime,audio_vision,100,1.0,-0.01390508033140634,-0.05067550643888827,0.021212040213628297,37,-0.010913303125673809,-0.05113494249747865,0.028314814638195643,-0.002991777190843502,-0.005994554468138316,2.8365758378620055e-05,0.5029486009565798,0.49220838989635535,0.514119726033024,0.02227713427713428,0.01772549335049335,0.027349259974259972,0.05884513084513086,0.04750401115401116,0.07049715787215788,18.302241956241957,17.049701190476192,19.5555055984556,0.020694122694122692,0.01873700128700129,0.022657818532818536,0.06109223509223509,0.05726682754182755,0.06464848777348778,16.701135564135562,15.928820785070785,17.462392299442303,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_sourceTime,text_audio,100,1.0,0.13965087270742027,0.11086805911581128,0.17086291475965698,37,0.07104170450923547,0.04617817302421454,0.09790063937091513,0.06860916809770604,0.05853303224689769,0.07936001767631712,0.6837595762555869,0.6552860960409499,0.712458437802354,,,,,,,,,,,,,,,,,,,0.07891848991848992,0.06314130201630203,0.09810974903474903,0.17487301587301587,0.14614791934791935,0.2047123552123552,11.77520634920635,10.68862012012012,12.995895302445302,0.055659373659373665,0.04583983268983269,0.06627339124839127,0.13317975117975117,0.11395983483483482,0.15161607679107675,12.579315744315746,11.378763942513944,13.799107142857137,,,,,,,,,,,,,,,,,,
|
||||
M3_sourceTime,text_vision,100,1.0,0.018218856013835664,-0.0019383159056188255,0.03824574691640572,37,0.01843536877672055,-0.0031358283705724124,0.0398007043003149,-0.00021651274608931568,-0.0033959447400838593,0.0026867472750431616,0.5073603928776801,0.5013866661304163,0.5138052809268234,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.02189274989274989,0.017497951522951527,0.026569176319176314,0.06272629772629773,0.053270045045045046,0.07303394465894468,18.673179751179752,17.663443983268984,19.694041184041183,0.01792406692406693,0.015639414414414417,0.02020035392535392,0.052900900900900896,0.04713513513513515,0.058622297297297295,17.553818961818966,16.37964121621622,18.782360049335054
|
||||
M4_sourceTime,audio_vision,100,1.0,0.00960363132716526,-0.026355777083579862,0.048860038747030224,37,0.008486620442388824,-0.028060019295580236,0.04573340321810003,0.0011170108847764362,-0.0023584936828478765,0.004587398100880476,0.5134115260777495,0.4984700587080242,0.529333335144605,0.02358944658944659,0.01981911196911197,0.027432882882882884,0.06169626769626772,0.05163449163449164,0.07249851994851997,17.65543586443586,16.274862580437578,19.030651254826253,0.0226001716001716,0.019407979407979413,0.026030244530244533,0.06180137280137281,0.05271195838695839,0.07151898326898326,17.76484513084513,16.36237923637924,19.1801465036465,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M4_sourceTime,text_audio,100,1.0,0.06588486519170021,0.03945950347046453,0.09289460044348818,37,0.042297942433741985,0.019241170794713554,0.06466475258021885,0.02358692279788703,0.01496262407228283,0.032121939048117505,0.6160321730354975,0.5843945260158524,0.6481647989887683,,,,,,,,,,,,,,,,,,,0.037331617331617335,0.03060465465465466,0.043963170313170305,0.10766709566709566,0.09256888674388675,0.12209738309738306,13.641798798798801,12.305344755469758,15.062401372801373,0.03636851136851138,0.027877606177606182,0.04520361432861432,0.10497468897468898,0.08676978764478764,0.12320139425139427,13.546082368082368,12.237779311454313,14.863573423423423,,,,,,,,,,,,,,,,,,
|
||||
M4_sourceTime,text_vision,100,1.0,0.002750558023057728,-0.02300323787854146,0.028440979736176288,37,-0.0017949415156420013,-0.03015163323847945,0.026173505636594735,0.004545499502936101,0.00029588325187894103,0.009067848319586198,0.5036754941681804,0.4941306089880723,0.5134271535022865,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.024956670956670958,0.020241580866580868,0.030214564564564565,0.06877262977262978,0.056675429000429,0.08342528957528957,17.827770913770916,16.393623906048905,19.236555115830114,0.018706134706134706,0.01641861861861862,0.020702370227370228,0.0541947661947662,0.049403925353925356,0.05915512655512656,18.49487387387387,17.468441087516084,19.530955534105537
|
||||
TSFA-global,audio_vision,100,1.0,0.058836026214662884,0.009433591336071618,0.10898509821607989,37,0.0299474197815315,-0.019026566930953497,0.07779842395164258,0.02888860643662733,0.019019952453169467,0.039115650392871,0.5811011281037877,0.5633139432698278,0.5997097862395071,0.05360531960531961,0.04721345988845988,0.06024396181896182,0.1473792363792364,0.1310375482625483,0.16456441441441436,12.399526383526384,11.223109202059202,13.630522844272843,0.026169884169884174,0.023153153153153153,0.02972972972972973,0.072981552981553,0.06513077005577005,0.0819059738309738,16.35457228657229,15.289285274560278,17.390430094380097,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-global,text_audio,100,1.0,0.6151585970459093,0.592303545010855,0.6363802687550169,37,0.3307069785888169,0.30723476551075196,0.3557131022926266,0.28445161780264955,0.2568491350423748,0.313431063751506,0.888037095225925,0.8713038621448196,0.9025196944665029,,,,,,,,,,,,,,,,,,,0.1913303303303303,0.17120793650793648,0.21155896610896607,0.44082539682539684,0.4059675246675246,0.4759347168597168,4.0553886743886745,3.421013191763192,4.85362648005148,0.13350965250965252,0.11927451737451739,0.1480651222651223,0.3463878163878164,0.31618583226083224,0.37751728871728873,6.496407550407551,5.68826003861004,7.341266816816816,,,,,,,,,,,,,,,,,,
|
||||
TSFA-global,text_vision,100,1.0,0.05193602225629566,0.00557772744820211,0.09913521469205716,37,0.018522642509243077,-0.031060642072022387,0.06214130753726415,0.03341337973851245,0.023362873667806743,0.044610356791469324,0.5860619295207061,0.5682458135804879,0.6058382392698549,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.0624924924924925,0.05556150793650794,0.0695357679107679,0.1712166452166452,0.15426744959244962,0.18985087945087942,10.319833118833117,9.234121578721577,11.574682368082371,0.027978978978978986,0.02543226083226084,0.030679751179751174,0.08042342342342342,0.07197972972972973,0.08928452380952381,16.16939510939511,14.841324195624193,17.51072467824968
|
||||
TSFA-main,audio_vision,100,1.0,0.014167320872974634,-0.010923848048597458,0.039202572867964074,37,0.00572396734721069,-0.022427993542694716,0.03323088554438458,0.008443353525896292,0.0026837436866675632,0.015297084395711374,0.5168712160108436,0.5063480574752184,0.5274386673004691,0.02177691977691978,0.01702814242814243,0.026475278850278845,0.059097383097383105,0.0487433075933076,0.06956501501501502,17.385808665808668,16.533373498498495,18.1887395002145,0.020783783783783785,0.017991763191763196,0.02361460746460746,0.059234234234234226,0.0514413770913771,0.06677416344916343,17.36473530673531,16.14145280995281,18.719550675675674,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-main,text_audio,100,1.0,0.44300997549055066,0.41801447102011324,0.4684682627939124,37,0.23093871736075175,0.2083505803346634,0.254020971881992,0.21207125773569485,0.18952878732104791,0.23568765088817542,0.8444317785680818,0.8258805004947956,0.8617723152745259,,,,,,,,,,,,,,,,,,,0.12499742599742601,0.11379501287001287,0.13663513513513512,0.3075298155298155,0.2829487880737881,0.3312437258687259,6.916232089232089,6.001308118833121,7.864674903474903,0.1002827112827113,0.08834907764907765,0.11306663449163447,0.2701518661518662,0.24654092664092664,0.29654121621621626,7.918441441441442,6.990601801801802,8.871822072072073,,,,,,,,,,,,,,,,,,
|
||||
TSFA-main,text_vision,100,1.0,0.0174827435109615,-0.014795200237153264,0.0498013264192671,37,0.007209132154900075,-0.027171939497727737,0.040232186743695476,0.010273611322660614,0.0044551992623352394,0.01675656997344102,0.5184451704744258,0.5070176356542713,0.5300536991588852,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.02143886743886744,0.017653732303732305,0.02514813384813385,0.06253453453453454,0.05233642213642214,0.07386306306306306,17.168172458172457,16.094917084942086,18.22205849420849,0.021965250965250968,0.019243018018018018,0.024749935649935648,0.06071342771342771,0.05453082368082369,0.0671651866151866,16.860575718575717,15.960335660660665,17.848634663234666
|
||||
TSFA-multiply,audio_vision,100,1.0,0.011799700871433188,-0.013416462369410848,0.03714225595848734,37,0.003975713585776283,-0.023566553326942314,0.029399206363613498,0.007823987230021269,0.002012223205404298,0.014442908524818563,0.5172232899693006,0.5074870466258432,0.5276387511371553,0.020090519090519093,0.01639286786786787,0.023955534105534106,0.05755426855426856,0.04768575718575719,0.06837712355212355,17.54742471042471,16.713855855855854,18.3979314993565,0.020211497211497214,0.017404579579579582,0.022875203775203776,0.058004719004719,0.05067953667953669,0.06470415057915058,16.951271986271987,15.84848443800944,18.024721492921493,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-multiply,text_audio,100,1.0,0.43704115381661734,0.4115881279842063,0.46182952157855794,37,0.22681150353265547,0.20526582196287743,0.2506810446160017,0.21022965143079514,0.186519824604155,0.23452360996737565,0.8422674450666473,0.8232177948179944,0.8604417612806575,,,,,,,,,,,,,,,,,,,0.12100386100386103,0.10705222007722007,0.13455019305019308,0.3025752895752896,0.27550450450450453,0.3296434684684685,7.047516087516087,6.080864478764479,8.009039800514799,0.10672972972972973,0.09416164736164735,0.11945048262548263,0.2784924924924925,0.2542594487344487,0.3048471042471042,7.982056199056198,7.081995849420849,8.870854268554268,,,,,,,,,,,,,,,,,,
|
||||
TSFA-multiply,text_vision,100,1.0,0.019027262118925484,-0.011798013240855429,0.05015302798114453,37,0.008701149625835541,-0.025909994276746277,0.04330936423482336,0.010326112528868553,0.004384441093969887,0.016339780587871982,0.5203762987911925,0.5091359831811294,0.5319951762695445,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.021473616473616473,0.017847393822393823,0.025207303732303732,0.06058687258687259,0.05200009652509653,0.06891801801801802,17.13536293436293,16.116098712998713,18.12695836550836,0.022200772200772205,0.01949655727155727,0.02511743886743887,0.05943286143286144,0.053154193479193486,0.0655993993993994,16.844067353067352,15.867397082797082,17.840103292578295
|
||||
TSFA-random,audio_vision,100,20.0,0.0003412659092388615,-0.012525598498225196,0.01250697673623247,37,-0.002757302584965167,-0.017743520338419794,0.011022231272467209,0.0030985684875587003,0.0007254310763673855,0.005789096312349675,0.520877531278429,0.5157918074214983,0.5265877221121911,0.03191679536679536,0.0298980518018018,0.0340041924066924,0.09458826683826685,0.08867857142857143,0.10115269626769628,15.031224281424278,14.405575359287857,15.620949209030458,0.021313685113685112,0.02049507185757186,0.022244750643500646,0.06297726297726298,0.06123326576576577,0.0648488315100815,16.687722222222224,15.855059311990566,17.524219135027884,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-random,text_audio,100,20.0,0.5176953808174132,0.49329636456565923,0.5421801408262926,37,0.27252678952364234,0.24728572903165366,0.2993041891233309,0.24516859096668722,0.22255121880186324,0.26788503176686185,0.8510175559155282,0.8342504583359005,0.8671632742516454,,,,,,,,,,,,,,,,,,,0.13294581724581728,0.12157137548262546,0.14623681306306305,0.35170619905619904,0.32670355051480054,0.37593875482625483,5.308989296439297,4.696646238202488,5.932726235521235,0.11373277563277563,0.1038839065851566,0.12520446160446164,0.2932850278850279,0.2702927622265122,0.3166170704633205,7.3795730802230795,6.716822666237666,8.059022419026167,,,,,,,,,,,,,,,,,,
|
||||
TSFA-random,text_vision,100,20.0,0.01558142443491039,-0.004607028207164052,0.0340684453611749,37,0.01131799393027936,-0.01173588358127808,0.03077004442264562,0.0042634304992695575,0.0006321330527004355,0.008231108258869975,0.523311412590402,0.5182214701491481,0.5286413477330866,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.03621218361218361,0.03320817889317888,0.03949472597597597,0.1053566495066495,0.09686440047190047,0.11374875321750319,14.090841570141572,13.219035881595882,14.95690251287001,0.021406113256113257,0.02049555823680824,0.022317504826254827,0.06158863148863149,0.05928467771342772,0.06384178035178034,16.919727498927497,15.876626134706132,18.00174117599742
|
||||
|
Binary file not shown.
|
After Width: | Height: | Size: 71 KiB |
@@ -0,0 +1,7 @@
|
||||
method,control,pair,clip_count,video_id_count,matched_vs_shifted_auc_video_macro_mean,matched_vs_shifted_auc_ci95_low,matched_vs_shifted_auc_ci95_high,same_minus_shifted_margin_video_macro_mean,same_minus_shifted_margin_ci95_low,same_minus_shifted_margin_ci95_high,exact_r1_left_to_right_video_macro_mean,exact_r1_left_to_right_ci95_low,exact_r1_left_to_right_ci95_high,within_pm1_r1_left_to_right_video_macro_mean,within_pm1_r1_left_to_right_ci95_low,within_pm1_r1_left_to_right_ci95_high
|
||||
M4_sourceTime,independent_within_clip_permutation,audio_vision,100,37,0.49986099215667035,0.49931733210509144,0.5003794440377403,-0.0003183416417442903,-0.0006530043625340906,-3.3575496045077684e-05,0.02070763620763621,0.019744644465894464,0.021695343736593736,0.059731810381810385,0.05825279922779924,0.06113831724581724
|
||||
M4_sourceTime,independent_within_clip_permutation,text_audio,100,37,0.4992583332049756,0.4971305291708881,0.5012287335156442,-0.0004793830947098817,-0.0017547874540129774,0.0007121573235354092,0.01981724581724582,0.018432171814671817,0.021092101029601035,0.05907595452595453,0.057171570141570136,0.06100200343200343
|
||||
M4_sourceTime,independent_within_clip_permutation,text_vision,100,37,0.4995061476417528,0.49898081806837713,0.5000416571403756,-0.0001465278630284633,-0.00048526501402030057,0.00016435085172832273,0.01881887601887602,0.01782144305019305,0.01977210102960103,0.058243114543114545,0.05672013513513514,0.059936018339768325
|
||||
TSFA-main,independent_within_clip_permutation,audio_vision,100,37,0.5002577534839172,0.4994096693338,0.5011686280803744,0.00012270166467700342,-0.0003552724864745554,0.0005955004762808745,0.019852552552552555,0.0187075,0.020917128378378377,0.059261347061347065,0.05789174603174604,0.060541009223509225
|
||||
TSFA-main,independent_within_clip_permutation,text_audio,100,37,0.5004550297305284,0.4981946165487771,0.5026825985666993,0.0006103920254983002,-0.0024510630057079466,0.0037619848284550215,0.01998494208494209,0.019009004182754183,0.02102998873873874,0.05908569283569285,0.05776156960531959,0.06049437419562419
|
||||
TSFA-main,independent_within_clip_permutation,text_vision,100,37,0.5000126508394793,0.49939359917466397,0.5006472318623715,-0.00016505359922095624,-0.0005699285037343316,0.00024382621948318463,0.019825032175032176,0.01893382561132561,0.020605167310167313,0.059049335049335065,0.05775769841269841,0.06029238685113685
|
||||
|
@@ -0,0 +1,26 @@
|
||||
left_method,right_method,metric_family,pair,metric,clip_count,video_id_count,left_minus_right_video_macro_mean,ci95_low,ci95_high
|
||||
TSFA-main,M4_sourceTime,overall,all_pairs,content_auc_mean,100,37,0.08220965725730794,0.07113723316940294,0.09569307390121884
|
||||
TSFA-main,M4_sourceTime,overall,all_pairs,canonical_pairwise_time_mae_mean,100,37,0.005933525736358895,0.005308267109580502,0.006529948953558749
|
||||
TSFA-main,M4_sourceTime,content_pair,text_audio,matched_vs_shifted_auc,100,37,0.22839960553258426,0.19753921496296828,0.2586728798778832
|
||||
TSFA-main,M4_sourceTime,content_pair,text_vision,matched_vs_shifted_auc,100,37,0.014769676306245455,0.00615762201696377,0.024147940037301717
|
||||
TSFA-main,M4_sourceTime,content_pair,audio_vision,matched_vs_shifted_auc,100,37,0.003459689933094183,-0.0127482317281187,0.01864119744744742
|
||||
TSFA-main,TSFA-random,overall,all_pairs,content_auc_mean,100,37,-0.00515277824366921,-0.011173873250387557,0.0007255270180419965
|
||||
TSFA-main,TSFA-random,overall,all_pairs,canonical_pairwise_time_mae_mean,100,37,-0.18037076895618454,-0.18509186583110665,-0.17551767054171094
|
||||
TSFA-main,TSFA-random,content_pair,text_audio,matched_vs_shifted_auc,100,37,-0.006585777347446219,-0.015532873664531726,0.0009897398589461357
|
||||
TSFA-main,TSFA-random,content_pair,text_vision,matched_vs_shifted_auc,100,37,-0.00486624211597616,-0.014786038407090595,0.004877792533937811
|
||||
TSFA-main,TSFA-random,content_pair,audio_vision,matched_vs_shifted_auc,100,37,-0.004006315267585209,-0.013151033853001922,0.005371947005658802
|
||||
TSFA-main,TSFA-global,overall,all_pairs,content_auc_mean,100,37,-0.05848399593235586,-0.06921582521604686,-0.04751669353941074
|
||||
TSFA-main,TSFA-global,overall,all_pairs,canonical_pairwise_time_mae_mean,100,37,-0.17289980508939898,-0.17836035039600534,-0.16750727197744386
|
||||
TSFA-main,TSFA-global,content_pair,text_audio,matched_vs_shifted_auc,100,37,-0.043605316657843264,-0.053833684686776434,-0.03223812133752294
|
||||
TSFA-main,TSFA-global,content_pair,text_vision,matched_vs_shifted_auc,100,37,-0.06761675904628031,-0.0849899430490521,-0.050819432782896884
|
||||
TSFA-main,TSFA-global,content_pair,audio_vision,matched_vs_shifted_auc,100,37,-0.06422991209294399,-0.0784464424435834,-0.050670400781670726
|
||||
TSFA-multiply,TSFA-main,overall,all_pairs,content_auc_mean,100,37,3.9622924596328686e-05,-0.0011158515658515594,0.0011833296608728741
|
||||
TSFA-multiply,TSFA-main,overall,all_pairs,canonical_pairwise_time_mae_mean,100,37,-0.0025367852317495707,-0.0027293981115725143,-0.0023532245292300245
|
||||
TSFA-multiply,TSFA-main,content_pair,text_audio,matched_vs_shifted_auc,100,37,-0.0021643335014345675,-0.003420222806393028,-0.0009433171987527146
|
||||
TSFA-multiply,TSFA-main,content_pair,text_vision,matched_vs_shifted_auc,100,37,0.0019311283167666208,-3.923633930282419e-05,0.003844595478837497
|
||||
TSFA-multiply,TSFA-main,content_pair,audio_vision,matched_vs_shifted_auc,100,37,0.00035207395845693497,-0.001707745444912998,0.0025183332791377994
|
||||
TSFA-main,M3_noSourceTime,overall,all_pairs,content_auc_mean,100,37,-0.08134204496416375,-0.09018397700115897,-0.07252863456636728
|
||||
TSFA-main,M3_noSourceTime,overall,all_pairs,canonical_pairwise_time_mae_mean,100,37,-0.18092685065902986,-0.18650572139607072,-0.17507314724507567
|
||||
TSFA-main,M3_noSourceTime,content_pair,text_audio,matched_vs_shifted_auc,100,37,-0.11151433985277072,-0.126396370047384,-0.09652581659832322
|
||||
TSFA-main,M3_noSourceTime,content_pair,text_vision,matched_vs_shifted_auc,100,37,-0.08788246281996281,-0.10456414269512673,-0.07031106495933623
|
||||
TSFA-main,M3_noSourceTime,content_pair,audio_vision,matched_vs_shifted_auc,100,37,-0.04462933221975775,-0.054448958632834964,-0.0345335877670984
|
||||
|
@@ -0,0 +1,260 @@
|
||||
{
|
||||
"created_utc": "2026-09-23T14:26:26.904353+00:00",
|
||||
"experiment": "TSFA: Temporal-Semantic Factorized Alignment",
|
||||
"sample_count": 100,
|
||||
"fold_count": 5,
|
||||
"heldout_partition_count": 100,
|
||||
"unique_video_id_count": 37,
|
||||
"folds": [
|
||||
{
|
||||
"fold": 1,
|
||||
"train_count": 80,
|
||||
"heldout_count": 20,
|
||||
"train_video_id_count": 31,
|
||||
"heldout_video_id_count": 6,
|
||||
"video_id_overlap": [],
|
||||
"validation_sample_ids": [
|
||||
"-3g5yACwYnA/13",
|
||||
"-3g5yACwYnA/3",
|
||||
"-3g5yACwYnA/2",
|
||||
"-3g5yACwYnA/9",
|
||||
"-egA8-b7-3M/26",
|
||||
"-egA8-b7-3M/17",
|
||||
"-egA8-b7-3M/18",
|
||||
"-egA8-b7-3M/16",
|
||||
"-egA8-b7-3M/13",
|
||||
"-egA8-b7-3M/1",
|
||||
"-egA8-b7-3M/6",
|
||||
"-egA8-b7-3M/9",
|
||||
"-egA8-b7-3M/20",
|
||||
"-I_e4mIh0yE/1",
|
||||
"-I_e4mIh0yE/3",
|
||||
"-qDkUB0GgYY/6",
|
||||
"-9y-fZ3swSY/0",
|
||||
"-9y-fZ3swSY/4",
|
||||
"-9y-fZ3swSY/8",
|
||||
"-UUCSKoHeMA/0"
|
||||
]
|
||||
},
|
||||
{
|
||||
"fold": 2,
|
||||
"train_count": 80,
|
||||
"heldout_count": 20,
|
||||
"train_video_id_count": 30,
|
||||
"heldout_video_id_count": 7,
|
||||
"video_id_overlap": [],
|
||||
"validation_sample_ids": [
|
||||
"-THoVjtIkeU/12",
|
||||
"-THoVjtIkeU/2",
|
||||
"-THoVjtIkeU/6",
|
||||
"-UuX1xuaiiE/1",
|
||||
"-UuX1xuaiiE/0",
|
||||
"-UuX1xuaiiE/3",
|
||||
"-UuX1xuaiiE/6",
|
||||
"-lzEya4AM_4/5",
|
||||
"-lzEya4AM_4/6",
|
||||
"-mqbVkbCndg/0",
|
||||
"-wny0OAz3g8/1",
|
||||
"-wny0OAz3g8/0",
|
||||
"-wny0OAz3g8/3",
|
||||
"-wny0OAz3g8/2",
|
||||
"-wny0OAz3g8/5",
|
||||
"-wny0OAz3g8/7",
|
||||
"-wny0OAz3g8/9",
|
||||
"-571d8cVauQ/0",
|
||||
"-571d8cVauQ/5",
|
||||
"-HeZS2-Prhc/2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"fold": 3,
|
||||
"train_count": 80,
|
||||
"heldout_count": 20,
|
||||
"train_video_id_count": 29,
|
||||
"heldout_video_id_count": 8,
|
||||
"video_id_overlap": [],
|
||||
"validation_sample_ids": [
|
||||
"-HwX2H8Z4hY/2",
|
||||
"-HwX2H8Z4hY/5",
|
||||
"-HwX2H8Z4hY/6",
|
||||
"-HwX2H8Z4hY/9",
|
||||
"-aqamKhZ1Ec/0",
|
||||
"-vxjVxOeScU/4",
|
||||
"-wMB_hJL-3o/7",
|
||||
"-hnBHBN8p5A/7",
|
||||
"-hnBHBN8p5A/6",
|
||||
"-AUZQgSxyPQ/2",
|
||||
"-RfYyzHpjk4/11",
|
||||
"-RfYyzHpjk4/8",
|
||||
"-RfYyzHpjk4/2",
|
||||
"-s9qJ7ATP7w/1",
|
||||
"-s9qJ7ATP7w/0",
|
||||
"-s9qJ7ATP7w/5",
|
||||
"-s9qJ7ATP7w/4",
|
||||
"-s9qJ7ATP7w/7",
|
||||
"-s9qJ7ATP7w/6",
|
||||
"-s9qJ7ATP7w/8"
|
||||
]
|
||||
},
|
||||
{
|
||||
"fold": 4,
|
||||
"train_count": 80,
|
||||
"heldout_count": 20,
|
||||
"train_video_id_count": 29,
|
||||
"heldout_video_id_count": 8,
|
||||
"video_id_overlap": [],
|
||||
"validation_sample_ids": [
|
||||
"-aNfi7CP8vM/7",
|
||||
"-iRBcNs9oI8/3",
|
||||
"-iRBcNs9oI8/7",
|
||||
"-iRBcNs9oI8/6",
|
||||
"-iRBcNs9oI8/9",
|
||||
"-iRBcNs9oI8/8",
|
||||
"-t217m2on-s/2",
|
||||
"-t217m2on-s/7",
|
||||
"-tPCytz4rww/11",
|
||||
"-tPCytz4rww/10",
|
||||
"-tPCytz4rww/12",
|
||||
"-tPCytz4rww/16",
|
||||
"-tPCytz4rww/18",
|
||||
"-UacrmKiTn4/10",
|
||||
"-UacrmKiTn4/4",
|
||||
"-uywlfIYOS8/4",
|
||||
"-6rXp3zJ3kc/8",
|
||||
"-ri04Z7vwnc/0",
|
||||
"-ri04Z7vwnc/2",
|
||||
"-ri04Z7vwnc/5"
|
||||
]
|
||||
},
|
||||
{
|
||||
"fold": 5,
|
||||
"train_count": 80,
|
||||
"heldout_count": 20,
|
||||
"train_video_id_count": 29,
|
||||
"heldout_video_id_count": 8,
|
||||
"video_id_overlap": [],
|
||||
"validation_sample_ids": [
|
||||
"-3nNcZdcdvU/5",
|
||||
"-NFrJFQijFE/1",
|
||||
"-NFrJFQijFE/2",
|
||||
"-a55Q6RWvTA/3",
|
||||
"-dxfTGcXJoc/1",
|
||||
"-dxfTGcXJoc/0",
|
||||
"-dxfTGcXJoc/2",
|
||||
"-dxfTGcXJoc/6",
|
||||
"-mJ2ud6oKI8/1",
|
||||
"-mJ2ud6oKI8/2",
|
||||
"-mJ2ud6oKI8/6",
|
||||
"-mJ2ud6oKI8/9",
|
||||
"-mJ2ud6oKI8/8",
|
||||
"-tANM6ETl_M/3",
|
||||
"-MeTTeMJBNc/0",
|
||||
"-MeTTeMJBNc/13",
|
||||
"-MeTTeMJBNc/7",
|
||||
"-yRb-Jum7EQ/1",
|
||||
"-yRb-Jum7EQ/5",
|
||||
"-yRb-Jum7EQ/6"
|
||||
]
|
||||
}
|
||||
],
|
||||
"alignment_models_retrained": false,
|
||||
"temporal_branch": "frozen M4_sourceTime D5 checkpoint per grouped fold",
|
||||
"semantic_branch_trained": true,
|
||||
"emotion_labels_used_for_alignment": false,
|
||||
"feature_extractors_changed": false,
|
||||
"feature_encoder_note": "Existing BERT text and DeiT image features are reused unchanged.",
|
||||
"example_sample_id": "-3g5yACwYnA/13",
|
||||
"parameters": {
|
||||
"seed": 42,
|
||||
"grid_size": 50,
|
||||
"hidden_size": 128,
|
||||
"heads_in_frozen_alignment": 4,
|
||||
"delta_normalized_time": 0.1,
|
||||
"hard_negative_offsets_slots": [
|
||||
-5,
|
||||
-3,
|
||||
-2,
|
||||
2,
|
||||
3,
|
||||
5
|
||||
],
|
||||
"semantic_epochs": 40,
|
||||
"semantic_learning_rate": 0.001,
|
||||
"local_contrastive_temperature": 0.1,
|
||||
"content_probe_epochs": 40,
|
||||
"content_probe_temperature": 0.1,
|
||||
"reconstruction_probe_epochs": 40,
|
||||
"shuffle_repeats": 20,
|
||||
"random_window_repeats": 20,
|
||||
"window_ablation_status": "delta sweep and 80-percent attention-mass window deferred until the seven-method MVP is reviewed"
|
||||
},
|
||||
"methods": [
|
||||
"M3_noSourceTime",
|
||||
"M3_sourceTime",
|
||||
"M4_sourceTime",
|
||||
"TSFA-main",
|
||||
"TSFA-multiply",
|
||||
"TSFA-random",
|
||||
"TSFA-global"
|
||||
],
|
||||
"protocol": {
|
||||
"semantic_query_key_inputs": "Frozen M4 attention-pooled text value output is the semantic query input; M4 pre-attention projected Audio/Vision content is the semantic key/value input. No explicit source-time code, absolute PE, slot ID, or latent positional vector is passed to semantic Q/K. Temporal selection can still encode time indirectly.",
|
||||
"baseline_content_values": "M4 and M3 Audio/Vision use each checkpoint's actual MHA attention output. M3 Text uses attention-pooled pre-attention text features to remove its source-time query residual.",
|
||||
"temporal_role": "Frozen M4 attention supplies candidate masks; TSFA-multiply additionally multiplies semantic probabilities by M4 temporal attention",
|
||||
"local_contrastive": "same-slot positives with within-clip slot-offset negatives at +/-2, +/-3, +/-5; symmetric Text-Audio and Text-Vision loss",
|
||||
"random_candidate": "same delta-width windows with independently sampled centers at inference; 20 held-out random draws, one train draw for the train-only probe",
|
||||
"global_candidate": "all valid source positions available to semantic attention",
|
||||
"content_probe": "same train-only symmetric within-clip InfoNCE probe applied to each method; held-out negatives are positions more than two slots away",
|
||||
"content_shuffle": "independently permute each modality's 50 projected content rows within each held-out clip; M4 and TSFA-main",
|
||||
"reconstruction": "fold-train decoder predicts one aligned content representation from the other two; shift one partner by signed offsets +/-1,2,5,10 on identical target support",
|
||||
"confidence_intervals": "95-percent video_id-cluster bootstrap; 2,000 repetitions for CSV summaries"
|
||||
},
|
||||
"input_paths": {
|
||||
"feature_dir": "/home/gloamxun/modeling_zhaocui/deep_learning/Q1/outputs/q1_features/features",
|
||||
"feature_manifest": "/home/gloamxun/modeling_zhaocui/deep_learning/Q1/outputs/audit/manifest.csv",
|
||||
"grouped_splits": "/home/gloamxun/modeling_zhaocui/deep_learning/Q1/outputs/method_comparison/splits.json",
|
||||
"checkpoint_root": "/home/gloamxun/modeling_zhaocui/deep_learning/Q1/outputs/alignment_debug/heldout",
|
||||
"m3_m4_checkpoint_variants": [
|
||||
"M3_noSourceTime",
|
||||
"M3_sourceTime",
|
||||
"M4_sourceTime"
|
||||
]
|
||||
},
|
||||
"device": "cuda",
|
||||
"gpu_name": "NVIDIA GeForce RTX 5070 Ti",
|
||||
"python": "3.14.7",
|
||||
"torch": "2.14.0+cu130",
|
||||
"elapsed_seconds": 129.8167963027954,
|
||||
"interpretation_limits": [
|
||||
"Same-slot labels are a temporal training convention, not independent semantic ground truth.",
|
||||
"Frozen M4 temporal attention was trained with a timestamp-derived Gaussian prior; time may enter content indirectly through candidate selection.",
|
||||
"AUC, retrieval, and shifted similarity measure within-clip content matchability, not human event alignment.",
|
||||
"Reconstruction predicts M4/TSFA pooled content values, not raw waveform or image pixels.",
|
||||
"Five folds use one alignment-model seed; bootstrap intervals quantify video-group sampling uncertainty, not seed uncertainty.",
|
||||
"Human event IoU/center error remains pending event annotation."
|
||||
],
|
||||
"input_sha256": {
|
||||
"feature_manifest": "bc07d5fb1ff104144cf845abf856c4e04da3caadda0c05d4b798a64a581bb35a",
|
||||
"grouped_splits": "ccdcab416488fd3d94b762696f70411ebcd02346417cb7f1a5be70e7a1a6cf4d",
|
||||
"uv_lock": "28ae0c736eef130a380022da82c79c473f467e2cb8dad4c199a85fd0b1faadf1",
|
||||
"features_combined": "ed99618a0f4ac943fd21c214a331aeff11ca6aa7419b48622ab25f8f8d8ae985",
|
||||
"feature_file_count": 100,
|
||||
"frozen_alignment_checkpoints": {
|
||||
"fold_01/M3_noSourceTime": "e54088fbc4432f41e9a09b167cd90a752c6bf678461e4d4ca553e1a7eb23fadc",
|
||||
"fold_01/M3_sourceTime": "377772c546ec1cb79359b4792de76f67df9700a1ab48f0dfc66451640733ef67",
|
||||
"fold_01/M4_sourceTime": "ad2e5fb0db3a8891a831ad3d2c85dde59f0563bc41dc05fff10655589d37d1fc",
|
||||
"fold_02/M3_noSourceTime": "2bd24ca12a8c6eb4335b54db5de7b0f9963752279f0bf4daa9c7641b251e00ae",
|
||||
"fold_02/M3_sourceTime": "64eca35e24962055c8e2d1520f62af291f395f8243438bc1609925d62794c673",
|
||||
"fold_02/M4_sourceTime": "f77c6a46ec6424b7a52907bca81a0842ec0066ba9dec38b8e10bfbbf455a1a62",
|
||||
"fold_03/M3_noSourceTime": "76dff4f2643a4fd46dc6d7a5d7d6a74ce8e119da53fed81d98b4c22d90af0052",
|
||||
"fold_03/M3_sourceTime": "f52e47bdbe8a3b3b360ad053a202570170f45cdfc6d07066f6e8558d31182125",
|
||||
"fold_03/M4_sourceTime": "de6fcfbaa57eab745b6ccfd7a19860589fccaa30b497813a3afdecb15956ac6c",
|
||||
"fold_04/M3_noSourceTime": "735354a326ab23a832370449862493bcab2452551858a6825c099c27ff982b58",
|
||||
"fold_04/M3_sourceTime": "304c7287f4d777b27a5354e4316802cdc6de75dd1dd20a419314255b67618a4e",
|
||||
"fold_04/M4_sourceTime": "d0e50814377b5b98fbf70c3ca48431756b22391823629dab74c3192b8bfe1a3d",
|
||||
"fold_05/M3_noSourceTime": "0f8d57bae2af04e0a7432a42a68b02b09c60bac4da68d4b7595905a31e1c8fe2",
|
||||
"fold_05/M3_sourceTime": "47bccf271f6046e695fd7ad7e16d9a018d944df5773b5a8e15af9be80bd9af41",
|
||||
"fold_05/M4_sourceTime": "787e28df330b9891f8ff24c6522d808bd0a1bac269f29c6f11dff55b8ba95d4e"
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 148 KiB |
@@ -0,0 +1,25 @@
|
||||
method,target_modality,shifted_modality,abs_delta,clip_count,video_id_count,aligned_mae_same_support_video_macro_mean,aligned_mae_same_support_ci95_low,aligned_mae_same_support_ci95_high,shifted_mae_video_macro_mean,shifted_mae_ci95_low,shifted_mae_ci95_high,gain_shift_minus_aligned_video_macro_mean,gain_shift_minus_aligned_ci95_low,gain_shift_minus_aligned_ci95_high
|
||||
M4_sourceTime,audio,vision,1,100,37,0.10648079388657068,0.09498403317711335,0.11971105591350258,0.10648057482407897,0.09536464065503321,0.1203203971314147,-2.190624917080248e-07,-3.002898652523641e-06,2.836719715064019e-06
|
||||
M4_sourceTime,audio,vision,10,100,37,0.10608212038697232,0.09495647062817791,0.11900391438347258,0.10613144132552779,0.09468061707438626,0.12023388864214406,4.932093855548141e-05,-0.000138402270899901,0.0002644478668511099
|
||||
M4_sourceTime,audio,vision,2,100,37,0.10634162971084272,0.0951935955060543,0.12024532359561485,0.10634258744158634,0.09515405380473113,0.12009526039769572,9.577307436201306e-07,-1.0696270178279515e-05,1.3246172074194955e-05
|
||||
M4_sourceTime,audio,vision,5,100,37,0.1060977863330044,0.09541860889488209,0.11980911076375722,0.1061057905801574,0.09548577428451295,0.1195104905764324,8.004247152986875e-06,-5.808767217863869e-05,7.687466521118136e-05
|
||||
M4_sourceTime,text,vision,1,100,37,0.3524187293592465,0.3221663890648929,0.3832589358215371,0.35242382845190495,0.3240157492065436,0.38444638677967397,5.09909265852445e-06,-8.073796445031578e-06,2.1147349556082562e-05
|
||||
M4_sourceTime,text,vision,10,100,37,0.34596756801060324,0.31742921319489337,0.37868989748536663,0.34664258137473164,0.31755937971955817,0.3774341924067551,0.0006750133641284713,-0.0002579232627475584,0.0017880953921477383
|
||||
M4_sourceTime,text,vision,2,100,37,0.35132473441234785,0.32286859659712197,0.38345827370335456,0.3513478712829614,0.3224631958370705,0.38120628674285656,2.3136870613531815e-05,-2.753329515642539e-05,8.594694638849325e-05
|
||||
M4_sourceTime,text,vision,5,100,37,0.3485910437432214,0.3205305428072932,0.3803397981259862,0.348778251139507,0.3203935584532843,0.3815424183815085,0.00018720739628561882,-9.496389538631425e-05,0.0005151327779809838
|
||||
M4_sourceTime,vision,audio,1,100,37,0.2231766281827664,0.20607600232587064,0.24236390166586727,0.22317987500195693,0.20554712626708696,0.2422070137882051,3.246819190537148e-06,-2.2809803989287967e-06,9.420666630299547e-06
|
||||
M4_sourceTime,vision,audio,10,100,37,0.22307425242673293,0.20607910972755006,0.24121378121482856,0.22325032634755695,0.2067523672321235,0.24305877808947715,0.00017607392082394369,-0.00022312340428489366,0.0006277892943623485
|
||||
M4_sourceTime,vision,audio,2,100,37,0.22316048174652228,0.20552624977176193,0.24224169822075334,0.22317257495856702,0.2061885128990303,0.2429836954945456,1.209321204478625e-05,-9.359554453552767e-06,3.6892148305301144e-05
|
||||
M4_sourceTime,vision,audio,5,100,37,0.2231394299643585,0.20618209030565493,0.24110680100715395,0.22321575703646734,0.20586565202003376,0.2427468816852832,7.632707210888122e-05,-4.880731381197847e-05,0.0002121602112554407
|
||||
TSFA-main,audio,vision,1,100,37,0.24988408831952214,0.22469871818983359,0.27840707807349385,0.24988695082411724,0.22501213166380882,0.2790530408493829,2.8625045951371134e-06,-1.742456965739765e-05,2.272696183944069e-05
|
||||
TSFA-main,audio,vision,10,100,37,0.24386757700414372,0.2190698984802626,0.2733444834273454,0.2444317758278318,0.21902110992468563,0.27219107996658004,0.000564198823688062,-0.00022641418153737327,0.001395855642402091
|
||||
TSFA-main,audio,vision,2,100,37,0.24895100756913924,0.22416956525924045,0.27810367903911887,0.24896510513695155,0.22274126205659267,0.27930326741638245,1.4097567812305108e-05,-4.732088382019858e-05,7.63355802100572e-05
|
||||
TSFA-main,audio,vision,5,100,37,0.2463452433058948,0.22097869036887005,0.2751587883843783,0.246497264087475,0.2214858425035993,0.2751850658218311,0.0001520207815802389,-0.00014739758679055951,0.0004554734628724275
|
||||
TSFA-main,text,vision,1,100,37,0.3476741555274325,0.32148611245427566,0.3737497650390369,0.3476541439183967,0.32086054265937813,0.37359475283999777,-2.0011609035824965e-05,-6.729367620210594e-05,3.311202856174665e-05
|
||||
TSFA-main,text,vision,10,100,37,0.33808821519346943,0.3123830481146549,0.36471129622609255,0.3382066619050689,0.31227041983330905,0.365546023769169,0.00011844671159944317,-0.001876128714584399,0.002146817830318111
|
||||
TSFA-main,text,vision,2,100,37,0.3463484496062007,0.3194491144852671,0.372250999602138,0.34635784703948913,0.31847836298931187,0.3729820835754293,9.397433288411396e-06,-0.0001571169486153489,0.00020347758379682152
|
||||
TSFA-main,text,vision,5,100,37,0.3427197299862511,0.31759720182711926,0.3687916805460675,0.34274824133142034,0.3161611675319563,0.37042456927433204,2.8511345169156534e-05,-0.0007572757773635113,0.0009358633216909969
|
||||
TSFA-main,vision,audio,1,100,37,0.30385903824985366,0.2804930920345874,0.32447569376476193,0.3038967083492334,0.28237160721884863,0.32535285142726206,3.767009937983119e-05,2.2140309296097684e-05,5.2699988083822036e-05
|
||||
TSFA-main,vision,audio,10,100,37,0.30352860244078117,0.2808613100727914,0.32450677675282535,0.30500827796376856,0.28273082537522215,0.32564273932073723,0.0014796755229874467,0.0004977929359841301,0.002395565682036881
|
||||
TSFA-main,vision,audio,2,100,37,0.30382711134055096,0.2824193253098451,0.32531113133968326,0.3039569732806986,0.2827239277182888,0.3254230913166988,0.0001298619401475382,7.685713367700474e-05,0.00018430574637678292
|
||||
TSFA-main,vision,audio,5,100,37,0.3037293283725082,0.2824902004786779,0.3239507917223273,0.30433064016570666,0.2829440499471156,0.32495928013412795,0.0006013117931984567,0.00028662674138598147,0.0009081425354786053
|
||||
|
Binary file not shown.
|
After Width: | Height: | Size: 170 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 86 KiB |
@@ -0,0 +1,113 @@
|
||||
method,modality,clip_count,video_id_count,span_ratio_video_macro_mean,span_ratio_ci95_low,span_ratio_ci95_high,span_abs_video_macro_mean,span_abs_ci95_low,span_abs_ci95_high,metric_family,direction,pairwise_time_mae_video_macro_mean,pairwise_time_mae_ci95_low,pairwise_time_mae_ci95_high,pairwise_signed_lag_video_macro_mean,pairwise_signed_lag_ci95_low,pairwise_signed_lag_ci95_high,pairwise_time_corr_video_macro_mean,pairwise_time_corr_ci95_low,pairwise_time_corr_ci95_high,near_similarity_offdiag_video_macro_mean,near_similarity_offdiag_ci95_low,near_similarity_offdiag_ci95_high,far_similarity_video_macro_mean,far_similarity_ci95_low,far_similarity_ci95_high,d_self_offdiag_video_macro_mean,d_self_offdiag_ci95_low,d_self_offdiag_ci95_high,gram_target_error_video_macro_mean,gram_target_error_ci95_low,gram_target_error_ci95_high,far_slot_leakage_video_macro_mean,far_slot_leakage_ci95_low,far_slot_leakage_ci95_high,cycle_kind,cycle_band_error_video_macro_mean,cycle_band_error_ci95_low,cycle_band_error_ci95_high,cycle_time_mae_video_macro_mean,cycle_time_mae_ci95_low,cycle_time_mae_ci95_high,triangle_relative_error_video_macro_mean,triangle_relative_error_ci95_low,triangle_relative_error_ci95_high,triangle_mean_absolute_residual_video_macro_mean,triangle_mean_absolute_residual_ci95_low,triangle_mean_absolute_residual_ci95_high
|
||||
M3_noSourceTime,audio,100,37,0.006845225875929812,-0.0050480489482916426,0.018641673964870674,0.036477922386970923,0.029721606294995286,0.04310413792211531,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_noSourceTime,text,100,37,0.8675032330277417,0.8513314214973553,0.8824147079299696,0.8675032330277417,0.8522296196594468,0.8833766268951103,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_noSourceTime,vision,100,37,-0.05248904935502325,-0.09102026382990809,-0.017373356121759984,0.1361241436408572,0.10828269435313069,0.16937841433957945,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_sourceTime,audio,100,37,0.7079631044011514,0.6640514063782721,0.7384979763997631,0.7079631044011514,0.6651266517684566,0.7387978141351451,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_sourceTime,text,100,37,0.8675032330277417,0.8519241948813162,0.8843134111737262,0.8675032330277417,0.8520634954683572,0.8832864640856598,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_sourceTime,vision,100,37,0.7103498443247823,0.6786859055886948,0.7367456721643969,0.7103498443247823,0.677706011592382,0.7352975597625276,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M4_sourceTime,audio,100,37,0.8157060578090206,0.8144226218056944,0.8168588184847212,0.8157060578090206,0.8144152058853479,0.8168682832834657,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M4_sourceTime,text,100,37,0.7210220398065448,0.7029622253382732,0.7391635974741213,0.7210220398065448,0.7032309030429688,0.7388928438031057,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M4_sourceTime,vision,100,37,0.8217648772803375,0.819098820782984,0.8244210758168445,0.8217648772803375,0.8191228203610597,0.8243458564337164,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-global,audio,100,37,0.011351551106837957,-0.03307923351329353,0.054755991186068075,0.1361641932261882,0.1071832420028644,0.16555720151779119,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-global,text,100,37,0.7210220398065448,0.7028018689779558,0.7385341768148932,0.7210220398065448,0.7016156508853206,0.7388307750319667,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-global,vision,100,37,0.007610485834433587,-0.016687070936055162,0.033969306453111976,0.0831604315308393,0.06578630184541784,0.10186488890564342,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-main,audio,100,37,0.7959866389347422,0.7814527810816472,0.8097362828007407,0.7959866389347422,0.7818257680156309,0.8106330514335037,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-main,text,100,37,0.7210220398065448,0.7032168935165822,0.7400873256704725,0.7210220398065448,0.7028208226665803,0.7398312410824844,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-main,vision,100,37,0.8207314282427551,0.8150341158499098,0.8263598218647393,0.8207314282427551,0.8149187170040457,0.8264408487068886,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-multiply,audio,100,37,0.8238465478704826,0.8087581509483757,0.8369243381130251,0.8238465478704826,0.8084151479166035,0.8377465327122026,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-multiply,text,100,37,0.7210220398065448,0.7018140627887535,0.7388406501835107,0.7210220398065448,0.7017115274716906,0.7390402836448626,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-multiply,vision,100,37,0.8564870598082609,0.8515092554836746,0.8619992878873571,0.8564870598082609,0.8514594765786335,0.8620531933055623,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-random,audio,100,37,-0.0166169608453845,-0.0356561105662797,0.003911856542961215,0.3092861632918937,0.29694859626772774,0.32168873174102913,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-random,text,100,37,0.7210220398065448,0.702517714202256,0.7388815238925032,0.7210220398065448,0.7032355236466896,0.7387423622324837,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-random,vision,100,37,0.009951817230022326,-0.009544235701251413,0.02983924649004834,0.31530318162219934,0.30375650755698547,0.327376648433025,span,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_noSourceTime,,100,37,,,,,,,pairwise,audio_to_text,0.24798689239530688,0.24575595320867452,0.250478564603917,-0.00905109914646526,-0.021659838618577246,0.0043487629126405004,0.06358454760547783,0.023034305728283397,0.10249668543842441,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_noSourceTime,,100,37,,,,,,,pairwise,audio_to_vision,0.2557268510896316,0.2522416884193085,0.2599540490992459,0.0010860293038389595,-0.021838431492360325,0.02637708897886559,0.023136812147259593,-0.017164462319666285,0.06361670928884612,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_noSourceTime,,100,37,,,,,,,pairwise,text_to_audio,0.23844018512711035,0.231815399826556,0.24513403638360018,0.00032643349356500825,-0.014206368505950655,0.013788368566719469,0.09157069878863097,0.017082456940376654,0.16326818673274548,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_noSourceTime,,100,37,,,,,,,pairwise,text_to_vision,0.2595995219781619,0.24768831061652558,0.2721962396716379,0.003549698052312745,-0.018195794363028227,0.025729048351632686,-0.07469627228957339,-0.1645982737378071,0.010895163182870584,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_noSourceTime,,100,37,,,,,,,pairwise,vision_to_audio,0.2517105464205569,0.24997783849782754,0.25325740669487573,0.00011944104763818307,-0.003998172286235301,0.004314578789024335,0.09065033078927832,0.005952490324212469,0.18367249135769936,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_noSourceTime,,100,37,,,,,,,pairwise,vision_to_text,0.26965636750629723,0.26312577044158164,0.27726571774760705,-0.005324066992981435,-0.019810339686355333,0.010465554443999218,-0.051735984712035554,-0.14065674992330893,0.032703681806886056,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_sourceTime,,100,37,,,,,,,pairwise,audio_to_text,0.0871124186546691,0.07843402180952203,0.0960482575299671,-0.00818185656226965,-0.02501026774573674,0.009866064174329241,0.9621272440842052,0.9404591626453755,0.9767108597657944,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_sourceTime,,100,37,,,,,,,pairwise,audio_to_vision,0.07549793127843017,0.06847063238991467,0.08375030573884443,-0.005625651477412908,-0.00969390107684296,-0.0018935553076301747,0.976149538484469,0.9633859880459819,0.9838961024157739,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_sourceTime,,100,37,,,,,,,pairwise,text_to_audio,0.11115512644896658,0.09863427941596914,0.12356354925002973,0.0029040672010041033,-0.017689209501853456,0.021674971143182058,0.8664779183674878,0.8310981906498358,0.8947861184932466,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_sourceTime,,100,37,,,,,,,pairwise,text_to_vision,0.1095721907332496,0.09817313494579982,0.12168360601054318,-0.00021640865527353695,-0.020344818529633008,0.019194392157659432,0.8786640862668412,0.8555847476303776,0.9012911708379915,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_sourceTime,,100,37,,,,,,,pairwise,vision_to_audio,0.08402641543628109,0.07674065296156558,0.09217593659353024,-0.0002759945725429673,-0.005976672227240552,0.005817899813735201,0.9615264449317401,0.9381665699920014,0.9756067583306635,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_sourceTime,,100,37,,,,,,,pairwise,vision_to_text,0.0949450733631415,0.08614114589590695,0.10400518407272195,-0.006395282222322348,-0.023404822078732648,0.011666152940526454,0.9565651139127582,0.9392471240397936,0.9696111837015309,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M4_sourceTime,,100,37,,,,,,,pairwise,audio_to_text,0.05986829347375184,0.05422676746679547,0.06582581936684692,0.001716346997700615,-0.006062248610466559,0.010155625110385253,0.9896387509611086,0.987113812806792,0.9918754847735682,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M4_sourceTime,,100,37,,,,,,,pairwise,audio_to_vision,0.024761989554881157,0.023888416321772084,0.025694809281355088,0.0009678201964747196,0.0004121556321871045,0.0015657592778028952,0.9963711557037225,0.9957911008064857,0.9968302062855288,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M4_sourceTime,,100,37,,,,,,,pairwise,text_to_audio,0.08413479872440241,0.07637751970403586,0.09244831443618172,0.003073342587004113,-0.008817055347750192,0.01405819189112211,0.9277882717821058,0.9106142612324963,0.943464413917021,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M4_sourceTime,,100,37,,,,,,,pairwise,text_to_vision,0.08428864072945397,0.07631423853160876,0.09282317706135165,0.0030182972199566836,-0.008150982694603395,0.013742263037027585,0.9277943139436867,0.9109454276967438,0.9437667482725357,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M4_sourceTime,,100,37,,,,,,,pairwise,vision_to_audio,0.03125012554416389,0.030335602973254293,0.032274370405520454,0.001699291733416523,0.000989529421278144,0.002399587124229476,0.994063490418897,0.9934018419958887,0.9946277486801368,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M4_sourceTime,,100,37,,,,,,,pairwise,vision_to_text,0.06582797990544002,0.06015672140269782,0.07191811912272289,0.002534441565149494,-0.0049269372098783275,0.01014627322915136,0.9873438070809311,0.9842728666509312,0.9898634520953487,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-global,,100,37,,,,,,,pairwise,audio_to_text,0.26426354529126855,0.2595954058664512,0.26927950613867524,0.016063363039220284,0.0023853956672613154,0.030029997969078993,0.01186519338523276,-0.017863932995588257,0.04408118069862936,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-global,,100,37,,,,,,,pairwise,audio_to_vision,0.251902193365334,0.24851626175956615,0.25607406149155476,0.011592058619731964,-0.0020735190406534266,0.026210686820579737,-0.011882097548093922,-0.041437837608961185,0.02005575593397121,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-global,,100,37,,,,,,,pairwise,text_to_audio,0.23919675578879268,0.22823033229896159,0.2502279509927412,0.011978075655547017,-0.007856421287997644,0.03205505955258488,0.13953505174336778,0.013684665832117467,0.26550892119657277,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-global,,100,37,,,,,,,pairwise,text_to_vision,0.23858647233188462,0.2303998823983567,0.24728701813446996,0.016471399245608764,-0.0030107452002579448,0.03881277808578741,0.028802362948568784,-0.10411775420285496,0.16248498491336846,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-global,,100,37,,,,,,,pairwise,vision_to_audio,0.2600867141079052,0.2555181459039208,0.26489728316899913,0.008798799409372195,-0.009991486618197236,0.027522525432970055,-0.04216116616051461,-0.11934540642052503,0.0328652169183437,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-global,,100,37,,,,,,,pairwise,vision_to_text,0.25645549371974924,0.25181104298018353,0.2616800858478354,0.011963496297170268,0.0027002137949521223,0.02234805357364569,0.011657691672614899,-0.06382264351518231,0.08717482762128388,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-main,,100,37,,,,,,,pairwise,audio_to_text,0.06742660977259347,0.06276358021854823,0.07237029198906914,0.001769438513107734,-0.005499522451483993,0.009628460807482234,0.9780673585858292,0.9740539184484459,0.9812739256641996,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-main,,100,37,,,,,,,pairwise,audio_to_vision,0.04204494685473837,0.04060578185676881,0.04347158780261231,0.0015684376347839893,-0.0005385557212511506,0.0036400813468450794,0.9876396523633675,0.9866859120783338,0.9885968762026767,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-main,,100,37,,,,,,,pairwise,text_to_audio,0.08441610461747974,0.07641255077281847,0.09256938825420448,0.0024054198545138397,-0.009065747922210807,0.012980256698477306,0.9275963343341015,0.9108534196618289,0.9432700846219623,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-main,,100,37,,,,,,,pairwise,text_to_vision,0.08452495474559611,0.07623102571244669,0.09289985937525803,0.0033036427730361436,-0.007851205779410854,0.013378385207246357,0.9274092030944906,0.9101469934454218,0.9433551123562987,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-main,,100,37,,,,,,,pairwise,vision_to_audio,0.030076064676432743,0.02785290171629763,0.03252893344676687,0.0005874198264963244,-0.0019138561045836968,0.003188964794821949,0.9926644930991895,0.9917709342590185,0.9934552026921571,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-main,,100,37,,,,,,,pairwise,vision_to_text,0.05890817948197322,0.053470287745186154,0.06463501870845278,0.001918705659802123,-0.0056993804592977496,0.009731904017812175,0.9857566395421482,0.98161901062293,0.9889256100291415,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-multiply,,100,37,,,,,,,pairwise,audio_to_text,0.06643963790591051,0.06155519203761027,0.07146318366180746,0.0016541105648428527,-0.005800171780808754,0.0094397165252077,0.9792184259320681,0.9752197300750737,0.9826263203417981,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-multiply,,100,37,,,,,,,pairwise,audio_to_vision,0.03748529088182362,0.03602949972109203,0.03887249988224152,0.0014145159042255155,-0.0004971187343514189,0.003414844650505779,0.9895384201014192,0.9886004595988371,0.9904223364933273,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-multiply,,100,37,,,,,,,pairwise,text_to_audio,0.08316689271595411,0.07493328226138725,0.09161901581232872,0.002656665242589696,-0.008838504815762998,0.01322843925099298,0.9277193243312696,0.9105277835589112,0.9435921704549525,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-multiply,,100,37,,,,,,,pairwise,text_to_vision,0.08272346692478777,0.07413101830975553,0.09082178262359049,0.00355221780618286,-0.00780020790439905,0.013463799558736902,0.92770141066974,0.9112267680307472,0.9435642584766777,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-multiply,,100,37,,,,,,,pairwise,vision_to_audio,0.026516252496326696,0.024272904323796834,0.02923892906045038,0.0005202078371720159,-0.0017816156849158036,0.0028475703813473827,0.9943015227769918,0.9935560002859497,0.9950439734823314,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-multiply,,100,37,,,,,,,pairwise,vision_to_text,0.059111147729345934,0.05318454866186355,0.06473659759075691,0.001869284025221193,-0.0058491839333650285,0.009862095021189055,0.9861672927109927,0.9819146359888574,0.9894506176914498,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-random,,100,37,,,,,,,pairwise,audio_to_text,0.2715302012965951,0.2681423482921384,0.2751148151494631,0.013213615803127883,0.001544596804972213,0.024384888097449788,-0.02156884837448982,-0.043228793267016175,0.0008287115908261907,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-random,,100,37,,,,,,,pairwise,audio_to_vision,0.26677251834192656,0.2644589349599799,0.26927277649842407,-0.0009915105457357748,-0.0036349776141373887,0.0018582449001258766,0.002305975374338194,-0.016209359818672282,0.021096996250978824,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-random,,100,37,,,,,,,pairwise,text_to_audio,0.24253396559006382,0.23648577481667343,0.24879672462738361,0.004271181421459896,-0.008669847073590682,0.016266469979640173,0.001809260826441729,-0.024212496786697527,0.02977938535091217,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-random,,100,37,,,,,,,pairwise,text_to_vision,0.24279182915437733,0.23618153703657,0.24918830521619556,0.002916123269236995,-0.011716715868149562,0.015546720870855023,-0.025966406801151545,-0.054607945323011,0.003413669136643002,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-random,,100,37,,,,,,,pairwise,vision_to_audio,0.26196779815139154,0.25996107208940733,0.2642539732400678,0.0027559449435993182,3.7462547767353755e-05,0.005297637651498782,-0.0016721355157619025,-0.025000031524045015,0.019068474158866765,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
TSFA-random,,100,37,,,,,,,pairwise,vision_to_text,0.2619983087285825,0.26002152342407947,0.2640970253943566,0.009418709922206357,0.001987751724951219,0.01781982791398481,-0.02090627930113151,-0.04115157465871522,-0.0008479818764069004,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
M3_noSourceTime,audio,100,37,,,,,,,self,,,,,,,,,,,0.8602562808430486,0.8390017979472224,0.8791960507765548,0.6306394423686962,0.6065736211847845,0.6559103310590667,0.22961683847435244,0.212597593561371,0.24839158745125373,1.5433979842530003,1.4939613439855566,1.5980224666375722,0.5914922344267804,0.5863327383501546,0.5961485323484096,,,,,,,,,,,,,
|
||||
M3_noSourceTime,text,100,37,,,,,,,self,,,,,,,,,,,0.500730224675529,0.44326381155953154,0.5546875932491161,0.0,0.0,0.0,0.500730224675529,0.44678261442106343,0.5536740821511082,0.7783799210361851,0.7588314988752327,0.7978312145643258,0.0,0.0,0.0,,,,,,,,,,,,,
|
||||
M3_noSourceTime,vision,100,37,,,,,,,self,,,,,,,,,,,0.8370010596898583,0.8128242535668444,0.8586645729538069,0.5354454345010294,0.49812339029733943,0.5758168318539664,0.3015556251888289,0.26835387949396683,0.33611183886562884,1.3908956425610952,1.3204375307722693,1.4698184122554232,0.5652966473540847,0.5525497540528237,0.5764936815046576,,,,,,,,,,,,,
|
||||
M3_sourceTime,audio,100,37,,,,,,,self,,,,,,,,,,,0.888331670085538,0.8783799815302573,0.8982361516689842,0.2859968780215105,0.2614315436352547,0.30991001303306515,0.6023347920640273,0.5718641086403762,0.629589423002457,1.1051953048433216,1.0611940801301807,1.1477056646407984,0.3858874333468643,0.36389107499473,0.4080694269041795,,,,,,,,,,,,,
|
||||
M3_sourceTime,text,100,37,,,,,,,self,,,,,,,,,,,0.500730224675529,0.4425607231213978,0.5514131309491841,0.0,0.0,0.0,0.500730224675529,0.44289256370891295,0.553421139441445,0.7783799210361851,0.7596998963592567,0.7978403791798905,0.0,0.0,0.0,,,,,,,,,,,,,
|
||||
M3_sourceTime,vision,100,37,,,,,,,self,,,,,,,,,,,0.8923011998859715,0.8829162316726851,0.90115625806856,0.30215556168475666,0.2766014128544894,0.32747475171660667,0.5901456382012148,0.5624948401289267,0.6188530100994244,1.113534676309553,1.072232565941515,1.1558847797411864,0.3964994452097854,0.37548961116022583,0.41767726895743396,,,,,,,,,,,,,
|
||||
M4_sourceTime,audio,100,37,,,,,,,self,,,,,,,,,,,0.9830707859067331,0.983043585033989,0.9830990215661007,0.0692560064447752,0.06915230942710256,0.06936442043075314,0.9138147794619581,0.9137097396996392,0.9139154768791944,0.5438185058326855,0.5430943251047008,0.5445142540617243,0.12587317163943731,0.1257309976018284,0.12601617989449548,,,,,,,,,,,,,
|
||||
M4_sourceTime,text,100,37,,,,,,,self,,,,,,,,,,,0.98463863452608,0.9837836830897783,0.9854255401052563,0.12195832206472494,0.10937889431437446,0.13486592677771062,0.8626803124613551,0.8505551729542101,0.8741516645446641,0.7332360007514397,0.7053039293689665,0.7605882309710607,0.19238998771979546,0.17759862017965225,0.2070004338350631,,,,,,,,,,,,,
|
||||
M4_sourceTime,vision,100,37,,,,,,,self,,,,,,,,,,,0.9828657132228624,0.9827565207179908,0.9829821773464078,0.07301019914488216,0.07189453444841783,0.07421181716145536,0.9098555140779804,0.9086199818433717,0.9109538990663043,0.5426103369833182,0.5395012878989437,0.5462207981931749,0.13257047995822666,0.13090495318430864,0.13445453758276793,,,,,,,,,,,,,
|
||||
TSFA-global,audio,100,37,,,,,,,self,,,,,,,,,,,0.9381095443274116,0.931651754282041,0.9441185921559399,0.2100733717905093,0.1826601718796808,0.23920233233183366,0.7280361725369022,0.7001579464958054,0.7535776749762602,0.863381418331564,0.8039491487313329,0.926914950609264,0.33487438012325216,0.31245989188931467,0.35649616064506184,,,,,,,,,,,,,
|
||||
TSFA-global,text,100,37,,,,,,,self,,,,,,,,,,,0.98463863452608,0.983766490951091,0.9853981353359472,0.12195832206472494,0.10965717998549025,0.1347841332424256,0.8626803124613551,0.850372569552986,0.8740441549811607,0.7332360007514397,0.7066870479875449,0.7613386633835179,0.19238998771979546,0.17703831482671462,0.20823798629303045,,,,,,,,,,,,,
|
||||
TSFA-global,vision,100,37,,,,,,,self,,,,,,,,,,,0.9889972492867095,0.9853616333200549,0.9921496089242945,0.7152139051917641,0.6681896821127071,0.7602635452078422,0.27378334409494554,0.23153592011200502,0.3169545411183176,1.807907897230697,1.7252640583459942,1.8897482017579965,0.5648694933764876,0.5505857506060692,0.5788789461690218,,,,,,,,,,,,,
|
||||
TSFA-main,audio,100,37,,,,,,,self,,,,,,,,,,,0.8234864222497095,0.8147252263635052,0.831638166641138,0.0018924057590828816,0.0014796837371070474,0.002338841789812211,0.8215940164906266,0.8128654426624269,0.8293378196210572,0.4197792178264907,0.39931766382912903,0.44031334323802956,0.005687440577387501,0.004467740371657415,0.0070866126639524945,,,,,,,,,,,,,
|
||||
TSFA-main,text,100,37,,,,,,,self,,,,,,,,,,,0.98463863452608,0.9837952570538889,0.9854168029916435,0.12195832206472494,0.11024324958022892,0.13513622841986347,0.8626803124613551,0.8502792958299866,0.8738112439977024,0.7332360007514397,0.7062337654689973,0.7604249710025476,0.19238998771979546,0.17820739335916275,0.20748713402377209,,,,,,,,,,,,,
|
||||
TSFA-main,vision,100,37,,,,,,,self,,,,,,,,,,,0.8753932551323013,0.8700816518650041,0.8800146684598407,0.004439952313729791,0.003958120248637802,0.004918994027823828,0.8709533028185714,0.8654470668263632,0.875883832637767,0.2922236461705599,0.27221210954296104,0.31353420473606874,0.010609511873370787,0.009417318597629527,0.011965775154103995,,,,,,,,,,,,,
|
||||
TSFA-multiply,audio,100,37,,,,,,,self,,,,,,,,,,,0.845473601064013,0.8371369589334942,0.8535547155860039,0.0011535688717004406,0.0008681368939363554,0.0014583059130755891,0.8443200321923128,0.8358434386626328,0.8519076978453295,0.4008834873616406,0.37780546574374,0.4226541568072099,0.0033878968038142357,0.0025584266488971052,0.0043033148150613335,,,,,,,,,,,,,
|
||||
TSFA-multiply,text,100,37,,,,,,,self,,,,,,,,,,,0.98463863452608,0.9837051549803183,0.9854353569171195,0.12195832206472494,0.11040081311310136,0.13455153639535794,0.8626803124613551,0.8512955021521286,0.8742087339006359,0.7332360007514397,0.7059750518908112,0.7605916492760649,0.19238998771979546,0.17717071592148534,0.20736117762042403,,,,,,,,,,,,,
|
||||
TSFA-multiply,vision,100,37,,,,,,,self,,,,,,,,,,,0.902764058566973,0.8974145599745504,0.9077503722502418,0.0022335280035617423,0.0019388695415776,0.0025267593271357434,0.9005305305634114,0.8948633558813284,0.9057063079847478,0.2410330598364526,0.2190705020750994,0.26486362144176023,0.005374162383074902,0.0045517751203657825,0.006298533655693178,,,,,,,,,,,,,
|
||||
TSFA-random,audio,100,37,,,,,,,self,,,,,,,,,,,0.1702136025448653,0.16723226208912714,0.17300510804100797,0.050006548425651956,0.04463245283092054,0.05590187111703176,0.12020705411921334,0.11476586705860664,0.125197794987648,0.9000364572774586,0.8908052350066653,0.9108907750588845,0.3193311199680904,0.29964326945087166,0.34036697059575866,,,,,,,,,,,,,
|
||||
TSFA-random,text,100,37,,,,,,,self,,,,,,,,,,,0.98463863452608,0.9837430297876442,0.9854105897288065,0.12195832206472494,0.11012758265837151,0.13410818320090137,0.8626803124613551,0.8498772919284686,0.874141656735737,0.7332360007514397,0.7050931335501408,0.7594350878347045,0.19238998771979546,0.17794175293831158,0.20700749691688433,,,,,,,,,,,,,
|
||||
TSFA-random,vision,100,37,,,,,,,self,,,,,,,,,,,0.18680846851618726,0.18398392204894654,0.18986701006886691,0.16210455704600274,0.1565246182058625,0.16698550554870373,0.02470391147018457,0.020263875415987974,0.029383917404735463,1.0959556252675098,1.0855062865988658,1.1058215053916503,0.5504916063539527,0.5429613692763432,0.5567723296495888,,,,,,,,,,,,,
|
||||
M3_noSourceTime,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_audio_vision_audio,0.9654507654379716,0.94782301334197,0.9844262926673437,0.24526603600206048,0.24460695902603002,0.24592368024056546,,,,,,
|
||||
M3_noSourceTime,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_text_audio_text,0.8658500951793273,0.8596118509176524,0.8712751183264686,0.21988855319734216,0.21315919945463932,0.22707554891642193,,,,,,
|
||||
M3_noSourceTime,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_text_vision_text,0.8312663326293602,0.8178681469090896,0.8442766436361115,0.20514106384912809,0.19733560193931374,0.2124968724218096,,,,,,
|
||||
M3_noSourceTime,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,triangle_TAV,,,,,,,0.635751241203901,0.6076922664336832,0.6627118953375897,0.019152084617695322,0.016176198013098882,0.02205733569262106
|
||||
M3_sourceTime,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_audio_vision_audio,0.8045496673993995,0.7971204875711047,0.8125584394999472,0.11888443345978468,0.10957197271767381,0.12804746974727776,,,,,,
|
||||
M3_sourceTime,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_text_audio_text,0.7470532880949274,0.7350454921488248,0.7583312459337788,0.10782301608973462,0.09783257218838352,0.11888516673233186,,,,,,
|
||||
M3_sourceTime,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_text_vision_text,0.7488389508139521,0.7375039579370523,0.7604845451779533,0.11137686125855244,0.10127015365499387,0.12215765169648239,,,,,,
|
||||
M3_sourceTime,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,triangle_TAV,,,,,,,0.39458475984546676,0.3873522859378844,0.40157171964300187,0.01299650427840006,0.011530419632056874,0.014404423079089927
|
||||
M4_sourceTime,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_audio_vision_audio,0.7194542968836412,0.7187292856075248,0.7202319735193489,0.04983903677260223,0.049023056283463394,0.050749148327950494,,,,,,
|
||||
M4_sourceTime,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_text_audio_text,0.8651297946131749,0.8462946860695034,0.8852096860574188,0.09564003516962925,0.08768005125014286,0.10423717662834384,,,,,,
|
||||
M4_sourceTime,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_text_vision_text,0.8642305913190416,0.8457638814878798,0.8844126405604584,0.09851611937621746,0.09029651463872992,0.10699334122957067,,,,,,
|
||||
M4_sourceTime,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,triangle_TAV,,,,,,,0.2262284485325185,0.22148258912718663,0.23134585693643994,0.006954086200606416,0.006118734019699923,0.007763513689622475
|
||||
TSFA-global,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_audio_vision_audio,1.7233751514850173,1.5320620802026454,1.910227031436825,0.2499189757629977,0.24813426973160263,0.2517913851678485,,,,,,
|
||||
TSFA-global,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_text_audio_text,0.8828054688922468,0.8655570599393629,0.9002497028299116,0.16017333436663472,0.15020650011568096,0.17071839324736315,,,,,,
|
||||
TSFA-global,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_text_vision_text,0.948073999704302,0.9353821128883112,0.961931287382611,0.22096955633926135,0.21213126171141727,0.22922736277343997,,,,,,
|
||||
TSFA-global,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,triangle_TAV,,,,,,,0.14486850608413515,0.12897322570694134,0.16042111551767863,0.003349657125055823,0.0027169777735415804,0.003995757491955577
|
||||
TSFA-main,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_audio_vision_audio,2.0587887072533735,1.883379123386877,2.239275068016061,0.04549742980449285,0.04396053159546751,0.04718275588930528,,,,,,
|
||||
TSFA-main,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_text_audio_text,0.8415374909118359,0.8187267280346352,0.8647714332096803,0.08541476449405673,0.07766053427838494,0.09406202419279779,,,,,,
|
||||
TSFA-main,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_text_vision_text,0.8473648847790903,0.8255998143909318,0.871333941141899,0.08711804315672145,0.07865623181447821,0.09561705139066572,,,,,,
|
||||
TSFA-main,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,triangle_TAV,,,,,,,0.11295304219180526,0.10829628941263399,0.11777329791566588,0.003259473054729029,0.0028647321008819352,0.003665811502401078
|
||||
TSFA-multiply,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_audio_vision_audio,2.067465096935341,1.8999584180770852,2.2458561639507786,0.04087040783248778,0.039240224738941,0.04256950418053263,,,,,,
|
||||
TSFA-multiply,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_text_audio_text,0.8402509149510545,0.8179641691851386,0.8632553287603144,0.08511486218799579,0.0765977381321801,0.09348774126063883,,,,,,
|
||||
TSFA-multiply,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_text_vision_text,0.8443214704351736,0.8227938391074004,0.8671506370987065,0.08644753408290066,0.07827521794931933,0.09534041757944388,,,,,,
|
||||
TSFA-multiply,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,triangle_TAV,,,,,,,0.11133536067721364,0.10623922285694568,0.11667533159216026,0.0031566936483280374,0.002768192325193071,0.0035930796529320273
|
||||
TSFA-random,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_audio_vision_audio,1.2376846358141187,1.1866995355978647,1.2929412415837394,0.22290692396408593,0.22159528393492708,0.2241383736708128,,,,,,
|
||||
TSFA-random,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_text_audio_text,0.8726010103709779,0.856940323413578,0.8882453329903173,0.15424833033806157,0.14485775852974697,0.16349338810652073,,,,,,
|
||||
TSFA-random,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,cycle_text_vision_text,0.9356607187725524,0.9246008385013482,0.9475927270482284,0.2136385162721102,0.20676204288370334,0.22075529878895422,,,,,,
|
||||
TSFA-random,,100,37,,,,,,,cycle_triangle,,,,,,,,,,,,,,,,,,,,,,,,,,triangle_TAV,,,,,,,0.2284731831664083,0.21969593281342203,0.2373126331123121,0.005839563997881332,0.005085714025060491,0.006556360621932
|
||||
|
Binary file not shown.
|
After Width: | Height: | Size: 186 KiB |
@@ -0,0 +1,43 @@
|
||||
{
|
||||
"seed": 42,
|
||||
"grid_size": 50,
|
||||
"hidden_size": 128,
|
||||
"heads_in_frozen_alignment": 4,
|
||||
"delta_normalized_time": 0.1,
|
||||
"hard_negative_offsets_slots": [
|
||||
-5,
|
||||
-3,
|
||||
-2,
|
||||
2,
|
||||
3,
|
||||
5
|
||||
],
|
||||
"semantic_epochs": 40,
|
||||
"semantic_learning_rate": 0.001,
|
||||
"local_contrastive_temperature": 0.1,
|
||||
"content_probe_epochs": 40,
|
||||
"content_probe_temperature": 0.1,
|
||||
"reconstruction_probe_epochs": 40,
|
||||
"shuffle_repeats": 20,
|
||||
"random_window_repeats": 20,
|
||||
"window_ablation_status": "delta sweep and 80-percent attention-mass window deferred until the seven-method MVP is reviewed",
|
||||
"semantic_query_key_inputs": "Frozen M4 attention-pooled text value output is the semantic query input; M4 pre-attention projected Audio/Vision content is the semantic key/value input. No explicit source-time code, absolute PE, slot ID, or latent positional vector is passed to semantic Q/K. Temporal selection can still encode time indirectly.",
|
||||
"baseline_content_values": "M4 and M3 Audio/Vision use each checkpoint's actual MHA attention output. M3 Text uses attention-pooled pre-attention text features to remove its source-time query residual.",
|
||||
"temporal_role": "Frozen M4 attention supplies candidate masks; TSFA-multiply additionally multiplies semantic probabilities by M4 temporal attention",
|
||||
"local_contrastive": "same-slot positives with within-clip slot-offset negatives at +/-2, +/-3, +/-5; symmetric Text-Audio and Text-Vision loss",
|
||||
"random_candidate": "same delta-width windows with independently sampled centers at inference; 20 held-out random draws, one train draw for the train-only probe",
|
||||
"global_candidate": "all valid source positions available to semantic attention",
|
||||
"content_probe": "same train-only symmetric within-clip InfoNCE probe applied to each method; held-out negatives are positions more than two slots away",
|
||||
"content_shuffle": "independently permute each modality's 50 projected content rows within each held-out clip; M4 and TSFA-main",
|
||||
"reconstruction": "fold-train decoder predicts one aligned content representation from the other two; shift one partner by signed offsets +/-1,2,5,10 on identical target support",
|
||||
"confidence_intervals": "95-percent video_id-cluster bootstrap; 2,000 repetitions for CSV summaries",
|
||||
"methods": [
|
||||
"M3_noSourceTime",
|
||||
"M3_sourceTime",
|
||||
"M4_sourceTime",
|
||||
"TSFA-main",
|
||||
"TSFA-multiply",
|
||||
"TSFA-random",
|
||||
"TSFA-global"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
method,modality,clip_count,video_id_count,mvr_epsilon_0_01_video_macro_mean,mvr_epsilon_0_01_ci95_low,mvr_epsilon_0_01_ci95_high,mvr_epsilon_0_02_video_macro_mean,mvr_epsilon_0_02_ci95_low,mvr_epsilon_0_02_ci95_high,mvr_epsilon_0_05_video_macro_mean,mvr_epsilon_0_05_ci95_low,mvr_epsilon_0_05_ci95_high,time_span_ratio_video_macro_mean,time_span_ratio_ci95_low,time_span_ratio_ci95_high,normalized_entropy_mean_video_macro_mean,normalized_entropy_mean_ci95_low,normalized_entropy_mean_ci95_high,source_coverage_rate_video_macro_mean,source_coverage_rate_ci95_low,source_coverage_rate_ci95_high
|
||||
M3_noSourceTime,audio,100,37,0.1523511850042463,0.12890654138103116,0.1775852091158214,0.11118027648639893,0.09314737478002783,0.13030062861695513,0.03717244941734738,0.026531859848186373,0.04761878496572374,0.006845225875929802,-0.004974720338403389,0.018367859269390833,0.9484751192262098,0.9413206608520659,0.9551741392240333,1.0,1.0,1.0
|
||||
M3_noSourceTime,text,100,37,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.8675032330277417,0.8521122687516515,0.8834943239261863,0.0013463322666157121,0.0,0.004038996799847136,1.0,1.0,1.0
|
||||
M3_noSourceTime,vision,100,37,0.19206480533011147,0.16843243024365476,0.21904841795147914,0.1745917929591399,0.15144481215909786,0.1997835736611246,0.1288914278710197,0.10981094432625045,0.14901438245826,-0.05248904935502326,-0.09002182426718092,-0.018003616275419482,0.806956850982112,0.7777992190084679,0.8363112281176653,1.0,1.0,1.0
|
||||
M3_sourceTime,audio,100,37,0.12775457673416857,0.10811011083970269,0.15060003151839885,0.11171039844509233,0.09435531741654192,0.1297671249967168,0.07801066372494944,0.06502721745068683,0.09146934835710345,0.7079631044011514,0.6658551697959926,0.7388692005361774,0.876785624402091,0.8714454313105274,0.8814951963640908,1.0,1.0,1.0
|
||||
M3_sourceTime,text,100,37,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.8675032330277417,0.8514982524616859,0.8832306376250671,0.0013463322666157121,0.0,0.004038996799847136,1.0,1.0,1.0
|
||||
M3_sourceTime,vision,100,37,0.12819977411814146,0.10739960076694771,0.152875036114832,0.11318563461420603,0.09480123709715546,0.13370256918726306,0.07447141018569589,0.062481876920652414,0.08859555984555983,0.7103498443247823,0.6770012181250491,0.7366984965947195,0.783255981912249,0.7721062307032261,0.7937545194100867,1.0,1.0,1.0
|
||||
M4_sourceTime,audio,100,37,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.8157060578090207,0.8144300662672872,0.8168691650761236,0.8461043286957146,0.8432685890105509,0.8494121036930785,1.0,1.0,1.0
|
||||
M4_sourceTime,text,100,37,0.0022181073201481363,0.0,0.005572452919391665,0.0003742809865258845,0.0,0.0009849499645418012,0.0,0.0,0.0,0.7210220398065448,0.7030382029586985,0.7392271515304444,0.6015413742024268,0.5867125625307114,0.6174784327556735,1.0,1.0,1.0
|
||||
M4_sourceTime,vision,100,37,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.8217648772803375,0.8191443827978445,0.8242396484969566,0.7198475360014294,0.7103728027872257,0.7303326531663908,1.0,1.0,1.0
|
||||
TSFA-global,audio,100,37,0.2408066959087367,0.21770717875309714,0.2635084793248058,0.14064997942548962,0.1231999054448034,0.157178588938793,0.03884773986814804,0.029588586399810893,0.04845730570220366,0.01135155110683782,-0.03307952783374511,0.05474875110063228,0.7009484166229463,0.6727335474417834,0.7300350355184457,1.0,1.0,1.0
|
||||
TSFA-global,text,100,37,0.0022181073201481363,0.0,0.005677251595618942,0.0003742809865258845,0.0,0.0009465369159246656,0.0,0.0,0.0,0.7210220398065448,0.7023545769979086,0.7389899179294055,0.6015413742024268,0.5875561023863175,0.6159531085332476,1.0,1.0,1.0
|
||||
TSFA-global,vision,100,37,0.13183358285399102,0.10105264885877131,0.16295599900191735,0.06452516656598291,0.043409021266164124,0.08763517891068912,0.017349565308748986,0.00888142515693536,0.027128378378378375,0.0076104858344335806,-0.01709729452706841,0.03299500925503682,0.902044345677258,0.8671364603137538,0.9334658138613803,1.0,1.0,1.0
|
||||
TSFA-main,audio,100,37,0.01236571848816747,0.007674894282037139,0.018037126047330123,0.0025122790428912874,0.0009413495127780841,0.004597779266146612,0.0,0.0,0.0,0.7959866389347421,0.7821903087901576,0.8092532415645509,0.5376040991299903,0.5180895213701402,0.5563726279490699,0.9991265907079654,0.9981073190065879,0.9998740929454016
|
||||
TSFA-main,text,100,37,0.0022181073201481363,0.0,0.005456622803561579,0.0003742809865258845,0.0,0.0009849499645418012,0.0,0.0,0.0,0.7210220398065448,0.7033838928902758,0.7389047563723402,0.6015413742024268,0.5877413970677293,0.6179347115827777,1.0,1.0,1.0
|
||||
TSFA-main,vision,100,37,0.0015050035458198724,0.0005253066477556274,0.0027079557691802587,0.00038610038610038604,0.0,0.0010479867622724766,0.0,0.0,0.0,0.8207314282427551,0.8149307422602344,0.8265787064699319,0.5102220348192638,0.48883208000021033,0.5325813152434384,0.998100693277735,0.9965188082802303,0.9993628555025613
|
||||
TSFA-multiply,audio,100,37,0.008198723504845955,0.004761904761904762,0.011943536101699364,0.002291650250833924,0.0006894649751792608,0.004294874320384522,0.0,0.0,0.0,0.8238465478704826,0.8092843768510277,0.8375431991484058,0.5374824091259056,0.5192627699574578,0.5557536982458872,0.9991146371356402,0.9980471424100044,0.9998737805676262
|
||||
TSFA-multiply,text,100,37,0.0022181073201481363,0.0,0.00556693719959026,0.0003742809865258845,0.0,0.0009849499645418012,0.0,0.0,0.0,0.7210220398065448,0.7032826453493484,0.7394791749401468,0.6015413742024268,0.5869907574297073,0.6161966829904635,1.0,1.0,1.0
|
||||
TSFA-multiply,vision,100,37,0.00011031439602868173,0.0,0.00033094318808604516,0.0,0.0,0.0,0.0,0.0,0.0,0.8564870598082609,0.8516115315250857,0.8619042260648179,0.5042486909404768,0.4827586923295134,0.5264450873938568,0.998100693277735,0.9964701640477626,0.9994824236000707
|
||||
TSFA-random,audio,100,37,0.4774993433666905,0.47491765489979776,0.4801365665957502,0.46498947197926793,0.46206071997653625,0.4679666873506158,0.43554342972710314,0.43223226105113854,0.43878350143146055,-0.016616960845384495,-0.03615306587688328,0.0035000853492482787,0.5331781007342047,0.5159025794331448,0.5498651267241725,0.9998828757861041,0.9998175312333477,0.9999458535193156
|
||||
TSFA-random,text,100,37,0.0022181073201481363,0.0,0.0058309037900874635,0.0003742809865258845,0.0,0.0009455519659601293,0.0,0.0,0.0,0.7210220398065448,0.7030686031936083,0.739263951757811,0.6015413742024268,0.5866909114370855,0.617601946399249,1.0,1.0,1.0
|
||||
TSFA-random,vision,100,37,0.484447464957669,0.482029311016556,0.486984872810128,0.47450730613995923,0.4721397031141929,0.477074162901969,0.4430162451080818,0.440237471436451,0.4457687994116566,0.009951817230022326,-0.00961620625239858,0.030069627676334455,0.49993764529426016,0.47863484579793103,0.5198129423960058,0.9997262801373744,0.9995674285759633,0.9998587347587298
|
||||
|
Binary file not shown.
|
After Width: | Height: | Size: 134 KiB |
Reference in New Issue
Block a user