Loading evaluate.py +2 −2 Original line number Diff line number Diff line Loading @@ -10,8 +10,8 @@ from pytrec_eval import RelevanceEvaluator, parse_qrel, parse_run RELEVANCE_JUDGEMENTS = { 'task1': 'qrel.V0.1.tsv', 'ntcir-11-math-2-main': 'NTCIR11_Math-qrels.dat', 'ntcir-12-mathir-arxiv-main': 'NTCIR12_Math-qrels_agg.dat', 'ntcir-11-math-2-main': 'NTCIR11_Math-qrels-test.dat', 'ntcir-12-mathir-arxiv-main': 'NTCIR12_Math-qrels_agg-test.dat', } TASK_README_HEAD = r''' This table contains the best result for every user. Loading ntcir-11-math-2-main/NTCIR11_Math-qrels-test.dat 0 → 100644 +500 −0 Original line number Diff line number Diff line NTCIR11-Math-41 xxx 0911.4293_1_52 2 NTCIR11-Math-41 xxx 1302.4087_1_18 2 NTCIR11-Math-41 xxx math-ph9908025_1_7 0 NTCIR11-Math-41 xxx 1201.1701_1_18 2 NTCIR11-Math-41 xxx math9912119_1_142 0 NTCIR11-Math-41 xxx 0711.3806_1_181 0 NTCIR11-Math-41 xxx 0906.2614_1_42 2 NTCIR11-Math-41 xxx math9908021_1_98 0 NTCIR11-Math-41 xxx 1008.4232_1_110 2 NTCIR11-Math-41 xxx 0906.0303_1_69 2 NTCIR11-Math-41 xxx 1203.3920_1_7 1 NTCIR11-Math-41 xxx 1105.4222_1_17 0 NTCIR11-Math-41 xxx 1211.5488_1_6 1 NTCIR11-Math-41 xxx 1206.2305_1_136 1 NTCIR11-Math-41 xxx 1102.3668_1_3 2 NTCIR11-Math-41 xxx 0804.2912_1_83 1 NTCIR11-Math-41 xxx 1302.4087_1_63 2 NTCIR11-Math-41 xxx 1001.1830_1_32 1 NTCIR11-Math-41 xxx 1109.5309_1_41 2 NTCIR11-Math-41 xxx hep-th0603207_1_43 0 NTCIR11-Math-41 xxx 1111.4845_1_23 2 NTCIR11-Math-41 xxx 1003.4765_1_83 3 NTCIR11-Math-41 xxx math0506127_1_45 2 NTCIR11-Math-41 xxx 0804.2912_1_85 1 NTCIR11-Math-41 xxx 1210.2087_1_197 2 NTCIR11-Math-41 xxx 1206.6567_1_37 2 NTCIR11-Math-41 xxx hep-th0603207_1_15 0 NTCIR11-Math-41 xxx 1006.5861_1_188 2 NTCIR11-Math-41 xxx 0903.0022_1_68 2 NTCIR11-Math-41 xxx 0804.0321_1_76 2 NTCIR11-Math-41 xxx math9908056_1_212 0 NTCIR11-Math-41 xxx math0607549_1_54 2 NTCIR11-Math-41 xxx 0704.0046_1_20 2 NTCIR11-Math-41 xxx 0809.4380_1_44 2 NTCIR11-Math-41 xxx 1311.7385_1_49 4 NTCIR11-Math-41 xxx 1305.7170_1_56 1 NTCIR11-Math-41 xxx 0810.2821_1_24 2 NTCIR11-Math-41 xxx 0810.2821_1_26 2 NTCIR11-Math-41 xxx 0810.2821_1_28 2 NTCIR11-Math-41 xxx 1110.4329_1_167 1 NTCIR11-Math-41 xxx 0711.3806_1_156 0 NTCIR11-Math-41 xxx 1105.3137_1_67 1 NTCIR11-Math-41 xxx 1005.1893_1_23 2 NTCIR11-Math-41 xxx math9911026_1_315 0 NTCIR11-Math-41 xxx 0810.2821_1_6 2 NTCIR11-Math-41 xxx 1112.4968_1_50 1 NTCIR11-Math-41 xxx hep-th0603207_1_52 0 NTCIR11-Math-41 xxx hep-th0603207_1_36 0 NTCIR11-Math-41 xxx 1105.4274_1_196 2 NTCIR11-Math-41 xxx 0810.2821_1_42 2 NTCIR11-Math-42 xxx math0201313_1_90 4 NTCIR11-Math-42 xxx 1010.1620_1_27 2 NTCIR11-Math-42 xxx 1105.2476_1_18 3 NTCIR11-Math-42 xxx math9909151_1_21 2 NTCIR11-Math-42 xxx math9703218_1_300 0 NTCIR11-Math-42 xxx math9909036_1_111 1 NTCIR11-Math-42 xxx math-ph0105015_1_21 2 NTCIR11-Math-42 xxx 1210.7436_1_1 1 NTCIR11-Math-42 xxx math0201314_1_20 4 NTCIR11-Math-42 xxx math0605172_1_326 2 NTCIR11-Math-42 xxx 1309.6552_1_1 0 NTCIR11-Math-42 xxx 0809.3317_1_84 1 NTCIR11-Math-42 xxx hep-th0703106_1_56 1 NTCIR11-Math-42 xxx 1009.5153_1_17 4 NTCIR11-Math-42 xxx math9908144_1_31 0 NTCIR11-Math-42 xxx math0507519_1_106 0 NTCIR11-Math-42 xxx 1108.1660_1_5 2 NTCIR11-Math-42 xxx math0212378_1_77 3 NTCIR11-Math-42 xxx 1202.0085_1_7 0 NTCIR11-Math-42 xxx 1108.5286_1_172 1 NTCIR11-Math-42 xxx 1211.4402_1_28 1 NTCIR11-Math-42 xxx 1201.0389_1_19 2 NTCIR11-Math-42 xxx 1205.3957_1_44 0 NTCIR11-Math-42 xxx 1211.4402_1_20 1 NTCIR11-Math-42 xxx 1209.6278_1_41 2 NTCIR11-Math-42 xxx 1211.4402_1_24 1 NTCIR11-Math-42 xxx 1212.4205_1_121 0 NTCIR11-Math-42 xxx 1209.6278_1_46 2 NTCIR11-Math-42 xxx hep-lat0001002_1_7 0 NTCIR11-Math-42 xxx hep-lat0001002_1_6 0 NTCIR11-Math-42 xxx hep-lat0001002_1_5 1 NTCIR11-Math-42 xxx math9909036_1_48 2 NTCIR11-Math-42 xxx 0705.0341_1_87 2 NTCIR11-Math-42 xxx 1206.0367_1_23 1 NTCIR11-Math-42 xxx physics9908014_1_40 0 NTCIR11-Math-42 xxx 1304.1245_1_28 0 NTCIR11-Math-42 xxx hep-lat0001002_1_9 1 NTCIR11-Math-42 xxx math0510364_1_127 2 NTCIR11-Math-42 xxx 1307.3810_1_1 1 NTCIR11-Math-42 xxx 0908.2797_1_13 2 NTCIR11-Math-42 xxx physics9908014_1_38 0 NTCIR11-Math-42 xxx 1104.2010_1_7 1 NTCIR11-Math-42 xxx math9909036_1_145 1 NTCIR11-Math-42 xxx 1106.1759_1_18 2 NTCIR11-Math-42 xxx math9912114_1_35 1 NTCIR11-Math-42 xxx 1206.0694_1_618 1 NTCIR11-Math-42 xxx math9909036_1_102 0 NTCIR11-Math-42 xxx 0911.2140_1_16 0 NTCIR11-Math-42 xxx 1002.1468_1_120 2 NTCIR11-Math-42 xxx math0302119_1_40 2 NTCIR11-Math-43 xxx 1009.3496_1_13 0 NTCIR11-Math-43 xxx math-ph9910031_1_41 0 NTCIR11-Math-43 xxx 0810.5666_1_29 0 NTCIR11-Math-43 xxx hep-th9804086_1_3 0 NTCIR11-Math-43 xxx hep-th9804086_1_2 1 NTCIR11-Math-43 xxx hep-th9802130_1_3 1 NTCIR11-Math-43 xxx 0904.1757_1_10 1 NTCIR11-Math-43 xxx hep-th0004105_1_1 1 NTCIR11-Math-43 xxx 1304.5883_1_10 1 NTCIR11-Math-43 xxx 1208.5230_1_103 0 NTCIR11-Math-43 xxx hep-th9305001_1_34 1 NTCIR11-Math-43 xxx 0706.4428_1_84 0 NTCIR11-Math-43 xxx hep-th9802130_1_2 0 NTCIR11-Math-43 xxx math9908104_1_287 0 NTCIR11-Math-43 xxx math0509073_1_104 0 NTCIR11-Math-43 xxx 0904.3524_1_91 3 NTCIR11-Math-43 xxx hep-th0510122_1_9 1 NTCIR11-Math-43 xxx 1303.3305_1_20 0 NTCIR11-Math-43 xxx 1109.5228_1_76 1 NTCIR11-Math-43 xxx 1104.5492_1_11 0 NTCIR11-Math-43 xxx math0301096_1_33 0 NTCIR11-Math-43 xxx math-ph0111035_1_18 2 NTCIR11-Math-43 xxx math-ph0609045_1_104 0 NTCIR11-Math-43 xxx 1111.1222_1_7 0 NTCIR11-Math-43 xxx 1112.0591_1_33 1 NTCIR11-Math-43 xxx 1112.0591_1_34 1 NTCIR11-Math-43 xxx math-ph0111035_1_16 2 NTCIR11-Math-43 xxx cs9909009_1_110 0 NTCIR11-Math-43 xxx math9912016_1_201 0 NTCIR11-Math-43 xxx 1301.2743_1_2 2 NTCIR11-Math-43 xxx 1308.2455_1_8 1 NTCIR11-Math-43 xxx math0203254_1_51 0 NTCIR11-Math-43 xxx hep-th0101139_1_13 2 NTCIR11-Math-43 xxx 1009.3308_1_51 0 NTCIR11-Math-43 xxx 0805.3849_1_114 1 NTCIR11-Math-43 xxx 0711.1783_1_35 4 NTCIR11-Math-43 xxx 1108.5094_1_12 4 NTCIR11-Math-43 xxx 0711.1783_1_33 3 NTCIR11-Math-43 xxx 0704.1401_1_107 0 NTCIR11-Math-43 xxx math9908104_1_294 0 NTCIR11-Math-43 xxx 1109.5228_1_68 0 NTCIR11-Math-43 xxx 0904.4332_1_19 0 NTCIR11-Math-43 xxx 1201.3768_1_50 1 NTCIR11-Math-43 xxx math0302011_1_122 0 NTCIR11-Math-43 xxx math0203291_1_4 0 NTCIR11-Math-43 xxx solv-int9606005_1_5 1 NTCIR11-Math-43 xxx 0810.1737_1_28 0 NTCIR11-Math-43 xxx hep-th0401109_1_87 3 NTCIR11-Math-43 xxx math-ph0111035_1_17 2 NTCIR11-Math-43 xxx hep-th9804186_1_22 1 NTCIR11-Math-44 xxx math0410360_1_35 0 NTCIR11-Math-44 xxx 1112.3862_1_8 2 NTCIR11-Math-44 xxx hep-th0410023_1_28 0 NTCIR11-Math-44 xxx math0309472_1_62 0 NTCIR11-Math-44 xxx cond-mat0004018_1_11 0 NTCIR11-Math-44 xxx 1010.2484_1_4 1 NTCIR11-Math-44 xxx 0904.1757_1_10 0 NTCIR11-Math-44 xxx 0804.1485_1_37 0 NTCIR11-Math-44 xxx chao-dyn9909001_1_105 4 NTCIR11-Math-44 xxx 1202.1539_1_62 0 NTCIR11-Math-44 xxx 1202.1628_1_44 0 NTCIR11-Math-44 xxx 0710.1628_1_17 0 NTCIR11-Math-44 xxx math0110086_1_157 4 NTCIR11-Math-44 xxx 1005.1528_1_2 3 NTCIR11-Math-44 xxx 1106.5530_1_66 0 NTCIR11-Math-44 xxx 1109.0956_1_129 2 NTCIR11-Math-44 xxx math0211301_1_17 3 NTCIR11-Math-44 xxx cond-mat0009019_1_71 0 NTCIR11-Math-44 xxx 0711.1306_1_127 0 NTCIR11-Math-44 xxx 0811.2477_1_18 1 NTCIR11-Math-44 xxx 0712.3951_1_29 2 NTCIR11-Math-44 xxx 1311.0717_1_1 3 NTCIR11-Math-44 xxx math-ph0112016_1_6 1 NTCIR11-Math-44 xxx 1112.3862_1_31 2 NTCIR11-Math-44 xxx math-ph0112016_1_3 0 NTCIR11-Math-44 xxx gr-qc9807060_1_14 0 NTCIR11-Math-44 xxx 0707.1946_1_105 0 NTCIR11-Math-44 xxx 1308.1628_1_111 0 NTCIR11-Math-44 xxx math0604527_1_262 0 NTCIR11-Math-44 xxx math-ph0112016_1_16 1 NTCIR11-Math-44 xxx math0504257_1_26 0 NTCIR11-Math-44 xxx math-ph0112016_1_10 1 NTCIR11-Math-44 xxx 1301.3128_1_49 0 NTCIR11-Math-44 xxx math0210330_1_3 4 NTCIR11-Math-44 xxx math0605227_1_1 3 NTCIR11-Math-44 xxx math9910043_1_18 0 NTCIR11-Math-44 xxx hep-th0001053_1_10 0 NTCIR11-Math-44 xxx hep-th0001222_1_129 0 NTCIR11-Math-44 xxx math0010221_1_48 0 NTCIR11-Math-44 xxx math9910116_1_366 0 NTCIR11-Math-44 xxx hep-th0610118_1_27 0 NTCIR11-Math-44 xxx nlin0012025_1_20 0 NTCIR11-Math-44 xxx 0705.4608_1_11 0 NTCIR11-Math-44 xxx 0707.1946_1_383 0 NTCIR11-Math-44 xxx hep-th9909201_1_77 0 NTCIR11-Math-44 xxx 1309.2605_1_1 2 NTCIR11-Math-44 xxx math9911126_1_117 0 NTCIR11-Math-44 xxx math0404335_1_160 4 NTCIR11-Math-44 xxx math-ph0609031_1_6 0 NTCIR11-Math-44 xxx 1009.2166_1_11 0 NTCIR11-Math-45 xxx gr-qc0201031_1_25 0 NTCIR11-Math-45 xxx 1211.3234_1_76 2 NTCIR11-Math-45 xxx hep-ph0004008_1_77 0 NTCIR11-Math-45 xxx 1211.3234_1_53 3 NTCIR11-Math-45 xxx math0608187_1_3 4 NTCIR11-Math-45 xxx math9909131_1_12 0 NTCIR11-Math-45 xxx math-ph0008030_1_41 0 NTCIR11-Math-45 xxx math9908088_1_20 1 NTCIR11-Math-45 xxx math0006205_1_77 0 NTCIR11-Math-45 xxx hep-th9910213_1_7 0 NTCIR11-Math-45 xxx 0901.3075_1_18 4 NTCIR11-Math-45 xxx 1212.1300_1_25 0 NTCIR11-Math-45 xxx hep-th9210064_1_31 0 NTCIR11-Math-45 xxx math0211110_1_111 0 NTCIR11-Math-45 xxx 1303.2526_1_51 4 NTCIR11-Math-45 xxx hep-ph0004008_1_12 0 NTCIR11-Math-45 xxx hep-th0001077_1_4 0 NTCIR11-Math-45 xxx 1210.5561_1_55 2 NTCIR11-Math-45 xxx hep-th9911216_1_178 0 NTCIR11-Math-45 xxx math0410550_1_2 4 NTCIR11-Math-45 xxx 1205.0371_1_25 1 NTCIR11-Math-45 xxx 1205.0371_1_26 1 NTCIR11-Math-45 xxx 1210.6925_1_80 4 NTCIR11-Math-45 xxx math0405048_1_2 3 NTCIR11-Math-45 xxx 0811.4652_1_14 0 NTCIR11-Math-45 xxx 0811.4652_1_13 2 NTCIR11-Math-45 xxx math0404181_1_5 0 NTCIR11-Math-45 xxx 0811.4652_1_10 0 NTCIR11-Math-45 xxx 1005.1231_1_3 1 NTCIR11-Math-45 xxx 1005.1231_1_2 4 NTCIR11-Math-45 xxx 1211.0699_1_92 3 NTCIR11-Math-45 xxx 0802.2806_1_106 0 NTCIR11-Math-45 xxx 1007.2676_1_3 2 NTCIR11-Math-45 xxx 1212.6221_1_58 2 NTCIR11-Math-45 xxx math-ph0008030_1_38 0 NTCIR11-Math-45 xxx hep-th0001077_1_10 0 NTCIR11-Math-45 xxx math0406323_1_6 0 NTCIR11-Math-45 xxx math0209178_1_43 0 NTCIR11-Math-45 xxx math0406323_1_8 0 NTCIR11-Math-45 xxx hep-th9602040_1_10 0 NTCIR11-Math-45 xxx 1302.4302_1_3 1 NTCIR11-Math-45 xxx 0805.2707_1_4 3 NTCIR11-Math-45 xxx 0805.2707_1_1 3 NTCIR11-Math-45 xxx math9908088_1_5 0 NTCIR11-Math-45 xxx 1107.2710_1_18 0 NTCIR11-Math-45 xxx 0909.1273_1_32 4 NTCIR11-Math-45 xxx 0902.3467_1_62 0 NTCIR11-Math-45 xxx math0210262_1_19 1 NTCIR11-Math-45 xxx 1307.2080_1_91 0 NTCIR11-Math-45 xxx math0312014_1_42 2 NTCIR11-Math-46 xxx math0609071_1_1 0 NTCIR11-Math-46 xxx 0707.2563_1_6 0 NTCIR11-Math-46 xxx 1303.2770_1_365 3 NTCIR11-Math-46 xxx 1202.4355_1_9 0 NTCIR11-Math-46 xxx 1005.1726_1_28 3 NTCIR11-Math-46 xxx 0709.3803_1_116 0 NTCIR11-Math-46 xxx 0906.5557_1_420 1 NTCIR11-Math-46 xxx gr-qc9909001_1_70 0 NTCIR11-Math-46 xxx nlin0205028_1_25 0 NTCIR11-Math-46 xxx hep-th9911067_1_211 0 NTCIR11-Math-46 xxx hep-th9911231_1_50 0 NTCIR11-Math-46 xxx math9909074_1_79 0 NTCIR11-Math-46 xxx math0410360_1_35 0 NTCIR11-Math-46 xxx hep-th9406180_1_85 0 NTCIR11-Math-46 xxx hep-th9406180_1_117 0 NTCIR11-Math-46 xxx 0706.0612_1_8 0 NTCIR11-Math-46 xxx hep-th9910080_1_36 1 NTCIR11-Math-46 xxx math9909074_1_78 0 NTCIR11-Math-46 xxx 0712.0747_1_15 0 NTCIR11-Math-46 xxx 0906.4253_1_68 0 NTCIR11-Math-46 xxx 1110.1116_1_103 0 NTCIR11-Math-46 xxx hep-th9412240_1_104 0 NTCIR11-Math-46 xxx 1002.4332_1_37 1 NTCIR11-Math-46 xxx 1401.2932_1_127 0 NTCIR11-Math-46 xxx 1212.2910_1_65 3 NTCIR11-Math-46 xxx 1110.1116_1_126 0 NTCIR11-Math-46 xxx 0906.4253_1_72 0 NTCIR11-Math-46 xxx hep-th0506211_1_59 0 NTCIR11-Math-46 xxx 1303.2770_1_334 3 NTCIR11-Math-46 xxx 1110.1116_1_163 0 NTCIR11-Math-46 xxx 1006.2428_1_14 0 NTCIR11-Math-46 xxx 1011.1754_1_124 0 NTCIR11-Math-46 xxx 0809.2865_1_13 0 NTCIR11-Math-46 xxx 1303.5475_1_47 2 NTCIR11-Math-46 xxx 1212.2910_1_53 3 NTCIR11-Math-46 xxx math9808104_1_348 0 NTCIR11-Math-46 xxx 1305.2567_1_1 0 NTCIR11-Math-46 xxx 1303.2770_1_354 3 NTCIR11-Math-46 xxx 0707.2563_1_27 0 NTCIR11-Math-46 xxx 0707.2563_1_25 0 NTCIR11-Math-46 xxx 1303.2770_1_374 3 NTCIR11-Math-46 xxx hep-th9406180_1_92 0 NTCIR11-Math-46 xxx 0802.3529_1_1 1 NTCIR11-Math-46 xxx math0504491_1_98 0 NTCIR11-Math-46 xxx 1106.2944_1_60 2 NTCIR11-Math-46 xxx hep-th9406180_1_71 0 NTCIR11-Math-46 xxx 1303.2770_1_383 3 NTCIR11-Math-46 xxx 1001.4216_1_79 3 NTCIR11-Math-46 xxx 0811.0081_1_200 4 NTCIR11-Math-46 xxx 1005.1726_1_108 4 NTCIR11-Math-47 xxx math9909075_1_129 0 NTCIR11-Math-47 xxx 1007.1319_1_254 0 NTCIR11-Math-47 xxx math0303138_1_10 2 NTCIR11-Math-47 xxx math0001190_1_13 1 NTCIR11-Math-47 xxx 1002.1229_1_82 4 NTCIR11-Math-47 xxx math0404535_1_27 1 NTCIR11-Math-47 xxx 1305.0363_1_59 0 NTCIR11-Math-47 xxx math0312316_1_21 0 NTCIR11-Math-47 xxx 1007.1319_1_46 0 NTCIR11-Math-47 xxx math0303138_1_6 0 NTCIR11-Math-47 xxx 1102.0659_1_11 3 NTCIR11-Math-47 xxx hep-lat9903022_1_34 0 NTCIR11-Math-47 xxx 1108.5164_1_5 0 NTCIR11-Math-47 xxx 0801.1898_1_62 0 NTCIR11-Math-47 xxx 0904.2672_1_22 2 NTCIR11-Math-47 xxx hep-ph9912505_1_11 0 NTCIR11-Math-47 xxx math0602672_1_87 4 NTCIR11-Math-47 xxx math0508546_1_6 4 NTCIR11-Math-47 xxx math9908131_1_25 0 NTCIR11-Math-47 xxx 0811.0044_1_5 4 NTCIR11-Math-47 xxx math0203226_1_179 4 NTCIR11-Math-47 xxx hep-th9406147_1_57 1 NTCIR11-Math-47 xxx math0303138_1_11 1 NTCIR11-Math-47 xxx 1305.6970_1_80 3 NTCIR11-Math-47 xxx gr-qc9912067_1_161 0 NTCIR11-Math-47 xxx 0907.4431_1_7 0 NTCIR11-Math-47 xxx 0905.0941_1_13 3 NTCIR11-Math-47 xxx 1303.7466_1_99 4 NTCIR11-Math-47 xxx hep-ph9911463_1_71 0 NTCIR11-Math-47 xxx 1204.4908_1_282 0 NTCIR11-Math-47 xxx math0003014_1_53 0 NTCIR11-Math-47 xxx 1112.1183_1_28 0 NTCIR11-Math-47 xxx hep-ph9905249_1_52 1 NTCIR11-Math-47 xxx math0303138_1_12 0 NTCIR11-Math-47 xxx math0303138_1_13 1 NTCIR11-Math-47 xxx 1007.1319_1_192 0 NTCIR11-Math-47 xxx hep-th0507128_1_36 0 NTCIR11-Math-47 xxx 0901.3075_1_25 3 NTCIR11-Math-47 xxx math0510417_1_18 3 NTCIR11-Math-47 xxx 1203.6264_1_63 4 NTCIR11-Math-47 xxx math0510417_1_19 4 NTCIR11-Math-47 xxx math9910116_1_19 0 NTCIR11-Math-47 xxx 1206.3190_1_113 0 NTCIR11-Math-47 xxx math9908154_1_129 0 NTCIR11-Math-47 xxx math0702349_1_305 0 NTCIR11-Math-47 xxx 0911.1720_1_52 0 NTCIR11-Math-47 xxx 1204.4910_1_207 0 NTCIR11-Math-47 xxx math0001190_1_8 1 NTCIR11-Math-47 xxx 1102.3640_1_44 0 NTCIR11-Math-47 xxx hep-th9304158_1_40 1 NTCIR11-Math-48 xxx math0511112_1_1 3 NTCIR11-Math-48 xxx 0808.3051_1_26 0 NTCIR11-Math-48 xxx math0511112_1_46 1 NTCIR11-Math-48 xxx 1307.5799_1_5 1 NTCIR11-Math-48 xxx 1204.1080_1_17 1 NTCIR11-Math-48 xxx 0909.1001_1_2 1 NTCIR11-Math-48 xxx math-ph9911031_1_28 0 NTCIR11-Math-48 xxx math0509718_1_43 4 NTCIR11-Math-48 xxx 1210.4532_1_2 1 NTCIR11-Math-48 xxx 0811.1849_1_3 1 NTCIR11-Math-48 xxx alg-geom9402011_1_4 1 NTCIR11-Math-48 xxx math0108133_1_78 1 NTCIR11-Math-48 xxx 0903.3325_1_5 1 NTCIR11-Math-48 xxx 1106.4075_1_24 0 NTCIR11-Math-48 xxx math-ph0607041_1_54 1 NTCIR11-Math-48 xxx math-ph0607041_1_55 1 NTCIR11-Math-48 xxx math9908074_1_6 0 NTCIR11-Math-48 xxx math0403180_1_39 0 NTCIR11-Math-48 xxx 0902.2332_1_28 1 NTCIR11-Math-48 xxx 0908.3332_1_40 1 NTCIR11-Math-48 xxx 0908.3332_1_43 0 NTCIR11-Math-48 xxx 0908.3332_1_20 1 NTCIR11-Math-48 xxx 0902.2332_1_20 1 NTCIR11-Math-48 xxx math0511112_1_4 1 NTCIR11-Math-48 xxx 0902.2332_1_24 1 NTCIR11-Math-48 xxx 0710.0862_1_23 4 NTCIR11-Math-48 xxx 0902.2332_1_18 0 NTCIR11-Math-48 xxx 1211.6173_1_130 1 NTCIR11-Math-48 xxx 0908.3334_1_8 1 NTCIR11-Math-48 xxx 0908.3334_1_9 1 NTCIR11-Math-48 xxx 1010.2241_1_96 2 NTCIR11-Math-48 xxx gr-qc9909061_1_24 0 NTCIR11-Math-48 xxx 0908.3332_1_7 1 NTCIR11-Math-48 xxx 0801.3888_1_5 2 NTCIR11-Math-48 xxx 1102.3992_1_7 1 NTCIR11-Math-48 xxx math0612834_1_97 1 NTCIR11-Math-48 xxx 0806.1593_1_5 1 NTCIR11-Math-48 xxx hep-th9908113_1_53 0 NTCIR11-Math-48 xxx 0903.1337_1_4 0 NTCIR11-Math-48 xxx 0902.2332_1_35 1 NTCIR11-Math-48 xxx 1210.6552_1_66 1 NTCIR11-Math-48 xxx 1309.3112_1_12 1 NTCIR11-Math-48 xxx 1002.2900_1_7 1 NTCIR11-Math-48 xxx math9904153_1_26 1 NTCIR11-Math-48 xxx 1310.5289_1_52 1 NTCIR11-Math-48 xxx hep-th9908113_1_55 0 NTCIR11-Math-48 xxx hep-th9908113_1_50 1 NTCIR11-Math-48 xxx 0905.3229_1_10 2 NTCIR11-Math-48 xxx 0902.2332_1_10 2 NTCIR11-Math-48 xxx 1302.3077_1_8 1 NTCIR11-Math-49 xxx 1002.3875_1_53 0 NTCIR11-Math-49 xxx gr-qc9909004_1_18 0 NTCIR11-Math-49 xxx 1303.5188_1_42 1 NTCIR11-Math-49 xxx 0902.1199_1_57 0 NTCIR11-Math-49 xxx 0811.4287_1_27 1 NTCIR11-Math-49 xxx hep-th0408105_1_109 0 NTCIR11-Math-49 xxx math9912104_1_113 0 NTCIR11-Math-49 xxx 1210.6400_1_11 1 NTCIR11-Math-49 xxx 1210.6400_1_12 0 NTCIR11-Math-49 xxx 1210.8173_1_30 0 NTCIR11-Math-49 xxx 0710.1699_1_159 0 NTCIR11-Math-49 xxx math9911098_1_5 0 NTCIR11-Math-49 xxx math0512271_1_34 2 NTCIR11-Math-49 xxx math0307329_1_173 0 NTCIR11-Math-49 xxx math0409372_1_108 1 NTCIR11-Math-49 xxx 1303.5188_1_4 2 NTCIR11-Math-49 xxx 1004.3374_1_63 0 NTCIR11-Math-49 xxx 1005.4299_1_84 1 NTCIR11-Math-49 xxx 1006.4323_1_16 1 NTCIR11-Math-49 xxx 0811.4287_1_70 0 NTCIR11-Math-49 xxx 0907.2375_1_138 0 NTCIR11-Math-49 xxx 1002.3875_1_47 1 NTCIR11-Math-49 xxx cond-mat0703214_1_247 0 NTCIR11-Math-49 xxx 1301.2441_1_60 0 NTCIR11-Math-49 xxx hep-th0203086_1_17 0 NTCIR11-Math-49 xxx math-ph0207011_1_10 0 NTCIR11-Math-49 xxx 0706.2735_1_9 2 NTCIR11-Math-49 xxx 1208.0971_1_23 1 NTCIR11-Math-49 xxx 1002.3875_1_48 0 NTCIR11-Math-49 xxx 1210.8173_1_29 1 NTCIR11-Math-49 xxx 1102.2298_1_80 0 NTCIR11-Math-49 xxx math-ph0207011_1_43 2 NTCIR11-Math-49 xxx 1009.4619_1_6 0 NTCIR11-Math-49 xxx 1105.5933_1_109 0 NTCIR11-Math-49 xxx 0711.0780_1_30 0 NTCIR11-Math-49 xxx 1007.0353_1_19 1 NTCIR11-Math-49 xxx hep-ph0312058_1_10 0 NTCIR11-Math-49 xxx 0901.4044_1_50 0 NTCIR11-Math-49 xxx math9910116_1_173 0 NTCIR11-Math-49 xxx 0901.4044_1_52 0 NTCIR11-Math-49 xxx hep-th9910129_1_45 0 NTCIR11-Math-49 xxx quant-ph0003107_1_3 1 NTCIR11-Math-49 xxx 1401.3279_1_17 0 NTCIR11-Math-49 xxx 1303.5188_1_17 2 NTCIR11-Math-49 xxx 1108.3151_1_57 0 NTCIR11-Math-49 xxx math9911019_1_95 0 NTCIR11-Math-49 xxx 1305.6688_1_8 0 NTCIR11-Math-49 xxx solv-int9606005_1_5 0 NTCIR11-Math-49 xxx 0903.2289_1_173 0 NTCIR11-Math-49 xxx 1001.1366_1_32 0 NTCIR11-Math-50 xxx 0708.2956_1_15 4 NTCIR11-Math-50 xxx cs0309011_1_9 2 NTCIR11-Math-50 xxx dg-ga9605004_1_198 0 NTCIR11-Math-50 xxx 0810.2697_1_66 1 NTCIR11-Math-50 xxx math-ph9910013_1_172 0 NTCIR11-Math-50 xxx hep-th9402033_1_7 0 NTCIR11-Math-50 xxx 1205.6869_1_155 2 NTCIR11-Math-50 xxx 1105.4940_1_21 4 NTCIR11-Math-50 xxx hep-th9702026_1_95 0 NTCIR11-Math-50 xxx 1001.1231_1_13 4 NTCIR11-Math-50 xxx 1005.2889_1_54 0 NTCIR11-Math-50 xxx math-ph9910013_1_182 0 NTCIR11-Math-50 xxx math9911055_1_110 0 NTCIR11-Math-50 xxx 1206.1535_1_36 2 NTCIR11-Math-50 xxx 1206.1535_1_52 3 NTCIR11-Math-50 xxx 1206.1535_1_54 3 NTCIR11-Math-50 xxx 1102.1169_1_9 4 NTCIR11-Math-50 xxx math-ph0311054_1_47 0 NTCIR11-Math-50 xxx hep-ph9306286_1_23 0 NTCIR11-Math-50 xxx 1308.4251_1_69 0 NTCIR11-Math-50 xxx hep-ph9911307_1_204 0 NTCIR11-Math-50 xxx 1011.3376_1_3 3 NTCIR11-Math-50 xxx 1208.2315_1_6 4 NTCIR11-Math-50 xxx 1303.4580_1_7 2 NTCIR11-Math-50 xxx 1208.2315_1_3 4 NTCIR11-Math-50 xxx 1309.2352_1_261 0 NTCIR11-Math-50 xxx 1110.2235_1_66 0 NTCIR11-Math-50 xxx 1110.2235_1_67 0 NTCIR11-Math-50 xxx 1205.6869_1_127 2 NTCIR11-Math-50 xxx 1106.1110_1_102 4 NTCIR11-Math-50 xxx math9911055_1_118 0 NTCIR11-Math-50 xxx 1205.6869_1_1 2 NTCIR11-Math-50 xxx 1208.2315_1_9 3 NTCIR11-Math-50 xxx 1301.1521_1_27 4 NTCIR11-Math-50 xxx 1205.6869_1_103 2 NTCIR11-Math-50 xxx 1011.3376_1_23 3 NTCIR11-Math-50 xxx 1206.1535_1_62 4 NTCIR11-Math-50 xxx 0805.3789_1_54 0 NTCIR11-Math-50 xxx 1001.2157_1_182 0 NTCIR11-Math-50 xxx 1206.1535_1_46 2 NTCIR11-Math-50 xxx math9911118_1_77 0 NTCIR11-Math-50 xxx 1206.1535_1_44 2 NTCIR11-Math-50 xxx 1209.2471_1_4 4 NTCIR11-Math-50 xxx 1005.3044_1_56 0 NTCIR11-Math-50 xxx 1206.1535_1_24 2 NTCIR11-Math-50 xxx 1209.2471_1_1 2 NTCIR11-Math-50 xxx 1001.1231_1_154 2 NTCIR11-Math-50 xxx 1001.1231_1_156 4 NTCIR11-Math-50 xxx 1102.1169_1_13 4 NTCIR11-Math-50 xxx 1001.1231_1_152 2 ntcir-11-math-2-main/NTCIR11_Math-qrels.dat→ntcir-11-math-2-main/NTCIR11_Math-qrels-train.dat +0 −500 File changed and moved.Preview size limit exceeded, changes collapsed. Show changes ntcir-11-math-2-main/README.md +4 −4 Original line number Diff line number Diff line Loading @@ -2,7 +2,7 @@ This table contains the best result for every user. | nDCG | User | Result name | |:-----|------|:------------| | 0.5499 | xstefan3 | example, key1=value1, key2=value2, etc | | 0.5499 | xnovot32 | example, key1=value1, key2=value2, etc | | 0.5499 | xluptak4 | example, key1=value1, key2=value2, etc | | 0.5499 | ayetiran | example, key1=value1, key2=value2, etc | | 0.5600 | xstefan3 | example, key1=value1, key2=value2, etc | | 0.5600 | xnovot32 | example, key1=value1, key2=value2, etc | | 0.5600 | xluptak4 | example, key1=value1, key2=value2, etc | | 0.5600 | ayetiran | example, key1=value1, key2=value2, etc | ntcir-11-math-2-main/ayetiran/README.md +1 −1 Original line number Diff line number Diff line Loading @@ -4,4 +4,4 @@ underscores (`_`) replaced with a comma and a space for improved readability. | nDCG | Result name | |------|:------------| | 0.5499 | example, key1=value1, key2=value2, etc | | 0.5600 | example, key1=value1, key2=value2, etc | Loading
evaluate.py +2 −2 Original line number Diff line number Diff line Loading @@ -10,8 +10,8 @@ from pytrec_eval import RelevanceEvaluator, parse_qrel, parse_run RELEVANCE_JUDGEMENTS = { 'task1': 'qrel.V0.1.tsv', 'ntcir-11-math-2-main': 'NTCIR11_Math-qrels.dat', 'ntcir-12-mathir-arxiv-main': 'NTCIR12_Math-qrels_agg.dat', 'ntcir-11-math-2-main': 'NTCIR11_Math-qrels-test.dat', 'ntcir-12-mathir-arxiv-main': 'NTCIR12_Math-qrels_agg-test.dat', } TASK_README_HEAD = r''' This table contains the best result for every user. Loading
ntcir-11-math-2-main/NTCIR11_Math-qrels-test.dat 0 → 100644 +500 −0 Original line number Diff line number Diff line NTCIR11-Math-41 xxx 0911.4293_1_52 2 NTCIR11-Math-41 xxx 1302.4087_1_18 2 NTCIR11-Math-41 xxx math-ph9908025_1_7 0 NTCIR11-Math-41 xxx 1201.1701_1_18 2 NTCIR11-Math-41 xxx math9912119_1_142 0 NTCIR11-Math-41 xxx 0711.3806_1_181 0 NTCIR11-Math-41 xxx 0906.2614_1_42 2 NTCIR11-Math-41 xxx math9908021_1_98 0 NTCIR11-Math-41 xxx 1008.4232_1_110 2 NTCIR11-Math-41 xxx 0906.0303_1_69 2 NTCIR11-Math-41 xxx 1203.3920_1_7 1 NTCIR11-Math-41 xxx 1105.4222_1_17 0 NTCIR11-Math-41 xxx 1211.5488_1_6 1 NTCIR11-Math-41 xxx 1206.2305_1_136 1 NTCIR11-Math-41 xxx 1102.3668_1_3 2 NTCIR11-Math-41 xxx 0804.2912_1_83 1 NTCIR11-Math-41 xxx 1302.4087_1_63 2 NTCIR11-Math-41 xxx 1001.1830_1_32 1 NTCIR11-Math-41 xxx 1109.5309_1_41 2 NTCIR11-Math-41 xxx hep-th0603207_1_43 0 NTCIR11-Math-41 xxx 1111.4845_1_23 2 NTCIR11-Math-41 xxx 1003.4765_1_83 3 NTCIR11-Math-41 xxx math0506127_1_45 2 NTCIR11-Math-41 xxx 0804.2912_1_85 1 NTCIR11-Math-41 xxx 1210.2087_1_197 2 NTCIR11-Math-41 xxx 1206.6567_1_37 2 NTCIR11-Math-41 xxx hep-th0603207_1_15 0 NTCIR11-Math-41 xxx 1006.5861_1_188 2 NTCIR11-Math-41 xxx 0903.0022_1_68 2 NTCIR11-Math-41 xxx 0804.0321_1_76 2 NTCIR11-Math-41 xxx math9908056_1_212 0 NTCIR11-Math-41 xxx math0607549_1_54 2 NTCIR11-Math-41 xxx 0704.0046_1_20 2 NTCIR11-Math-41 xxx 0809.4380_1_44 2 NTCIR11-Math-41 xxx 1311.7385_1_49 4 NTCIR11-Math-41 xxx 1305.7170_1_56 1 NTCIR11-Math-41 xxx 0810.2821_1_24 2 NTCIR11-Math-41 xxx 0810.2821_1_26 2 NTCIR11-Math-41 xxx 0810.2821_1_28 2 NTCIR11-Math-41 xxx 1110.4329_1_167 1 NTCIR11-Math-41 xxx 0711.3806_1_156 0 NTCIR11-Math-41 xxx 1105.3137_1_67 1 NTCIR11-Math-41 xxx 1005.1893_1_23 2 NTCIR11-Math-41 xxx math9911026_1_315 0 NTCIR11-Math-41 xxx 0810.2821_1_6 2 NTCIR11-Math-41 xxx 1112.4968_1_50 1 NTCIR11-Math-41 xxx hep-th0603207_1_52 0 NTCIR11-Math-41 xxx hep-th0603207_1_36 0 NTCIR11-Math-41 xxx 1105.4274_1_196 2 NTCIR11-Math-41 xxx 0810.2821_1_42 2 NTCIR11-Math-42 xxx math0201313_1_90 4 NTCIR11-Math-42 xxx 1010.1620_1_27 2 NTCIR11-Math-42 xxx 1105.2476_1_18 3 NTCIR11-Math-42 xxx math9909151_1_21 2 NTCIR11-Math-42 xxx math9703218_1_300 0 NTCIR11-Math-42 xxx math9909036_1_111 1 NTCIR11-Math-42 xxx math-ph0105015_1_21 2 NTCIR11-Math-42 xxx 1210.7436_1_1 1 NTCIR11-Math-42 xxx math0201314_1_20 4 NTCIR11-Math-42 xxx math0605172_1_326 2 NTCIR11-Math-42 xxx 1309.6552_1_1 0 NTCIR11-Math-42 xxx 0809.3317_1_84 1 NTCIR11-Math-42 xxx hep-th0703106_1_56 1 NTCIR11-Math-42 xxx 1009.5153_1_17 4 NTCIR11-Math-42 xxx math9908144_1_31 0 NTCIR11-Math-42 xxx math0507519_1_106 0 NTCIR11-Math-42 xxx 1108.1660_1_5 2 NTCIR11-Math-42 xxx math0212378_1_77 3 NTCIR11-Math-42 xxx 1202.0085_1_7 0 NTCIR11-Math-42 xxx 1108.5286_1_172 1 NTCIR11-Math-42 xxx 1211.4402_1_28 1 NTCIR11-Math-42 xxx 1201.0389_1_19 2 NTCIR11-Math-42 xxx 1205.3957_1_44 0 NTCIR11-Math-42 xxx 1211.4402_1_20 1 NTCIR11-Math-42 xxx 1209.6278_1_41 2 NTCIR11-Math-42 xxx 1211.4402_1_24 1 NTCIR11-Math-42 xxx 1212.4205_1_121 0 NTCIR11-Math-42 xxx 1209.6278_1_46 2 NTCIR11-Math-42 xxx hep-lat0001002_1_7 0 NTCIR11-Math-42 xxx hep-lat0001002_1_6 0 NTCIR11-Math-42 xxx hep-lat0001002_1_5 1 NTCIR11-Math-42 xxx math9909036_1_48 2 NTCIR11-Math-42 xxx 0705.0341_1_87 2 NTCIR11-Math-42 xxx 1206.0367_1_23 1 NTCIR11-Math-42 xxx physics9908014_1_40 0 NTCIR11-Math-42 xxx 1304.1245_1_28 0 NTCIR11-Math-42 xxx hep-lat0001002_1_9 1 NTCIR11-Math-42 xxx math0510364_1_127 2 NTCIR11-Math-42 xxx 1307.3810_1_1 1 NTCIR11-Math-42 xxx 0908.2797_1_13 2 NTCIR11-Math-42 xxx physics9908014_1_38 0 NTCIR11-Math-42 xxx 1104.2010_1_7 1 NTCIR11-Math-42 xxx math9909036_1_145 1 NTCIR11-Math-42 xxx 1106.1759_1_18 2 NTCIR11-Math-42 xxx math9912114_1_35 1 NTCIR11-Math-42 xxx 1206.0694_1_618 1 NTCIR11-Math-42 xxx math9909036_1_102 0 NTCIR11-Math-42 xxx 0911.2140_1_16 0 NTCIR11-Math-42 xxx 1002.1468_1_120 2 NTCIR11-Math-42 xxx math0302119_1_40 2 NTCIR11-Math-43 xxx 1009.3496_1_13 0 NTCIR11-Math-43 xxx math-ph9910031_1_41 0 NTCIR11-Math-43 xxx 0810.5666_1_29 0 NTCIR11-Math-43 xxx hep-th9804086_1_3 0 NTCIR11-Math-43 xxx hep-th9804086_1_2 1 NTCIR11-Math-43 xxx hep-th9802130_1_3 1 NTCIR11-Math-43 xxx 0904.1757_1_10 1 NTCIR11-Math-43 xxx hep-th0004105_1_1 1 NTCIR11-Math-43 xxx 1304.5883_1_10 1 NTCIR11-Math-43 xxx 1208.5230_1_103 0 NTCIR11-Math-43 xxx hep-th9305001_1_34 1 NTCIR11-Math-43 xxx 0706.4428_1_84 0 NTCIR11-Math-43 xxx hep-th9802130_1_2 0 NTCIR11-Math-43 xxx math9908104_1_287 0 NTCIR11-Math-43 xxx math0509073_1_104 0 NTCIR11-Math-43 xxx 0904.3524_1_91 3 NTCIR11-Math-43 xxx hep-th0510122_1_9 1 NTCIR11-Math-43 xxx 1303.3305_1_20 0 NTCIR11-Math-43 xxx 1109.5228_1_76 1 NTCIR11-Math-43 xxx 1104.5492_1_11 0 NTCIR11-Math-43 xxx math0301096_1_33 0 NTCIR11-Math-43 xxx math-ph0111035_1_18 2 NTCIR11-Math-43 xxx math-ph0609045_1_104 0 NTCIR11-Math-43 xxx 1111.1222_1_7 0 NTCIR11-Math-43 xxx 1112.0591_1_33 1 NTCIR11-Math-43 xxx 1112.0591_1_34 1 NTCIR11-Math-43 xxx math-ph0111035_1_16 2 NTCIR11-Math-43 xxx cs9909009_1_110 0 NTCIR11-Math-43 xxx math9912016_1_201 0 NTCIR11-Math-43 xxx 1301.2743_1_2 2 NTCIR11-Math-43 xxx 1308.2455_1_8 1 NTCIR11-Math-43 xxx math0203254_1_51 0 NTCIR11-Math-43 xxx hep-th0101139_1_13 2 NTCIR11-Math-43 xxx 1009.3308_1_51 0 NTCIR11-Math-43 xxx 0805.3849_1_114 1 NTCIR11-Math-43 xxx 0711.1783_1_35 4 NTCIR11-Math-43 xxx 1108.5094_1_12 4 NTCIR11-Math-43 xxx 0711.1783_1_33 3 NTCIR11-Math-43 xxx 0704.1401_1_107 0 NTCIR11-Math-43 xxx math9908104_1_294 0 NTCIR11-Math-43 xxx 1109.5228_1_68 0 NTCIR11-Math-43 xxx 0904.4332_1_19 0 NTCIR11-Math-43 xxx 1201.3768_1_50 1 NTCIR11-Math-43 xxx math0302011_1_122 0 NTCIR11-Math-43 xxx math0203291_1_4 0 NTCIR11-Math-43 xxx solv-int9606005_1_5 1 NTCIR11-Math-43 xxx 0810.1737_1_28 0 NTCIR11-Math-43 xxx hep-th0401109_1_87 3 NTCIR11-Math-43 xxx math-ph0111035_1_17 2 NTCIR11-Math-43 xxx hep-th9804186_1_22 1 NTCIR11-Math-44 xxx math0410360_1_35 0 NTCIR11-Math-44 xxx 1112.3862_1_8 2 NTCIR11-Math-44 xxx hep-th0410023_1_28 0 NTCIR11-Math-44 xxx math0309472_1_62 0 NTCIR11-Math-44 xxx cond-mat0004018_1_11 0 NTCIR11-Math-44 xxx 1010.2484_1_4 1 NTCIR11-Math-44 xxx 0904.1757_1_10 0 NTCIR11-Math-44 xxx 0804.1485_1_37 0 NTCIR11-Math-44 xxx chao-dyn9909001_1_105 4 NTCIR11-Math-44 xxx 1202.1539_1_62 0 NTCIR11-Math-44 xxx 1202.1628_1_44 0 NTCIR11-Math-44 xxx 0710.1628_1_17 0 NTCIR11-Math-44 xxx math0110086_1_157 4 NTCIR11-Math-44 xxx 1005.1528_1_2 3 NTCIR11-Math-44 xxx 1106.5530_1_66 0 NTCIR11-Math-44 xxx 1109.0956_1_129 2 NTCIR11-Math-44 xxx math0211301_1_17 3 NTCIR11-Math-44 xxx cond-mat0009019_1_71 0 NTCIR11-Math-44 xxx 0711.1306_1_127 0 NTCIR11-Math-44 xxx 0811.2477_1_18 1 NTCIR11-Math-44 xxx 0712.3951_1_29 2 NTCIR11-Math-44 xxx 1311.0717_1_1 3 NTCIR11-Math-44 xxx math-ph0112016_1_6 1 NTCIR11-Math-44 xxx 1112.3862_1_31 2 NTCIR11-Math-44 xxx math-ph0112016_1_3 0 NTCIR11-Math-44 xxx gr-qc9807060_1_14 0 NTCIR11-Math-44 xxx 0707.1946_1_105 0 NTCIR11-Math-44 xxx 1308.1628_1_111 0 NTCIR11-Math-44 xxx math0604527_1_262 0 NTCIR11-Math-44 xxx math-ph0112016_1_16 1 NTCIR11-Math-44 xxx math0504257_1_26 0 NTCIR11-Math-44 xxx math-ph0112016_1_10 1 NTCIR11-Math-44 xxx 1301.3128_1_49 0 NTCIR11-Math-44 xxx math0210330_1_3 4 NTCIR11-Math-44 xxx math0605227_1_1 3 NTCIR11-Math-44 xxx math9910043_1_18 0 NTCIR11-Math-44 xxx hep-th0001053_1_10 0 NTCIR11-Math-44 xxx hep-th0001222_1_129 0 NTCIR11-Math-44 xxx math0010221_1_48 0 NTCIR11-Math-44 xxx math9910116_1_366 0 NTCIR11-Math-44 xxx hep-th0610118_1_27 0 NTCIR11-Math-44 xxx nlin0012025_1_20 0 NTCIR11-Math-44 xxx 0705.4608_1_11 0 NTCIR11-Math-44 xxx 0707.1946_1_383 0 NTCIR11-Math-44 xxx hep-th9909201_1_77 0 NTCIR11-Math-44 xxx 1309.2605_1_1 2 NTCIR11-Math-44 xxx math9911126_1_117 0 NTCIR11-Math-44 xxx math0404335_1_160 4 NTCIR11-Math-44 xxx math-ph0609031_1_6 0 NTCIR11-Math-44 xxx 1009.2166_1_11 0 NTCIR11-Math-45 xxx gr-qc0201031_1_25 0 NTCIR11-Math-45 xxx 1211.3234_1_76 2 NTCIR11-Math-45 xxx hep-ph0004008_1_77 0 NTCIR11-Math-45 xxx 1211.3234_1_53 3 NTCIR11-Math-45 xxx math0608187_1_3 4 NTCIR11-Math-45 xxx math9909131_1_12 0 NTCIR11-Math-45 xxx math-ph0008030_1_41 0 NTCIR11-Math-45 xxx math9908088_1_20 1 NTCIR11-Math-45 xxx math0006205_1_77 0 NTCIR11-Math-45 xxx hep-th9910213_1_7 0 NTCIR11-Math-45 xxx 0901.3075_1_18 4 NTCIR11-Math-45 xxx 1212.1300_1_25 0 NTCIR11-Math-45 xxx hep-th9210064_1_31 0 NTCIR11-Math-45 xxx math0211110_1_111 0 NTCIR11-Math-45 xxx 1303.2526_1_51 4 NTCIR11-Math-45 xxx hep-ph0004008_1_12 0 NTCIR11-Math-45 xxx hep-th0001077_1_4 0 NTCIR11-Math-45 xxx 1210.5561_1_55 2 NTCIR11-Math-45 xxx hep-th9911216_1_178 0 NTCIR11-Math-45 xxx math0410550_1_2 4 NTCIR11-Math-45 xxx 1205.0371_1_25 1 NTCIR11-Math-45 xxx 1205.0371_1_26 1 NTCIR11-Math-45 xxx 1210.6925_1_80 4 NTCIR11-Math-45 xxx math0405048_1_2 3 NTCIR11-Math-45 xxx 0811.4652_1_14 0 NTCIR11-Math-45 xxx 0811.4652_1_13 2 NTCIR11-Math-45 xxx math0404181_1_5 0 NTCIR11-Math-45 xxx 0811.4652_1_10 0 NTCIR11-Math-45 xxx 1005.1231_1_3 1 NTCIR11-Math-45 xxx 1005.1231_1_2 4 NTCIR11-Math-45 xxx 1211.0699_1_92 3 NTCIR11-Math-45 xxx 0802.2806_1_106 0 NTCIR11-Math-45 xxx 1007.2676_1_3 2 NTCIR11-Math-45 xxx 1212.6221_1_58 2 NTCIR11-Math-45 xxx math-ph0008030_1_38 0 NTCIR11-Math-45 xxx hep-th0001077_1_10 0 NTCIR11-Math-45 xxx math0406323_1_6 0 NTCIR11-Math-45 xxx math0209178_1_43 0 NTCIR11-Math-45 xxx math0406323_1_8 0 NTCIR11-Math-45 xxx hep-th9602040_1_10 0 NTCIR11-Math-45 xxx 1302.4302_1_3 1 NTCIR11-Math-45 xxx 0805.2707_1_4 3 NTCIR11-Math-45 xxx 0805.2707_1_1 3 NTCIR11-Math-45 xxx math9908088_1_5 0 NTCIR11-Math-45 xxx 1107.2710_1_18 0 NTCIR11-Math-45 xxx 0909.1273_1_32 4 NTCIR11-Math-45 xxx 0902.3467_1_62 0 NTCIR11-Math-45 xxx math0210262_1_19 1 NTCIR11-Math-45 xxx 1307.2080_1_91 0 NTCIR11-Math-45 xxx math0312014_1_42 2 NTCIR11-Math-46 xxx math0609071_1_1 0 NTCIR11-Math-46 xxx 0707.2563_1_6 0 NTCIR11-Math-46 xxx 1303.2770_1_365 3 NTCIR11-Math-46 xxx 1202.4355_1_9 0 NTCIR11-Math-46 xxx 1005.1726_1_28 3 NTCIR11-Math-46 xxx 0709.3803_1_116 0 NTCIR11-Math-46 xxx 0906.5557_1_420 1 NTCIR11-Math-46 xxx gr-qc9909001_1_70 0 NTCIR11-Math-46 xxx nlin0205028_1_25 0 NTCIR11-Math-46 xxx hep-th9911067_1_211 0 NTCIR11-Math-46 xxx hep-th9911231_1_50 0 NTCIR11-Math-46 xxx math9909074_1_79 0 NTCIR11-Math-46 xxx math0410360_1_35 0 NTCIR11-Math-46 xxx hep-th9406180_1_85 0 NTCIR11-Math-46 xxx hep-th9406180_1_117 0 NTCIR11-Math-46 xxx 0706.0612_1_8 0 NTCIR11-Math-46 xxx hep-th9910080_1_36 1 NTCIR11-Math-46 xxx math9909074_1_78 0 NTCIR11-Math-46 xxx 0712.0747_1_15 0 NTCIR11-Math-46 xxx 0906.4253_1_68 0 NTCIR11-Math-46 xxx 1110.1116_1_103 0 NTCIR11-Math-46 xxx hep-th9412240_1_104 0 NTCIR11-Math-46 xxx 1002.4332_1_37 1 NTCIR11-Math-46 xxx 1401.2932_1_127 0 NTCIR11-Math-46 xxx 1212.2910_1_65 3 NTCIR11-Math-46 xxx 1110.1116_1_126 0 NTCIR11-Math-46 xxx 0906.4253_1_72 0 NTCIR11-Math-46 xxx hep-th0506211_1_59 0 NTCIR11-Math-46 xxx 1303.2770_1_334 3 NTCIR11-Math-46 xxx 1110.1116_1_163 0 NTCIR11-Math-46 xxx 1006.2428_1_14 0 NTCIR11-Math-46 xxx 1011.1754_1_124 0 NTCIR11-Math-46 xxx 0809.2865_1_13 0 NTCIR11-Math-46 xxx 1303.5475_1_47 2 NTCIR11-Math-46 xxx 1212.2910_1_53 3 NTCIR11-Math-46 xxx math9808104_1_348 0 NTCIR11-Math-46 xxx 1305.2567_1_1 0 NTCIR11-Math-46 xxx 1303.2770_1_354 3 NTCIR11-Math-46 xxx 0707.2563_1_27 0 NTCIR11-Math-46 xxx 0707.2563_1_25 0 NTCIR11-Math-46 xxx 1303.2770_1_374 3 NTCIR11-Math-46 xxx hep-th9406180_1_92 0 NTCIR11-Math-46 xxx 0802.3529_1_1 1 NTCIR11-Math-46 xxx math0504491_1_98 0 NTCIR11-Math-46 xxx 1106.2944_1_60 2 NTCIR11-Math-46 xxx hep-th9406180_1_71 0 NTCIR11-Math-46 xxx 1303.2770_1_383 3 NTCIR11-Math-46 xxx 1001.4216_1_79 3 NTCIR11-Math-46 xxx 0811.0081_1_200 4 NTCIR11-Math-46 xxx 1005.1726_1_108 4 NTCIR11-Math-47 xxx math9909075_1_129 0 NTCIR11-Math-47 xxx 1007.1319_1_254 0 NTCIR11-Math-47 xxx math0303138_1_10 2 NTCIR11-Math-47 xxx math0001190_1_13 1 NTCIR11-Math-47 xxx 1002.1229_1_82 4 NTCIR11-Math-47 xxx math0404535_1_27 1 NTCIR11-Math-47 xxx 1305.0363_1_59 0 NTCIR11-Math-47 xxx math0312316_1_21 0 NTCIR11-Math-47 xxx 1007.1319_1_46 0 NTCIR11-Math-47 xxx math0303138_1_6 0 NTCIR11-Math-47 xxx 1102.0659_1_11 3 NTCIR11-Math-47 xxx hep-lat9903022_1_34 0 NTCIR11-Math-47 xxx 1108.5164_1_5 0 NTCIR11-Math-47 xxx 0801.1898_1_62 0 NTCIR11-Math-47 xxx 0904.2672_1_22 2 NTCIR11-Math-47 xxx hep-ph9912505_1_11 0 NTCIR11-Math-47 xxx math0602672_1_87 4 NTCIR11-Math-47 xxx math0508546_1_6 4 NTCIR11-Math-47 xxx math9908131_1_25 0 NTCIR11-Math-47 xxx 0811.0044_1_5 4 NTCIR11-Math-47 xxx math0203226_1_179 4 NTCIR11-Math-47 xxx hep-th9406147_1_57 1 NTCIR11-Math-47 xxx math0303138_1_11 1 NTCIR11-Math-47 xxx 1305.6970_1_80 3 NTCIR11-Math-47 xxx gr-qc9912067_1_161 0 NTCIR11-Math-47 xxx 0907.4431_1_7 0 NTCIR11-Math-47 xxx 0905.0941_1_13 3 NTCIR11-Math-47 xxx 1303.7466_1_99 4 NTCIR11-Math-47 xxx hep-ph9911463_1_71 0 NTCIR11-Math-47 xxx 1204.4908_1_282 0 NTCIR11-Math-47 xxx math0003014_1_53 0 NTCIR11-Math-47 xxx 1112.1183_1_28 0 NTCIR11-Math-47 xxx hep-ph9905249_1_52 1 NTCIR11-Math-47 xxx math0303138_1_12 0 NTCIR11-Math-47 xxx math0303138_1_13 1 NTCIR11-Math-47 xxx 1007.1319_1_192 0 NTCIR11-Math-47 xxx hep-th0507128_1_36 0 NTCIR11-Math-47 xxx 0901.3075_1_25 3 NTCIR11-Math-47 xxx math0510417_1_18 3 NTCIR11-Math-47 xxx 1203.6264_1_63 4 NTCIR11-Math-47 xxx math0510417_1_19 4 NTCIR11-Math-47 xxx math9910116_1_19 0 NTCIR11-Math-47 xxx 1206.3190_1_113 0 NTCIR11-Math-47 xxx math9908154_1_129 0 NTCIR11-Math-47 xxx math0702349_1_305 0 NTCIR11-Math-47 xxx 0911.1720_1_52 0 NTCIR11-Math-47 xxx 1204.4910_1_207 0 NTCIR11-Math-47 xxx math0001190_1_8 1 NTCIR11-Math-47 xxx 1102.3640_1_44 0 NTCIR11-Math-47 xxx hep-th9304158_1_40 1 NTCIR11-Math-48 xxx math0511112_1_1 3 NTCIR11-Math-48 xxx 0808.3051_1_26 0 NTCIR11-Math-48 xxx math0511112_1_46 1 NTCIR11-Math-48 xxx 1307.5799_1_5 1 NTCIR11-Math-48 xxx 1204.1080_1_17 1 NTCIR11-Math-48 xxx 0909.1001_1_2 1 NTCIR11-Math-48 xxx math-ph9911031_1_28 0 NTCIR11-Math-48 xxx math0509718_1_43 4 NTCIR11-Math-48 xxx 1210.4532_1_2 1 NTCIR11-Math-48 xxx 0811.1849_1_3 1 NTCIR11-Math-48 xxx alg-geom9402011_1_4 1 NTCIR11-Math-48 xxx math0108133_1_78 1 NTCIR11-Math-48 xxx 0903.3325_1_5 1 NTCIR11-Math-48 xxx 1106.4075_1_24 0 NTCIR11-Math-48 xxx math-ph0607041_1_54 1 NTCIR11-Math-48 xxx math-ph0607041_1_55 1 NTCIR11-Math-48 xxx math9908074_1_6 0 NTCIR11-Math-48 xxx math0403180_1_39 0 NTCIR11-Math-48 xxx 0902.2332_1_28 1 NTCIR11-Math-48 xxx 0908.3332_1_40 1 NTCIR11-Math-48 xxx 0908.3332_1_43 0 NTCIR11-Math-48 xxx 0908.3332_1_20 1 NTCIR11-Math-48 xxx 0902.2332_1_20 1 NTCIR11-Math-48 xxx math0511112_1_4 1 NTCIR11-Math-48 xxx 0902.2332_1_24 1 NTCIR11-Math-48 xxx 0710.0862_1_23 4 NTCIR11-Math-48 xxx 0902.2332_1_18 0 NTCIR11-Math-48 xxx 1211.6173_1_130 1 NTCIR11-Math-48 xxx 0908.3334_1_8 1 NTCIR11-Math-48 xxx 0908.3334_1_9 1 NTCIR11-Math-48 xxx 1010.2241_1_96 2 NTCIR11-Math-48 xxx gr-qc9909061_1_24 0 NTCIR11-Math-48 xxx 0908.3332_1_7 1 NTCIR11-Math-48 xxx 0801.3888_1_5 2 NTCIR11-Math-48 xxx 1102.3992_1_7 1 NTCIR11-Math-48 xxx math0612834_1_97 1 NTCIR11-Math-48 xxx 0806.1593_1_5 1 NTCIR11-Math-48 xxx hep-th9908113_1_53 0 NTCIR11-Math-48 xxx 0903.1337_1_4 0 NTCIR11-Math-48 xxx 0902.2332_1_35 1 NTCIR11-Math-48 xxx 1210.6552_1_66 1 NTCIR11-Math-48 xxx 1309.3112_1_12 1 NTCIR11-Math-48 xxx 1002.2900_1_7 1 NTCIR11-Math-48 xxx math9904153_1_26 1 NTCIR11-Math-48 xxx 1310.5289_1_52 1 NTCIR11-Math-48 xxx hep-th9908113_1_55 0 NTCIR11-Math-48 xxx hep-th9908113_1_50 1 NTCIR11-Math-48 xxx 0905.3229_1_10 2 NTCIR11-Math-48 xxx 0902.2332_1_10 2 NTCIR11-Math-48 xxx 1302.3077_1_8 1 NTCIR11-Math-49 xxx 1002.3875_1_53 0 NTCIR11-Math-49 xxx gr-qc9909004_1_18 0 NTCIR11-Math-49 xxx 1303.5188_1_42 1 NTCIR11-Math-49 xxx 0902.1199_1_57 0 NTCIR11-Math-49 xxx 0811.4287_1_27 1 NTCIR11-Math-49 xxx hep-th0408105_1_109 0 NTCIR11-Math-49 xxx math9912104_1_113 0 NTCIR11-Math-49 xxx 1210.6400_1_11 1 NTCIR11-Math-49 xxx 1210.6400_1_12 0 NTCIR11-Math-49 xxx 1210.8173_1_30 0 NTCIR11-Math-49 xxx 0710.1699_1_159 0 NTCIR11-Math-49 xxx math9911098_1_5 0 NTCIR11-Math-49 xxx math0512271_1_34 2 NTCIR11-Math-49 xxx math0307329_1_173 0 NTCIR11-Math-49 xxx math0409372_1_108 1 NTCIR11-Math-49 xxx 1303.5188_1_4 2 NTCIR11-Math-49 xxx 1004.3374_1_63 0 NTCIR11-Math-49 xxx 1005.4299_1_84 1 NTCIR11-Math-49 xxx 1006.4323_1_16 1 NTCIR11-Math-49 xxx 0811.4287_1_70 0 NTCIR11-Math-49 xxx 0907.2375_1_138 0 NTCIR11-Math-49 xxx 1002.3875_1_47 1 NTCIR11-Math-49 xxx cond-mat0703214_1_247 0 NTCIR11-Math-49 xxx 1301.2441_1_60 0 NTCIR11-Math-49 xxx hep-th0203086_1_17 0 NTCIR11-Math-49 xxx math-ph0207011_1_10 0 NTCIR11-Math-49 xxx 0706.2735_1_9 2 NTCIR11-Math-49 xxx 1208.0971_1_23 1 NTCIR11-Math-49 xxx 1002.3875_1_48 0 NTCIR11-Math-49 xxx 1210.8173_1_29 1 NTCIR11-Math-49 xxx 1102.2298_1_80 0 NTCIR11-Math-49 xxx math-ph0207011_1_43 2 NTCIR11-Math-49 xxx 1009.4619_1_6 0 NTCIR11-Math-49 xxx 1105.5933_1_109 0 NTCIR11-Math-49 xxx 0711.0780_1_30 0 NTCIR11-Math-49 xxx 1007.0353_1_19 1 NTCIR11-Math-49 xxx hep-ph0312058_1_10 0 NTCIR11-Math-49 xxx 0901.4044_1_50 0 NTCIR11-Math-49 xxx math9910116_1_173 0 NTCIR11-Math-49 xxx 0901.4044_1_52 0 NTCIR11-Math-49 xxx hep-th9910129_1_45 0 NTCIR11-Math-49 xxx quant-ph0003107_1_3 1 NTCIR11-Math-49 xxx 1401.3279_1_17 0 NTCIR11-Math-49 xxx 1303.5188_1_17 2 NTCIR11-Math-49 xxx 1108.3151_1_57 0 NTCIR11-Math-49 xxx math9911019_1_95 0 NTCIR11-Math-49 xxx 1305.6688_1_8 0 NTCIR11-Math-49 xxx solv-int9606005_1_5 0 NTCIR11-Math-49 xxx 0903.2289_1_173 0 NTCIR11-Math-49 xxx 1001.1366_1_32 0 NTCIR11-Math-50 xxx 0708.2956_1_15 4 NTCIR11-Math-50 xxx cs0309011_1_9 2 NTCIR11-Math-50 xxx dg-ga9605004_1_198 0 NTCIR11-Math-50 xxx 0810.2697_1_66 1 NTCIR11-Math-50 xxx math-ph9910013_1_172 0 NTCIR11-Math-50 xxx hep-th9402033_1_7 0 NTCIR11-Math-50 xxx 1205.6869_1_155 2 NTCIR11-Math-50 xxx 1105.4940_1_21 4 NTCIR11-Math-50 xxx hep-th9702026_1_95 0 NTCIR11-Math-50 xxx 1001.1231_1_13 4 NTCIR11-Math-50 xxx 1005.2889_1_54 0 NTCIR11-Math-50 xxx math-ph9910013_1_182 0 NTCIR11-Math-50 xxx math9911055_1_110 0 NTCIR11-Math-50 xxx 1206.1535_1_36 2 NTCIR11-Math-50 xxx 1206.1535_1_52 3 NTCIR11-Math-50 xxx 1206.1535_1_54 3 NTCIR11-Math-50 xxx 1102.1169_1_9 4 NTCIR11-Math-50 xxx math-ph0311054_1_47 0 NTCIR11-Math-50 xxx hep-ph9306286_1_23 0 NTCIR11-Math-50 xxx 1308.4251_1_69 0 NTCIR11-Math-50 xxx hep-ph9911307_1_204 0 NTCIR11-Math-50 xxx 1011.3376_1_3 3 NTCIR11-Math-50 xxx 1208.2315_1_6 4 NTCIR11-Math-50 xxx 1303.4580_1_7 2 NTCIR11-Math-50 xxx 1208.2315_1_3 4 NTCIR11-Math-50 xxx 1309.2352_1_261 0 NTCIR11-Math-50 xxx 1110.2235_1_66 0 NTCIR11-Math-50 xxx 1110.2235_1_67 0 NTCIR11-Math-50 xxx 1205.6869_1_127 2 NTCIR11-Math-50 xxx 1106.1110_1_102 4 NTCIR11-Math-50 xxx math9911055_1_118 0 NTCIR11-Math-50 xxx 1205.6869_1_1 2 NTCIR11-Math-50 xxx 1208.2315_1_9 3 NTCIR11-Math-50 xxx 1301.1521_1_27 4 NTCIR11-Math-50 xxx 1205.6869_1_103 2 NTCIR11-Math-50 xxx 1011.3376_1_23 3 NTCIR11-Math-50 xxx 1206.1535_1_62 4 NTCIR11-Math-50 xxx 0805.3789_1_54 0 NTCIR11-Math-50 xxx 1001.2157_1_182 0 NTCIR11-Math-50 xxx 1206.1535_1_46 2 NTCIR11-Math-50 xxx math9911118_1_77 0 NTCIR11-Math-50 xxx 1206.1535_1_44 2 NTCIR11-Math-50 xxx 1209.2471_1_4 4 NTCIR11-Math-50 xxx 1005.3044_1_56 0 NTCIR11-Math-50 xxx 1206.1535_1_24 2 NTCIR11-Math-50 xxx 1209.2471_1_1 2 NTCIR11-Math-50 xxx 1001.1231_1_154 2 NTCIR11-Math-50 xxx 1001.1231_1_156 4 NTCIR11-Math-50 xxx 1102.1169_1_13 4 NTCIR11-Math-50 xxx 1001.1231_1_152 2
ntcir-11-math-2-main/NTCIR11_Math-qrels.dat→ntcir-11-math-2-main/NTCIR11_Math-qrels-train.dat +0 −500 File changed and moved.Preview size limit exceeded, changes collapsed. Show changes
ntcir-11-math-2-main/README.md +4 −4 Original line number Diff line number Diff line Loading @@ -2,7 +2,7 @@ This table contains the best result for every user. | nDCG | User | Result name | |:-----|------|:------------| | 0.5499 | xstefan3 | example, key1=value1, key2=value2, etc | | 0.5499 | xnovot32 | example, key1=value1, key2=value2, etc | | 0.5499 | xluptak4 | example, key1=value1, key2=value2, etc | | 0.5499 | ayetiran | example, key1=value1, key2=value2, etc | | 0.5600 | xstefan3 | example, key1=value1, key2=value2, etc | | 0.5600 | xnovot32 | example, key1=value1, key2=value2, etc | | 0.5600 | xluptak4 | example, key1=value1, key2=value2, etc | | 0.5600 | ayetiran | example, key1=value1, key2=value2, etc |
ntcir-11-math-2-main/ayetiran/README.md +1 −1 Original line number Diff line number Diff line Loading @@ -4,4 +4,4 @@ underscores (`_`) replaced with a comma and a space for improved readability. | nDCG | Result name | |------|:------------| | 0.5499 | example, key1=value1, key2=value2, etc | | 0.5600 | example, key1=value1, key2=value2, etc |
