Skip to content
GitLab
Commit 7e0a6c1b authored by Vít Novotný's avatar Vít Novotný
Browse files

Update ARQMath task 1 relevance judgements from V0.1 to V1.0

parent 9b484337
Pipeline #57492 failed with stage
in 42 seconds
Hide whitespace changes
Inline Side-by-side
scripts/configuration.py
View file @ 7e0a6c1b
RELEVANCE_JUDGEMENTS = {
'task1': 'qrel.V0.1-test.tsv',
'task1': 'qrel.V1.0-test.tsv',
'ntcir-11-math-2-main': 'NTCIR11_Math-qrels-test.dat',
'ntcir-12-mathir-arxiv-main': 'NTCIR12_Math-qrels_agg-test.dat',
'ntcir-12-mathir-math-wiki-formula': 'NTCIR12_MathWikiFrm-qrels_agg-test.dat',
......
task1/ayetiran/example_key1=value1_key2=value2_etc.tsv
View file @ 7e0a6c1b
1 Q0 70741 1 0.9 RUN1
1 Q0 70739 2 0.8 RUN1
1 Q0 672516 3 0.7 RUN1
1 Q0 897705 4 0.6 RUN1
1 Q0 982759 5 0.5 RUN1
1 Q0 574514 6 0.4 RUN1
1 Q0 743738 7 0.3 RUN1
1 Q0 860842 8 0.2 RUN1
2 Q0 263828 1 0.9 RUN1
2 Q0 264299 2 0.8 RUN1
2 Q0 264315 3 0.7 RUN1
2 Q0 264329 4 0.6 RUN1
2 Q0 319993 5 0.5 RUN1
2 Q0 319916 6 0.4 RUN1
2 Q0 319917 7 0.3 RUN1
3 Q0 2008712 1 0.9 RUN1
3 Q0 2008449 2 0.8 RUN1
3 Q0 2008650 3 0.7 RUN1
3 Q0 2008609 4 0.6 RUN1
3 Q0 2008616 5 0.5 RUN1
3 Q0 2008631 6 0.4 RUN1
3 Q0 1116378 7 0.3 RUN1
3 Q0 1116368 8 0.2 RUN1
3 Q0 1116370 9 0.1 RUN1
A.31 Q0 70741 1 0.9 RUN1
A.31 Q0 70739 2 0.8 RUN1
A.31 Q0 672516 3 0.7 RUN1
A.31 Q0 897705 4 0.6 RUN1
A.31 Q0 982759 5 0.5 RUN1
A.31 Q0 574514 6 0.4 RUN1
A.31 Q0 743738 7 0.3 RUN1
A.31 Q0 860842 8 0.2 RUN1
A.101 Q0 263828 1 0.9 RUN1
A.101 Q0 264299 2 0.8 RUN1
A.101 Q0 264315 3 0.7 RUN1
A.101 Q0 264329 4 0.6 RUN1
A.101 Q0 319993 5 0.5 RUN1
A.101 Q0 319916 6 0.4 RUN1
A.101 Q0 319917 7 0.3 RUN1
A.78 Q0 2008712 1 0.9 RUN1
A.78 Q0 2008449 2 0.8 RUN1
A.78 Q0 2008650 3 0.7 RUN1
A.78 Q0 2008609 4 0.6 RUN1
A.78 Q0 2008616 5 0.5 RUN1
A.78 Q0 2008631 6 0.4 RUN1
A.78 Q0 1116378 7 0.3 RUN1
A.78 Q0 1116368 8 0.2 RUN1
A.78 Q0 1116370 9 0.1 RUN1
task1/qrel.V0.1-test.tsv deleted 100644 → 0
View file @ 9b484337
3 0 493764 3
3 0 493782 3
3 0 1282155 0
3 0 1282180 1
3 0 1282116 1
3 0 1282112 1
3 0 1282166 3
3 0 1282114 0
3 0 2008628 2
3 0 2008712 3
3 0 2008449 1
3 0 2008650 1
3 0 2008609 2
3 0 2008616 1
3 0 2008631 2
3 0 1116378 2
3 0 1116368 1
3 0 1116370 1
task1/qrel.V0.1-train.tsv deleted 100644 → 0
View file @ 9b484337
1 0 70741 3
1 0 70739 3
1 0 672516 2
1 0 897705 3
1 0 982759 2
1 0 574514 3
1 0 743738 2
1 0 860842 3
1 0 1596444 3
1 0 1623400 1
1 0 2170920 1
1 0 2227543 1
1 0 2362771 2
1 0 2968174 2
1 0 48202 2
1 0 48164 2
1 0 48235 2
1 0 48172 2
1 0 1282180 0
1 0 48181 2
1 0 1639289 1
1 0 1116368 0
1 0 48162 1
1 0 264329 0
1 0 439044 0
1 0 48219 2
1 0 48241 2
1 0 472635 1
1 0 48165 2
1 0 48167 2
1 0 48184 1
1 0 98328 1
1 0 48260 1
1 0 69435 1
1 0 692232 3
1 0 1018719 1
1 0 1609339 1
1 0 2602592 2
1 0 53781 1
1 0 53790 2
1 0 53784 2
1 0 53779 1
1 0 616373 1
1 0 616321 2
1 0 616514 2
1 0 616315 1
1 0 168305 0
1 0 168290 2
1 0 168286 2
1 0 168323 1
2 0 263828 3
2 0 264299 1
2 0 264315 2
2 0 264329 2
2 0 319993 0
2 0 319916 3
2 0 319917 2
2 0 319938 3
2 0 319919 2
2 0 2780928 2
2 0 496898 1
2 0 496909 2
2 0 876137 2
2 0 876221 1
2 0 1489896 1
2 0 439132 1
2 0 439044 1
2 0 439027 2
2 0 439055 1
task1/qrel.V1.0-test.tsv 0 → 100644
View file @ 7e0a6c1b
A.78 0 493764 3
A.78 0 493782 3
A.78 0 1282155 0
A.78 0 1282180 1
A.78 0 1282116 1
A.78 0 1282112 1
A.78 0 1282166 3
A.78 0 1282114 0
A.78 0 2008628 2
A.78 0 2008712 3
A.78 0 2008449 1
A.78 0 2008650 1
A.78 0 2008609 2
A.78 0 2008616 1
A.78 0 2008631 2
A.78 0 1116378 2
A.78 0 1116368 1
A.78 0 1116370 1
task1/qrel.V1.0-train.tsv 0 → 100644
View file @ 7e0a6c1b
A.31 0 70741 3
A.31 0 70739 3
A.31 0 672516 2
A.31 0 897705 3
A.31 0 982759 2
A.31 0 574514 3
A.31 0 743738 2
A.31 0 860842 3
A.31 0 1596444 3
A.31 0 1623400 1
A.31 0 2170920 1
A.31 0 2227543 1
A.31 0 2362771 2
A.31 0 2968174 2
A.31 0 48202 2
A.31 0 48164 2
A.31 0 48235 2
A.31 0 48172 2
A.31 0 1282180 0
A.31 0 48181 2
A.31 0 1639289 1
A.31 0 1116368 0
A.31 0 48162 1
A.31 0 264329 0
A.31 0 439044 0
A.31 0 48219 2
A.31 0 48241 2
A.31 0 472635 1
A.31 0 48165 2
A.31 0 48167 2
A.31 0 48184 1
A.31 0 98328 1
A.31 0 48260 1
A.31 0 69435 1
A.31 0 692232 3
A.31 0 1018719 1
A.31 0 1609339 1
A.31 0 2602592 2
A.31 0 53781 1
A.31 0 53790 2
A.31 0 53784 2
A.31 0 53779 1
A.31 0 616373 1
A.31 0 616321 2
A.31 0 616514 2
A.31 0 616315 1
A.31 0 168305 0
A.31 0 168290 2
A.31 0 168286 2
A.31 0 168323 1
A.101 0 263828 3
A.101 0 264299 1
A.101 0 264315 2
A.101 0 264329 2
A.101 0 319993 0
A.101 0 319916 3
A.101 0 319917 2
A.101 0 319938 3
A.101 0 319919 2
A.101 0 2780928 2
A.101 0 496898 1
A.101 0 496909 2
A.101 0 876137 2
A.101 0 876221 1
A.101 0 1489896 1
A.101 0 439132 1
A.101 0 439044 1
A.101 0 439027 2
A.101 0 439055 1
task1/xluptak4/example_key1=value1_key2=value2_etc.tsv
View file @ 7e0a6c1b
1 Q0 70741 1 0.9 RUN1
1 Q0 70739 2 0.8 RUN1
1 Q0 672516 3 0.7 RUN1
1 Q0 897705 4 0.6 RUN1
1 Q0 982759 5 0.5 RUN1
1 Q0 574514 6 0.4 RUN1
1 Q0 743738 7 0.3 RUN1
1 Q0 860842 8 0.2 RUN1
2 Q0 263828 1 0.9 RUN1
2 Q0 264299 2 0.8 RUN1
2 Q0 264315 3 0.7 RUN1
2 Q0 264329 4 0.6 RUN1
2 Q0 319993 5 0.5 RUN1
2 Q0 319916 6 0.4 RUN1
2 Q0 319917 7 0.3 RUN1
3 Q0 2008712 1 0.9 RUN1
3 Q0 2008449 2 0.8 RUN1
3 Q0 2008650 3 0.7 RUN1
3 Q0 2008609 4 0.6 RUN1
3 Q0 2008616 5 0.5 RUN1
3 Q0 2008631 6 0.4 RUN1
3 Q0 1116378 7 0.3 RUN1
3 Q0 1116368 8 0.2 RUN1
3 Q0 1116370 9 0.1 RUN1
A.31 Q0 70741 1 0.9 RUN1
A.31 Q0 70739 2 0.8 RUN1
A.31 Q0 672516 3 0.7 RUN1
A.31 Q0 897705 4 0.6 RUN1
A.31 Q0 982759 5 0.5 RUN1
A.31 Q0 574514 6 0.4 RUN1
A.31 Q0 743738 7 0.3 RUN1
A.31 Q0 860842 8 0.2 RUN1
A.101 Q0 263828 1 0.9 RUN1
A.101 Q0 264299 2 0.8 RUN1
A.101 Q0 264315 3 0.7 RUN1
A.101 Q0 264329 4 0.6 RUN1
A.101 Q0 319993 5 0.5 RUN1
A.101 Q0 319916 6 0.4 RUN1
A.101 Q0 319917 7 0.3 RUN1
A.78 Q0 2008712 1 0.9 RUN1
A.78 Q0 2008449 2 0.8 RUN1
A.78 Q0 2008650 3 0.7 RUN1
A.78 Q0 2008609 4 0.6 RUN1
A.78 Q0 2008616 5 0.5 RUN1
A.78 Q0 2008631 6 0.4 RUN1
A.78 Q0 1116378 7 0.3 RUN1
A.78 Q0 1116368 8 0.2 RUN1
A.78 Q0 1116370 9 0.1 RUN1
task1/xnovot32/example_key1=value1_key2=value2_etc.tsv
View file @ 7e0a6c1b
1 Q0 70741 1 0.9 RUN1
1 Q0 70739 2 0.8 RUN1
1 Q0 672516 3 0.7 RUN1
1 Q0 897705 4 0.6 RUN1
1 Q0 982759 5 0.5 RUN1
1 Q0 574514 6 0.4 RUN1
1 Q0 743738 7 0.3 RUN1
1 Q0 860842 8 0.2 RUN1
2 Q0 263828 1 0.9 RUN1
2 Q0 264299 2 0.8 RUN1
2 Q0 264315 3 0.7 RUN1
2 Q0 264329 4 0.6 RUN1
2 Q0 319993 5 0.5 RUN1
2 Q0 319916 6 0.4 RUN1
2 Q0 319917 7 0.3 RUN1
3 Q0 2008712 1 0.9 RUN1
3 Q0 2008449 2 0.8 RUN1
3 Q0 2008650 3 0.7 RUN1
3 Q0 2008609 4 0.6 RUN1
3 Q0 2008616 5 0.5 RUN1
3 Q0 2008631 6 0.4 RUN1
3 Q0 1116378 7 0.3 RUN1
3 Q0 1116368 8 0.2 RUN1
3 Q0 1116370 9 0.1 RUN1
A.31 Q0 70741 1 0.9 RUN1
A.31 Q0 70739 2 0.8 RUN1
A.31 Q0 672516 3 0.7 RUN1
A.31 Q0 897705 4 0.6 RUN1
A.31 Q0 982759 5 0.5 RUN1
A.31 Q0 574514 6 0.4 RUN1
A.31 Q0 743738 7 0.3 RUN1
A.31 Q0 860842 8 0.2 RUN1
A.101 Q0 263828 1 0.9 RUN1
A.101 Q0 264299 2 0.8 RUN1
A.101 Q0 264315 3 0.7 RUN1
A.101 Q0 264329 4 0.6 RUN1
A.101 Q0 319993 5 0.5 RUN1
A.101 Q0 319916 6 0.4 RUN1
A.101 Q0 319917 7 0.3 RUN1
A.78 Q0 2008712 1 0.9 RUN1
A.78 Q0 2008449 2 0.8 RUN1
A.78 Q0 2008650 3 0.7 RUN1
A.78 Q0 2008609 4 0.6 RUN1
A.78 Q0 2008616 5 0.5 RUN1
A.78 Q0 2008631 6 0.4 RUN1
A.78 Q0 1116378 7 0.3 RUN1
A.78 Q0 1116368 8 0.2 RUN1
A.78 Q0 1116370 9 0.1 RUN1
task1/xstefan3/example_key1=value1_key2=value2_etc.tsv
View file @ 7e0a6c1b
1 Q0 70741 1 0.9 RUN1
1 Q0 70739 2 0.8 RUN1
1 Q0 672516 3 0.7 RUN1
1 Q0 897705 4 0.6 RUN1
1 Q0 982759 5 0.5 RUN1
1 Q0 574514 6 0.4 RUN1
1 Q0 743738 7 0.3 RUN1
1 Q0 860842 8 0.2 RUN1
2 Q0 263828 1 0.9 RUN1
2 Q0 264299 2 0.8 RUN1
2 Q0 264315 3 0.7 RUN1
2 Q0 264329 4 0.6 RUN1
2 Q0 319993 5 0.5 RUN1
2 Q0 319916 6 0.4 RUN1
2 Q0 319917 7 0.3 RUN1
3 Q0 2008712 1 0.9 RUN1
3 Q0 2008449 2 0.8 RUN1
3 Q0 2008650 3 0.7 RUN1
3 Q0 2008609 4 0.6 RUN1
3 Q0 2008616 5 0.5 RUN1
3 Q0 2008631 6 0.4 RUN1
3 Q0 1116378 7 0.3 RUN1
3 Q0 1116368 8 0.2 RUN1
3 Q0 1116370 9 0.1 RUN1
A.31 Q0 70741 1 0.9 RUN1
A.31 Q0 70739 2 0.8 RUN1
A.31 Q0 672516 3 0.7 RUN1
A.31 Q0 897705 4 0.6 RUN1
A.31 Q0 982759 5 0.5 RUN1
A.31 Q0 574514 6 0.4 RUN1
A.31 Q0 743738 7 0.3 RUN1
A.31 Q0 860842 8 0.2 RUN1
A.101 Q0 263828 1 0.9 RUN1
A.101 Q0 264299 2 0.8 RUN1
A.101 Q0 264315 3 0.7 RUN1
A.101 Q0 264329 4 0.6 RUN1
A.101 Q0 319993 5 0.5 RUN1
A.101 Q0 319916 6 0.4 RUN1
A.101 Q0 319917 7 0.3 RUN1
A.78 Q0 2008712 1 0.9 RUN1
A.78 Q0 2008449 2 0.8 RUN1
A.78 Q0 2008650 3 0.7 RUN1
A.78 Q0 2008609 4 0.6 RUN1
A.78 Q0 2008616 5 0.5 RUN1
A.78 Q0 2008631 6 0.4 RUN1
A.78 Q0 1116378 7 0.3 RUN1
A.78 Q0 1116368 8 0.2 RUN1
A.78 Q0 1116370 9 0.1 RUN1
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment