Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Sign in
Toggle navigation
Menu
Open sidebar
Michal Štefánik
ARQMath-eval
Commits
e69e56f8
Commit
e69e56f8
authored
Apr 06, 2020
by
Vít Novotný
Browse files
Create scripts.common.remove_nonjudged_topics_and_documents
parent
8d76eab9
Pipeline
#57592
failed with stage
in 41 seconds
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
scripts/common.py
View file @
e69e56f8
...
...
@@ -7,7 +7,7 @@ import numpy as np
from
.configuration
import
EVALUATORS
,
PARSED_RELEVANCE_JUDGEMENTS
def
ndcg
(
parsed_run
,
task
=
'task1'
,
subset
=
'train'
):
def
remove_nonjudged_topics_and_documents
(
parsed_run
,
task
,
subset
):
only_judged_parsed_run
=
deepcopy
(
parsed_run
)
parsed_relevance_judgements
=
PARSED_RELEVANCE_JUDGEMENTS
[
subset
][
task
]
for
topic_name
,
results
in
parsed_run
.
items
():
...
...
@@ -18,7 +18,12 @@ def ndcg(parsed_run, task='task1', subset='train'):
for
document_name
in
results
.
keys
():
if
document_name
not
in
judgements
:
del
only_judged_parsed_run
[
topic_name
][
document_name
]
return
only_judged_parsed_run
def
ndcg
(
parsed_run
,
task
=
'task1'
,
subset
=
'train'
):
evaluator
=
EVALUATORS
[
subset
][
task
]
only_judged_parsed_run
=
remove_nonjudged_topics_and_documents
(
parsed_run
,
task
,
subset
)
evaluation
=
evaluator
.
evaluate
(
only_judged_parsed_run
)
ndcg
=
np
.
mean
([
measures
[
'ndcg'
]
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment