Skip to content
GitLab
Menu
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Sign in
Toggle navigation
Menu
Open sidebar
Michal Štefánik
ARQMath-eval
Commits
69913b98
Commit
69913b98
authored
Aug 25, 2020
by
xstefan3
Browse files
evaluate_run @ evaluate.py: support for ranks starting at zero
parent
394a85c8
Pipeline
#62916
canceled with stage
Changes
1
Pipelines
2
Hide whitespace changes
Inline
Side-by-side
scripts/evaluate.py
View file @
69913b98
...
...
@@ -79,11 +79,15 @@ def evaluate_run(filename, subset):
'received %d-tuples: %s'
%
(
n
,
first_line
)
)
parsed_result
=
dict
()
rank_offset
=
0
for
line
in
lines
:
topic_id
,
result_id
,
*
_
,
rank
,
__
,
___
=
line
# support indexing of ranks starting from 0
if
int
(
rank
)
==
0
:
rank_offset
=
1
if
topic_id
not
in
parsed_result
:
parsed_result
[
topic_id
]
=
dict
()
parsed_result
[
topic_id
][
result_id
]
=
1.0
/
int
(
rank
)
parsed_result
[
topic_id
][
result_id
]
=
1.0
/
(
int
(
rank
)
+
rank_offset
)
ndcg
=
get_ndcg
(
parsed_result
,
task
,
subset
)
print
(
'%.3f'
%
ndcg
)
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment