Commit 804f854f authored by Vít Novotný's avatar Vít Novotný
Browse files

Extend the README (cont.)

parent aeac52de
Pipeline #58970 canceled with stage
......@@ -26,8 +26,8 @@ Each task comes with a number of *subsets*:
### Usage
#### Evaluating your model with various parameters
Place your results in [the trec\_eval format][treceval-format] into the
`results.csv` file. To evaluate your results on the train set, execute the
following commands:
`results.csv` file. To evaluate your results e.g. on the `train` subset of the
`task1-votes` task, execute the following commands:
``` sh
$ pip install git+https://gitlab.fi.muni.cz/xstefan3/arqmath-eval@master
......@@ -37,17 +37,19 @@ $ python
>>>
>>> with open('results.csv', 'rt') as f:
>>> results = parse_run(f)
>>> get_ndcg(task='task', subset='train')
>>> get_ndcg(task='task1-votes', subset='train')
0.5876
```
#### Placing your results to the leaderboard
Place your results in [the trec\_eval format][treceval-format] into your
dedicated directory *task/user*. To evaluate your results on the test set and
publish the results into the leaderboard, execute the following commands:
dedicated directory *task/user*, e.g. `task1-votes/xnovot32` for the user
@xnovot32 and the `task1-votes` task. To evaluate your results on the `test`
set of the `task1-votes` and publish the results into the leaderboard, execute
the following commands:
``` sh
$ git add task/user/result.tsv # track your new result with Git
$ git add task1-votes/xnovot32/result.tsv # track your new result with Git
$ pip install git+https://gitlab.fi.muni.cz/xstefan3/arqmath-eval@master
$ python -m scripts.evaluate # run the evaluation
$ git add -u # add the updated leaderboard to Git
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment