Named Entity Experiments:152f26f413ead4c1e17a9aa04e3f5f1fa309c96e commitshttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commits/152f26f413ead4c1e17a9aa04e3f5f1fa309c96e2022-09-19T16:11:25+02:00https://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/152f26f413ead4c1e17a9aa04e3f5f1fa309c96eUpdate evaluation of NER models2022-09-19T16:11:25+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/d167b7be8d29a73f91645d95764b1200c4e9545eDo not expect `category_map` to know all categories in `GROUPS.values()`2022-09-19T14:44:18+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/b400288e6f22efacaaddd4d731a0eef7be63dd73Make `AggregateMeanFScoreEvaluator` hashable and totally ordered2022-09-19T14:19:06+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/b36d1d79bc4215853aeb8dcd76121ddf4af7246cRename `AggregateMeanFScore` to `AggregateMeanFScoreEvaluator`2022-09-19T14:03:44+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/6c760a64742173074272f7ae63ad0893293b2c1fExplicitly use sets in `ahisto_named_entity_search.recognition.evaluator`2022-09-18T14:39:42+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/7e5701ebc6b60a7870ef832f061d8261c7451c1eAdd `*-ORG` and `*-MISC` to `AggregateMeanFScore.GROUPS`2022-09-18T14:36:23+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/63feb75bb139b300c04ef83b45e9dd5ef824e492Add `ahisto_named_entity_search.recognition.evaluator`2022-09-17T00:48:22+02:00Vit Novotnywitiko@mail.muni.cz
Adds the `AggregateMeanFScore` class, which can be used to evaluate NER
models on different categories and to aggregate the scores for clusters
of categories such as B-PER and I-PER.https://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/171526e68b703fbca7e921c005aeb0bbbb8e13aaUse testing dataset to evaluate NER models2022-09-16T11:39:30+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/3aa935def8eb672c6599640f98d153c56be84d01Add TODOs to `03_train_ner_models.ipynb`2022-09-15T13:35:25+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/2388bc47097199451e7246291940bc5ff2ecc38dEvaluate baseline model in `03_train_ner_models.ipynb`2022-09-15T12:08:32+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/2287e9badf0c1b41b2b77256ba7169f9b0969155Add `labels` parameter to `NerModel.__init__()`2022-09-15T12:08:09+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/a15f4ea01c0b44449d0273fa1254a14345299e52Add `NerModel.tokenizer`2022-09-15T10:57:43+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/69c56bf1aa209149791edcf11987775288c0ffb7Add evaluation results for all but one model to `03_train_ner_models.ipynb`2022-09-13T19:47:25+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/cd193fa8111c00a77cfb9c23a4469e5289db369aFix style check2022-09-12T13:43:02+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/f7aef48c33893bd8d00eb1fcb1e6e0b4a46855c7Fix type check2022-09-12T13:42:55+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/7770581dd51efde75ec3378919eb109f7f04cee0Add `03_train_ner_models.ipynb`2022-09-12T13:34:22+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/abb3e082bfe559bee71228912b1e13ae87eb1859Fix typos in `ahisto_named_entity_search.recognition.model`2022-09-12T13:14:45+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/47e65c6258bc15891ffd3a8ff7c95ba9d693a4b6Update `*.ipynb`2022-09-11T18:22:05+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/27e52d2cd29f44ba19de92a8978f40c4b908c467Add `NerModel.test()`2022-09-11T18:22:05+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/e8e1d2d40a26863f7364a2f2a03450bff60294a4Fix GPU selection script2022-09-11T17:57:30+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/98350de5cb86d5b68709108d037cda92d6b52d4dMake `NerModel.train_and_save()` return model2022-09-11T17:56:56+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/7263450d7015b9429c2ca935ebc29c59a33ca984Make `NerModel.__init__()` lazy and add `str()` and `repr()`2022-09-11T17:56:40+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/b78567bbf86cdc6a0d7acb17f3f9386a0c2ef2b8Prune old checkpoints2022-08-22T11:13:47+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/72d22741bfab6e6286862dbbeede35cec3127e31Fix joblog format in `scripts/03_train_ner_models.sh`2022-08-16T18:48:45+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/51be26ee9dabc3d174c9b904e4ae398451e2c9dcIncrease delay in `scripts/03_train_ner_models.sh`2022-08-16T18:43:17+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/837ffec0e2cec41b0cc4bd3f374888ab4d37c611Fix off-by-one error in schedules2022-08-16T17:14:34+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/ff8e4281e78f06af1ca4ef25e09581a918eb5aa7Increase early stopping patience for training NER models2022-08-16T03:19:20+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/832adbaa95b7fb0c55d65c528397a33c6eac0763Prevent future objectives from affecting early stopping (cont.)2022-08-15T18:07:31+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/a3022d220e244187d588c8b7adf4d2fa14f5577aStart taking GPUs from the tail of IDs2022-08-15T17:48:50+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/eb609384a5d536f19386f91f86973f3beb2ad031Remove extra forward slash2022-08-15T17:47:16+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/6e472d42211d5c14742394742b305aed9e627a00Add GNU parallel joblog to training NER models2022-08-15T17:34:19+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/1046301de53fe1d328a7b3b576445815b904de09Train using NER tags in BIO format2022-08-15T17:27:12+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/16f1e88d76916525cc3066f9462667bb366d0162Use 4 jobs to train NER model at most2022-08-15T17:13:58+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/590372a42f1258033da8eb63fda36d6e37c96c7aPrevent future objectives from affecting early stopping2022-08-15T17:13:35+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/c4c8ffd37554c7418f4cc4de5bf6e65b524da805Use fine-tuning schedule to train NER models2022-08-15T15:57:58+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/04b847a2a139b16ecd91d64e9c6d7b1b41d631c6Evaluate more often with fewer samples2022-08-15T15:57:58+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/b3810c32b0273222bf4a741b4ad74515898723c0Use parallel schedule to train NER models2022-08-13T12:56:11+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/3d99edcc43b0b13046e9af6acfd6eac9cb05f0f8Separate NerModel training schedules to `recognition.schedule`2022-08-13T12:56:11+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/63d085bde44ad3fe085107e1bee50b599e5203b9Automatically select GPU in `scripts/03_train_ner_models.sh`2022-08-11T21:08:09+02:00Vit Novotnywitiko@mail.muni.czhttps://gitlab.fi.muni.cz/nlp/ahisto-modules/ner-eval/-/commit/7c0a40444e5a124f24367c46c94c02f10be33ca4Add `_fair-sequential` suffix to `03_train_ner_models.py`2022-08-10T19:42:38+02:00Vit Novotnywitiko@mail.muni.cz