INJECT allows for export of logs from an exercise for analysis purposes.
The log files use the [JSONL](https://jsonlines.org/) format.
Bellow is description of the structure and schema of logs.

### Structure
```
- logs/
  - team-id/
    - uploaded_files/
        - files uploaded by team-id and for team-id during the exercise
    - inject_states.jsonl
    - questionnaire_states.jsonl
    - action_logs.jsonl
    - milestones.jsonl
    - emails.jsonl
  - team-id+1/
    ...
  - definition_files/
    - files used by the definition
  - exercise.jsonl
  - teams.jsonl
  - exercise_injects.jsonl
  - exercise_milestones.jsonl
  - exercise_tools.jsonl
  - email_participants.jsonl
  - exercise_questionnaires.jsonl
  - exercise_learning_objectives.jsonl
  - file_infos.jsonl
  - instructors.jsonl
```

## Description and format of individual files

Many of the fields described here are also described
in the definition format which can be found
[here](https://gitlab.fi.muni.cz/inject/backend/-/blob/main/definitions/README.md?ref_type=heads).

### Timestamps

All fields with the `timestamp` type are timestamps in `ISO 8601` format.

### Common objects

#### Control

- **milestone_condition**: _string_
- **activate_milestone**: _string_
- **deactivate_milestone**: _string_
- **roles**: _string_

#### Content

- **raw**: _string_ - raw text, possibly markdown
- **rendered**: _string_ - raw text converted to html elements
- **file_info_id**: _uuid_ - id of the file attachment

#### Confirmation

- **text**: _string_
- **control**: _control_

## Exercise files

### exercise.jsonl

Contains information about the exercise. The object has the following format:
- **exercise_id**: _int_ - id of the exercise
- **exercise_start**: _timestamp_ - time when the exercise was started
- **finished**: _bool_ - flag that signifies that the exercise actually finished,
  this is true when the entire duration of the exercise was passed
- **definition_id**: _int_ - id of the definition this exercise was created from

### teams.jsonl

Contains information about all the exercise teams. Each object has the following format:
- **team_id**: _int_ - id of the team
- **name**: _string_ - name of the team
- **role**: _string_ - role of the team, empty string if no role
- **exercise_id**: _int_ - id of the exercise the team belongs to
- **finish_time**: _timestamp_ - time when the team reached one of the _final_ milestones
- **users**: _a list of user objects_ - a list of trainees assigned to this team
  - **user_id**: _uuid_ - id of the user
  - **username**: _optional string_ - username of the user, not present if the logs were anonymized

### instructors.jsonl

Contains all instructors assigned to the exercise. Each object has the following format:
- **user_id**: _uuid_ - id of the user
- **username**: _optional string_ - username of the user, not present if the logs were anonymized

### exercise_injects.jsonl

Contains all injects for this exercise. Each object has the following format:
- **inject_id**: _int_ - id of the inject
- **name**: _string_
- **time**: _int_
- **delay**: _int_
- **organization**: _string_
- **type**: _(`Info`, `Email`)_
- **target**: _channel object_
  - **channel_id**: _int_ - id of the channel
  - **name**: _string_
  - **display_name**: _string_
  - **description**: _string_
  - **type**: _(`Info`, `Email`)_
- **alternatives**: _list of alternative objects_

#### _Info_ inject alternative

- **alternative_id**: _int_ - id of the alternative
- **name**: _string_
- **content**: _content_
- **control**: _control_
- **confirmation**: _optional confirmation object_

#### _Email_ inject alternative

- **alternative_id**: _int_ - id of the alternative
- **name**: _string_
- **subject**: _string_
- **content**: _content_
- **control**: _control_
- **extra_copies**: _int_

### exercise_milestones.jsonl

Contains all milestones for this exercise, which are referenced by milestones in each team.
Each object has the following format:
- **milestone_id**: _int_ - id of the milestone
- **name**: _string_
- **display_name**: _string_
- **description**: _string_
- **tags**: _list of strings_
- **roles**: _string_
- **file_names**: _string_ - space separated list of file names
- **initial_state**: _bool_

### exercise_tools.jsonl

Contains all tools for this exercise, which are referenced by action logs in each team.
Each object has the following format:
- **tool_id**: _int_ - id of the tool
- **name**: _string_
- **tooltip_description**: _string_
- **default_response**: _content_
- **roles**: _string_
- **requires_input**: _bool_
- **responses**: _list of response objects_
  - **param**: _string_
  - **content**: _content_
  - **control**: _control_
  - **time**: _int_
  - **regex**: _bool_

### exercise_questionnaires.jsonl

Contains all questionnaires for this exercise. Each object has the following format:
- **questionnaire_id**: _int_ - id of the questionnaire
- **title**: _string_
- **content**: _content_ 
- **time**: _int_
- **control**: _control_
- **questions**: _list of question objects_
  - **question_id**: _int_ - id of the question
  - **content**: _content_
  - **type**: _(`Radio`, `Free-form`)_
  - **details**: _additional details depending on the question type_

#### _Radio_ question details

- **labels**: _list of strings_
- **correct**: _int_
- **max**: _int_

#### _Free-form_ question details

- **related_milestone_ids**: _list of int_

### exercise_learning_objectives.jsonl

Contains all learning objectives for the exercise. Each object has the following format:
- **objective_id**: _int_ - id of the learning objective
- **name**: _string_
- **description**: _string_
- **tags**: _list of strings_
- **activities**: _list of activity objects_
  - **activity_id**: _int_ - id of the activity
  - **name**: _string_
  - **description**: _string_
  - **tags**: _list of strings_
  - **milestone_ids**: _list of int_

### email_participants.jsonl

Contains all email participants for this exercise, which are referenced by email threads and emails
in each team. Each object has the following format:
- **participant_id**: _int_ - id of the email participant
- **address**: _string_
- **exercise_id**: _int_ - id of the exercise this participant belongs to
- **team_id**: _optional int_ - team which this participant represents, null if it does not belong to any team
- **definition_address**: _optional definition address object_ - email address from the definition
  which this participant represents, null if it belongs to a team, definition address format:
  - **email_address_id**: _int_ - id of the definition address
  - **address**: _string_
  - **description**: _string_
  - **team_visible**: _boolean_
  - **control**: _control_

### file_infos.jsonl

Contains all file infos for this exercise. Each object has the following format:
- **file_id**: _uuid_ - id of the file, this is also the name of the file on the file system
- **file_name**: _string_ - original name of the file


## Individual team files

### inject_states.jsonl

Contains all the inject states for the team. Each object has the following format:
- **team_id**: _int_ - id of the team
- **inject_id**: _int_ - id of the inject
- **status**: _(`Unsent`, `Delayed`, `Sent`)_ - status of the inject
- **alternative_id**: _int_ - id of the sent alternative

### questionnaire_states.jsonl

Contains all the questionnaire states for the team. Each object has the following format:
- **questionnaire_id**: _int_ - id of the questionnaire
- **team_id**: _int_ - id of the team
- **status**: _(`Unsent`, `Sent`, `Answered`, `Reviewed`)_ - status of the questionnaire
- **timestamp_sent**: _optional timestamp_
- **timestamp_answered**: _optional timestamp_
- **timestamp_reviewed**: _optional timestamp_
- **reviewed_by_id**: _optional uuid_ - id of the instructor that reviewed the questionnaire
- **answers**: _list of answer objects_
  - **question_id**: _int_ - id of the question
  - **answer**: _string_ - submitted answer
  - **is_correct**: _optional bool_ - if specified, flag that marks the answer as correct according to the definition

### milestones.jsonl

Contains all milestone states for the team. Each object has the following format:
- **milestone_id**: _int_ - id of the milestone this state belongs to,
  the referenced milestone can be found in [exercise_milestones.jsonl](#exercisemilestonesjsonl)
- **reached**: _bool_ - state of the milestone
- **timestamp_reached**: _optional timestamp_ - time of the last state change for this milestone
- **history**: _list of history objects_
  - **reached**: _bool_ - state of the milestone at that point
  - **timestamp_from**: _timestamp_ - timestamp of the state change

### emails.jsonl

Optional file, included only if the email feature is enabled. Contains all email threads and emails
for the team. Each object has the following format:
- **thread_id**: _int_ - id of the email thread
- **subject**: _string_
- **timestamp**: _timestamp_ - time when this email thread was created
- **participants**: _list of ints_ - list of email participant ids that belong to the thread,
  the referenced participants can be found in [email_participants.jsonl](#emailparticipantsjsonl)
- **emails**: _list of email object_ - list of emails sent to this thread, email object format:
  - **email_id**: _int_ - id of the email
  - **thread_id**: _int_ - id of the thread this email was sent to
  - **sender_id**: _int_ - id of the email participant that sent the email
  - **user_id**: _uuid_ - id of the user that sent the email
  - **timestamp**: _timestamp_ - time when this email was sent
  - **content**: _content_

### action_logs.jsonl

Contains all the action logs for the team. Each object has the following format:
- **action_log_id**: _int_ - id of the action log
- **type**: _(`Inject`, `Custom Inject`, `Form`, `Tool`, `Email`)_ - type of the action log
- **timestamp**: _timestamp_ - time when this action log was created
- **details**: _additional details depending on the action log type_

#### _Inject_ details

- **inject_id**: _int_ - id of the inject
- **content**: _content_
- **confirmation**: _optional confirmation object_
- **confirmed**: _optional bool_ - included only if the alternative contains a confirmation,
  specifies whether it has been confirmed
- **confirmed_by_id**: _optional uuid_ - id of the user that confirmed this inject
- **timestamp_confirmed**: _optional timestamp_ - timestamp when this inject was confirmed

#### _Custom_ inject details

- **content**: _content_
- **user_id**: _uuid_ - id of the instructor that sent this custom inject

#### _Form_ inject details

- **questionnaire_state_id**: _int_ - id of the sent questionnaire state

#### _Tool_ inject details

- **tool_id**: _int_ - id of the used tool
- **argument**: _string_ - argument provided to the tool
- **content**: _content_
- **user_id**: _uuid_ - id of the user that used the tool

#### _Email_ inject details

Same fields as described in `emails.jsonl` in the `emails` field.

## Comparing logs from multiple exercises

The logs are constructed in a way that should allow for simple comparison of logs from multiple exercises with the same definition.
It has to be the exact same upload of the definition, otherwise the IDs of `injects`, `tools` and other definition data will not match.

Between different runs of the same definition,
the `exercise_injects.jsonl`, `exercise_milestones.jsonl` and `exercise_tools.jsonl` files will be the same.
The `email_participants.jsonl` will not match because email participants need to be re-generated for every exercise.
However, the definition address they are linked to will have the same ID in all runs.
