Merge branch 'COPY-e6d167a8' into 'main'

INJECT allows for export of logs from an exercise for analysis purposes. The log files use the

[JSONL](https://jsonlines.org/) format. Bellow is description of the structure and schema of logs.

INJECT allows for export of logs from an exercise for analysis purposes.

The log files use the [JSONL](https://jsonlines.org/) format.

Bellow is description of the structure and schema of logs.

### Structure

    - files used by the definition

  - exercise.jsonl

  - teams.jsonl

  - exercise_injects.jsonl

  - exercise_milestones.jsonl

  - instructors.jsonl

  - file_infos.jsonl

  - exercise_tools.jsonl

  - email_participants.jsonl

  - exercise_questionnaires.jsonl

  - exercise_milestones.jsonl

  - exercise_learning_objectives.jsonl

  - file_infos.jsonl

  - instructors.jsonl

  - exercise_injects.jsonl

  - exercise_channels.jsonl

  - email_participants.jsonl

```

## Description and format of individual files

Many of the fields described here are also described in the definition format which can be found

Many of the fields described here do not contain a description.

That is because these fields are direct copies of the fields described in the definition format,

which can be found

[here](https://gitlab.fi.muni.cz/inject/backend/-/blob/main/definitions/README.md?ref_type=heads).

### Timestamps

#### Control

- **milestone_condition**: _string_

- **activate_milestone**: _string_

- **deactivate_milestone**: _string_

- **roles**: _string_

- **activate_milestone**: _list of strings_

- **deactivate_milestone**: _list of strings_

- **roles**: _list of strings_

#### Content

- **raw**: _string_ - raw text, possibly markdown

- **rendered**: _string_ - raw text converted to html elements

- **file_info_id**: _uuid_ - id of the file attachment

- **attachments**: _list of uuids_ - list of file info ids which are attached to this content

#### Confirmation

- **text**: _string_

- **control**: _control_

#### Overlay

- **duration**: _optional int_

## Exercise files

### exercise.jsonl

Contains information about the exercise. The object has the following format:

Contains information about the exercise.

The object has the following format:

- **exercise_id**: _int_ - id of the exercise

- **exercise_start**: _timestamp_ - time when the exercise was started

- **finished**: _bool_ - flag that signifies that the exercise actually finished, this is true when

    the entire duration of the exercise was passed

- **definition_id**: _int_ - id of the definition this exercise was created from

- **states**: _list of state objects_ - these objects represent the state of an exercise for the

    teams, one for normal exercises, multiple for on-demand exercises

    - **exercise_state_id**: _int_ - id of the exercise state

    - **status**: _(`Not Started`, `Stopped`, `Running`, `Finished`)_ - the status of the exercise

    - **start_time**: _optional timestamp_ - timestamp when this exercise state was started

    - **finish_time**: _optional timestamp_ - timestamp when this exercise state was finished

    - **elapsed_s**: _int_ - number of in-exercise seconds that elapsed in this exercise state

### teams.jsonl

Contains information about all the exercise teams. Each object has the following format:

Contains information about all the exercise teams.

Each object has the following format:

- **team_id**: _int_ - id of the team

- **name**: _string_ - name of the team

- **users**: _a list of user objects_ - a list of trainees assigned to this team

    - **user_id**: _uuid_ - id of the user

    - **username**: _optional string_ - username of the user, not present if the logs were anonymized

- **achieved_score**: _int_ - score achieved by the team during the exercise, includes score from

    instructor comments

- **total_score**: _int_ - total achievable score during the exercise, does not include score from

    instructor comments

- **exercise_state_id**: _int_ - id of the exercise state for this team

### instructors.jsonl

Contains all instructors assigned to the exercise. Each object has the following format:

Contains all instructors assigned to the exercise.

Each object has the following format:

- **user_id**: _uuid_ - id of the user

- **username**: _optional string_ - username of the user, not present if the logs were anonymized

### exercise_injects.jsonl

Contains all injects for this exercise. Each object has the following format:

Contains all injects for this exercise.

Each object has the following format:

- **inject_id**: _int_ - id of the inject

- **name**: _string_

- **display_name**: _string_

- **time**: _int_

- **delay**: _int_

- **organization**: _string_

### exercise_milestones.jsonl

Contains all milestones for this exercise, which are referenced by milestones in each team. Each

object has the following format:

Contains all milestones for this exercise, which are referenced by milestones in each team.

Each object has the following format:

- **milestone_id**: _int_ - id of the milestone

- **name**: _string_

- **display_name**: _string_

- **description**: _string_

- **tags**: _list of strings_

- **roles**: _string_

- **file_names**: _string_ - space separated list of file names

- **file_names**: _list of strings_ - space separated list of file names

- **initial_state**: _bool_

- **score**: _int_ - score added to the team when this milestone is reached

### exercise_tools.jsonl

Contains all tools for this exercise, which are referenced by action logs in each team. Each object

has the following format:

Contains all tools for this exercise, which are referenced by action logs in each team.

Each object has the following format:

- **tool_id**: _int_ - id of the tool

- **name**: _string_

- **tooltip_description**: _string_

- **default_response**: _content_

- **roles**: _string_

- **roles**: _list of strings_

- **requires_input**: _bool_

- **responses**: _list of response objects_

    - **response_id**: _int_ - id of the response

    - **param**: _string_

    - **content**: _content_

    - **control**: _control_

### exercise_questionnaires.jsonl

Contains all questionnaires for this exercise. Each object has the following format:

Contains all questionnaires for this exercise.

Each object has the following format:

- **questionnaire_id**: _int_ - id of the questionnaire

- **title**: _string_

- **name**: _string_

- **content**: _content_

- **time**: _int_

- **control**: _control_

- **repeatable**: _optional repeatable object_

    - **max_attempts**: _int_

    - **on_fail**: _optional control_

- **questions**: _list of question objects_

    - **question_id**: _int_ - id of the question

    - **content**: _content_

    - **type**: _(`Radio`, `Free-form`)_

    - **type**: _(`Radio`, `Free-form`, `Auto-free-form`, `Multiple-choice`)_

    - **details**: _additional details depending on the question type_

#### _Radio_ question details

- **related_milestone_ids**: _list of int_

#### _Auto-free-form_ question details

- **correct_answer**: _string_

- **correct**: _optional control_

- **incorrect**: _optional control_

#### _Multiple-choice_ question details

- **labels**: _list of strings_

- **correct**: _list of ints_

- **exact_match** _boolean_

### exercise_learning_objectives.jsonl

Contains all learning objectives for the exercise. Each object has the following format:

Contains all learning objectives for the exercise.

Each object has the following format:

- **objective_id**: _int_ - id of the learning objective

- **name**: _string_

- **description**: _string_

- **tags**: _list of strings_

- **total_score**: _int_ - total score achievable by completing all activities in this objective

- **activities**: _list of activity objects_

    - **activity_id**: _int_ - id of the activity

    - **name**: _string_

    - **description**: _string_

    - **tags**: _list of strings_

    - **milestone_ids**: _list of int_

    - **total_score**: _int_ - total score achievable by reaching all milestones linked to this

        activity

### exercise_channels.jsonl

Contains all channels that exist for the exercise.

Each object has the following format:

- **channel_id**: _int_ - id of the channel

- **name**: _string_

- **display_name**: _string_

- **description**: _string_

- **type**: _(`Info`, `Email`, `Tool`, `Form`)_

### email_participants.jsonl

Contains all email participants for this exercise, which are referenced by email threads and emails

in each team. Each object has the following format:

Contains all email participants for this exercise, which are referenced by email threads and emails in each team.

Each object has the following format:

- **participant_id**: _int_ - id of the email participant

- **address**: _string_

    - **address**: _string_

    - **description**: _string_

    - **team_visible**: _boolean_

    - **organization**: _string_

    - **control**: _control_

### file_infos.jsonl

Contains all file infos for this exercise. Each object has the following format:

Contains all file infos for this exercise.

Each object has the following format:

- **file_id**: _uuid_ - id of the file, this is also the name of the file on the file system

- **file_name**: _string_ - original name of the file

- **uploaded_by_id**: _optional uuid_ - uuid of the user that uploaded this file, null if the file

    was not uploaded

- **uploaded_at**: _optional timestamp_ - timestamp when this file was uploaded, null if the file

    was not uploaded

## Individual team files

### inject_states.jsonl

Contains all the inject states for the team. Each object has the following format:

Contains all the inject states for the team.

Each object has the following format:

- **team_id**: _int_ - id of the team

- **inject_id**: _int_ - id of the inject

- **status**: _(`Unsent`, `Delayed`, `Sent`)_ - status of the inject

- **alternative_id**: _int_ - id of the sent alternative

- **alternative_id**: _optional int_ - id of the sent alternative

### questionnaire_states.jsonl

Contains all the questionnaire states for the team. Each object has the following format:

Contains all the questionnaire states for the team.

Each object has the following format:

- **questionnaire_state_id**: _int_ - id of the questionnaire state

- **questionnaire_id**: _int_ - id of the questionnaire

- **team_id**: _int_ - id of the team

- **status**: _(`Unsent`, `Sent`, `Answered`, `Reviewed`)_ - status of the questionnaire

- **timestamp_answered**: _optional timestamp_

- **timestamp_reviewed**: _optional timestamp_

- **reviewed_by_id**: _optional uuid_ - id of the instructor that reviewed the questionnaire

- **submissions**: _list of submission objects_

    - **answers**: _list of answer objects_

        - **question_id**: _int_ - id of the question

    - **answer**: _string_ - submitted answer

    - **is_correct**: _optional bool_ - if specified, flag that marks the answer as correct according

        to the definition

        - **answer**: _list of strings_ - submitted answers for this question

        - **correct**: _(`Correct`, `Incorrect`, `Partially Correct`, `Unknown`)_ - correctness of the

            question as determined by the backend

        - **attempt**: _int_ - attempt number of this answer

    - **attempt**: _int_ - attempt number of this submission

    - **accepted**: _boolean_ - flag that determines whether this submission was accepted by the

        platform, controlled by the repeatable field on the questionnaire

### milestones.jsonl

Contains all milestone states for the team. Each object has the following format:

Contains all milestone states for the team.

Each object has the following format:

- **milestone_state_id**: _int_ - id of the milestone state

- **milestone_id**: _int_ - id of the milestone this state belongs to, the referenced milestone can

    be found in [exercise_milestones.jsonl](#exercisemilestonesjsonl)

- **reached**: _bool_ - state of the milestone

- **timestamp_reached**: _optional timestamp_ - time of the last state change for this milestone

- **history**: _list of history objects_

    - **reached**: _bool_ - state of the milestone at that point

    - **timestamp_from**: _timestamp_ - timestamp of the state change

### emails.jsonl

Optional file, included only if the email feature is enabled. Contains all email threads and emails

for the team. Each object has the following format:

Optional file, included only if the email feature is enabled.

Contains all email threads and emails for the team.

Each object has the following format:

- **thread_id**: _int_ - id of the email thread

- **subject**: _string_

    - **email_id**: _int_ - id of the email

    - **thread_id**: _int_ - id of the thread this email was sent to

    - **sender_id**: _int_ - id of the email participant that sent the email

    - **user_id**: _uuid_ - id of the user that sent the email

    - **timestamp**: _timestamp_ - time when this email was sent

    - **content**: _content_

### action_logs.jsonl

Contains all the action logs for the team. Each object has the following format:

Contains all the action logs for the team.

Each object has the following format:

- **action_log_id**: _int_ - id of the action log

- **type**: _(`Inject`, `Custom Inject`, `Form`, `Tool`, `Email`)_ - type of the action log

- **type**: _(`Inject`, `Custom Inject`, `Tool`, `Email`, `Form`, `Form Submission`, `Form Review`,

    `Confirmation`, `File Download`, `Milestone Modification`)_ - type of the action log

- **timestamp**: _timestamp_ - time when this action log was created

- **channel_id**: _int_ - id of the channel this action log was sent to

- **instructor_comment**: _optional instructor comment_

    - **comment**: _string_ - the content of the comment

    - **score**: _int_ - score value assigned to the comment, this value is added to the team's

        achieved score, can be negative

    - **created_by_id**: _optional uuid_ - uuid of the instructor that created this comment

    - **created_at**: _timestamp_ - timestamp when this comment was created

    - **edited_by_id**: _optional uuid_ - uuid of the instructor that edited this comment

    - **edited_at**: _optional timestamp_ - timestamp when this comment was edited

- **previous_log_id**: _optional int_ - optional id of an action log that is connected to this

    action log

- **user_id**: _optional uuid_ - optional uuid of the user that performed this action, null for

    automatic actions performed by the platform

- **in_exercise_time**: _int_ - the in-exercise-time value when this action happened

- **details**: _additional details depending on the action log type_

#### _Inject_ details

- **inject_id**: _int_ - id of the inject

- **content**: _content_

- **confirmation**: _optional confirmation object_

- **confirmed**: _optional bool_ - included only if the alternative contains a confirmation,

    specifies whether it has been confirmed

- **confirmed_by_id**: _optional uuid_ - id of the user that confirmed this inject

- **timestamp_confirmed**: _optional timestamp_ - timestamp when this inject was confirmed

- **overlay**: _optional overlay_

#### _Custom_ inject details

#### _Custom Inject_ details

- **content**: _content_

- **user_id**: _uuid_ - id of the instructor that sent this custom inject

- **overlay**: _optional overlay_

#### _Form_ inject details

- **questionnaire_state_id**: _int_ - id of the sent questionnaire state

#### _Tool_ inject details

#### _Tool_ details

- **tool_id**: _int_ - id of the used tool

- **argument**: _string_ - argument provided to the tool

- **content**: _content_

- **user_id**: _uuid_ - id of the user that used the tool

- **selected_response_id**: _optional int_ - id of the selected tool response

#### _Email_ inject details

#### _Email_ details

Same fields as described in `emails.jsonl` in the `emails` field.

#### _Form_ details

- **questionnaire_state_id**: _int_ - id of the sent questionnaire state

#### _Form Submission_ details

Same fields as described in `questionnaire_states.jsonl` in the `submissions` field.

#### _Form Review_ details

This object currently contains no additional fields.

#### _Confirmation_ details

This object currently contains no additional fields.

#### _File Download_ details

- **file_info_id**: _uuid_ - uuid of the file info that was downloaded

#### _Milestone Modification_ details

- **activated_milestone_states**: _list of ints_ - ids of the milestone states that were activated

- **deactivated_milestone_states**: _list of ints_ - ids of the milestone states that were

    deactivated

- **cause**: _(0, 2, 4)_ - the cause for this milestone modification, 0 for trainee action, 2 for

    instructor action, 4 for automatic action

## Comparing logs from multiple exercises

The logs are constructed in a way that should allow for simple comparison of logs from multiple

exercises with the same definition. It has to be the exact same upload of the definition, otherwise

the IDs of `injects`, `tools` and other definition data will not match.

exercises with the same definition.

It has to be the exact same upload of the definition,

otherwise the IDs of `injects`, `tools` and other definition data will not match.

Between different runs of the same definition, the `exercise_injects.jsonl`,

`exercise_milestones.jsonl` and `exercise_tools.jsonl` files will be the same. The

`email_participants.jsonl` will not match because email participants need to be re-generated for

every exercise. However, the definition address they are linked to will have the same ID in all

runs.

`exercise_milestones.jsonl` and `exercise_tools.jsonl` files will be the same.

The `email_participants.jsonl` will not match because email participants

need to be re-generated for every exercise.

However, the definition address they are linked to will have the same ID in all runs.