Loading docs/tech/log-format.md +174 −69 Original line number Diff line number Diff line INJECT allows for export of logs from an exercise for analysis purposes. The log files use the [JSONL](https://jsonlines.org/) format. Bellow is description of the structure and schema of logs. INJECT allows for export of logs from an exercise for analysis purposes. The log files use the [JSONL](https://jsonlines.org/) format. Bellow is description of the structure and schema of logs. ### Structure Loading @@ -19,19 +20,22 @@ INJECT allows for export of logs from an exercise for analysis purposes. The log - files used by the definition - exercise.jsonl - teams.jsonl - exercise_injects.jsonl - exercise_milestones.jsonl - instructors.jsonl - file_infos.jsonl - exercise_tools.jsonl - email_participants.jsonl - exercise_questionnaires.jsonl - exercise_milestones.jsonl - exercise_learning_objectives.jsonl - file_infos.jsonl - instructors.jsonl - exercise_injects.jsonl - exercise_channels.jsonl - email_participants.jsonl ``` ## Description and format of individual files Many of the fields described here are also described in the definition format which can be found Many of the fields described here do not contain a description. That is because these fields are direct copies of the fields described in the definition format, which can be found [here](https://gitlab.fi.muni.cz/inject/backend/-/blob/main/definitions/README.md?ref_type=heads). ### Timestamps Loading @@ -43,36 +47,46 @@ All fields with the `timestamp` type are timestamps in `ISO 8601` format. #### Control - **milestone_condition**: _string_ - **activate_milestone**: _string_ - **deactivate_milestone**: _string_ - **roles**: _string_ - **activate_milestone**: _list of strings_ - **deactivate_milestone**: _list of strings_ - **roles**: _list of strings_ #### Content - **raw**: _string_ - raw text, possibly markdown - **rendered**: _string_ - raw text converted to html elements - **file_info_id**: _uuid_ - id of the file attachment - **attachments**: _list of uuids_ - list of file info ids which are attached to this content #### Confirmation - **text**: _string_ - **control**: _control_ #### Overlay - **duration**: _optional int_ ## Exercise files ### exercise.jsonl Contains information about the exercise. The object has the following format: Contains information about the exercise. The object has the following format: - **exercise_id**: _int_ - id of the exercise - **exercise_start**: _timestamp_ - time when the exercise was started - **finished**: _bool_ - flag that signifies that the exercise actually finished, this is true when the entire duration of the exercise was passed - **definition_id**: _int_ - id of the definition this exercise was created from - **states**: _list of state objects_ - these objects represent the state of an exercise for the teams, one for normal exercises, multiple for on-demand exercises - **exercise_state_id**: _int_ - id of the exercise state - **status**: _(`Not Started`, `Stopped`, `Running`, `Finished`)_ - the status of the exercise - **start_time**: _optional timestamp_ - timestamp when this exercise state was started - **finish_time**: _optional timestamp_ - timestamp when this exercise state was finished - **elapsed_s**: _int_ - number of in-exercise seconds that elapsed in this exercise state ### teams.jsonl Contains information about all the exercise teams. Each object has the following format: Contains information about all the exercise teams. Each object has the following format: - **team_id**: _int_ - id of the team - **name**: _string_ - name of the team Loading @@ -82,20 +96,28 @@ Contains information about all the exercise teams. Each object has the following - **users**: _a list of user objects_ - a list of trainees assigned to this team - **user_id**: _uuid_ - id of the user - **username**: _optional string_ - username of the user, not present if the logs were anonymized - **achieved_score**: _int_ - score achieved by the team during the exercise, includes score from instructor comments - **total_score**: _int_ - total achievable score during the exercise, does not include score from instructor comments - **exercise_state_id**: _int_ - id of the exercise state for this team ### instructors.jsonl Contains all instructors assigned to the exercise. Each object has the following format: Contains all instructors assigned to the exercise. Each object has the following format: - **user_id**: _uuid_ - id of the user - **username**: _optional string_ - username of the user, not present if the logs were anonymized ### exercise_injects.jsonl Contains all injects for this exercise. Each object has the following format: Contains all injects for this exercise. Each object has the following format: - **inject_id**: _int_ - id of the inject - **name**: _string_ - **display_name**: _string_ - **time**: _int_ - **delay**: _int_ - **organization**: _string_ Loading Loading @@ -127,30 +149,31 @@ Contains all injects for this exercise. Each object has the following format: ### exercise_milestones.jsonl Contains all milestones for this exercise, which are referenced by milestones in each team. Each object has the following format: Contains all milestones for this exercise, which are referenced by milestones in each team. Each object has the following format: - **milestone_id**: _int_ - id of the milestone - **name**: _string_ - **display_name**: _string_ - **description**: _string_ - **tags**: _list of strings_ - **roles**: _string_ - **file_names**: _string_ - space separated list of file names - **file_names**: _list of strings_ - space separated list of file names - **initial_state**: _bool_ - **score**: _int_ - score added to the team when this milestone is reached ### exercise_tools.jsonl Contains all tools for this exercise, which are referenced by action logs in each team. Each object has the following format: Contains all tools for this exercise, which are referenced by action logs in each team. Each object has the following format: - **tool_id**: _int_ - id of the tool - **name**: _string_ - **tooltip_description**: _string_ - **default_response**: _content_ - **roles**: _string_ - **roles**: _list of strings_ - **requires_input**: _bool_ - **responses**: _list of response objects_ - **response_id**: _int_ - id of the response - **param**: _string_ - **content**: _content_ - **control**: _control_ Loading @@ -159,17 +182,21 @@ has the following format: ### exercise_questionnaires.jsonl Contains all questionnaires for this exercise. Each object has the following format: Contains all questionnaires for this exercise. Each object has the following format: - **questionnaire_id**: _int_ - id of the questionnaire - **title**: _string_ - **name**: _string_ - **content**: _content_ - **time**: _int_ - **control**: _control_ - **repeatable**: _optional repeatable object_ - **max_attempts**: _int_ - **on_fail**: _optional control_ - **questions**: _list of question objects_ - **question_id**: _int_ - id of the question - **content**: _content_ - **type**: _(`Radio`, `Free-form`)_ - **type**: _(`Radio`, `Free-form`, `Auto-free-form`, `Multiple-choice`)_ - **details**: _additional details depending on the question type_ #### _Radio_ question details Loading @@ -182,25 +209,52 @@ Contains all questionnaires for this exercise. Each object has the following for - **related_milestone_ids**: _list of int_ #### _Auto-free-form_ question details - **correct_answer**: _string_ - **correct**: _optional control_ - **incorrect**: _optional control_ #### _Multiple-choice_ question details - **labels**: _list of strings_ - **correct**: _list of ints_ - **exact_match** _boolean_ ### exercise_learning_objectives.jsonl Contains all learning objectives for the exercise. Each object has the following format: Contains all learning objectives for the exercise. Each object has the following format: - **objective_id**: _int_ - id of the learning objective - **name**: _string_ - **description**: _string_ - **tags**: _list of strings_ - **total_score**: _int_ - total score achievable by completing all activities in this objective - **activities**: _list of activity objects_ - **activity_id**: _int_ - id of the activity - **name**: _string_ - **description**: _string_ - **tags**: _list of strings_ - **milestone_ids**: _list of int_ - **total_score**: _int_ - total score achievable by reaching all milestones linked to this activity ### exercise_channels.jsonl Contains all channels that exist for the exercise. Each object has the following format: - **channel_id**: _int_ - id of the channel - **name**: _string_ - **display_name**: _string_ - **description**: _string_ - **type**: _(`Info`, `Email`, `Tool`, `Form`)_ ### email_participants.jsonl Contains all email participants for this exercise, which are referenced by email threads and emails in each team. Each object has the following format: Contains all email participants for this exercise, which are referenced by email threads and emails in each team. Each object has the following format: - **participant_id**: _int_ - id of the email participant - **address**: _string_ Loading @@ -213,30 +267,39 @@ in each team. Each object has the following format: - **address**: _string_ - **description**: _string_ - **team_visible**: _boolean_ - **organization**: _string_ - **control**: _control_ ### file_infos.jsonl Contains all file infos for this exercise. Each object has the following format: Contains all file infos for this exercise. Each object has the following format: - **file_id**: _uuid_ - id of the file, this is also the name of the file on the file system - **file_name**: _string_ - original name of the file - **uploaded_by_id**: _optional uuid_ - uuid of the user that uploaded this file, null if the file was not uploaded - **uploaded_at**: _optional timestamp_ - timestamp when this file was uploaded, null if the file was not uploaded ## Individual team files ### inject_states.jsonl Contains all the inject states for the team. Each object has the following format: Contains all the inject states for the team. Each object has the following format: - **team_id**: _int_ - id of the team - **inject_id**: _int_ - id of the inject - **status**: _(`Unsent`, `Delayed`, `Sent`)_ - status of the inject - **alternative_id**: _int_ - id of the sent alternative - **alternative_id**: _optional int_ - id of the sent alternative ### questionnaire_states.jsonl Contains all the questionnaire states for the team. Each object has the following format: Contains all the questionnaire states for the team. Each object has the following format: - **questionnaire_state_id**: _int_ - id of the questionnaire state - **questionnaire_id**: _int_ - id of the questionnaire - **team_id**: _int_ - id of the team - **status**: _(`Unsent`, `Sent`, `Answered`, `Reviewed`)_ - status of the questionnaire Loading @@ -244,28 +307,33 @@ Contains all the questionnaire states for the team. Each object has the followin - **timestamp_answered**: _optional timestamp_ - **timestamp_reviewed**: _optional timestamp_ - **reviewed_by_id**: _optional uuid_ - id of the instructor that reviewed the questionnaire - **submissions**: _list of submission objects_ - **answers**: _list of answer objects_ - **question_id**: _int_ - id of the question - **answer**: _string_ - submitted answer - **is_correct**: _optional bool_ - if specified, flag that marks the answer as correct according to the definition - **answer**: _list of strings_ - submitted answers for this question - **correct**: _(`Correct`, `Incorrect`, `Partially Correct`, `Unknown`)_ - correctness of the question as determined by the backend - **attempt**: _int_ - attempt number of this answer - **attempt**: _int_ - attempt number of this submission - **accepted**: _boolean_ - flag that determines whether this submission was accepted by the platform, controlled by the repeatable field on the questionnaire ### milestones.jsonl Contains all milestone states for the team. Each object has the following format: Contains all milestone states for the team. Each object has the following format: - **milestone_state_id**: _int_ - id of the milestone state - **milestone_id**: _int_ - id of the milestone this state belongs to, the referenced milestone can be found in [exercise_milestones.jsonl](#exercisemilestonesjsonl) - **reached**: _bool_ - state of the milestone - **timestamp_reached**: _optional timestamp_ - time of the last state change for this milestone - **history**: _list of history objects_ - **reached**: _bool_ - state of the milestone at that point - **timestamp_from**: _timestamp_ - timestamp of the state change ### emails.jsonl Optional file, included only if the email feature is enabled. Contains all email threads and emails for the team. Each object has the following format: Optional file, included only if the email feature is enabled. Contains all email threads and emails for the team. Each object has the following format: - **thread_id**: _int_ - id of the email thread - **subject**: _string_ Loading @@ -276,17 +344,32 @@ for the team. Each object has the following format: - **email_id**: _int_ - id of the email - **thread_id**: _int_ - id of the thread this email was sent to - **sender_id**: _int_ - id of the email participant that sent the email - **user_id**: _uuid_ - id of the user that sent the email - **timestamp**: _timestamp_ - time when this email was sent - **content**: _content_ ### action_logs.jsonl Contains all the action logs for the team. Each object has the following format: Contains all the action logs for the team. Each object has the following format: - **action_log_id**: _int_ - id of the action log - **type**: _(`Inject`, `Custom Inject`, `Form`, `Tool`, `Email`)_ - type of the action log - **type**: _(`Inject`, `Custom Inject`, `Tool`, `Email`, `Form`, `Form Submission`, `Form Review`, `Confirmation`, `File Download`, `Milestone Modification`)_ - type of the action log - **timestamp**: _timestamp_ - time when this action log was created - **channel_id**: _int_ - id of the channel this action log was sent to - **instructor_comment**: _optional instructor comment_ - **comment**: _string_ - the content of the comment - **score**: _int_ - score value assigned to the comment, this value is added to the team's achieved score, can be negative - **created_by_id**: _optional uuid_ - uuid of the instructor that created this comment - **created_at**: _timestamp_ - timestamp when this comment was created - **edited_by_id**: _optional uuid_ - uuid of the instructor that edited this comment - **edited_at**: _optional timestamp_ - timestamp when this comment was edited - **previous_log_id**: _optional int_ - optional id of an action log that is connected to this action log - **user_id**: _optional uuid_ - optional uuid of the user that performed this action, null for automatic actions performed by the platform - **in_exercise_time**: _int_ - the in-exercise-time value when this action happened - **details**: _additional details depending on the action log type_ #### _Inject_ details Loading @@ -294,39 +377,61 @@ Contains all the action logs for the team. Each object has the following format: - **inject_id**: _int_ - id of the inject - **content**: _content_ - **confirmation**: _optional confirmation object_ - **confirmed**: _optional bool_ - included only if the alternative contains a confirmation, specifies whether it has been confirmed - **confirmed_by_id**: _optional uuid_ - id of the user that confirmed this inject - **timestamp_confirmed**: _optional timestamp_ - timestamp when this inject was confirmed - **overlay**: _optional overlay_ #### _Custom_ inject details #### _Custom Inject_ details - **content**: _content_ - **user_id**: _uuid_ - id of the instructor that sent this custom inject - **overlay**: _optional overlay_ #### _Form_ inject details - **questionnaire_state_id**: _int_ - id of the sent questionnaire state #### _Tool_ inject details #### _Tool_ details - **tool_id**: _int_ - id of the used tool - **argument**: _string_ - argument provided to the tool - **content**: _content_ - **user_id**: _uuid_ - id of the user that used the tool - **selected_response_id**: _optional int_ - id of the selected tool response #### _Email_ inject details #### _Email_ details Same fields as described in `emails.jsonl` in the `emails` field. #### _Form_ details - **questionnaire_state_id**: _int_ - id of the sent questionnaire state #### _Form Submission_ details Same fields as described in `questionnaire_states.jsonl` in the `submissions` field. #### _Form Review_ details This object currently contains no additional fields. #### _Confirmation_ details This object currently contains no additional fields. #### _File Download_ details - **file_info_id**: _uuid_ - uuid of the file info that was downloaded #### _Milestone Modification_ details - **activated_milestone_states**: _list of ints_ - ids of the milestone states that were activated - **deactivated_milestone_states**: _list of ints_ - ids of the milestone states that were deactivated - **cause**: _(0, 2, 4)_ - the cause for this milestone modification, 0 for trainee action, 2 for instructor action, 4 for automatic action ## Comparing logs from multiple exercises The logs are constructed in a way that should allow for simple comparison of logs from multiple exercises with the same definition. It has to be the exact same upload of the definition, otherwise the IDs of `injects`, `tools` and other definition data will not match. exercises with the same definition. It has to be the exact same upload of the definition, otherwise the IDs of `injects`, `tools` and other definition data will not match. Between different runs of the same definition, the `exercise_injects.jsonl`, `exercise_milestones.jsonl` and `exercise_tools.jsonl` files will be the same. The `email_participants.jsonl` will not match because email participants need to be re-generated for every exercise. However, the definition address they are linked to will have the same ID in all runs. `exercise_milestones.jsonl` and `exercise_tools.jsonl` files will be the same. The `email_participants.jsonl` will not match because email participants need to be re-generated for every exercise. However, the definition address they are linked to will have the same ID in all runs. Loading
docs/tech/log-format.md +174 −69 Original line number Diff line number Diff line INJECT allows for export of logs from an exercise for analysis purposes. The log files use the [JSONL](https://jsonlines.org/) format. Bellow is description of the structure and schema of logs. INJECT allows for export of logs from an exercise for analysis purposes. The log files use the [JSONL](https://jsonlines.org/) format. Bellow is description of the structure and schema of logs. ### Structure Loading @@ -19,19 +20,22 @@ INJECT allows for export of logs from an exercise for analysis purposes. The log - files used by the definition - exercise.jsonl - teams.jsonl - exercise_injects.jsonl - exercise_milestones.jsonl - instructors.jsonl - file_infos.jsonl - exercise_tools.jsonl - email_participants.jsonl - exercise_questionnaires.jsonl - exercise_milestones.jsonl - exercise_learning_objectives.jsonl - file_infos.jsonl - instructors.jsonl - exercise_injects.jsonl - exercise_channels.jsonl - email_participants.jsonl ``` ## Description and format of individual files Many of the fields described here are also described in the definition format which can be found Many of the fields described here do not contain a description. That is because these fields are direct copies of the fields described in the definition format, which can be found [here](https://gitlab.fi.muni.cz/inject/backend/-/blob/main/definitions/README.md?ref_type=heads). ### Timestamps Loading @@ -43,36 +47,46 @@ All fields with the `timestamp` type are timestamps in `ISO 8601` format. #### Control - **milestone_condition**: _string_ - **activate_milestone**: _string_ - **deactivate_milestone**: _string_ - **roles**: _string_ - **activate_milestone**: _list of strings_ - **deactivate_milestone**: _list of strings_ - **roles**: _list of strings_ #### Content - **raw**: _string_ - raw text, possibly markdown - **rendered**: _string_ - raw text converted to html elements - **file_info_id**: _uuid_ - id of the file attachment - **attachments**: _list of uuids_ - list of file info ids which are attached to this content #### Confirmation - **text**: _string_ - **control**: _control_ #### Overlay - **duration**: _optional int_ ## Exercise files ### exercise.jsonl Contains information about the exercise. The object has the following format: Contains information about the exercise. The object has the following format: - **exercise_id**: _int_ - id of the exercise - **exercise_start**: _timestamp_ - time when the exercise was started - **finished**: _bool_ - flag that signifies that the exercise actually finished, this is true when the entire duration of the exercise was passed - **definition_id**: _int_ - id of the definition this exercise was created from - **states**: _list of state objects_ - these objects represent the state of an exercise for the teams, one for normal exercises, multiple for on-demand exercises - **exercise_state_id**: _int_ - id of the exercise state - **status**: _(`Not Started`, `Stopped`, `Running`, `Finished`)_ - the status of the exercise - **start_time**: _optional timestamp_ - timestamp when this exercise state was started - **finish_time**: _optional timestamp_ - timestamp when this exercise state was finished - **elapsed_s**: _int_ - number of in-exercise seconds that elapsed in this exercise state ### teams.jsonl Contains information about all the exercise teams. Each object has the following format: Contains information about all the exercise teams. Each object has the following format: - **team_id**: _int_ - id of the team - **name**: _string_ - name of the team Loading @@ -82,20 +96,28 @@ Contains information about all the exercise teams. Each object has the following - **users**: _a list of user objects_ - a list of trainees assigned to this team - **user_id**: _uuid_ - id of the user - **username**: _optional string_ - username of the user, not present if the logs were anonymized - **achieved_score**: _int_ - score achieved by the team during the exercise, includes score from instructor comments - **total_score**: _int_ - total achievable score during the exercise, does not include score from instructor comments - **exercise_state_id**: _int_ - id of the exercise state for this team ### instructors.jsonl Contains all instructors assigned to the exercise. Each object has the following format: Contains all instructors assigned to the exercise. Each object has the following format: - **user_id**: _uuid_ - id of the user - **username**: _optional string_ - username of the user, not present if the logs were anonymized ### exercise_injects.jsonl Contains all injects for this exercise. Each object has the following format: Contains all injects for this exercise. Each object has the following format: - **inject_id**: _int_ - id of the inject - **name**: _string_ - **display_name**: _string_ - **time**: _int_ - **delay**: _int_ - **organization**: _string_ Loading Loading @@ -127,30 +149,31 @@ Contains all injects for this exercise. Each object has the following format: ### exercise_milestones.jsonl Contains all milestones for this exercise, which are referenced by milestones in each team. Each object has the following format: Contains all milestones for this exercise, which are referenced by milestones in each team. Each object has the following format: - **milestone_id**: _int_ - id of the milestone - **name**: _string_ - **display_name**: _string_ - **description**: _string_ - **tags**: _list of strings_ - **roles**: _string_ - **file_names**: _string_ - space separated list of file names - **file_names**: _list of strings_ - space separated list of file names - **initial_state**: _bool_ - **score**: _int_ - score added to the team when this milestone is reached ### exercise_tools.jsonl Contains all tools for this exercise, which are referenced by action logs in each team. Each object has the following format: Contains all tools for this exercise, which are referenced by action logs in each team. Each object has the following format: - **tool_id**: _int_ - id of the tool - **name**: _string_ - **tooltip_description**: _string_ - **default_response**: _content_ - **roles**: _string_ - **roles**: _list of strings_ - **requires_input**: _bool_ - **responses**: _list of response objects_ - **response_id**: _int_ - id of the response - **param**: _string_ - **content**: _content_ - **control**: _control_ Loading @@ -159,17 +182,21 @@ has the following format: ### exercise_questionnaires.jsonl Contains all questionnaires for this exercise. Each object has the following format: Contains all questionnaires for this exercise. Each object has the following format: - **questionnaire_id**: _int_ - id of the questionnaire - **title**: _string_ - **name**: _string_ - **content**: _content_ - **time**: _int_ - **control**: _control_ - **repeatable**: _optional repeatable object_ - **max_attempts**: _int_ - **on_fail**: _optional control_ - **questions**: _list of question objects_ - **question_id**: _int_ - id of the question - **content**: _content_ - **type**: _(`Radio`, `Free-form`)_ - **type**: _(`Radio`, `Free-form`, `Auto-free-form`, `Multiple-choice`)_ - **details**: _additional details depending on the question type_ #### _Radio_ question details Loading @@ -182,25 +209,52 @@ Contains all questionnaires for this exercise. Each object has the following for - **related_milestone_ids**: _list of int_ #### _Auto-free-form_ question details - **correct_answer**: _string_ - **correct**: _optional control_ - **incorrect**: _optional control_ #### _Multiple-choice_ question details - **labels**: _list of strings_ - **correct**: _list of ints_ - **exact_match** _boolean_ ### exercise_learning_objectives.jsonl Contains all learning objectives for the exercise. Each object has the following format: Contains all learning objectives for the exercise. Each object has the following format: - **objective_id**: _int_ - id of the learning objective - **name**: _string_ - **description**: _string_ - **tags**: _list of strings_ - **total_score**: _int_ - total score achievable by completing all activities in this objective - **activities**: _list of activity objects_ - **activity_id**: _int_ - id of the activity - **name**: _string_ - **description**: _string_ - **tags**: _list of strings_ - **milestone_ids**: _list of int_ - **total_score**: _int_ - total score achievable by reaching all milestones linked to this activity ### exercise_channels.jsonl Contains all channels that exist for the exercise. Each object has the following format: - **channel_id**: _int_ - id of the channel - **name**: _string_ - **display_name**: _string_ - **description**: _string_ - **type**: _(`Info`, `Email`, `Tool`, `Form`)_ ### email_participants.jsonl Contains all email participants for this exercise, which are referenced by email threads and emails in each team. Each object has the following format: Contains all email participants for this exercise, which are referenced by email threads and emails in each team. Each object has the following format: - **participant_id**: _int_ - id of the email participant - **address**: _string_ Loading @@ -213,30 +267,39 @@ in each team. Each object has the following format: - **address**: _string_ - **description**: _string_ - **team_visible**: _boolean_ - **organization**: _string_ - **control**: _control_ ### file_infos.jsonl Contains all file infos for this exercise. Each object has the following format: Contains all file infos for this exercise. Each object has the following format: - **file_id**: _uuid_ - id of the file, this is also the name of the file on the file system - **file_name**: _string_ - original name of the file - **uploaded_by_id**: _optional uuid_ - uuid of the user that uploaded this file, null if the file was not uploaded - **uploaded_at**: _optional timestamp_ - timestamp when this file was uploaded, null if the file was not uploaded ## Individual team files ### inject_states.jsonl Contains all the inject states for the team. Each object has the following format: Contains all the inject states for the team. Each object has the following format: - **team_id**: _int_ - id of the team - **inject_id**: _int_ - id of the inject - **status**: _(`Unsent`, `Delayed`, `Sent`)_ - status of the inject - **alternative_id**: _int_ - id of the sent alternative - **alternative_id**: _optional int_ - id of the sent alternative ### questionnaire_states.jsonl Contains all the questionnaire states for the team. Each object has the following format: Contains all the questionnaire states for the team. Each object has the following format: - **questionnaire_state_id**: _int_ - id of the questionnaire state - **questionnaire_id**: _int_ - id of the questionnaire - **team_id**: _int_ - id of the team - **status**: _(`Unsent`, `Sent`, `Answered`, `Reviewed`)_ - status of the questionnaire Loading @@ -244,28 +307,33 @@ Contains all the questionnaire states for the team. Each object has the followin - **timestamp_answered**: _optional timestamp_ - **timestamp_reviewed**: _optional timestamp_ - **reviewed_by_id**: _optional uuid_ - id of the instructor that reviewed the questionnaire - **submissions**: _list of submission objects_ - **answers**: _list of answer objects_ - **question_id**: _int_ - id of the question - **answer**: _string_ - submitted answer - **is_correct**: _optional bool_ - if specified, flag that marks the answer as correct according to the definition - **answer**: _list of strings_ - submitted answers for this question - **correct**: _(`Correct`, `Incorrect`, `Partially Correct`, `Unknown`)_ - correctness of the question as determined by the backend - **attempt**: _int_ - attempt number of this answer - **attempt**: _int_ - attempt number of this submission - **accepted**: _boolean_ - flag that determines whether this submission was accepted by the platform, controlled by the repeatable field on the questionnaire ### milestones.jsonl Contains all milestone states for the team. Each object has the following format: Contains all milestone states for the team. Each object has the following format: - **milestone_state_id**: _int_ - id of the milestone state - **milestone_id**: _int_ - id of the milestone this state belongs to, the referenced milestone can be found in [exercise_milestones.jsonl](#exercisemilestonesjsonl) - **reached**: _bool_ - state of the milestone - **timestamp_reached**: _optional timestamp_ - time of the last state change for this milestone - **history**: _list of history objects_ - **reached**: _bool_ - state of the milestone at that point - **timestamp_from**: _timestamp_ - timestamp of the state change ### emails.jsonl Optional file, included only if the email feature is enabled. Contains all email threads and emails for the team. Each object has the following format: Optional file, included only if the email feature is enabled. Contains all email threads and emails for the team. Each object has the following format: - **thread_id**: _int_ - id of the email thread - **subject**: _string_ Loading @@ -276,17 +344,32 @@ for the team. Each object has the following format: - **email_id**: _int_ - id of the email - **thread_id**: _int_ - id of the thread this email was sent to - **sender_id**: _int_ - id of the email participant that sent the email - **user_id**: _uuid_ - id of the user that sent the email - **timestamp**: _timestamp_ - time when this email was sent - **content**: _content_ ### action_logs.jsonl Contains all the action logs for the team. Each object has the following format: Contains all the action logs for the team. Each object has the following format: - **action_log_id**: _int_ - id of the action log - **type**: _(`Inject`, `Custom Inject`, `Form`, `Tool`, `Email`)_ - type of the action log - **type**: _(`Inject`, `Custom Inject`, `Tool`, `Email`, `Form`, `Form Submission`, `Form Review`, `Confirmation`, `File Download`, `Milestone Modification`)_ - type of the action log - **timestamp**: _timestamp_ - time when this action log was created - **channel_id**: _int_ - id of the channel this action log was sent to - **instructor_comment**: _optional instructor comment_ - **comment**: _string_ - the content of the comment - **score**: _int_ - score value assigned to the comment, this value is added to the team's achieved score, can be negative - **created_by_id**: _optional uuid_ - uuid of the instructor that created this comment - **created_at**: _timestamp_ - timestamp when this comment was created - **edited_by_id**: _optional uuid_ - uuid of the instructor that edited this comment - **edited_at**: _optional timestamp_ - timestamp when this comment was edited - **previous_log_id**: _optional int_ - optional id of an action log that is connected to this action log - **user_id**: _optional uuid_ - optional uuid of the user that performed this action, null for automatic actions performed by the platform - **in_exercise_time**: _int_ - the in-exercise-time value when this action happened - **details**: _additional details depending on the action log type_ #### _Inject_ details Loading @@ -294,39 +377,61 @@ Contains all the action logs for the team. Each object has the following format: - **inject_id**: _int_ - id of the inject - **content**: _content_ - **confirmation**: _optional confirmation object_ - **confirmed**: _optional bool_ - included only if the alternative contains a confirmation, specifies whether it has been confirmed - **confirmed_by_id**: _optional uuid_ - id of the user that confirmed this inject - **timestamp_confirmed**: _optional timestamp_ - timestamp when this inject was confirmed - **overlay**: _optional overlay_ #### _Custom_ inject details #### _Custom Inject_ details - **content**: _content_ - **user_id**: _uuid_ - id of the instructor that sent this custom inject - **overlay**: _optional overlay_ #### _Form_ inject details - **questionnaire_state_id**: _int_ - id of the sent questionnaire state #### _Tool_ inject details #### _Tool_ details - **tool_id**: _int_ - id of the used tool - **argument**: _string_ - argument provided to the tool - **content**: _content_ - **user_id**: _uuid_ - id of the user that used the tool - **selected_response_id**: _optional int_ - id of the selected tool response #### _Email_ inject details #### _Email_ details Same fields as described in `emails.jsonl` in the `emails` field. #### _Form_ details - **questionnaire_state_id**: _int_ - id of the sent questionnaire state #### _Form Submission_ details Same fields as described in `questionnaire_states.jsonl` in the `submissions` field. #### _Form Review_ details This object currently contains no additional fields. #### _Confirmation_ details This object currently contains no additional fields. #### _File Download_ details - **file_info_id**: _uuid_ - uuid of the file info that was downloaded #### _Milestone Modification_ details - **activated_milestone_states**: _list of ints_ - ids of the milestone states that were activated - **deactivated_milestone_states**: _list of ints_ - ids of the milestone states that were deactivated - **cause**: _(0, 2, 4)_ - the cause for this milestone modification, 0 for trainee action, 2 for instructor action, 4 for automatic action ## Comparing logs from multiple exercises The logs are constructed in a way that should allow for simple comparison of logs from multiple exercises with the same definition. It has to be the exact same upload of the definition, otherwise the IDs of `injects`, `tools` and other definition data will not match. exercises with the same definition. It has to be the exact same upload of the definition, otherwise the IDs of `injects`, `tools` and other definition data will not match. Between different runs of the same definition, the `exercise_injects.jsonl`, `exercise_milestones.jsonl` and `exercise_tools.jsonl` files will be the same. The `email_participants.jsonl` will not match because email participants need to be re-generated for every exercise. However, the definition address they are linked to will have the same ID in all runs. `exercise_milestones.jsonl` and `exercise_tools.jsonl` files will be the same. The `email_participants.jsonl` will not match because email participants need to be re-generated for every exercise. However, the definition address they are linked to will have the same ID in all runs.
