pwndocImportAutomator tagshttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags2023-02-06T15:46:12+01:00https://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2023-02-05release-2023-02-05<p data-sourcepos="1:1-1:9" dir="auto">Changelog</p>
<ul data-sourcepos="3:1-5:57" dir="auto">
<li data-sourcepos="3:1-3:152">Switch from pwndoc to pwndoc-ng (see <a data-sourcepos="3:40-3:151" href="https://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator#version-2023-02-05">relevant section of readme.md</a>)</li>
<li data-sourcepos="4:1-4:53">Fix scope in CSV and JSON output of PwnDoc findings</li>
<li data-sourcepos="5:1-5:57">Output PwnDoc findings in JSON without unicode escaping</li>
</ul>2023-02-06T15:46:12+01:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2023-01-04release-2023-01-04<p data-sourcepos="1:1-1:13" dir="auto">Release note:</p>
<ul data-sourcepos="3:1-3:58" dir="auto">
<li data-sourcepos="3:1-3:58">Fix (and test) export of audit findings as CSV and JSON</li>
</ul>2023-01-04T19:55:44+01:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-12-20release-2022-12-20<p data-sourcepos="1:1-1:13" dir="auto">Release notes</p>
<ul data-sourcepos="2:1-3:64" dir="auto">
<li data-sourcepos="2:1-2:60">Add new finding sorting option - CVSS first, then by title</li>
<li data-sourcepos="3:1-3:64">Fix docx template - table of findings (show critical findings)</li>
</ul>2022-12-20T14:11:34+01:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-12-18release-2022-12-18<p data-sourcepos="1:1-1:9" dir="auto">Changelog</p>
<ul data-sourcepos="2:1-5:24" dir="auto">
<li data-sourcepos="2:1-2:39">Fix sorting of IPv4 and IPv6 together</li>
<li data-sourcepos="3:1-3:15">Speedup tests</li>
<li data-sourcepos="4:1-4:74">Take port for Websocket dynamically inside the browser, not during build</li>
<li data-sourcepos="5:1-5:24">Cleanup frontend build</li>
</ul>2022-12-19T00:44:28+01:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-12-12release-2022-12-12<p data-sourcepos="1:1-1:14" dir="auto">Release notes:</p>
<ul data-sourcepos="3:1-8:48" dir="auto">
<li data-sourcepos="3:1-3:50">Fix single threaded init (avoid race conditions)</li>
<li data-sourcepos="4:1-4:52">Remove unused docx templates, add English template</li>
<li data-sourcepos="5:1-5:46">Add table of open ports to default templates</li>
<li data-sourcepos="6:1-6:40">Add template with CVSS severity colors</li>
<li data-sourcepos="7:1-7:62">PwnDoc: Persist fake CVSS score set based on custom severity</li>
<li data-sourcepos="8:1-8:48">Default import locale to PwnDoc's audit locale</li>
</ul>2022-12-12T13:45:16+01:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-12-04release-2022-12-04<p data-sourcepos="1:1-1:13" dir="auto">Release notes</p>
<ul data-sourcepos="3:1-7:0" dir="auto">
<li data-sourcepos="3:1-3:67">Avoid race conditions during first run - init using single thread</li>
<li data-sourcepos="4:1-4:48">Switch scan2report submodule from ssh to https</li>
<li data-sourcepos="5:1-5:61">Add instructions for first run - insecure, but fast version</li>
<li data-sourcepos="6:1-7:0">Import scan2report public templates during first run</li>
</ul>
<p data-sourcepos="8:1-8:22" dir="auto">Various smaller fixes.</p>2022-12-04T19:05:01+01:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-12-01release-2022-12-01<p data-sourcepos="1:1-1:9" dir="auto">Changelog</p>
<ul data-sourcepos="3:1-11:0" dir="auto">
<li data-sourcepos="3:1-3:18">Test Scan2Report</li>
<li data-sourcepos="4:1-7:61">Update Scan2Report after <a data-sourcepos="4:28-4:128" href="https://gitlab.fi.muni.cz/cybersec/tns/scan2report/-/releases/release-2022-12-01">major refactoring</a>
<ul data-sourcepos="5:3-7:61">
<li data-sourcepos="5:3-7:61">Relevant for Importer:
<ul data-sourcepos="6:5-7:61">
<li data-sourcepos="6:5-6:153">Change location of Scan2Report templates and configuration (usage of Importer should not overwrite the the data from Scan2Report for easier update)</li>
<li data-sourcepos="7:5-7:61">Pass config using <code data-sourcepos="7:26-7:42">Scan2ReportConfig</code> instead of <code data-sourcepos="7:57-7:60">argv</code>
</li>
</ul>
</li>
</ul>
</li>
<li data-sourcepos="8:1-8:27">Release under MIT license</li>
<li data-sourcepos="9:1-9:106">Don't reorder findings unless they are over 1MB (keep original order from Scan2Report: severity -> name)</li>
<li data-sourcepos="10:1-11:0">Update backup script</li>
</ul>
<p data-sourcepos="12:1-12:141" dir="auto">Note that this release may require <a data-sourcepos="12:36-12:141" href="https://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator#version-2022-12-01">a manual step to update</a></p>2022-12-01T11:20:41+01:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-11-15release-2022-11-15<p data-sourcepos="1:1-1:13" dir="auto">Release notes</p>
<ul data-sourcepos="2:1-3:96" dir="auto">
<li data-sourcepos="2:1-2:49">Order findings in docx and UI by severity_value</li>
<li data-sourcepos="3:1-3:96">Fix description of grouped findings (order of Description A, PoC A, Description B, PoC B, ...)</li>
</ul>2022-11-15T00:43:54+01:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-11-14release-2022-11-14<p data-sourcepos="1:1-1:13" dir="auto">Release notes</p>
<ul data-sourcepos="2:1-2:24" dir="auto">
<li data-sourcepos="2:1-2:24">Fix PwnDoc API re-auth</li>
</ul>2022-11-13T23:00:18+01:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-11-13release-2022-11-13<p data-sourcepos="1:1-1:20" dir="auto">Highlighted changes:</p>
<ul data-sourcepos="2:1-11:0" dir="auto">
<li data-sourcepos="2:1-2:63">Fix template versioning (don’t create unnecessary versions)</li>
<li data-sourcepos="3:1-3:70">Fix multilang template imports (don’t escape old template locales)</li>
<li data-sourcepos="4:1-4:46">Fix list of findings which should be grouped</li>
<li data-sourcepos="5:1-5:57">Described most common workflows directly in Importer UI</li>
<li data-sourcepos="6:1-6:39">Improvements to documentation on Wiki</li>
<li data-sourcepos="7:1-7:44">Add caching for HTML formatting (speed up)</li>
<li data-sourcepos="8:1-8:49">Add CSV and JSON export for findings from audit</li>
<li data-sourcepos="9:1-9:71">Put whole scan2report description into both description and Proof/PoC</li>
<li data-sourcepos="10:1-11:0">Sort IPs and ports in table of opened ports</li>
</ul>
<p data-sourcepos="12:1-12:14" dir="auto">Other changes:</p>
<ul data-sourcepos="13:1-25:0" dir="auto">
<li data-sourcepos="13:1-13:46">Add error msg for duplicate FID inside audit</li>
<li data-sourcepos="14:1-14:28">Set height for JSON editor</li>
<li data-sourcepos="15:1-15:37">Stop step timer if processing fails</li>
<li data-sourcepos="16:1-16:35">Safeguard BeautifulSoup formatter</li>
<li data-sourcepos="17:1-17:55">Add button to directly save proposed template version</li>
<li data-sourcepos="18:1-18:80">Return status messages as webpages, not string (i.e. show web menu everywhere)</li>
<li data-sourcepos="19:1-19:23">Added many more tests</li>
<li data-sourcepos="20:1-20:30">Add missing test input files</li>
<li data-sourcepos="21:1-21:26">Modified PwnDoc API auth</li>
<li data-sourcepos="22:1-22:41">Added links between PwnDoc and Importer</li>
<li data-sourcepos="23:1-23:115">Drop non-critical fields (original_description_from_scan/template) if the finding is otherwise too big for import</li>
<li data-sourcepos="24:1-25:0">Upload small findings first (speed improvement)</li>
</ul>
<p data-sourcepos="26:1-26:49" dir="auto">And many more improvements in the implementation.</p>2022-11-13T22:43:34+01:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-11-02release-2022-11-02<p data-sourcepos="1:1-1:13" dir="auto">Release notes</p>
<ul data-sourcepos="3:1-5:49" dir="auto">
<li data-sourcepos="3:1-3:59">Delete template in Importer if deleted in PwnDoc (bugfix)</li>
<li data-sourcepos="4:1-4:60">Add debug action: Re-synchronize FID to PwnDoc ID mappings</li>
<li data-sourcepos="5:1-5:49">Show warning/errors flashes for debug endpoints</li>
</ul>2022-11-02T22:17:15+01:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-10-31release-2022-10-31<p data-sourcepos="1:1-1:14" dir="auto">Release notes:</p>
<ul data-sourcepos="3:1-8:0" dir="auto">
<li data-sourcepos="3:1-3:57">move Importer DB update trigger to PwnDoc template save</li>
<li data-sourcepos="4:1-4:31">add list of template versions</li>
<li data-sourcepos="5:1-5:46">add page with warnings for template problems</li>
<li data-sourcepos="6:1-6:66">additional fixes to pwndoc HTML cleanup (and covered with tests)</li>
<li data-sourcepos="7:1-8:0">test grouping, aliases, and DB (db test indicates an existing problem)</li>
</ul>2022-10-31T14:15:54+01:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-10-27release-2022-10-27<p data-sourcepos="1:1-1:14" dir="auto">Release notes:</p>
<ul data-sourcepos="3:1-11:0" dir="auto">
<li data-sourcepos="3:1-3:44">Update PwnDoc to 0.5.3 (general bug fixes)</li>
<li data-sourcepos="4:1-4:114">Pin PwnDoc Frontend Node version to 16 (until we fix it <a data-sourcepos="4:59-4:113" href="https://github.com/pwndoc/pwndoc/issues/383" rel="nofollow noreferrer noopener" target="_blank">upstream</a>)</li>
<li data-sourcepos="5:1-5:44">Support audit titles with Czech characters</li>
<li data-sourcepos="6:1-6:54">Test template conversion (detect appearing newlines)</li>
<li data-sourcepos="7:1-7:34">Fix conversion of checkbox field</li>
<li data-sourcepos="8:1-8:32">Add a page for JSON comparison</li>
<li data-sourcepos="9:1-9:40">Add Template versioning history lookup</li>
<li data-sourcepos="10:1-11:0">Add button to synchronize custom fields across locales of templates</li>
</ul>
<h3 data-sourcepos="12:1-12:22" dir="auto">
<a id="user-content-manual-update-step" class="anchor" href="#manual-update-step" aria-hidden="true"></a>Manual update step</h3>
<p data-sourcepos="13:1-13:142" dir="auto">This version changes password requirements. It's recommended to manually change the password in PwnDoc and the change it in <code data-sourcepos="13:126-13:135">docker.env</code> file.</p>
<ul data-sourcepos="14:1-14:89" dir="auto">
<li data-sourcepos="14:1-14:89"><em data-sourcepos="14:3-14:89">Password must be at least 8 characters with minimum 1 Uppercase, Lowercase and Number</em></li>
</ul>2022-10-27T14:39:06+02:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-10-19release-2022-10-19<p data-sourcepos="1:1-1:14" dir="auto">Release notes:</p>
<ul data-sourcepos="3:1-6:42" dir="auto">
<li data-sourcepos="3:1-3:75">Fix bug on import: ignore_pluginoutput is a string representation of bool</li>
<li data-sourcepos="4:1-4:44">Sort keys of scan2report JSON output files</li>
<li data-sourcepos="5:1-5:56">Reinitialize scan2report global variables on every run</li>
<li data-sourcepos="6:1-6:42">Start every import tests with a clean DB</li>
</ul>2022-10-19T22:54:46+02:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-10-17release-2022-10-17<p data-sourcepos="1:1-1:14" dir="auto">Release notes:</p>
<p data-sourcepos="3:1-3:46" dir="auto">Improvements to the Finding importing process:</p>
<ul data-sourcepos="5:1-9:0" dir="auto">
<li data-sourcepos="5:1-5:114">Support upload of bigger/more findings (updating to MongoDB to 4.4.17 and using allowDiskUsage for aggregations)</li>
<li data-sourcepos="6:1-6:133">Skip individual Findings that failed to be uploaded to Audit, don't fail the whole upload. Present a message with reasons to users.</li>
<li data-sourcepos="7:1-7:61">Add support for ignore_pluginoutput of Scan2Report Template</li>
<li data-sourcepos="8:1-9:0">Add a page to Importer which can show individual Findings from the scan2report output. This can be helpful if the some findings failed to be uploaded.</li>
</ul>
<p data-sourcepos="10:1-10:71" dir="auto">The above mentioned features are result of the other following changes:</p>
<ul data-sourcepos="12:1-18:0" dir="auto">
<li data-sourcepos="12:1-12:85">Test Finding import using publicly available scan files, as well as some edge cases</li>
<li data-sourcepos="13:1-13:83">Add support for adding new custom fields to PwnDoc during Importer version update</li>
<li data-sourcepos="14:1-14:48">Add support for custom fields of type Checkbox</li>
<li data-sourcepos="15:1-15:32">Removed unused “us” locale</li>
<li data-sourcepos="16:1-16:68">Make docker build faster by limiting the context (<code data-sourcepos="16:54-16:66">.dockerignore</code>)</li>
<li data-sourcepos="17:1-18:0">Update backup script to include semi-permanent data</li>
</ul>
<p data-sourcepos="19:1-19:47" dir="auto">Known issues related to modified functionality:</p>
<ul data-sourcepos="20:1-21:96" dir="auto">
<li data-sourcepos="20:1-20:122">there is still not easy way how to copy a template in one language (for example og) to another language (for example cs)</li>
<li data-sourcepos="21:1-21:96">the "common properties" (severity, ignore_pluginoutput, ...) are not yet shared across locales</li>
</ul>2022-10-17T22:52:08+02:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-10-07release-2022-10-07<p data-sourcepos="1:1-1:13" dir="auto">Release notes</p>
<ul data-sourcepos="3:1-6:42" dir="auto">
<li data-sourcepos="3:1-3:31">Add tests for Finding parsing</li>
<li data-sourcepos="4:1-4:31">Fix imports from config files</li>
<li data-sourcepos="5:1-5:54">Fix aborts which crashed instead of giving error msg</li>
<li data-sourcepos="6:1-6:42">Update PwnDoc to solve timeout on import</li>
</ul>2022-10-07T14:39:27+02:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-09-29release-2022-09-29<p data-sourcepos="1:1-1:14" dir="auto">Release notes:</p>
<ul data-sourcepos="3:1-13:0" dir="auto">
<li data-sourcepos="3:1-5:115">Přidána nginx reverse proxy
<ul data-sourcepos="4:3-5:115">
<li data-sourcepos="4:3-4:29">Přidána TLS Client Auth</li>
<li data-sourcepos="5:3-5:115">Importer přesunut pod stejnou doménu na Path <code data-sourcepos="5:53-5:69">/import_automator</code>, jako PwnDoc. Předpokládaný port je 8443.</li>
</ul>
</li>
<li data-sourcepos="6:1-6:62">Nezobrazovat v JSON editoru HTTP error, když request selže</li>
<li data-sourcepos="7:1-7:47">Docker-compose soubor rozdělen na Prod a Dev</li>
<li data-sourcepos="8:1-8:71">Prodlouženy timeouty background jobů na 30 minut (viz. <code data-sourcepos="8:61-8:69">config.py</code>)</li>
<li data-sourcepos="9:1-9:50">Opraveno hlášení, když background job selže</li>
<li data-sourcepos="10:1-10:72">Přidán workaround, když některý template nelze přidat do PwnDocu</li>
<li data-sourcepos="11:1-11:48">Výrazně zrychleno čistění HTML pro PwnDoc</li>
<li data-sourcepos="12:1-13:0">Další drobné opravy (viz. commity)</li>
</ul>
<p data-sourcepos="14:1-14:7" dir="auto">Update:</p>
<ul data-sourcepos="16:1-17:0" dir="auto">
<li data-sourcepos="16:1-17:0">Tento update vyžaduje manuální krok - zapnutí nebo vypnutí TLS auth.</li>
</ul>
<p data-sourcepos="18:1-18:60" dir="auto">Webová instance je nově na <a href="https://pwndoc.borysek.eu:8443/" rel="nofollow noreferrer noopener" target="_blank">https://pwndoc.borysek.eu:8443/</a></p>2022-09-29T14:17:12+02:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-09-22release-2022-09-22<p data-sourcepos="1:1-1:14" dir="auto">Release notes:</p>
<ul data-sourcepos="3:1-7:85" dir="auto">
<li data-sourcepos="3:1-3:58">Zafixování verzí knihoven pro udržení kompatibility</li>
<li data-sourcepos="4:1-4:87">Oprava deprecated parametru attachment_filename pro Flask (blokovalo stažení logů)</li>
<li data-sourcepos="5:1-5:59">Přidán background processing pro importování nálezů</li>
<li data-sourcepos="6:1-6:72">Přidán workaround pro timeout PwnDocu při generování docx reportu</li>
<li data-sourcepos="7:1-7:85">Oprava různých dalších chyb (více detailů lze případně vidět v commitech)</li>
</ul>2022-09-22T20:36:31+02:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-08-25release-2022-08-25<p data-sourcepos="1:1-1:14" dir="auto">Release notes:</p>
<ul data-sourcepos="2:1-3:42" dir="auto">
<li data-sourcepos="2:1-2:53">Oprava chyby se stahováním logů jako ZIP archivu</li>
<li data-sourcepos="3:1-3:42">PwnDoc beautified logs přidány do ZIPu</li>
</ul>2022-08-25T20:05:17+02:00Ondřej BorýsekBorysekOndrej@users.noreply.github.comhttps://gitlab.fi.muni.cz/cybersec/tns/pwndocimportautomator/-/tags/release-2022-08-24release-2022-08-24<p data-sourcepos="1:1-1:13" dir="auto">Release notes</p>
<ul data-sourcepos="2:1-3:55" dir="auto">
<li data-sourcepos="2:1-2:126">Data pro tabulku otevřených portů se skladují nově i lokálně, a lze je pomocí debug endpointu znovunahrát do auditu</li>
<li data-sourcepos="3:1-3:55">schováno políčko Scope v definici auditu v PwnDocu</li>
</ul>2022-08-24T23:49:40+02:00Ondřej BorýsekBorysekOndrej@users.noreply.github.com