Add FAQ to documentation

Explain difference between security warnings and violations in a new FAQ section of the documentation.

Closes #7797.

components/shared_code/src/shared_data_model/metrics.py

@@ -348,6 +348,9 @@
         description="The number of security warnings about the software.",
         rationale="Monitor security warnings about the software, its source code, dependencies, or "
         "infrastructure so vulnerabilities can be fixed before they end up in production.",
+        rationale_urls=[
+            "faq.md#what-is-the-difference-between-violations-and-warnings",
+        ],
         unit=Unit.SECURITY_WARNINGS,
         near_target="5",
         sources=[
@@ -729,6 +732,7 @@
             "https://docs.sonarqube.org/latest/user-guide/rules/",
             "https://martinfowler.com/bliki/CodeSmell.html",
             "https://www.w3.org/standards/webdesign/accessibility",
+            "faq.md#what-is-the-difference-between-violations-and-warnings",
         ],
         unit=Unit.VIOLATIONS,
         sources=["axecsv", "axe_core", "axe_html_reporter", "manual_number", "ojaudit", "sarif_json", "sonarqube"],

docs/src/changelog.md

@@ -21,6 +21,7 @@ If your currently installed *Quality-time* version is v4.10.0 or older, please r
 ### Added
 
 - Allow for hiding the legend card via the Settings panel.
+- Explain difference between security warnings and violations in a new FAQ section of the documentation. Closes [#7797](https://github.com/ICTU/quality-time/issues/7797).
 - When adding a metric to a subject, add an option to choose from all metric types in addition to the metric types officially supported by the subject type. Closes [#8176](https://github.com/ICTU/quality-time/issues/8176).
 - When a report has a configured issue tracker, show a card in the dashboard with the number of issues per issue status (open, in progress, done). The card can be hidden via the Settings panel. Clicking the card or setting "Visible metrics" to "Metrics with issues" in the Settings panel hides metrics without issues. Closes [#8222](https://github.com/ICTU/quality-time/issues/8222).
 

docs/src/create_reference_md.py

@@ -23,7 +23,7 @@ def markdown_paragraph(text: str) -> str:
 def markdown_link(url: HttpUrl | str, anchor: HttpUrl | str | None = None) -> str:
     """Return a Markdown link."""
     anchor = anchor or url
-    return f"[{anchor}]({url})"
+    return f"[{anchor}]({url})" if str(url).startswith("http") else f"[]({url})"
 
 
 def definition_list(term: str, *definitions: str) -> str:

docs/src/faq.md

@@ -0,0 +1,11 @@
+# Frequently asked questions
+
+## What is the difference between violations and warnings?
+
+*Quality-time* has a metric called "Violations" and a metric called "Security warnings". What is the difference?
+
+The metric "Violations" is used to measure the number of violations of programming rules. When measuring violations, there is always some set of rules checked by the source and the metric counts the number of violations of the rules. For example, SonarQube checks source code against rules grouped in quality profile and Axe checks software for violations of WCAG guidelines.
+
+The metric "Security warnings" is used to measure the number of potential security vulnerabilities in software. These can be insecure constructs used in the source code or known vulnerabilities in third party dependencies. For example, SonarQube checks source code for SQL-injection vulnerabilities and the OWASP Dependency Checker reports dependencies with known security vulnerabilities.
+
+There can be overlap between the two metrics in the sense that violations can be security risks, and that security warnings can be discovered by checking source code against rules. The difference between the metrics is that "Violations" always reports violations of programming rules, which are not necessarily security risks, and "Security warnings" always reports security risks, which are not necessarily rule violations.

docs/src/index.md

@@ -28,6 +28,7 @@ intro.md
 :maxdepth: 1
 :caption: User documentation 📗
 usage.md
+faq.md
 reference.md
 ```
 ````