From 0fcadeca5d9873d39f0a5752992e19992d2c9fe6 Mon Sep 17 00:00:00 2001 From: Adam Korczynski Date: Tue, 9 Jan 2024 21:16:15 +0000 Subject: [PATCH] Add sentence about what a finding is Signed-off-by: Adam Korczynski --- probes/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/probes/README.md b/probes/README.md index 3fe1075964e5..0d50a0b2f7e2 100644 --- a/probes/README.md +++ b/probes/README.md @@ -6,7 +6,7 @@ A probe is an assessment of a focused, specific supply-chain security risk typic Each probe has its own directory in `scorecard/probes`. The probes follow a camelcase naming convention that describe the exact supply-chain security risk a particular probe assesses. -Probes can return multiple or a single `finding.Outcome`. Probes should be designed in such a way that a `finding.OutcomePositive` reflects a positive result, and `finding.OutcomeNegative` reflects a negative result. Scorecard has other `finding.Outcome` types available for other results; For example, the `finding.OutcomeNotAvailable` is often used for scenarios, where Scorecard cannot assess a project with a given probe. In addition, probes should also be named in such a way that they answer "yes" or "no", and where "yes" answers positively to the supply-chain security risk, and "no" answers negatively. For example, probes that check for SAST tools in the CI are called `toolXXXInstalled` so that `finding.OutcomePositive` reflects that it is positive to use the given tool, and that "yes" reflects what Scorecard considers the positive outcome. For some probes, this can be a bit trickier to do; The `notArchived` probe checks whether a project is archived, however, Scorecard considers archived projects to be negative, and the probe cannot be called `isArchived`. These naming conventions are not hard rules but merely guidelines. Note that probes do not do any formal evaluation such a scoring; This is left to the evaluation part once the outcomes have been produced by the probes. +Probes can return multiple or a single finding, where a finding is a piece of data with an outcome, message, and optionally a location. Probes should be designed in such a way that a `finding.OutcomePositive` reflects a positive result, and `finding.OutcomeNegative` reflects a negative result. Scorecard has other `finding.Outcome` types available for other results; For example, the `finding.OutcomeNotAvailable` is often used for scenarios, where Scorecard cannot assess a project with a given probe. In addition, probes should also be named in such a way that they answer "yes" or "no", and where "yes" answers positively to the supply-chain security risk, and "no" answers negatively. For example, probes that check for SAST tools in the CI are called `toolXXXInstalled` so that `finding.OutcomePositive` reflects that it is positive to use the given tool, and that "yes" reflects what Scorecard considers the positive outcome. For some probes, this can be a bit trickier to do; The `notArchived` probe checks whether a project is archived, however, Scorecard considers archived projects to be negative, and the probe cannot be called `isArchived`. These naming conventions are not hard rules but merely guidelines. Note that probes do not do any formal evaluation such a scoring; This is left to the evaluation part once the outcomes have been produced by the probes. A probe consists of three files: