TG2-MEASURE_VALIDATIONTESTS_NOTCOMPLIANT

[iDigBioBot]

TestField	Value
GUID	453844ae-9df4-439f-8e24-c52498eca84a
Label	MEASURE_VALIDATIONTESTS_NOTCOMPLIANT
Description	A count of the number of distinct VALIDATION tests that have a Response.status="NOT_COMPLIANT" for a given record.
TestType	Measure
Darwin Core Class	bdq:Validation
Information Elements ActedUpon
Information Elements Consulted	bdq:AllValidationTestsRunOnSingleRecord
Expected Response	INTERNAL_PREREQUISITES_NOT_MET if no tests of type VALIDATION were attempted to be run; REPORT of the number of tests of output type VALIDATION run against the record that have Response.result="NOT_COMPLIANT"
Data Quality Dimension	Reliability
Term-Actions	VALIDATIONTESTS_NOTCOMPLIANT
Parameter(s)
Source Authority
Specification Last Updated	2024-08-22
Examples	[Response.status=RUN_HAS_RESULT, Response.result="37", Response.comment="37 VALIDATION tests had Response.status="NOT_COMPLIANT."]
Source	Lee Belbin
References
Example Implementations (Mechanisms)
Link to Specification Source Code
Notes	We have three individual measures for pass (MEASURE_VALIDATIONTESTS_COMPLIANT (45fb49eb-4a1b-4b49-876f-15d5034dfc73)), fail (MEASURE_VALIDATIONTESTS_NOTCOMPLIANT (453844ae-9df4-439f-8e24-c52498eca84a)), and PREREQUISITES_NOT_MET (49a94636-a562-4e6b-803c-665c80628a3d). To get the total number of tests that were attempted, add all three measures. To get the total number of tests that ran, add NOT_COMPLIANT (fail) and COMPLIANT (pass).

[iDigBioBot]

Comment by Paul Morris (@chicoreus) migrated from spreadsheet:
Either we are using the framework or we are not. If we are using the framework, we must be consistent with it. Validations have results COMPLIANT or NOT_COMPLIANT. Validations have result status values that inform whether the test produced a result or whether for some reason it was not able to run. Therefore we need measures at the data set level of proportion of data compliant, proportion of data not compliant, and proportion of data where the validation could not be run.

[iDigBioBot]

Comment by Paul Morris (@chicoreus) migrated from spreadsheet:
Name should reflect single-record nature

[iDigBioBot]

Comment by Paul Morris (@chicoreus) migrated from spreadsheet:
Test name should reflect the framework: SINGLE_RECORD_VALIDATION_COMPLIANT_COUNT or something like that.

[godfoder]

@chicoreus Does this look OK to you?

[chicoreus]

Single or MultiRecord? See comment by @ArthurChapman in #134

[Tasilee]

@ArthurChapman, as per #134, this is a SINGLE RECORD test (and the resulting count can be accumulated across multiple record/datasets/collections etc).

[tucotuco]

I suggest the Description:

'The number of distinct VALIDATION tests that have a Response.status="NOT_COMPLIANT" for a given record.'

in place of:

'The number of VALIDATION type tests run on a record that have a Response.status="NOT_COMPLIANT".'

[ArthurChapman]

Updated wording of Notes to be consistent with #134 and #135 and to removed internal GitHub cross references.

[Tasilee]

Splitting bdqffdq:Information Elements into "Information Elements ActedUpon" and "Information Elements Consulted"

[chicoreus]

AllDarwinCoreTerms needs to be replaced by a list of relevant validations, in the form (used in the multirecord measures):

bdq:VALIDATION_BASISOFRECORD_NOTEMPTY.Response, as it is the results of validations on the single records that are the information elements for this test, not the darwin core terms.

We should probably also split this test into one test for each use case, with information elements matching the validations found in that use case.

[Tasilee]

I agree @chicoreus , but there are two issue. First up, we probably should not list CORE VALIDATION tests as these may change depending on context. Second, we don't agree about splitting on use case.

I have changed Information Element Consulted from "All DarinCoreTerms" to "All CORE tests of type VALIDATION that were run"

[Tasilee]

Changed Information Elements Consulted to "bdq:AllValidationTestRunOnSingleRecord"

[Tasilee]

Changed Expected Response from

INTERNAL_PREREQUISITES_NOT_MET if no tests of type VALIDATION were attempted to be run; REPORT of the number of tests of output type VALIDATION run against the record that have Response.result="NOT_COMPLIANT"; otherwise NOT_REPORTED

to

INTERNAL_PREREQUISITES_NOT_MET if no tests of type VALIDATION were attempted to be run; REPORT of the number of tests of output type VALIDATION run against the record that have Response.result="NOT_COMPLIANT"

as you can't reach "NOT_REPORTED" anyway.