Releases: solid-contrib/conformance-test-harness
Releases · solid-contrib/conformance-test-harness
Release 1.1.10
Minor changes
- Switch to UBI for base image
- Update dependencies and enable OWASP dependency checks
Release 1.1.8
- Update libraries for better Graal support and Karate websocket improvements.
Release 1.1.7
What's Changed
- Add retry mechanism for HTTP requests so results are more consistent
- Add Sonar to CI
- Fix code smells and update license
Full Changelog: v1.1.5...v1.1.7
Release 1.1.5
Improvements
- When comments are linked to a test subject or a specification requirement, show them in the report.
- Config changes detailed in USAGE.md:
- Remove SETUP_ROOT_ACL option.
- Allow individual IDP for users which will override SOLID_IDENTITY_PROVIDER.
- Make RESOURCE_SERVER_ROOT optional - if omitted, the first available storage from the WebID profile is used. It is
maintained for backwards compatibility. If a different test pod is required this can be given as an absolute URL in
TEST_CONTAINER. - TEST_CONTAINER is more flexible:
- An absolute URL to the storage location.
- A relative path applied to the storage location (either user's default storage or RESOURCE_SERVER_ROOT).
- Can be omitted if the test should use the root container of the user's default storage location.
- Update Karate library to benefit from improved logging of the body in HTTP requests & responses.
Release 1.1.4
Minor changes
- When a background step is reused in multiple scenarios, ensure the log output is available in each scenario.
- Allow tests to exit with the
earl:cantTell
outcome when a precondition in not met for the test to continue. - Improve assertions at the scenario level to apply the correct outcomes.
- Improve assertions at the feature level to derive outcomes from the scenarios that make up the feature.
- Break scenario counts into each requirement level and show in results summary table.
Release 1.1.3
Fix
- Issues with login authentication mechanism and PKCE support.
Release 1.1.2
Minor changes
- Get ACL link on container creation to avoid unnecessary HEAD request.
- Fix issue attempting to get ACL link when resource creation failed.
- Improve JSON log format: structured scores and ISO8601 timestamp
Release 1.1.0
Fix
- Fix reporting error when creating assertions on skipped ScenarioOutline tests.
Improvements
- In ACP matcher support, replace acp:group with a simple list of agents.
- Add result counts to the report and command line output for each combination of requirement level and outcome.
- Add result counts to the command line output for the combined passes and fails of mandatory requirements.
- Add toggle for client requirements and initially hide them.
- Various report improvements (styling and annotation related).
Release 1.0.14
Features
- Use tags on features or scenarios to specify when they depend on optional features. Testers can then choose to skip
tests with these tags. Skipped features or scenarios are marked with suitable assertions in the results report. - The docker image has moved to https://hub.docker.com/r/solidproject/conformance-test-harness.
Release 1.0.13
Features
- Add
send
methods to theSolidClient
API to allow testers to send requests with full control over the headers and
the ability to use methods not defined by the HTTP specification. - Push comments (for features, scenarios and steps) found in the test files through to the results reports.
Minor changes
- Shorten the generated names of test resources.
- Prevent unnecessary data from annotated specifications being included in the Turtle report.