Skip to content

Commit

Permalink
Validate uploaded BOMs against CycloneDX schema
Browse files Browse the repository at this point in the history
Closes DependencyTrack#3218

Signed-off-by: nscuro <[email protected]>
  • Loading branch information
nscuro committed Mar 10, 2024
1 parent f8a6a0c commit bd28ebe
Show file tree
Hide file tree
Showing 12 changed files with 877 additions and 7 deletions.
11 changes: 11 additions & 0 deletions docs/_posts/2024-xx-xx-v4.11.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@ predictable, and log messages emitted during processing contain additional conte
Because the new implementation can have a big impact on how Dependency-Track behaves regarding BOM uploads,
it is disabled by default for this release. It may be enabled by setting the environment variable `BOM_PROCESSING_TASK_V2_ENABLED`
to `true`. Users are highly encouraged to do so.
* **BOM Validation**. Historically, Dependency-Track did not validate uploaded BOMs and VEXs against the CycloneDX
schema. While this allowed BOMs to be processed that did not strictly adhere to the schema, it could lead to confusion
when uploaded files were accepted, but then failed to be ingested during asynchronous processing. Starting with this
release, uploaded files will be rejected if they fail schema validation. Note that this may reveal issues in BOM
generators that currently produce invalid CycloneDX documents. Validation may be turned off by setting the
environment variable `BOM_VALIDATION_ENABLED` to `false`.

**Features:**

Expand Down Expand Up @@ -39,6 +45,7 @@ to `true`. Users are highly encouraged to do so.
* Align retry configuration and behavior across analyzers - [apiserver/#3494]
* Add auto-generated changelog to GitHub releases - [apiserver/#3502]
* Bump SPDX license list to v3.23 - [apiserver/#3508]
* Validate uploaded BOMs against CycloneDX schema prior to processing them - [apiserver/#3522]
* Show component count in projects list - [frontend/#683]
* Add current *fail*, *warn*, and *info* values to bottom of policy violation metrics - [frontend/#707]
* Remove unused policy violation widget - [frontend/#710]
Expand Down Expand Up @@ -80,6 +87,9 @@ to `true`. Users are highly encouraged to do so.

**Upgrade Notes:**

* To enable the optimized BOM ingestion, set the environment variable `BOM_PROCESSING_TASK_V2_ENABLED` to `true`
* Validation of uploaded BOMs and VEXs is enabled per default, but can be disabled by setting the environment
variable `BOM_VALIDATION_ENABLED` to `false`
* The `CWE` table is dropped automatically upon upgrade
* The default logging configuration ([logback.xml]) was updated to include the [Mapped Diagnostic Context] (MDC)
* Users who [customized their logging configuration] are recommended to follow this change
Expand Down Expand Up @@ -162,6 +172,7 @@ Special thanks to everyone who contributed code to implement enhancements and fi
[apiserver/#3511]: https://github.com/DependencyTrack/dependency-track/pull/3511
[apiserver/#3512]: https://github.com/DependencyTrack/dependency-track/pull/3512
[apiserver/#3513]: https://github.com/DependencyTrack/dependency-track/pull/3513
[apiserver/#3522]: https://github.com/DependencyTrack/dependency-track/pull/3522

[frontend/#682]: https://github.com/DependencyTrack/frontend/pull/682
[frontend/#683]: https://github.com/DependencyTrack/frontend/pull/683
Expand Down
3 changes: 2 additions & 1 deletion src/main/java/org/dependencytrack/common/ConfigKey.java
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,8 @@ public enum ConfigKey implements Config.Key {
REPO_META_ANALYZER_CACHE_STAMPEDE_BLOCKER_LOCK_BUCKETS("repo.meta.analyzer.cacheStampedeBlocker.lock.buckets", 1000),
REPO_META_ANALYZER_CACHE_STAMPEDE_BLOCKER_MAX_ATTEMPTS("repo.meta.analyzer.cacheStampedeBlocker.max.attempts", 10),
SYSTEM_REQUIREMENT_CHECK_ENABLED("system.requirement.check.enabled", true),
BOM_PROCESSING_TASK_V2_ENABLED("bom.processing.task.v2.enabled", false);
BOM_PROCESSING_TASK_V2_ENABLED("bom.processing.task.v2.enabled", false),
BOM_VALIDATION_ENABLED("bom.validation.enabled", true);

private final String propertyName;
private final Object defaultValue;
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,208 @@
/*
* This file is part of Dependency-Track.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
* Copyright (c) Steve Springett. All Rights Reserved.
*/
package org.dependencytrack.parser.cyclonedx;

import alpine.common.logging.Logger;
import com.fasterxml.jackson.core.JsonParseException;
import com.fasterxml.jackson.core.JsonToken;
import com.fasterxml.jackson.databind.json.JsonMapper;
import org.codehaus.stax2.XMLInputFactory2;
import org.cyclonedx.CycloneDxSchema;
import org.cyclonedx.exception.ParseException;
import org.cyclonedx.parsers.JsonParser;
import org.cyclonedx.parsers.Parser;
import org.cyclonedx.parsers.XmlParser;

import javax.xml.stream.XMLInputFactory;
import javax.xml.stream.XMLStreamException;
import javax.xml.stream.XMLStreamReader;
import javax.xml.stream.events.XMLEvent;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Optional;

import static org.cyclonedx.CycloneDxSchema.NS_BOM_10;
import static org.cyclonedx.CycloneDxSchema.NS_BOM_11;
import static org.cyclonedx.CycloneDxSchema.NS_BOM_12;
import static org.cyclonedx.CycloneDxSchema.NS_BOM_13;
import static org.cyclonedx.CycloneDxSchema.NS_BOM_14;
import static org.cyclonedx.CycloneDxSchema.NS_BOM_15;
import static org.cyclonedx.CycloneDxSchema.Version.VERSION_10;
import static org.cyclonedx.CycloneDxSchema.Version.VERSION_11;
import static org.cyclonedx.CycloneDxSchema.Version.VERSION_12;
import static org.cyclonedx.CycloneDxSchema.Version.VERSION_13;
import static org.cyclonedx.CycloneDxSchema.Version.VERSION_14;
import static org.cyclonedx.CycloneDxSchema.Version.VERSION_15;

/**
* @since 4.11.0
*/
public class CycloneDxValidator {

private static final Logger LOGGER = Logger.getLogger(CycloneDxValidator.class);
private static final CycloneDxValidator INSTANCE = new CycloneDxValidator();

private final JsonMapper jsonMapper = new JsonMapper();

CycloneDxValidator() {
}

public static CycloneDxValidator getInstance() {
return INSTANCE;
}

public void validate(final byte[] bomBytes) {
final FormatAndVersion formatAndVersion = detectFormatAndSchemaVersion(bomBytes);

final Parser bomParser = switch (formatAndVersion.format()) {
case JSON -> new JsonParser();
case XML -> new XmlParser();
};

final List<ParseException> validationErrors;
try {
validationErrors = bomParser.validate(bomBytes, formatAndVersion.version());
} catch (IOException e) {
throw new RuntimeException("Failed to validate BOM", e);
}

if (!validationErrors.isEmpty()) {
throw new InvalidBomException("Schema validation failed", validationErrors.stream()
.map(ParseException::getMessage)
.toList());
}
}

private FormatAndVersion detectFormatAndSchemaVersion(final byte[] bomBytes) {
try {
final CycloneDxSchema.Version version = detectSchemaVersionFromJson(bomBytes);
return new FormatAndVersion(Format.JSON, version);
} catch (JsonParseException e) {
if (LOGGER.isDebugEnabled()) {
LOGGER.debug("Failed to parse BOM as JSON", e);
}
} catch (IOException e) {
throw new RuntimeException(e);
}

try {
final CycloneDxSchema.Version version = detectSchemaVersionFromXml(bomBytes);
return new FormatAndVersion(Format.XML, version);
} catch (XMLStreamException e) {
if (LOGGER.isDebugEnabled()) {
LOGGER.debug("Failed to parse BOM as XML", e);
}
}

throw new InvalidBomException("BOM is neither valid JSON nor XML");
}

private CycloneDxSchema.Version detectSchemaVersionFromJson(final byte[] bomBytes) throws IOException {
try (final com.fasterxml.jackson.core.JsonParser jsonParser = jsonMapper.createParser(bomBytes)) {
JsonToken currentToken = jsonParser.nextToken();
if (currentToken != JsonToken.START_OBJECT) {
final String currentTokenAsString = Optional.ofNullable(currentToken)
.map(JsonToken::asString).orElse(null);
throw new JsonParseException(jsonParser, "Expected token %s, but got %s"
.formatted(JsonToken.START_OBJECT.asString(), currentTokenAsString));
}

CycloneDxSchema.Version schemaVersion = null;
while (jsonParser.nextToken() != JsonToken.END_OBJECT) {
final String fieldName = jsonParser.getCurrentName();
if ("specVersion".equals(fieldName)) {
if (jsonParser.nextToken() == JsonToken.VALUE_STRING) {
final String specVersion = jsonParser.getValueAsString();
schemaVersion = switch (jsonParser.getValueAsString()) {
case "1.0", "1.1" ->
throw new InvalidBomException("JSON is not supported for specVersion %s".formatted(specVersion));
case "1.2" -> VERSION_12;
case "1.3" -> VERSION_13;
case "1.4" -> VERSION_14;
case "1.5" -> VERSION_15;
default ->
throw new InvalidBomException("Unrecognized specVersion %s".formatted(specVersion));
};
}
}

if (schemaVersion != null) {
return schemaVersion;
}
}

throw new InvalidBomException("Unable to determine schema version from JSON");
}
}

private CycloneDxSchema.Version detectSchemaVersionFromXml(final byte[] bomBytes) throws XMLStreamException {
final XMLInputFactory xmlInputFactory = XMLInputFactory2.newFactory();
final var bomBytesStream = new ByteArrayInputStream(bomBytes);
final XMLStreamReader xmlStreamReader = xmlInputFactory.createXMLStreamReader(bomBytesStream);

CycloneDxSchema.Version schemaVersion = null;
while (xmlStreamReader.hasNext()) {
if (xmlStreamReader.next() == XMLEvent.START_ELEMENT) {
if (!"bom".equalsIgnoreCase(xmlStreamReader.getLocalName())) {
continue;
}

final var namespaceUrisSeen = new ArrayList<String>();
for (int i = 0; i < xmlStreamReader.getNamespaceCount(); i++) {
final String namespaceUri = xmlStreamReader.getNamespaceURI(i);
namespaceUrisSeen.add(namespaceUri);

schemaVersion = switch (namespaceUri) {
case NS_BOM_10 -> VERSION_10;
case NS_BOM_11 -> VERSION_11;
case NS_BOM_12 -> VERSION_12;
case NS_BOM_13 -> VERSION_13;
case NS_BOM_14 -> VERSION_14;
case NS_BOM_15 -> VERSION_15;
default -> null;
};
}

if (schemaVersion == null) {
throw new InvalidBomException("Unable to determine schema version from XML namespaces %s"
.formatted(namespaceUrisSeen));
}

break;
}
}

if (schemaVersion == null) {
throw new InvalidBomException("Unable to determine schema version from XML");
}

return schemaVersion;
}

private enum Format {
JSON,
XML
}

private record FormatAndVersion(Format format, CycloneDxSchema.Version version) {
}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
/*
* This file is part of Dependency-Track.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
* Copyright (c) Steve Springett. All Rights Reserved.
*/
package org.dependencytrack.parser.cyclonedx;

import java.util.Collections;
import java.util.List;

/**
* @since 4.11.0
*/
public class InvalidBomException extends RuntimeException {

private final List<String> validationErrors;

InvalidBomException(final String message) {
this(message, (Throwable) null);
}

InvalidBomException(final String message, final Throwable cause) {
this(message, cause, Collections.emptyList());
}

InvalidBomException(final String message, final List<String> validationErrors) {
this(message, null, validationErrors);
}

private InvalidBomException(final String message, final Throwable cause, final List<String> validationErrors) {
super(message, cause);
this.validationErrors = validationErrors;
}

public List<String> getValidationErrors() {
return validationErrors;
}

}
Loading

0 comments on commit bd28ebe

Please sign in to comment.