Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AJ-1157: Spring Boot 3.2.1 #440

Merged
merged 15 commits into from
Jan 3, 2024
8 changes: 2 additions & 6 deletions build.gradle
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import org.springframework.boot.gradle.plugin.SpringBootPlugin

plugins {
id 'org.springframework.boot' version '2.7.18' apply false
id 'org.springframework.boot' version '3.2.1' apply false
id 'io.spring.dependency-management' version '1.1.4' apply false
id 'com.google.cloud.tools.jib' version '3.2.1' apply false
id "org.sonarqube" version "4.4.1.3373" apply false
Expand Down Expand Up @@ -36,12 +36,8 @@ subprojects {
imports {
mavenBom(SpringBootPlugin.BOM_COORDINATES)
}

dependencies {
dependency 'org.yaml:snakeyaml:2.0'
dependency 'org.webjars:webjars-locator-core:0.53'
dependency 'ch.qos.logback:logback-classic:1.2.13+' // CVE-2023-6378
dependency 'ch.qos.logback:logback-core:1.2.13+' // CVE-2023-6378
dependency 'org.liquibase:liquibase-core:4.25.1'
}
}
}
Expand Down
6 changes: 3 additions & 3 deletions client/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,10 @@ ext {

// default to javax, since WDS is currently on Spring Boot 2. -Pjakarta=true overrides
// the default and creates a Jakarta-based client.
if (project.hasProperty("jakarta") && "true".equalsIgnoreCase(project.getProperty("jakarta"))) {
apply from: "dependencies-jakarta.gradle"
} else {
if (project.hasProperty("jakarta") && "false".equalsIgnoreCase(project.getProperty("jakarta"))) {
apply from: "dependencies-javax.gradle"
} else {
apply from: "dependencies-jakarta.gradle"
}

apply from: 'artifactory.gradle'
Expand Down
6 changes: 3 additions & 3 deletions client/swagger.gradle
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
dependencies {
//Need to include libraries for generated code to work
api 'com.google.code.gson:gson:2.10.1'
api 'com.google.code.gson:gson'
api 'io.gsonfire:gson-fire:1.9.0'
api 'com.squareup.okhttp3:okhttp:4.12.0'
api 'com.squareup.okhttp3:logging-interceptor:4.12.0'
api 'com.squareup.okhttp3:okhttp'
api 'com.squareup.okhttp3:logging-interceptor'
}

openApiValidate {
Expand Down
53 changes: 21 additions & 32 deletions service/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ jib {
}

ext {
jersey_version = "2.36"
parquet_mr_version = "1.13.1"
hadoop_version = "3.3.6"
}
Expand All @@ -57,29 +56,29 @@ dependencies {
implementation 'org.springframework.boot:spring-boot-starter-actuator'
implementation 'io.micrometer:micrometer-registry-prometheus'
implementation 'org.springframework.integration:spring-integration-jdbc'
implementation 'org.aspectj:aspectjweaver:1.8.9' // required by spring-retry, not used directly
implementation 'org.aspectj:aspectjweaver' // required by spring-retry, not used directly
implementation 'com.google.guava:guava:32.1.3-jre'
implementation 'org.postgresql:postgresql'
implementation 'org.webjars:webjars-locator-core' // versioned by spring dependency management
implementation 'com.fasterxml.jackson.dataformat:jackson-dataformat-csv:2.13.5'
implementation 'com.fasterxml.jackson.dataformat:jackson-dataformat-csv'
implementation 'io.sentry:sentry-logback:6.34.0'
implementation 'org.liquibase:liquibase-core:4.21.1'
implementation 'org.liquibase:liquibase-core'
implementation 'javax.cache:cache-api'
implementation 'org.ehcache:ehcache:3.10.8'
implementation 'org.ehcache:ehcache:3.10.8:jakarta'
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the ehcache version is managed by Spring Dependency Management, so ideally we'd leave the version number off here. However, it requires the jakarta classifier, and we can't specify a classifier without also specifying a version using this syntax, so I left the version in.

implementation group: 'org.ehcache', name: 'ehcache', classifier: 'jakarta'

would work, but results in IntelliJ complaining about the missing version.

implementation 'org.hashids:hashids:1.0.3'
implementation 'javax.ws.rs:javax.ws.rs-api:2.1.1'
implementation 'jakarta.ws.rs:jakarta.ws.rs-api'
implementation 'com.google.mug:mug:6.6'

// required by openapi-generated models and api interfaces
implementation 'javax.validation:validation-api'
implementation 'jakarta.validation:jakarta.validation-api'
implementation 'io.swagger.core.v3:swagger-annotations:2.2.16'

// Terra libraries
implementation group: 'org.broadinstitute.dsde.workbench', name: 'sam-client_2.13', version: '0.1-08b6588'
implementation group: 'org.broadinstitute.dsde.workbench', name: 'sam-client_2.13', version: '0.1-752b4f3'
implementation group: 'org.broadinstitute.dsde.workbench', name: 'leonardo-client_2.13', version: '1.3.6-22ee00b'
implementation 'com.squareup.okhttp3:okhttp:4.12.0' // required by Sam client
implementation "bio.terra:datarepo-client:1.537.0-SNAPSHOT"
implementation "bio.terra:workspace-manager-client:0.254.717-SNAPSHOT"
implementation 'com.squareup.okhttp3:okhttp' // required by Sam client
implementation "bio.terra:datarepo-jakarta-client:1.557.0-SNAPSHOT"
implementation "bio.terra:workspace-manager-client:0.254.983-SNAPSHOT"
implementation "bio.terra:java-pfb-library:0.15.0"
implementation project(path: ':client')

Expand All @@ -106,15 +105,15 @@ dependencies {
implementation "org.apache.hadoop:hadoop-common:$hadoop_version", withoutHadoopExcludes
implementation "org.apache.hadoop:hadoop-mapreduce-client-core:$hadoop_version", withoutHadoopExcludes

// hk2 is required to use WSM client, but not correctly exposed by the client
implementation "org.glassfish.jersey.inject:jersey-hk2:$jersey_version"
implementation "org.glassfish.jersey.core:jersey-client:$jersey_version"
implementation "org.glassfish.jersey.core:jersey-common:$jersey_version"
// Jersey libs, required by WSM client (and maybe other clients?), version managed by Spring
implementation "org.glassfish.jersey.inject:jersey-hk2"
implementation "org.glassfish.jersey.core:jersey-client"
implementation "org.glassfish.jersey.core:jersey-common"

annotationProcessor 'org.springframework.boot:spring-boot-configuration-processor'
runtimeOnly 'org.webjars.npm:swagger-ui-dist:5.9.0'
testImplementation 'org.springframework.boot:spring-boot-starter-test'
testImplementation 'org.junit.jupiter:junit-jupiter:5.8.2'
testImplementation 'org.junit.jupiter:junit-jupiter'
testImplementation project(':client')
testImplementation 'au.com.dius.pact.consumer:junit5:4.6.1'
testImplementation 'org.junit-pioneer:junit-pioneer:2.1.0'
Expand All @@ -127,18 +126,6 @@ dependencies {
implementation('org.apache.commons:commons-compress:1.24.0') {
because("CVE-2023-42503")
}


// this is redundant (and ineffective) without an explicit override in build.gradle
// dependencyManagement, but left here for documentation purposes
implementation('ch.qos.logback:logback-classic:1.2.13+') {
because("CVE-2023-6378")
}
// this is redundant (and ineffective) without an explicit override in build.gradle
// dependencyManagement, but left here for documentation purposes
implementation('ch.qos.logback:logback-core:1.2.13+') {
because("CVE-2023-6378")
}
}
}

Expand All @@ -159,10 +146,12 @@ tasks.register('generateApiInterfaces', GenerateTask) {
supportingFilesConstrainedTo.set([]) // empty array means generate none
skipOperationExample = true // example responses require the ApiUtil.java supporting file
configOptions.set([
useJakartaEe : "true", // for Spring Boot 3
interfaceOnly : "true", // Java interfaces only; no controllers or implementation classes
useTags : "true", // use the OpenAPI tag to classify apis, not the first segment of the api path
hideGenerationTimestamp: "true" // hidden to prevent unnecessary diff noise on unrelated changes
])

}


Expand Down Expand Up @@ -252,10 +241,10 @@ testing {
implementation project(':client')
implementation 'org.springframework.boot:spring-boot-starter-test'
implementation 'org.springframework.boot:spring-boot-starter-jdbc'
implementation 'org.liquibase:liquibase-core:4.21.1'
implementation "org.glassfish.jersey.core:jersey-client:$jersey_version"
implementation "org.glassfish.jersey.core:jersey-common:$jersey_version"
implementation "org.glassfish.jersey.inject:jersey-hk2:$jersey_version"
implementation 'org.liquibase:liquibase-core'
implementation "org.glassfish.jersey.core:jersey-client"
implementation "org.glassfish.jersey.core:jersey-common"
implementation "org.glassfish.jersey.inject:jersey-hk2"
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,7 @@ private String requestRemoteBackup(UUID trackingId) {
} catch (WorkspaceDataServiceException wdsE) {
if (wdsE.getCause() != null
&& wdsE.getCause() instanceof RestException restException
&& restException.getStatus() == HttpStatus.NOT_FOUND) {
&& restException.getStatusCode() == HttpStatus.NOT_FOUND) {
LOGGER.error(
"Remote source WDS in workspace {} does not support cloning", sourceWorkspaceId);
cloneDao.terminateCloneToError(
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
package org.databiosphere.workspacedataservice.controller;

import com.google.common.annotations.VisibleForTesting;
import jakarta.servlet.http.HttpServletRequest;
import java.util.ArrayList;
import java.util.Date;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import org.apache.commons.lang3.StringUtils;
import org.jetbrains.annotations.NotNull;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.HttpStatusCode;
import org.springframework.http.MediaType;
import org.springframework.http.ProblemDetail;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ControllerAdvice;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.context.request.ServletWebRequest;
import org.springframework.web.context.request.WebRequest;
import org.springframework.web.method.annotation.MethodArgumentTypeMismatchException;
import org.springframework.web.servlet.mvc.method.annotation.ResponseEntityExceptionHandler;

@ControllerAdvice
public class GlobalExceptionHandler extends ResponseEntityExceptionHandler {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a lot of new/novel code -- was this logic handled somehow for us automatically by something that got removed in the upgrade to SpringBoot 3 or did I miss a move in the red parts of the diff?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also confused what this is all about. Did you hand write this code?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apologies, I meant to document this in the PR!

Spring Boot 3 / Spring Framework 6 enables, by default, the RFC 7807 "Problem Details for HTTP APIs" standard for structuring its error responses. In WDS today on Spring Boot 2.7, Spring uses the ErrorResponse structure instead.

If we blindly accepted the change to use Problem Details responses, we'd break anyone using the Java or Python clients to handle error messages, and potentially (probably?) break Terra UI when it tries to parse error messages from WDS.

So, in this PR, I have set mvc.problemdetails.enabled: false in application.yml to maintain the previous behavior. But that's not sufficient, because in multiple places WDS's exception classes extend ResponseStatusException (example here). When those exceptions are serialized to the response, they serialize using the Problem Details structure. Therefore, I've also added this global exception handler to intercept any Problem Details being sent to the response and transform them to ErrorResponse instead.

You can see this in practice if you run this branch locally by turning off the global exception handler (comment out the @ControllerAdvice annotation) and doing something like sending an invalid UUID or an invalid version string in various APIs. You will also see a unit test fail because it asserts on the old error messages.

@calypsomatic yes, this is hand-written code.


private final String ERROR = "error";
private final String MESSAGE = "message";
private final String PATH = "path";
private final String STATUS = "status";
private final String TIMESTAMP = "timestamp";

/**
* Override to explicitly translate Problem Details structures back to {@link
* org.databiosphere.workspacedata.model.ErrorResponse} structures. This ensures backwards
* compatibility for clients.
*
* <p>Even though {@code }spring.mvc.problemdetails.enabled} is set to false in config, many
* exception classes extend {@link org.springframework.web.server.ResponseStatusException}, and
* that exception serializes to a problem detail in the response.
*
* @param body the body to use for the response
* @param headers the headers to use for the response
* @param statusCode the status code to use for the response
* @param request the current request
* @return the {@code ResponseEntity} instance to use
*/
@NotNull
@Override
protected ResponseEntity<Object> createResponseEntity(
Object body,
@NotNull HttpHeaders headers,
@NotNull HttpStatusCode statusCode,
@NotNull WebRequest request) {
if (body instanceof ProblemDetail problemDetail) {
Map<String, Object> errorBody = new LinkedHashMap<>();
errorBody.put(ERROR, problemDetail.getTitle());
errorBody.put(MESSAGE, problemDetail.getDetail());
errorBody.put(STATUS, problemDetail.getStatus());
errorBody.put(TIMESTAMP, new Date());

if (request instanceof ServletWebRequest servletWebRequest) {
errorBody.put(PATH, servletWebRequest.getRequest().getRequestURI());
}
return new ResponseEntity<>(errorBody, headers, statusCode);
}

return super.createResponseEntity(body, headers, statusCode, request);
}

/**
* Handler for MethodArgumentTypeMismatchException. This exception contains the real message we
* want to display inside the nested cause. This handler ditches the top-level message in favor of
* that nested one.
*
* @param ex the exception in question
* @param servletRequest the request
* @return the desired response
*/
@ExceptionHandler({MethodArgumentTypeMismatchException.class})
public ResponseEntity<Map<String, Object>> handleAllExceptions(
Exception ex, HttpServletRequest servletRequest) {
Map<String, Object> errorBody = new LinkedHashMap<>();
errorBody.put(ERROR, HttpStatus.BAD_REQUEST.getReasonPhrase());
errorBody.put(PATH, servletRequest.getRequestURI());
errorBody.put(STATUS, HttpStatus.BAD_REQUEST.value());
errorBody.put(TIMESTAMP, new Date());

// MethodArgumentTypeMismatchException nested exceptions contains
// the real message we want to display. Gather all the nested messages and display the last one.
List<String> errorMessages = gatherNestedErrorMessages(ex, new ArrayList<>());
if (!errorMessages.isEmpty()) {
errorBody.put(MESSAGE, errorMessages.get(errorMessages.size() - 1));
} else {
errorBody.put(MESSAGE, "Unexpected error: " + ex.getClass().getName());
}

return ResponseEntity.badRequest().contentType(MediaType.APPLICATION_JSON).body(errorBody);
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In these methods, I wanted to use the ErrorResponse class instead of a raw Map<String, Object>. However, ErrorResponse does not serialize to json consistent with other classes. We could use its generated .toJson() method, but that uses gson instead of Jackson. Or, we could serialize it with Jackson …. but then, the timestamp field is serialized as a string instead of a Date, and we'd have to do our own custom Date formatting inside the exception handler. So, I'm sticking with the raw Map. Hopefully we upgrade to Problem Details soon and this code can mostly go away.


@VisibleForTesting
List<String> gatherNestedErrorMessages(@NotNull Throwable t, @NotNull List<String> accumulator) {
if (StringUtils.isNotBlank(t.getMessage())) {
accumulator.add(t.getMessage());
}
if (t.getCause() == null) {
return accumulator;
}
return gatherNestedErrorMessages(t.getCause(), accumulator);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

import bio.terra.datarepo.api.RepositoryApi;
import bio.terra.datarepo.client.ApiClient;
import javax.ws.rs.client.Client;
import jakarta.ws.rs.client.Client;
import org.apache.commons.lang3.StringUtils;
import org.databiosphere.workspacedataservice.sam.TokenContextUtil;
import org.databiosphere.workspacedataservice.shared.model.BearerToken;
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading