Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible regression between 1.3 and 1.4 regarding UTF-8 handling? (Spring 5.1.12) #317

Closed
eranation opened this issue Jan 21, 2020 · 0 comments · Fixed by #331
Closed

Possible regression between 1.3 and 1.4 regarding UTF-8 handling? (Spring 5.1.12) #317

eranation opened this issue Jan 21, 2020 · 0 comments · Fixed by #331
Labels
Milestone

Comments

@eranation
Copy link
Contributor

  • Framework version: 1.4
  • Implementations: Spring

Scenario

I upgrade a project from 1.3.2 to 1.4. I have some responses that return UTF-8 encoded JSON strings, e.g.

/**
 * test utf (for unit testing only)
 */
@GetMapping(path = "/utf-test", produces = "application/json; charset=utf-8")
Map<String, ?> testUTF() {
  Map<String, Object> ret = new HashMap<>();
  ret.put("message", "שלום");
  return ret;
}

p.s. Adding the following didn't seem to change much

LambdaContainerHandler.getContainerConfig().setDefaultContentCharset("UTF-8");

As well as this (to application.properties)

spring.http.encoding.charset=UTF-8
spring.http.encoding.enabled=true
spring.http.encoding.force=true

Expected behavior

response should return as {"message": "שלום"}
(this is what I get in 1.3.2)

Actual behavior

response returns as: {"message":"ש���"}
(this is what I get when upgrading to 1.4)

Steps to reproduce

I tried this with both java 1.8 and java 11 using these core dependencies

<spring.version>5.1.12.RELEASE</spring.version>
<jackson.version>2.10.1</jackson.version>

with nothing changed, it works in 1.3.2, and broke in 1.4, not sure if I'm doing something wrong...

Full log output

N / A

@eranation eranation changed the title Possible regression between 1.3 and 1.4? Possible regression between 1.3 and 1.4 regarding UTF-8 handling? (Spring 5.1.12) Jan 21, 2020
sapessi added a commit that referenced this issue Mar 30, 2020
* fix issue 317 - use charset from request

* update dependencies

* update build dependencies, remove spring boot 2.0.x

* restoring ci config

Co-authored-by: Stefano Buliani <[email protected]>
@sapessi sapessi added the bug label Mar 31, 2020
@sapessi sapessi added this to the Release 1.5 milestone Mar 31, 2020
@sapessi sapessi linked a pull request Apr 7, 2020 that will close this issue
sapessi added a commit that referenced this issue Apr 8, 2020
* Bump spring.version in /aws-serverless-java-container-spring (#319)

Bumps `spring.version` from 5.1.9.RELEASE to 5.2.3.RELEASE.

Updates `spring-webmvc` from 5.1.9.RELEASE to 5.2.3.RELEASE
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](spring-projects/spring-framework@v5.1.9.RELEASE...v5.2.3.RELEASE)

Updates `spring-test` from 5.1.9.RELEASE to 5.2.3.RELEASE
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](spring-projects/spring-framework@v5.1.9.RELEASE...v5.2.3.RELEASE)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump spring-webflux in /aws-serverless-java-container-springboot2 (#318)

Bumps [spring-webflux](https://github.com/spring-projects/spring-framework) from 5.1.9.RELEASE to 5.2.0.RELEASE.
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](spring-projects/spring-framework@v5.1.9.RELEASE...v5.2.0.RELEASE)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* ci: Fixing Spring build to use 5.2 as latest

* chore(deps): Bump Spring 5.1 path release to address a security vulnerability

* chore(deps): Fixing usual spring dependency mess with exlusions out of the spring-security package used in the tests

* Fix for issue #317 (#323)

* fix issue 317 - use charset from request

* update dependencies

* update build dependencies, remove spring boot 2.0.x

* restoring ci config

Co-authored-by: Stefano Buliani <[email protected]>

* test: Fixed Spring security tests for SpringBoot 2, added validation tests and updated servlet tests to use the new servletApplication option

* fix: Avoid flushing the response buffer if we are dispatching the request asynchronously. This was causing race conditions in the SpringBoot 2 WebFlux implementation - requests that had to run through security or validation filters took longer and the library flushed an empty request, which caused the status code to default to 200. This fix addresses issues #279, #304, and #306

* chore(deps): Bump spring dependency version and added webmvc optional dependency to truly support Servlet-only server

* feat: New application type parameter to SpringBootLambdaContainerHandler that tells the framework whether to start a reactive or servlet-based embedded server. Also added a new servletApplication method to the builder object.

* test: Fixed UTF-8 encoding test

* ci: Fixed dependencies for CI run on SpringBoot 2

* ci: More Spring dependency convergence issues during CI

* fix: Added null-check on getServerName in case the multi-value headers property is null. Unlikely outside of tests but better safe than sorry. This addresses #327

* fix: Changed servlet initialization mechanism so that servlet that requests load on startup are initialized right away, as part of the initialization() method call in LambdaServletContainerHandler. Also centralized the lazy Servlet initialization to the ServletExecutionFilter so that we don't have code scattered all around. This begins to address #287

* feat: Added new 0-parameter constructor for async initializer that uses the actual JVM start time to calculate the timeout milliseconds. Also added the new method to the builder object and deprecated the current method that receives a milliseconds epoch parameter. I'm not deprecating the constructor of the async initializer class that receives the parameter as it may still be useful for tests. This change was suggested in #287

* fix: Updated SpringBoot 1.x handler to use the new servlet initialization mechanism

* ci: switch SpringBoot slow integration test to use a custom async time since the JVM is reused for both tests in the and we cannot reuse the actual JVM init time

* feat: New models for HTTP API support for #329

* feat: First implementation of HTTP API servlet request, request reader, and security context writer - continuing to address #329

* test: Basic unit tests for the new HTTP API support in core library (#329)

* feat: Updated log formatter to support both versions (1 and 2) of the proxy request model (#329)

* feat: Further generified request readers to read to a generic HttpServletRequest rather than specific implementations of it. This makes it easier to create container handler implementations that support HTTP API, API Gateway, and ALB (#329)

* test: Fixed tests for new logged and generified request readers

* feat: Added HTTP API support to Jersey implementation with new getHttpApiV2ProxyHandler method (#329)

* feat: Added HTTP API support to Spark implementation (#329)

* feat: Added HTTP API support to Spring implementation (#329)

* feat: HTTP API support in SpringBoot 2 implementation. bug: Fixed an issue with the implementation of AsyncContext where it wasn't dispatching if the handler wasn't set

* feat: First pass of HTTP API support in struts 2 implementation (#329)

* fix: Added support for HTTP APIs to the request dispatcher

* chore(deps): Dependency bump all around. Rotated Jersey ci versions

* fix: Updated stream handling logic to work with reactive applications as suggested in #316

* test: Added unit test to replicate #333

* feat: New configuration parameter to skip exception mapping and allow exception to bubble up from #307

* fix: Fixed spotbugs issue in RuntimeException cast

* test: Added tests for more complex content types mentioned in issue #315

* docs: Updated samples to support SAM CLI operations out of the box to address #293 and switched to HTTP API by default

* feat: Updated archetypes to work out of the box with the SAM CLI, continuing to address #293

* chore: License header pass on the entire project

* fix: Set default value for setDisableException mapper in config to false

* fix: Updated default initialization timeout to 20 seconds

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Eran Medan <[email protected]>
jogep pushed a commit to jogep/aws-serverless-java-container that referenced this issue Dec 8, 2020
* fix issue 317 - use charset from request

* update dependencies

* update build dependencies, remove spring boot 2.0.x

* restoring ci config

Co-authored-by: Stefano Buliani <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants