Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebClient 에서 Exceeded limit on max bytes to buffer : 262144 에러가 발생하는 경우 #167

Open
occidere opened this issue Mar 13, 2022 · 0 comments

Comments

@occidere
Copy link
Owner

상황

  • WebClient 를 통해 대용량 데이터를 가져오는 경우 아래와 같은 에러 발생
    Caused by: org.springframework.core.io.buffer.DataBufferLimitException: Exceeded limit on max bytes to buffer : 262144
        at org.springframework.core.io.buffer.LimitedDataBufferList.raiseLimitException(LimitedDataBufferList.java:99) ~[spring-core-5.3.16.jar:5.3.16]
        Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 

원인

  • Spring boot가 2.2.x, Spring framework : 5.2.x 가 되면서 WebClient에 설정되는 default codec의 buffer size가 변경된 것으로 추정
  • 예전엔 아래와 같이 config 를 변경해서 해결할 수 있었으나, default codec 값이 hard coded 되어서 config 로 이제는 수정 불가
    spring:
      codec:
        max-in-memory-size: 100MB

해결

  • WebClient 빌드 시 ExchangeStrategies 를 사용하여 codec 을 주입해야 함
    @Bean
    fun webClient() = WebClient.builder()
        .baseUrl(baseUrl)
        .defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_FORM_URLENCODED_VALUE)
        .defaultHeader(HttpHeaders.USER_AGENT, "Mozilla/5.0")
        .exchangeStrategies(
            ExchangeStrategies.builder()
                .codecs { configurer ->
                    configurer.defaultCodecs()
                        .maxInMemorySize(-1) // unlimited mem size
                }.build()
        ).build()

참고

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant