[BE] Redis Rate Limiter 사용시, Lua 스크립트를 읽지 못하는 현상

[mclub4]

What is the bug?

Redis Rate Limiter를 이용하여 Api Rate Limiter를 구현하려고 하는데, Lua 스크립트를 읽지 못해서 오류가 나는 현상

Under what circumstances does the bug occur?

    @Bean
    public RouteLocator gatewayRoutes(RouteLocatorBuilder builder,
                                      AuthorizationHeaderFilter authFilter,
                                      RequestRateLimitFilter limitFilter) {
        return builder.routes()
                .route("spring", r -> r.path("/api/**")
                        .filters(f->f
                                .filter(authFilter.apply(config -> {config.setRequiredRole("role_user");}))
                                .filter(limitFilter.apply(config -> {
                                    config.setRateLimiter(redisRateLimiter());
                                    config.setRouteId("22");
                                }))
                        )
                        .uri("http://localhost:8080"))
                .build();
    }

    @Bean
    public RedisRateLimiter redisRateLimiter() {
        return new RedisRateLimiter(0, 1, 1); 
    }

위와같이 authFilter를 통과하고, 그다음 api rate limiter를 구현하기 위해 limitFilter를 통과하도록 구성했습니다. limit filter는 아래와 같이 구성했습니다.

@Override
    public GatewayFilter apply(Config config) {
        log.info("여기 필터 지나는지 확인 1111");
        GatewayFilter filter = (exchange, chain) -> {
            KeyResolver keyResolver = getOrDefault(config.keyResolver, defaultKeyResolver);
            RedisRateLimiter rateLimiter = getOrDefault(config.rateLimiter, defaultRateLimiter);
            String routeId = config.getRouteId();
            log.info("여기 필터 지나는지 확인 2222222");

            return keyResolver.resolve(exchange)
                    .doOnNext(key -> log.info("Resolved key: {}", key))
                    .flatMap(key -> {
                        log.info("Calling rate limiter with routeId: {} and key: {}", routeId, key);
                        return rateLimiter.isAllowed(routeId, key);
                    })
                    .flatMap(rateLimitResponse -> {
                        log.info("Rate limiter response: {}", rateLimitResponse);
                        log.info("여기 필터 지나는지 확인 2222222");
                        if (rateLimitResponse.isAllowed()) {
                            return chain.filter(exchange);
                        } else {
                            throw new BusinessException(TOO_MANY_REQUESTS);
                        }
                    });
        };

        return filter;
    }

그런데 rateLimiter.isAllowed(routeId, key)를 하는 과정에서, Token Bucket Algorithm을 적용하여 Api Rate Limiter를 적용하는 구조인데, Race Condition을 방지하기 위해서 Lua 스크립트를 통해서 진행한다고 합니다.

하지만, 해당 경로로 요청을 보낼 시,

2024-05-06T15:59:16.674+09:00 DEBUG 23448 --- [back-gateway] [ioEventLoop-5-1] o.s.c.g.f.ratelimit.RedisRateLimiter     : Error calling rate limiter lua

org.springframework.data.redis.RedisSystemException: Error in execution
	at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:52) ~[spring-data-redis-3.2.5.jar:3.2.5]
	at org.springframework.data.redis.connection.lettuce.LettuceReactiveRedisConnection.lambda$translateException$0(LettuceReactiveRedisConnection.java:242) ~[spring-data-redis-3.2.5.jar:3.2.5]
	at reactor.core.publisher.Flux.lambda$onErrorMap$27(Flux.java:7267) ~[reactor-core-3.6.5.jar:3.6.5]
	at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94) ~[reactor-core-3.6.5.jar:3.6.5]
	at reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onError(MonoFlatMapMany.java:256) ~[reactor-core-3.6.5.jar:3.6.5]
	at io.lettuce.core.RedisPublisher$ImmediateSubscriber.onError(RedisPublisher.java:895) ~[lettuce-core-6.3.2.RELEASE.jar:6.3.2.RELEASE/8941aea]
	at io.lettuce.core.RedisPublisher$State.onError(RedisPublisher.java:716) ~[lettuce-core-6.3.2.RELEASE.jar:6.3.2.RELEASE/8941aea]
	at io.lettuce.core.RedisPublisher$RedisSubscription.onError(RedisPublisher.java:357) ~[lettuce-core-6.3.2.RELEASE.jar:6.3.2.RELEASE/8941aea]
	at io.lettuce.core.RedisPublisher$SubscriptionCommand.onError(RedisPublisher.java:801) ~[lettuce-core-6.3.2.RELEASE.jar:6.3.2.RELEASE/8941aea]
	at io.lettuce.core.RedisPublisher$SubscriptionCommand.doOnComplete(RedisPublisher.java:761) ~[lettuce-core-6.3.2.RELEASE.jar:6.3.2.RELEASE/8941aea]
	at io.lettuce.core.protocol.CommandWrapper.complete(CommandWrapper.java:65) ~[lettuce-core-6.3.2.RELEASE.jar:6.3.2.RELEASE/8941aea]
	at io.lettuce.core.protocol.CommandHandler.complete(CommandHandler.java:745) ~[lettuce-core-6.3.2.RELEASE.jar:6.3.2.RELEASE/8941aea]
	at io.lettuce.core.protocol.CommandHandler.decode(CommandHandler.java:680) ~[lettuce-core-6.3.2.RELEASE.jar:6.3.2.RELEASE/8941aea]
	at io.lettuce.core.protocol.CommandHandler.channelRead(CommandHandler.java:597) ~[lettuce-core-6.3.2.RELEASE.jar:6.3.2.RELEASE/8941aea]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) ~[netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) ~[netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.109.Final.jar:4.1.109.Final]
	at java.base/java.lang.Thread.run(Thread.java:840) ~[na:na]
Caused by: io.lettuce.core.RedisCommandExecutionException: ERR Error running script (call to f_775635cca91092477cbcee85fd800accb6adc4b2): @user_script:1: user_script:1: attempt to call field 'replicate_commands' (a nil value) 
	at io.lettuce.core.internal.ExceptionFactory.createExecutionException(ExceptionFactory.java:147) ~[lettuce-core-6.3.2.RELEASE.jar:6.3.2.RELEASE/8941aea]
	at io.lettuce.core.internal.ExceptionFactory.createExecutionException(ExceptionFactory.java:116) ~[lettuce-core-6.3.2.RELEASE.jar:6.3.2.RELEASE/8941aea]
	... 21 common frames omitted

attempt to call field 'replicate_commands' (a nil value) 라는 오류가 발생하게 됩니다. Lua 스크립트를 Redis가 제대로 읽지 못하는 것으로 추정됩니다. 하지만 원인을 모르겠습니다.

Redis 버전때문인가 생각이 들어서 Redis 3.2, 4.2, 5.0, 7.2 버전으로 전부 굴려봤지만 항상 동일한 오류가 뜹니다...
아무리 검색해봐도 원인을 알 수 없습니다...

자세한 코드는 back-gateway에 있는 spring cloud gateway 코드 참조해주시면 됩니다.

Expected Result

정상적으로 Token Bucket Algorithm이 적용되어야 합니다.

Reference Materials (Optional)

[ 참고자료 ]

참고자료 1
참고자료 2
참고자료 3
참고자료 4

[mclub4]

Spring Cloud Gateway 라이브러리에 있는 lua script 복사해서 내 로컬에 내 Redis에 실행을 해봤더니...

redis-cli --eval my_script.lua

위 명령어로 테스트 해봤을때는 첫번째 줄에서 오류가 안났었습니다. 즉, Lua Script가 작동한다는 것입니다.
그래서 아무래도 redis 라이브러리가 제대로 내 redis를 가르키지 못하는 것 같다고 느꼈습니다.

그래서

 @Bean
    @Primary
    public ReactiveRedisConnectionFactory reactiveRedisConnectionFactory() {
        return new LettuceConnectionFactory("redis", 6379);
    }

    @Bean
    @Primary
    public ReactiveRedisTemplate<String, Object> reactiveRedisTemplate(ReactiveRedisConnectionFactory factory) {
        RedisSerializationContext<String, Object> serializationContext = RedisSerializationContext
                .<String, Object>newSerializationContext(new StringRedisSerializer())
                .hashKey(new StringRedisSerializer())
                .hashValue(new StringRedisSerializer())
                .build();

        return new ReactiveRedisTemplate<>(factory, serializationContext);
    }

이런식으로 제 Redis를 확실히 맵핑을 시켰더니

잘 작동합니다!

[mclub4]

추후 코드 정리해서 PR 올리도록 하겠습니다.