Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NullPointerException if calling Lambda via CloudWatch event #237

Closed
huksley opened this issue Feb 11, 2019 · 13 comments
Closed

NullPointerException if calling Lambda via CloudWatch event #237

huksley opened this issue Feb 11, 2019 · 13 comments

Comments

@huksley
Copy link

huksley commented Feb 11, 2019

  • Framework version: 1.3
  • Implementations: Spring Boot

Scenario

Adding following event to MyServiceFunction

CheckWebsiteScheduledEvent:
  Type: Schedule
  Properties:
    Schedule: rate(5 minutes)

results in NullPointerException generated when CloudWatch periodically calls Lambda at AwsProxyHttpServletRequestReader.java:48

servletRequest.setAttribute(ALB_CONTEXT_PROPERTY, request.getRequestContext().getElb());

Expected behavior

No NullPointerException in Cloudformation logs

Actual behavior

Actually running Lambda + provide some way to execute code on such event.

Steps to reproduce

Add event to sam.yaml, deploy, observe CloudWatch logstream.

Full log output

2019-02-11 13:54:19.463 ERROR 1 --- [ main] c.a.s.p.internal.LambdaContainerHandler : Error while handling request

java.lang.NullPointerException: null
at com.amazonaws.serverless.proxy.internal.servlet.AwsProxyHttpServletRequestReader.readRequest(AwsProxyHttpServletRequestReader.java:48) ~[aws-serverless-java-container-core-1.3.jar:na]
at com.amazonaws.serverless.proxy.internal.servlet.AwsProxyHttpServletRequestReader.readRequest(AwsProxyHttpServletRequestReader.java:28) ~[aws-serverless-java-container-core-1.3.jar:na]
at com.amazonaws.serverless.proxy.internal.LambdaContainerHandler.proxy(LambdaContainerHandler.java:174) [aws-serverless-java-container-core-1.3.jar:na]
at com.amazonaws.serverless.proxy.internal.LambdaContainerHandler.proxyStream(LambdaContainerHandler.java:209) [aws-serverless-java-container-core-1.3.jar:na]
at my.service.StreamLambdaHandler.handleRequest(StreamLambdaHandler.java:37) [task/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_181]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_181]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_181]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_181]
at lambdainternal.EventHandlerLoader$StreamMethodRequestHandler.handleRequest(EventHandlerLoader.java:350) [LambdaSandboxJava-1.0.jar:na]
at lambdainternal.EventHandlerLoader$2.call(EventHandlerLoader.java:888) [LambdaSandboxJava-1.0.jar:na]
at lambdainternal.AWSLambda.startRuntime(AWSLambda.java:293) [LambdaSandboxJava-1.0.jar:na]
at lambdainternal.AWSLambda.<clinit>(AWSLambda.java:64) [LambdaSandboxJava-1.0.jar:na]
at java.lang.Class.forName0(Native Method) [na:1.8.0_181]
at java.lang.Class.forName(Class.java:348) [na:1.8.0_181]
at lambdainternal.LambdaRTEntry.main(LambdaRTEntry.java:104) [LambdaJavaRTEntry-1.0.jar:na]
@sapessi
Copy link
Collaborator

sapessi commented Feb 11, 2019

What does the scheduled event do? Does it invoke the same Lambda function or does it send an HTTP request?

@huksley
Copy link
Author

huksley commented Feb 11, 2019

What does the scheduled event do? Does it invoke the same Lambda function or does it send an HTTP request?

It invokes the same Lambda function.
https://github.com/huksley/serverless-java-spring-boot/blob/master/sam-template.yaml#L44

@sapessi
Copy link
Collaborator

sapessi commented Feb 11, 2019

Ok. I'll add a null check for this particular exception. However, this framework is only meant to work with API Gateway or ALB proxy events, it will not recognize events from other sources (such as CloudWatch scheduled events). To make this work, you will either have to implement your own RequestReader and ResponseWriter for the event types or handle it in the handler method before calling the proxy function on the framework.

@huksley
Copy link
Author

huksley commented Feb 12, 2019

To make this work, you will either have to implement your own RequestReader and ResponseWriter for the event types or handle it in the handler method before calling the proxy function on the framework.

Is there any examples how to handle this? As I understand right now there is single entry point handleRequest(InputStream, OutputStream, Context context) which handles all incoming events. How to distinguish type of event from this method?

@sapessi
Copy link
Collaborator

sapessi commented Feb 12, 2019

I'm assuming you are using the scheduled event just to keep the function "warm". If that's the case, I would probably recommend just catching the exception and returning successfully. Even though there is an exception, you are achieving your goal which is to force Lambda to keep instances of the function around.

The most important this to remember is to catch the exception and exit the function successfully. If you don't, Lambda will treat a cold start exception as an initialization failure and will discard the instance of the function.

sapessi added a commit that referenced this issue Feb 12, 2019
…ain an HTTP method and a request context we throw an . This addresses #237 by making the error more explicit.
@huksley
Copy link
Author

huksley commented Feb 13, 2019

@sapessi Thanks for a important note about function exit!

However I am genuinely interested in able to use all 3 types of events in Spring Boot application deployed as AWS Lambda - event (SNS) processing, cron tasks, API gw endpoints

@sapessi sapessi added the docs label Feb 13, 2019
@sapessi
Copy link
Collaborator

sapessi commented Feb 13, 2019

Understood. I'll write a short version of the steps here. I'll use this issue to remind myself to expand the docs on how to achieve this.

In the examples below I use the StreamHandler instead of the typed one because Lambda's built-in serialization mechanism does not support Jackson annotations. This frameworks uses a Jackson instance with annotations to parse the authorizer context in the proxy event. Also, I wrote the Java code below directly in the GitHub comment window so forgive me for typos/errors and missing try/catch.

You have a couple of options:
Option 1
In your stream handler, read the full stream content into a String. Then, use an ObjectMapper to unmarshal that String into one of the event types you need - either AwsProxyRequest, SNSEvent, or ScheduledEvent. I'll leave the logic on how to decide which type of event it is up to you. With the unmarshalled event type, you can find a way to convert it into an AwsProxyRequest - perhaps you can use the AwsProxyRequestBuilder. Then, with your proxy request object, you can invoke the proxy method of your LambdaContainerHandler. Something like this:

public class StreamLambdaHandler implements RequestStreamHandler {
    private static SpringLambdaContainerHandler<AwsProxyRequest, AwsProxyResponse> handler;
    static {
        try {
            handler = SpringLambdaContainerHandler.getAwsProxyHandler(PetStoreSpringAppConfig.class);
        } catch (ContainerInitializationException e) {
            // if we fail here. We re-throw the exception to force another cold start
            e.printStackTrace();
            throw new RuntimeException("Could not initialize Spring framework", e);
        }
    }

    @Override
    public void handleRequest(InputStream inputStream, OutputStream outputStream, Context context)
            throws IOException {
        // use Apache IOUtils to read the full contents of the stream to a String so that we can 
        // re-use it multiple times.
        StringWriter eventWriter = new StringWriter();
        IOUtils.copy(inputStream, eventWriter, LambdaContainerHandler.getContainerConfig().getDefaultContentCharset());
        String eventString = writer.toString();

        // next, we try to deserialize this into the various event types
       AwsProxyRequest req = LambdaContainerHandler.getObjectMapper().readValue(eventString, AwsProxyRequest.class);
       if (req.getHttpMethod() == "" or "".equals(req.getHttpMethod()) { // it's not a proxy request event
           SNSEvent sns = LambdaContainerHandler.getObjectMapper().readValue(eventString, SNSEvent.class);
           if (isValidSnsEvent(sns)) { // logic for this up to you
                req = snsEventToRequest(sns); // also up to you
           }
       }
       AwsProxyResponse resp = handler.proxy(req, context);
       LambdaContainerHandler.getObjectMapper().writeValue(outputStream, resp);
    }
}

Option 2
This is a little more involved. The library defines two abstract classes: RequestReader and ResponseWriter. The LambdaContainerHandler object uses these readers and writers to turn an event into a request type that the implementation can support. The default implementation of this included in the library is the AwsProxyHttpServletRequestReader. You could create a custom reader/writer for each event types you want to support, then use the constructor of the SpringLambdaContainerHandler class to pass in your custom request readers/writers.

While option 2 seems cleaner and more Java-ish, it involves a lot more code. For a simple solution, I would probably opt for option 1.

@huksley
Copy link
Author

huksley commented Feb 14, 2019

@sapessi Thanks for a thorough description!
I think I will go with Option 1 for a start, I will get back to you once done to share the results.

@huksley
Copy link
Author

huksley commented Feb 19, 2019

Added workaround so SNS/SQS/CloudWatch events are not sent through to AWSProxy code as-is.
huksley/serverless-java-spring-boot@ea93648

Good idea to convert them to AwsProxyRequest but I think such logic is heavily application-depended (i.e. which endpoint to hit?) or can/need to be configured somehow.

@sapessi
Copy link
Collaborator

sapessi commented Feb 26, 2019

@huksley I've added a new page to the docs about supporting custom event types. Hopefully this helps explain what is happening behind the scenes

@sapessi sapessi closed this as completed Feb 26, 2019
@huksley
Copy link
Author

huksley commented Feb 26, 2019

@sapessi Thanks! Really appreciated it! Maybe you are too much verbose but there is no such thing for docs)

@franciszabala
Copy link

franciszabala commented Nov 5, 2019

Is the doc for custom event still valid?

public class CustomRequestReader extends RequestReader<CustomEvent, AwsProxyHttpServletRequest> {
    @Override
    public AwsProxyHttpServletRequest readRequest(CustomEvent request, SecurityContext securityContext, Context lambdaContext, ContainerConfig config) {
        ...
        /* Can not return an object **AwsProxyHttpServletRequest** because instantiating it requires the first argument as AwsProxyRequest and needing to have a custom HttpServletRequest */
    }

    @Override
    protected Class<? extends CustomEventType> getRequestClass() {
        return CustomEvent.class;
    }
}

I can not return an object AwsProxyHttpServletRequest because instantiating it requires the first argument as AwsProxyRequest and needs a custom HttpServletRequest. I have to create a custom HttpServletRequest.

I also encountered an "Cannot infer type arguments for SpringBootLambdaContainerHandler<>" error.

Here's a link. Maybe you guys would like to check out just in case: https://github.com/franciszabala/aws-lambda-springboot2.0

@sapessi
Copy link
Collaborator

sapessi commented Nov 8, 2019

Hi @franciszabala. Yes, the docs are still valid. The AwsProxyHttpServletRequest is an internal class that is meant to be generated using an AwsProxyRequest object - we use it as part of our reader implementation.

You have a couple of options:

  1. Create your own custom implementation of HttpServletRequest and output that.
  2. Use the AwsProxyRequestBuilder object to generate an AwsProxyRequest from your custom event that you can then use to initialize the AwsProxyHttpServletRequest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants