Java Spark Produces 404 for the root url

[skryvets]

• Framework version: 1.1
• Implementations: Spark

Scenario

Very basic "Hello World" app.

Folder Structure

|--src
    |--main
        |--java
           SparkResources.java
           StreamLambdaHandler.java
pom.xml
sam.yaml

SparkResources

import static spark.Spark.*;

public class SparkResources {
    public static void defineResources() {
        get("/", (req, res) -> "This is root page");
        get("/example", (req, res) -> "This is example page");
    }
}

Expected behavior

1. User should be able to see "This is root page" when navigates to the root url "/"
2. User should be able to see "This is example page" when navigates to the "/example"

Actual behavior

1. User sees "404 page not found" when navigates to the root url "/"
2. User sees "This is example page" when navigates to the "/example" (this one works correctly)

Steps to reproduce

1. Clone the repo
   [email protected]:sergeome/java-spark-aws-lambda.git
2. mvn clean package
3. sam local start-api --template sam.yaml
4. Navigate to both Urls.

Full log output

For the first time user navigates to the "/", for the second time user navigates to the "/example"

$ sam local start-api --template sam.yaml
2018/05/22 21:18:06 Connected to Docker 1.37
2018/05/22 21:18:06 Fetching lambci/lambda:java8 image for java8 runtime...
java8: Pulling from lambci/lambda
Digest: sha256:6d8025500d2b06054470cb787d54d5bda8f1362fdaf0079510414947cf8cc0a1
Status: Image is up to date for lambci/lambda:java8

Mounting StreamLambdaHandler::handleRequest (java8) at http://127.0.0.1:3000/{proxy+} [OPTIONS GET HEAD POST PUT DELETE PATCH]

You can now browse to the above endpoints to invoke your functions.
You do not need to restart/reload SAM CLI while working on your functions,
changes will be reflected instantly/automatically. You only need to restart
SAM CLI if you update your AWS SAM template.

2018/05/22 21:18:13 Invoking StreamLambdaHandler::handleRequest (java8)
2018/05/22 21:18:13 Decompressing spark-cloud/target/spark-cloud-1.0.jar
2018/05/22 21:18:13 Mounting /private/var/folders/96/ykvzpbyx6sn6srvd5j48xmnm0000gn/T/aws-sam-local-1527041893269176000 as /var/task:ro inside runtime container
[main] INFO com.amazonaws.serverless.proxy.internal.LambdaContainerHandler - Starting Lambda Container Handler
[Thread-0] INFO com.amazonaws.serverless.proxy.spark.embeddedserver.LambdaEmbeddedServer - Spark called configureWebSockets. However, web sockets are not supported
[Thread-0] INFO com.amazonaws.serverless.proxy.spark.embeddedserver.LambdaEmbeddedServer - Starting Spark server, ignoring port and host
[Thread-0] INFO com.amazonaws.serverless.proxy.spark.embeddedserver.LambdaEmbeddedServer - Called join method, nothing to do here since Lambda only runs a single event per container
START RequestId: 4f3421ae-087e-435b-b8da-8d8007294676 Version: $LATEST
[main] INFO spark.http.matching.MatcherFilter - The requested route [/favicon.ico] has not been mapped in Spark for Accept: [image/webp,image/apng,image/*,*/*;q=0.8]
[main] INFO com.amazonaws.serverless.proxy.internal.LambdaContainerHandler - 127.0.0.1:51053 - [01/01/1970:00:00:00Z] "GET /favicon.ico null" 404 48 "-" "-" combined
END RequestId: 4f3421ae-087e-435b-b8da-8d8007294676
REPORT RequestId: 4f3421ae-087e-435b-b8da-8d8007294676  Duration: 160.74 ms     Billed Duration: 200 ms Memory Size: 512 MB     Max Memory Used: 2 MB   
2018/05/22 21:18:20 Invoking StreamLambdaHandler::handleRequest (java8)
2018/05/22 21:18:20 Decompressing spark-cloud/target/spark-cloud-1.0.jar
2018/05/22 21:18:20 Mounting /private/var/folders/96/ykvzpbyx6sn6srvd5j48xmnm0000gn/T/aws-sam-local-1527041900174297000 as /var/task:ro inside runtime container
[main] INFO com.amazonaws.serverless.proxy.internal.LambdaContainerHandler - Starting Lambda Container Handler
[Thread-0] INFO com.amazonaws.serverless.proxy.spark.embeddedserver.LambdaEmbeddedServer - Spark called configureWebSockets. However, web sockets are not supported
[Thread-0] INFO com.amazonaws.serverless.proxy.spark.embeddedserver.LambdaEmbeddedServer - Starting Spark server, ignoring port and host
[Thread-0] INFO com.amazonaws.serverless.proxy.spark.embeddedserver.LambdaEmbeddedServer - Called join method, nothing to do here since Lambda only runs a single event per container
START RequestId: 5031b3aa-7565-4a63-bde6-451978f77419 Version: $LATEST
[main] INFO com.amazonaws.serverless.proxy.internal.LambdaContainerHandler - 127.0.0.1:51053 - [01/01/1970:00:00:00Z] "GET /example null" 200 20 "-" "-" combined
END RequestId: 5031b3aa-7565-4a63-bde6-451978f77419
REPORT RequestId: 5031b3aa-7565-4a63-bde6-451978f77419  Duration: 157.26 ms     Billed Duration: 200 ms Memory Size: 512 MB     Max Memory Used: 2 MB   

[sapessi]

Hi @sergeome, the issue seems to be in the SAM template. When you configure API Gateway with this event structure below API Gateway only accepts requests in sub-resources - ie there needs to be a /something.

Events:
        GetResource:
          Type: Api
          Properties:
            Path: /{proxy+}
            Method: any

To make the GET method on the root work, you should configure the following structure:

Events:
        GetSubResource:
          Type: Api
          Properties:
            Path: /{proxy+}
            Method: any
        GetRootResource:
          Type: Api
          Properties:
            Path: /
            Method: any

[skryvets]

Hi, @sapessi. Thank you a lot for looking at that problem.
Sorry for this issue in SAM template.

I modified sam.yaml as you suggested and pushed my changes to the repo. However, the root url "/" still produces 404 (although it is routed correctly right now I suppose).

I suspect it might be an issue with my environment setup. Can you please confirm that it does work for you in this repo? Thank you.

This is log when SAM is starting:

$ sam local start-api --template sam.yaml
A newer version of the AWS SAM CLI is available!
Your version:   0.2.11
Latest version: 0.3.0
See https://github.com/awslabs/aws-sam-local for upgrade instructions

2018/05/23 19:49:52 Connected to Docker 1.37
2018/05/23 19:49:52 Fetching lambci/lambda:java8 image for java8 runtime...
java8: Pulling from lambci/lambda
Digest: sha256:6d8025500d2b06054470cb787d54d5bda8f1362fdaf0079510414947cf8cc0a1
Status: Image is up to date for lambci/lambda:java8

Mounting StreamLambdaHandler::handleRequest (java8) at http://127.0.0.1:3000/ [OPTIONS GET HEAD POST PUT DELETE PATCH]
Mounting StreamLambdaHandler::handleRequest (java8) at http://127.0.0.1:3000/{proxy+} [OPTIONS GET HEAD POST PUT DELETE PATCH]

You can now browse to the above endpoints to invoke your functions.
You do not need to restart/reload SAM CLI while working on your functions,

This is log when invoking root url "/":

2018/05/23 19:50:47 Invoking StreamLambdaHandler::handleRequest (java8)
2018/05/23 19:50:47 Decompressing /Users/sergeome/spark-cloud/target/spark-cloud-1.0.jar
2018/05/23 19:50:47 Mounting /private/var/folders/96/ykvzpbyx6sn6srvd5j48xmnm0000gn/T/aws-sam-local-1527123047114197000 as /var/task:ro inside runtime container
[main] INFO com.amazonaws.serverless.proxy.internal.LambdaContainerHandler - Starting Lambda Container Handler
[Thread-0] INFO com.amazonaws.serverless.proxy.spark.embeddedserver.LambdaEmbeddedServer - Spark called configureWebSockets. However, web sockets are not supported
[Thread-0] INFO com.amazonaws.serverless.proxy.spark.embeddedserver.LambdaEmbeddedServer - Starting Spark server, ignoring port and host
[Thread-0] INFO com.amazonaws.serverless.proxy.spark.embeddedserver.LambdaEmbeddedServer - Called join method, nothing to do here since Lambda only runs a single event per container
START RequestId: d7ebd4d7-9d9c-48f0-93a2-0492e2207615 Version: $LATEST
[main] INFO spark.http.matching.MatcherFilter - The requested route [] has not been mapped in Spark for Accept: [text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8]
[main] INFO com.amazonaws.serverless.proxy.internal.LambdaContainerHandler - 127.0.0.1:49542 - [01/01/1970:00:00:00Z] "GET  null" 404 48 "-" "-" combined
END RequestId: d7ebd4d7-9d9c-48f0-93a2-0492e2207615
REPORT RequestId: d7ebd4d7-9d9c-48f0-93a2-0492e2207615  Duration: 165.11 ms     Billed Duration: 200 ms Memory Size: 512 MB     Max Memory Used: 2 MB   

This is log when invoking "/example":

2018/05/23 19:54:34 Invoking StreamLambdaHandler::handleRequest (java8)
2018/05/23 19:54:34 Decompressing /Users/sergeome/spark-cloud/target/spark-cloud-1.0.jar
2018/05/23 19:54:35 Mounting /private/var/folders/96/ykvzpbyx6sn6srvd5j48xmnm0000gn/T/aws-sam-local-1527123274836129000 as /var/task:ro inside runtime container
[main] INFO com.amazonaws.serverless.proxy.internal.LambdaContainerHandler - Starting Lambda Container Handler
[Thread-0] INFO com.amazonaws.serverless.proxy.spark.embeddedserver.LambdaEmbeddedServer - Spark called configureWebSockets. However, web sockets are not supported
[Thread-0] INFO com.amazonaws.serverless.proxy.spark.embeddedserver.LambdaEmbeddedServer - Starting Spark server, ignoring port and host
[Thread-0] INFO com.amazonaws.serverless.proxy.spark.embeddedserver.LambdaEmbeddedServer - Called join method, nothing to do here since Lambda only runs a single event per container
START RequestId: 4685da5d-a4bf-4eee-ab73-d613621f57d1 Version: $LATEST
[main] INFO com.amazonaws.serverless.proxy.internal.LambdaContainerHandler - 127.0.0.1:49599 - [01/01/1970:00:00:00Z] "GET /example null" 200 20 "-" "-" combined
END RequestId: 4685da5d-a4bf-4eee-ab73-d613621f57d1
REPORT RequestId: 4685da5d-a4bf-4eee-ab73-d613621f57d1  Duration: 163.24 ms     Billed Duration: 200 ms Memory Size: 512 MB     Max Memory Used: 2 MB   
2018/05/23 19:54:38 Invoking StreamLambdaHandler::handleRequest (java8)
2018/05/23 19:54:38 Decompressing /Users/sergeome/spark-cloud/target/spark-cloud-1.0.jar
2018/05/23 19:54:38 Mounting /private/var/folders/96/ykvzpbyx6sn6srvd5j48xmnm0000gn/T/aws-sam-local-1527123278045600000 as /var/task:ro inside runtime container
[main] INFO com.amazonaws.serverless.proxy.internal.LambdaContainerHandler - Starting Lambda Container Handler
[Thread-0] INFO com.amazonaws.serverless.proxy.spark.embeddedserver.LambdaEmbeddedServer - Spark called configureWebSockets. However, web sockets are not supported
[Thread-0] INFO com.amazonaws.serverless.proxy.spark.embeddedserver.LambdaEmbeddedServer - Starting Spark server, ignoring port and host
[Thread-0] INFO com.amazonaws.serverless.proxy.spark.embeddedserver.LambdaEmbeddedServer - Called join method, nothing to do here since Lambda only runs a single event per container
START RequestId: 01e1a4ff-1335-4b7d-bf8e-70416a92eeab Version: $LATEST
[main] INFO spark.http.matching.MatcherFilter - The requested route [/favicon.ico] has not been mapped in Spark for Accept: [image/webp,image/apng,image/*,*/*;q=0.8]
[main] INFO com.amazonaws.serverless.proxy.internal.LambdaContainerHandler - 127.0.0.1:49599 - [01/01/1970:00:00:00Z] "GET /favicon.ico null" 404 48 "-" "-" combined
END RequestId: 01e1a4ff-1335-4b7d-bf8e-70416a92eeab
REPORT RequestId: 01e1a4ff-1335-4b7d-bf8e-70416a92eeab  Duration: 146.77 ms     Billed Duration: 200 ms Memory Size: 512 MB     Max Memory Used: 2 MB  

This is how it looks in the browser

Again, thank you very much for your help!

[sapessi]

Interesting. Looks like the path property of the proxy event is set to "" and not "/". Have you tried running this in Lambda (not SAM local)? Does it work there? Just trying to figure out whether the fix is only in the library or in SAM local as well.

[skryvets]

Just deployed. The same results as on local with sam.

https://3b8d6xmvy7.execute-api.us-east-1.amazonaws.com/dev/ - Not Found
https://3b8d6xmvy7.execute-api.us-east-1.amazonaws.com/dev/example - "This is example page"

[sapessi]

Thanks @sergeome, I'll investigate this week.

[sapessi]

Hey @sergeome, any chance you can test with the 1.2-SNAPSHOT in the core branch of this repo? Simply download or clone the core branch to your local box then run mvn install. Once that's installed in your local cache you should be able to just change the dependency in your pom to 1.2-SNAPSHOT

[sapessi]

release 1.1.1 is going out to maven central