Deploying with the CDK (or plain Cloudformation)

[plumdog]

Is your feature request related to a problem? Please describe.
I'm not using the serverless framework, I'm using the CDK. I want to be able to deploy a next.js app to APIGateway/Lambda/S3. I think that this project will help me to do that, but I'm not sure how.

Describe the solution you'd like
An overview of how I might go about generating code using this project and what things need to be deployed to AWS to make the app work.

Describe alternatives you've considered
I have tried using https://github.com/vincent-herlemont/next-aws-lambda-webpack-plugin, which has generated a lambda per page (and a shared lambda layer), but to make it work I need to traverse the lambda directories it creates, and determine the APIGateway config for it. I've also found that for the index route, I get an error like:

TypeError: Cannot read property 'match' of null
    at renderReqToHTML (/var/task/index.js:25929:30)
    at Module.render (/var/task/index.js:26000:28)

Further, one lambda per page seems unnecessary (at least, I'm not sure what the advantages are), though I'm happy to be told it is best practice.

Additional context
None. Happy to put together an example repo if that would be helpful.

[plumdog]

I think I have identified the error I was getting as an issue in next.js - see vercel/next.js#10953, so my approach may be viable.

[danielcondemarin]

@plumdog I’ve started working on support for aws-cdk (see https://github.com/danielcondemarin/serverless-next.js/tree/aws-cdk-support) although is still in progress so don’t expect it to work.

In terms of one lambda per page or not, I plan to provide the same architecture as the next component, that is SSR and API pages deployed to Lambda@Edge and static pages to S3 so there wouldn’t be API gateway. Later on, I may add the ability to deploy api gateway backed by one lambda per page (like the next plugin). The central idea is to give users building blocks (aka Constructs in CDK) they can use to design and deploy their next app however they want.

I’m hoping to create a formal RFC with the official proposal this week.

[plumdog]

Per vercel/next.js#10953, the issue is not with next.js, but rather with the value set for req.url here https://github.com/danielcondemarin/serverless-next.js/blob/8da92e2d8e23e361c24904e4579e9d4e02845126/packages/next-aws-lambda/lib/compatLayer.js#L15-L18

This is definitely a separate issue to my original CDK question, so I'll create a separate issue. But, fwiw, a way of dodging this issue is to put Cloudfront in front of API Gateway - Cloudfront defaults to route the "default" object, ie, the thing you get when you access the domain with no path, to /index.html. So provided you make API Gateway respond to /index.html, nothing from Cloudfront will ever encounter this error.

[OFranke]

Hey @danielcondemarin!
Would you mind giving a little more context on what needs to be done to make the whole thing work via CDK? In form of tickets, todos, an architecture diagram?

I just jumped into CDK and plan to somehow bring my nextjs frontend to AWS. I will do it any way, not sure how exactly yet - but this here seems like a good chance to make a useful contribution.

I'd appreciate some guidance by you :)

[danielcondemarin]

Hey @danielcondemarin!
Would you mind giving a little more context on what needs to be done to make the whole thing work via CDK? In form of tickets, todos, an architecture diagram?

I just jumped into CDK and plan to somehow bring my nextjs frontend to AWS. I will do it any way, not sure how exactly yet - but this here seems like a good chance to make a useful contribution.

I'd appreciate some guidance by you :)

Hi @OFranke ! Sorry it took me so long to reply to this. If you're still interested in contributing I can put together a summary of what needs doing? Thanks!

I have been doing a lot of ground work already to pull out of the next component "reusable pieces" that can be used in the AWS CDK package (or possibly other frameworks).

[OFranke]

Sorry @danielcondemarin, didn’t check my mentions. I’d be happy to take care of that. Will take me some time though but I am working on a production infrastructure that’s definitely going to be optimized regarding cost of hosting at some point. Right now I just hosted the nextjs app as ECS Container via cdk which is for sure not the best way to do.

[plumdog]

Something to beware regarding this is aws/aws-cdk#3463

Each CDK asset contributes 3 parameters, and there's a maximum of 60 in a Cloudformation stack, so that's a limit of 20.

This puts some constraints on the one-lambda-per-page architecture, until that issue is solved. I think the answer here is probably to wait for CDK to resolve that limitation. However, in the meantime, I think would require using a single shared (large) asset is used by all Lambdas, which is not ideal.

[danielcondemarin]

Something to beware regarding this is aws/aws-cdk#3463

Each CDK asset contributes 3 parameters, and there's a maximum of 60 in a Cloudformation stack, so that's a limit of 20.

This puts some constraints on the one-lambda-per-page architecture, until that issue is solved. I think the answer here is probably to wait for CDK to resolve that limitation. However, in the meantime, I think would require using a single shared (large) asset is used by all Lambdas, which is not ideal.

@plumdog The architecture I am planning to go for is using Lambda@Edge and CloudFront. That should be well under the limit of CloudFormation resources. Yes it means all server side rendered pages in one Lambda but that’s fine, the next.js team released a new build target optimised for this use 🙂
Later on I’d like to support API Gateway + Lambda like the plugin but using the new build target.

[plumdog]

the next.js team released a new build target optimised for this use

@danielcondemarin What build target is this? Only "server" and "serverless" are listed here https://nextjs.org/docs/api-reference/next.config.js/build-target and I am hopeful it could help with my API Gateway + Lambda assets problem.

[danielcondemarin]

the next.js team released a new build target optimised for this use

@danielcondemarin What build target is this? Only "server" and "serverless" are listed here https://nextjs.org/docs/api-reference/next.config.js/build-target and I am hopeful it could help with my API Gateway + Lambda assets problem.

The target is still experimental, you can see the PR for it here. That's probably why is not listed in the docs.
Although is experimental is actually what Vercel Now uses under the hood as you can see here.
The difference between "serverless" and "serverless-trace" is that the latter doesn't bundle node_modules for each page, which leads to less code duplication therefore smaller lambda artefacts. To support the target in a project like this, I had to pack the node_modules and upload them to Lambda so they're available during SSR. I have added support for it on the nextjs component, currently released on the latest alpha. See #236 (comment).

Also, it sounds like you're using the plugin version of this project which doesn't support the serverless-trace target.

[dbartholomae]

Any progress here? This topic affects me as well and I might have some time to help with a PR or two :)

[elendirx]

We used serverless framework heavily at work and it served us well, but serverless-components looks more like a desperate monetization attempt with no real benefit.

I would like to create similar architecture with CDK. Where to start? Should I just pick up the old branch or do we have some recent code?

[asterikx]

@plumdog I’ve started working on support for aws-cdk (see https://github.com/danielcondemarin/serverless-next.js/tree/aws-cdk-support) although is still in progress so don’t expect it to work.

@danielcondemarin the link doesn't work anymore. I'm very interested in this too. Is there any progress?

[asterikx]

I checked the link again, and today it works. Weird.

Anyways @danielcondemarin let me know if you need any help with the CDK stuff.

I have seen the NextjsConstruct here (I'll probably define my own, since I need additional origins for API GW, AppSync, and my CMS).

To generate the handler code, I followed the instructions from the lambda-at-edge package.

I created a script:

// generate-handlers.js

const path = require('path');
const { Builder } = require('@sls-next/lambda-at-edge');

const nextConfigDir = __dirname; // '/dir/to/my/nextapp';
const outputDir = path.join(nextConfigDir, '.serverless_nextjs');

const builder = new Builder(nextConfigDir, outputDir, {
  cmd: './node_modules/.bin/next',
  cwd: process.cwd(),
  env: {},
  args: ['build']
});

(async function () {
  return await builder.build();
})();

and then invoked it using node generate-handlers.js from the root of my Next.js app.

After creating the folder structure, the script just hangs and nothing happens. Do you have any idea what could be causing this?

EDIT: turns out I had commented args: ['build'], which causes next to start the development server instead creating a built.

[luismanuel001]

@danielcondemarin can we get an update on this? I'm also very interested on this, and may be able to help with it depending on the plan and current progress of this.

[GerardKetuma]

I checked the link again, and today it works. Weird.

Anyways @danielcondemarin let me know if you need any help with the CDK stuff.

I have seen the NextjsConstruct here (I'll probably define my own, since I need additional origins for API GW, AppSync, and my CMS).

To generate the handler code, I followed the instructions from the lambda-at-edge package.

I created a script:

// generate-handlers.js

const path = require('path');
const { Builder } = require('@sls-next/lambda-at-edge');

const nextConfigDir = __dirname; // '/dir/to/my/nextapp';
const outputDir = path.join(nextConfigDir, '.serverless_nextjs');

const builder = new Builder(nextConfigDir, outputDir, {
  cmd: './node_modules/.bin/next',
  cwd: process.cwd(),
  env: {},
  args: ['build']
});

(async function () {
  return await builder.build();
})();

and then invoked it using node generate-handlers.js from the root of my Next.js app.

After creating the folder structure, the script just hangs and nothing happens. Do you have any idea what could be causing this?

EDIT: turns out I had commented args: ['build'], which causes next to start the development server instead creating a built.

@asterikx Did you get it to work? I'm planning on going down this path.

[asterikx]

@GerardKetuma Yes and no. My setup worked, but I abandoned it due to unacceptable cold start times (which were likely caused by my project dependencies)

[crash7]

@danielcondemarin any way I can give a hand with this? I was planning on building something like this (nextjs + cdk) but with AG and Cloudfront, as a side project using the base packages of this project.
Maybe I can also contribute to your efforts towards cdk?
Let me know!

[dphang]

If you use deploy: false input running serverless command (or serverless --component=<component> build) will just build all outputs into the .serverless_nextjs directory, but not deploy, so you could actually use it with your own IaC like CDK or CDK for Terraform, you'd just need to manually provision the infrastructure yourselves (with all cache behaviors, etc.) from the build outputs.

We are looking into possibly migrating the deployment to CDK for Terraform (or other IaC) to solve a couple of pain points we've seen so far. Stay tuned.

[simonireilly]

That's really promising.

Looking at the existing branch there are a few Todo comments around cdk bundling.

This aws blog shows bundling for a static Nuxt.Js site. This could take care of managing the serverless --component<component> build step on synth/deploy.

https://aws.amazon.com/blogs/devops/building-apps-with-aws-cdk/

Then setting up the asset paths for S3 artefacts will be very straight forward as @dphang points out the src will be in the outdir.

CDK will be a nice API for customizing the stack without developing abstraction over Cloudfront, looking forward to progress.

[simonireilly]

Here is an example with the CDK which has worked for me:

https://github.com/simonireilly/nextjs-aws-cdk, would be happy to feed into https://github.com/serverless-nextjs/serverless-next.js/tree/aws-cdk-support if it is still under development @danielcondemarin, @dphang ?

This deploys well for me, but you know, it won't reach the speed of a serverless component due to cloud formation restrictions >> https://d3a8vzm1ccycvj.cloudfront.net/

[apoorvmote]

I am also looking to deploy Nextjs with CDK and/or cloudformation. I am new to this so please bare with me.
Can we deploy Nextjs (with SSR) without @sls-next/lambda-at-edge? Also why are we using lambda@edge for SSR and API? In both cases the datasource is stored at single location e.g. Dynamodb Table at us-east-1. So what is the advantage if with lambda@edge vs lambda@us-east-1?

I have been using HTTP API instead of Rest API and deploying with CDK. And HTTP API cannot be at edge location but they are performing very good from single us-east-1 location. I am also looking for possiblity of SSR with HTTP API.

[dphang]

I think this can be resolved since we have (experiment) CDK construct which has been implemented, so you can try that and hopefully it makes managing all the resources easier and more configurable.

For other issues like using Lambda@Edge vs. Lambda, I think that was original design and unfortunately the core code has been coupled with lambda-at-edge over the years. However we are working to decouple it so it can be platform-agnostic and work with Lambda/Cloudflare Workers/etc. in the near future.