Tracking ISR issues

[dphang]

Just creating this to track any bugs or issues with the newly-released Incremental Static Regeneration feature -

• Improving e2e tests (flaky tests, more test coverage for locales, basepath etc?)

[dphang]

@kirkness the ISR tests have been fairly flaky (mostly failing but sometimes 1/3 or 2/3 pass), if you have a chance please take a look otherwise I can check it out later on the weekend: https://github.com/serverless-nextjs/serverless-next.js/actions/runs/863012949

[kirkness]

Hi @dphang, keen to have a look at this, however, I won't be able to get much time on it until early next week, likely Tuesday.

[kirkness]

Related: #1099

[dphang]

No worries, yeah it would be good to add more test coverage. The last few runs seem to not be flaky, though.

[Miroko]

Not sure if I should open an issue about this or write straight to this thread...

We are getting the following debug log information when trying to deploy the "@sls-next/serverless-component": "1.20.0-alpha.16" (at the time of writing the latest alpha version) to AWS.

Our deploy script uses an existing serverless state made with previous deployment. That successfully deployed version uses SSR only, otherwise the configuration and component version are exactly the same. We are just trying to switch to use ISR.

Any help with this would be appreciated and thanks for the great work you have been doing for the ISR support!

  DEBUG ─ The QueueUrl has changed
  DEBUG ─ Deleting previous queue
  error:
  MissingRequiredParameter: Missing required key 'QueueUrl' in params
    at ParamValidator.fail (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/param_validator.js:50:37)
    at ParamValidator.validateStructure (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/param_validator.js:61:14)
    at ParamValidator.validateMember (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/param_validator.js:88:21)
    at ParamValidator.validate (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/param_validator.js:34:10)
    at Request.VALIDATE_PARAMETERS (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/event_listeners.js:132:42)
    at Request.callListeners (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at callNextListener (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/sequential_executor.js:96:12)
    at /builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/event_listeners.js:86:9
    at finish (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/config.js:386:7)
    at /builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/config.js:404:9 {
  code: 'MissingRequiredParameter',
  time: 2021-06-01T11:55:41.540Z
}

[a-game]

Not sure if I should open an issue about this or write straight to this thread...

We are getting the following debug log information when trying to deploy the "@sls-next/serverless-component": "1.20.0-alpha.16" (at the time of writing the latest alpha version) to AWS.

Our deploy script uses an existing serverless state made with previous deployment. That successfully deployed version uses SSR only, otherwise the configuration and component version are exactly the same. We are just trying to switch to use ISR.

Any help with this would be appreciated and thanks for the great work you have been doing for the ISR support!

  DEBUG ─ The QueueUrl has changed
  DEBUG ─ Deleting previous queue
  error:
  MissingRequiredParameter: Missing required key 'QueueUrl' in params
    at ParamValidator.fail (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/param_validator.js:50:37)
    at ParamValidator.validateStructure (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/param_validator.js:61:14)
    at ParamValidator.validateMember (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/param_validator.js:88:21)
    at ParamValidator.validate (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/param_validator.js:34:10)
    at Request.VALIDATE_PARAMETERS (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/event_listeners.js:132:42)
    at Request.callListeners (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at callNextListener (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/sequential_executor.js:96:12)
    at /builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/event_listeners.js:86:9
    at finish (/builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/config.js:386:7)
    at /builds/sok-digikehitys/raflaamo/raflaamo/frontend/node_modules/aws-sdk/lib/config.js:404:9 {
  code: 'MissingRequiredParameter',
  time: 2021-06-01T11:55:41.540Z
}

I'm having the same problem here and I have a theory about the cause (It's really just a shot in the dark).

I think the errors occurs at

    // triggerStaticRegeneration.ts
    await sqs.send(
        new SendMessageCommand({
            QueueUrl: `https://sqs.${region}.amazonaws.com/${bucketName}.fifo`,
            ...
        })
    );

bucketName comes from s3BucketNameFromEventRequest.ts which tries to parse it from an AWSLambda.CloudFrontRequest

export const s3BucketNameFromEventRequest = (
  request: AWSLambda.CloudFrontRequest
): string | undefined => {
  const { region, domainName } = request.origin?.s3 || {};
  return domainName?.replace(`.s3.${region}.amazonaws.com`, "");
};

Looking at the the docs for AWSLambda.CloudFrontRequest the s3 origin looks like this: awsexamplebucket.s3.us-east-1.amazonaws.com

This will result in s3BucketNameFromEventRequest returning awsexamplebucket as the bucketName.

So QueueUrl would be https://sqs.us-east-1.amazonaws.com/awsexamplebucket.fifo

That QueueUrl would be incorrect in my case. The created SQS has this url: https://sqs.us-east-1.amazonaws.com/12345/awsexamplebucket.fifo

Conclusion

I suspect that the account id 12345 is missing from the parsed bucketName but needed for a valid SQS url.

[valurhrafn]

Some issues I have come across implementing the ISR functionality:

• Cannot set a specific region in serverless.yml
  • tried setting eu-west-1 as a region and got an error
• if trailingSlash: true is set in next.config.js then the regeneration lambda does not know what file to update and in some cases creates a .html file which gets updates
  • I fixed it by changing the regex in the regeneration lambda that creates the filename so it removes the trailing slash first, also tried turning off the trailing slash setting in next.config.js which also works.
• If using a predefined IAM policy, I had to manually add the SQS permissions to the policy. (maybe that is normal, not sure).
• I am unable to get revalidate to work if it is defined in index.tsx e.g. the frontpage of the next.js application, if I duplicate the index.tsx page as something else like frontpage.tsx, revalidate works as expected. I have not found a workaround for this.

[kirkness]

I'm having the same problem here and I have a theory about the cause (It's really just a shot in the dark).

I think the errors occurs at

    // triggerStaticRegeneration.ts
    await sqs.send(
        new SendMessageCommand({
            QueueUrl: `https://sqs.${region}.amazonaws.com/${bucketName}.fifo`,
            ...
        })
    );

bucketName comes from s3BucketNameFromEventRequest.ts which tries to parse it from an AWSLambda.CloudFrontRequest

export const s3BucketNameFromEventRequest = (
  request: AWSLambda.CloudFrontRequest
): string | undefined => {
  const { region, domainName } = request.origin?.s3 || {};
  return domainName?.replace(`.s3.${region}.amazonaws.com`, "");
};

Looking at the the docs for AWSLambda.CloudFrontRequest the s3 origin looks like this: awsexamplebucket.s3.us-east-1.amazonaws.com

This will result in s3BucketNameFromEventRequest returning awsexamplebucket as the bucketName.

So QueueUrl would be https://sqs.us-east-1.amazonaws.com/awsexamplebucket.fifo

That QueueUrl would be incorrect in my case. The created SQS has this url: https://sqs.us-east-1.amazonaws.com/12345/awsexamplebucket.fifo

Conclusion

I suspect that the account id 12345 is missing from the parsed bucketName but needed for a valid SQS url.

I think its this line which is run on deployment:

serverless-next.js/packages/serverless-components/aws-sqs/serverless.js

Line 70 in d5bbdc6

await deleteQueue({ sqs, queueUrl: this.state.url });

I'll put in a fix for this.

[mikejcooper]

Revalidate not working as expected (I think).

Running 1.20.0-alpha.18 in eu-west-2, the request sequence/behaviour is as follows:

• 1st: Treats getStaticProps like getServerSideProps, blocking until contents of getStaticProps have returned (Cloudfront miss)
• 2 to n: Cloudfront hit (sometimes)
• nth: Treats getStaticProps like getServerSideProps, blocking until contents of getStaticProps have returned (Cloudfront miss)
• ... cycle repeats

nth request is when Revalidate / s-maxage is reached.

Below is the small example I ran the text on:

export const getStaticProps: GetStaticProps = async () => {
  const apolloClient = initializeApollo()

  const getAuctionsResult = await apolloClient.mutate<GetAuctionsQuery, GetAuctionsQueryVariables>({
    mutation: GetAuctionsDocument,
  })

  return {
    revalidate: 15,
    props: {
      date: new Date().toLocaleString(),
      auctions: JSON.parse(JSON.stringify(getAuctionsResult.data?.auction)) || [],
    },
  }
}

const AboutPage = (props: { date: string; auctions: any[] }) => (
  <div>
    <p>This is the about page, and the time is about {props.date}</p>
    <p></p>
    <p>This is the about page, and the time is about {JSON.stringify(props.auctions[0])}</p>
  </div>
)

Also having issues on other more complex pages where it's a cache miss every time.

Any thoughts?

[dphang]

@mikejcooper Unfortunately I'm not really familiar with apollo client, is it a problem without it as well?

[danjiro]

Great work on this so far guys. I have some questions. Is revalidate supposed to be functional with 1.20.0-alpha.23? I tried to deploy with it but it doesn't seem to be working correctly. The SQS isn't sending/receiving anything and im getting some 503 errors in console as it tries to fetch index.json multiple times. I have my revalidate on my index.js file and duplicated it to a test.js file and when accessing the test route also get 503:

503 ERROR
The request could not be satisfied.
The Lambda function associated with the CloudFront distribution is invalid or doesn't have the required permissions. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.

Am i missing any specific configurations that is needed?

[dphang]

Great work on this so far guys. I have some questions. Is revalidate supposed to be functional with 1.20.0-alpha.23? I tried to deploy with it but it doesn't seem to be working correctly. The SQS isn't sending/receiving anything and im getting some 503 errors in console as it tries to fetch index.json multiple times. I have my revalidate on my index.js file and duplicated it to a test.js file and when accessing the test route also get 503:

503 ERROR

The request could not be satisfied.

The Lambda function associated with the CloudFront distribution is invalid or doesn't have the required permissions. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.

If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.

Am i missing any specific configurations that is needed?

Please post the logs (503 is very generic and doesn't tell anything about the actual issue). Without knowing anything, I'm gonna guess there could be a permissions issue

[teban3010]

Hi, I'm having the same issue, the error shows on the browser and there are no logs, or at least I couldn't find any.

In my case, I'm doing the deployment on a github action, here is the serverless.yml file:

name: api-hub-web

nextApp:
  component: "@sls-next/[email protected]"
  inputs:
    bucketName: web-dev-s3
    description: '*lambda-type*@Edge for dev-api-hub-web'
    name:
      defaultLambda: web-dev-lambda
      apiLambda: web-dev-lambda
    publicDirectoryCache: false
    runtime:
      defaultLambda: 'nodejs12.x'
      apiLambda: 'nodejs12.x'
    logLambdaExecutionTimes: true
    minifyHandlers: true
    memory: 2048
    timeout: 30
    policy: 'arn:aws:iam::<acount>:policy/next-lambdas-execution-policy'

The deployment permissions are the same as https://github.com/serverless-nextjs/serverless-next.js#aws-permissions-for-deployment and the lambda policy is the following:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:*",
                "logs:CreateLogStream",
                "sqs:*",
                "logs:CreateLogGroup",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        }
    ]
}

I'm using serverless 2.46.0 and next 10.2.0.

[adamdottv]

In testing ISR with the https://github.com/dapperlabs/nextjs-isr-testing repo, I also am seeing 503 errors on retrieving JSON data files. I found this in the default lambda logs:

ERROR	Invoke Error 	
{
    "errorType": "TypeError",
    "errorMessage": "serdePlugin.lib.decodeHTML is not a function",
    "$metadata": {
        "attempts": 1,
        "totalRetryDelay": 0
    },
    "stack": [
        "TypeError: serdePlugin.lib.decodeHTML is not a function",
        "    at Object.tagValueProcessor (/var/task/index-b8afa7fd.js:3725:111)",
        "    at processTagValue (/var/task/serdePlugin-45449631.js:588:19)",
        "    at Object.getTraversalObj (/var/task/serdePlugin-45449631.js:709:31)",
        "    at Object.exports.parse (/var/task/serdePlugin-45449631.js:1742:39)",
        "    at /var/task/index-b8afa7fd.js:3720:48",
        "    at processTicksAndRejections (internal/process/task_queues.js:97:5)"
    ]
}

[kirkness]

Hi, I'm having the same issue, the error shows on the browser and there are no logs, or at least I couldn't find any.

In my case, I'm doing the deployment on a github action, here is the serverless.yml file:

name: api-hub-web

nextApp:
  component: "@sls-next/[email protected]"
  inputs:
    bucketName: web-dev-s3
    description: '*lambda-type*@Edge for dev-api-hub-web'
    name:
      defaultLambda: web-dev-lambda
      apiLambda: web-dev-lambda
    publicDirectoryCache: false
    runtime:
      defaultLambda: 'nodejs12.x'
      apiLambda: 'nodejs12.x'
    logLambdaExecutionTimes: true
    minifyHandlers: true
    memory: 2048
    timeout: 30
    policy: 'arn:aws:iam::<acount>:policy/next-lambdas-execution-policy'

The deployment permissions are the same as https://github.com/serverless-nextjs/serverless-next.js#aws-permissions-for-deployment and the lambda policy is the following:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:*",
                "logs:CreateLogStream",
                "sqs:*",
                "logs:CreateLogGroup",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        }
    ]
}

I'm using serverless 2.46.0 and next 10.2.0.

Hey @teban3010, make sure to switch to the region in the console that you are viewing the failing page from, a few more deets here: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-cloudwatch-metrics-logging.html#lambda-cloudwatch-logs

[teban3010]

Hey @teban3010, make sure to switch to the region in the console that you are viewing the failing page from, a few more deets here: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-cloudwatch-metrics-logging.html#lambda-cloudwatch-logs

@kirkness thanks for that, I have the same error as @adamelmore on the lambdas

ERROR Invoke Error

{
    "errorType": "TypeError",
    "errorMessage": "n.lib.decodeHTML is not a function",
    "$metadata": {
        "attempts": 1,
        "totalRetryDelay": 0
    },
    "stack": [
        "TypeError: n.lib.decodeHTML is not a function",
        "    at Object.tagValueProcessor (/var/task/index-b8afa7fd.js:1:59241)",
        "    at y (/var/task/serdePlugin-45449631.js:1:7924)",
        "    at Object.h.getTraversalObj (/var/task/serdePlugin-45449631.js:1:9674)",
        "    at Object.t.parse (/var/task/serdePlugin-45449631.js:1:21540)",
        "    at /var/task/index-b8afa7fd.js:1:59096",
        "    at processTicksAndRejections (internal/process/task_queues.js:97:5)"
    ]
}

UPDATE: This happens on an update of a deployment that uses a previous version of the component and on a new one as well.

[dphang]

If you check affected dist files, it looks like it's trying to call a non-existent function:

index-b8afa7fd.js

var parseBody = function (streamBody, context) {
    return collectBodyString(streamBody, context).then(function (encoded) {
        if (encoded.length) {
            var parsedObj = serdePlugin.parser.parse(encoded, {
                attributeNamePrefix: "",
                ignoreAttributes: false,
                parseNodeValue: false,
                trimValues: false,
                tagValueProcessor: function (val, tagName) { return (val.trim() === "" ? "" : serdePlugin.lib.decodeHTML(val)); },
            });
            var textNodeName = "#text";
            var key = Object.keys(parsedObj)[0];
            var parsedObjToReturn = parsedObj[key];
            if (parsedObjToReturn[textNodeName]) {
                parsedObjToReturn[key] = parsedObjToReturn[textNodeName];
                delete parsedObjToReturn[textNodeName];
            }
            return serdePlugin.getValueFromTextNode(parsedObjToReturn);
        }
        return {};
    });
};

When I checked serdePlugin.lib it seems lib is initialized empty in the serdePlugin.js file. Although the decodeHTML function should actually be from the entities library. See source: https://github.com/aws/aws-sdk-js-v3/blob/fd6b7ebd7d3fc4a2ceb3b016404de73413e1fb2f/clients/client-sqs/protocols/Aws_query.ts#L104.

Not sure why it's not being bundled correctly by Rollup?

[dphang]

Seems there might be a problem with @rollup/plugin-commonjs: 19.0.0, after 18.1.0 it broke the bundling and those decode functions are no longer bundled: https://github.com/rollup/plugins/blob/master/packages/commonjs/CHANGELOG.md. Need to investigate why

Ref PR: rollup/plugins#658

It can't be easily caught in unit tests (Rollup bundling issue), but possibly should add an e2e test for this? cc: @kirkness. Meanwhile I will downgrade to 18.1.0, which I tested and it bundles correctly. And also exclude it from dependabot updates.

[dphang]

The above fix is already published, please try it out and see if it works for you.

[teban3010]

I've tried out, it works at first but then I receive the same error on the default lambda. The only difference is that now if I press F5, some times I see the page and some times I have a 503 error. On the network tab, I see a lot of 503 of the other pages json. The other thing is that the page is not being updated after the revalidate time.

ERROR Invoke Error

{
    "errorType": "TypeError",
    "errorMessage": "n.lib.decodeHTML is not a function",
    "$metadata": {
        "attempts": 1,
        "totalRetryDelay": 0
    },
    "stack": [
        "TypeError: n.lib.decodeHTML is not a function",
        "    at Object.tagValueProcessor (/var/task/index-b8afa7fd.js:1:59241)",
        "    at y (/var/task/serdePlugin-45449631.js:1:7924)",
        "    at Object.h.getTraversalObj (/var/task/serdePlugin-45449631.js:1:9674)",
        "    at Object.t.parse (/var/task/serdePlugin-45449631.js:1:21540)",
        "    at /var/task/index-b8afa7fd.js:1:59096",
        "    at processTicksAndRejections (internal/process/task_queues.js:97:5)"
    ]
}

[dphang]

I've tried out, it works at first but then I receive the same error on the default lambda. The only difference is that now if I press F5, some times I see the page and some times I have a 503 error. On the network tab, I see a lot of 503 of the other pages json. The other thing is that the page is not being updated after the revalidate time.

ERROR Invoke Error

{
    "errorType": "TypeError",
    "errorMessage": "n.lib.decodeHTML is not a function",
    "$metadata": {
        "attempts": 1,
        "totalRetryDelay": 0
    },
    "stack": [
        "TypeError: n.lib.decodeHTML is not a function",
        "    at Object.tagValueProcessor (/var/task/index-b8afa7fd.js:1:59241)",
        "    at y (/var/task/serdePlugin-45449631.js:1:7924)",
        "    at Object.h.getTraversalObj (/var/task/serdePlugin-45449631.js:1:9674)",
        "    at Object.t.parse (/var/task/serdePlugin-45449631.js:1:21540)",
        "    at /var/task/index-b8afa7fd.js:1:59096",
        "    at processTicksAndRejections (internal/process/task_queues.js:97:5)"
    ]
}

Based on the hash on your latest log of the affected file: index-b8afa7fd.js it looks to be you are still using the same version as before (it's same as reported above in all logs, before the fix). The new hashes of bundled files where SQS is invoked in the fixed version (which has the decodeHTML function) should be:

• serdePlugin-7f212178.js
• index-855e9789.js

Please check if you had updated properly to the latest alpha version, if you specify the latest version exactly in serverless.yml file, I think it should have no problems (not sure if you use just alpha tag from npm, there could potentially be caching problems?). The latest version is 1.20.0-alpha.25, you can always find in npm: https://www.npmjs.com/package/@sls-next/serverless-component/v/1.20.0-alpha.25. From the version tab I see 0 downloads so far of the latest version (either npm is not updated so fast or there are really no downloads yet). Also it does take some time for function updates to propagate?

[teban3010]

You are right, I used version 24. Now it's not returning 503, but it's not doing the ISR, I added a date property with revalidate: 10 but I alway have the same date. When I checked the logs of the lambda that has de SQS trigger I have this error

ERROR	Invoke Error 	
{
    "errorType": "Error",
    "errorMessage": "Cannot find module './pages/.js'\nRequire stack:\n- /var/task/index.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js",
    "code": "MODULE_NOT_FOUND",
    "requireStack": [
        "/var/task/index.js",
        "/var/runtime/UserFunction.js",
        "/var/runtime/index.js"
    ],
    "stack": [
        "Error: Cannot find module './pages/.js'",
        "Require stack:",
        "- /var/task/index.js",
        "- /var/runtime/UserFunction.js",
        "- /var/runtime/index.js",
        "    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:815:15)",
        "    at Function.Module._load (internal/modules/cjs/loader.js:667:27)",
        "    at Module.require (internal/modules/cjs/loader.js:887:19)",
        "    at require (internal/modules/cjs/helpers.js:74:18)",
        "    at /var/task/index.js:1:4565",
        "    at Array.map (<anonymous>)",
        "    at Runtime.exports.handler (/var/task/index.js:1:4409)",
        "    at Runtime.handleOnce (/var/runtime/Runtime.js:66:25)"
    ]
}

[dphang]

Not sure which line is failing but it's strange to be requiring a pages/.js file, not sure why it's coming empty.

Could you please unminify the Lambda so it's easier to understand which line is failing in the SQS lambda (regeneration). And also post how your pages directory structure looks like, manifests etc if possible. Guessing it's because of the SQS-triggered lambda just tries to require the page path passed from the SQS message data after it gets triggered here:

serverless-next.js/packages/libs/lambda-at-edge/src/default-handler.ts

Lines 287 to 292 in 135e006

	const { throttle } = await triggerStaticRegeneration({
	basePath,
	request,
	response,
	pagePath: staticRoute.page
	});

Sending event:

serverless-next.js/packages/libs/lambda-at-edge/src/lib/triggerStaticRegeneration.ts

Lines 42 to 58 in 135e006

	await sqs.send(
	new SendMessageCommand({
	QueueUrl: `https://sqs.${region}.amazonaws.com/${bucketName}.fifo`,
	MessageBody: JSON.stringify(regenerationEvent), // This is not used, however it is a required property
	// We only want to trigger the regeneration once for every previous
	// update. This will prevent the case where this page is being
	// requested again whilst its already started to regenerate.
	MessageDeduplicationId:
	options.response.headers["etag"]?.[0].value ||
	new Date(options.response.headers["last-modified"]?.[0].value)
	.getTime()
	.toString(),
	// Only deduplicate based on the object, i.e. we can generate
	// different pages in parallel, just not the same one
	MessageGroupId: options.request.uri
	})
	);

Parsing the event and requiring page:

serverless-next.js/packages/libs/lambda-at-edge/src/regeneration-handler.ts

Lines 13 to 23 in 135e006

	const regenerationEvent: RegenerationEvent = JSON.parse(record.body);
	const manifest: OriginRequestDefaultHandlerManifest = Manifest;
	const { req, res } = lambdaAtEdgeCompat(
	{ request: regenerationEvent.cloudFrontEventRequest },
	{ enableHTTPCompression: manifest.enableHTTPCompression }
	);
	// eslint-disable-next-line @typescript-eslint/no-var-requires
	const page = require(`./${regenerationEvent.pagePath}`);

I wonder if there's a special character that is not being passed properly through SQS message body?

[teban3010]

I believe that I remove the minify from the serverless.yml file, but I see the exact same error. This is my serverless.yml file

name: web

nextApp:
  component: "@sls-next/[email protected]"
  inputs:
    bucketName: web-s3
    description: '*lambda-type*@Edge for web'
    name:
      defaultLambda: web-default-lambda
      apiLambda: web-api-lambda
      imageLambda: web-image-lambda
    publicDirectoryCache: false
    runtime:
      defaultLambda: 'nodejs12.x'
      apiLambda: 'nodejs12.x'
    logLambdaExecutionTimes: true
    # minifyHandlers: true
    memory: 2048
    timeout: 30
    policy: 'arn:aws:iam::599903031295:policy/next-lambdas-execution-policy'

And this is my pages directory structure:

.
├── pages
│   ├── _app.tsx
│   ├── _document.tsx
│   ├── account.tsx
│   ├── api
│   │   └── auth
│   │       └── [...nextauth].ts
│   ├── contact.tsx
│   ├── index.tsx
│   └── products.tsx

I'm not sure where can I get the manifest.

[dphang]

@teban3010 we need which line it is failing at. Manifests are the JSON files generated in .serverless_nextjs/*-lambda directory, they tell the Lambda how to route. Basically the point to know that is so we can figure out:

1. Which line of code is it failing exactly
2. What page name is being accessed and the manifests also help to see if there is some issue generating them (they are used to do the routing).

If you could provide some repro step as well (or a repo) then I can also debug myself. Otherwise it's hard to guess where is failing.

[teban3010]

Sorry, I didn't get that at first. I'm doing all the deployment using a github action, so I didn't have access to those files because were created on the server.

Now, I did the deployment from my machine, and got those files. Unfortunately, I cannot provide access to the repository, but I will try to put something in place so you can test it out.

For the deployment, I'm just running npx serverless and the component version is on the serverless.yml file. I'm running it from macOS.

The .serverless_next folder structure is the following:

.
├── api-lambda
│   ├── index-9e574644.js
│   ├── index.js
│   ├── manifest.json
│   ├── pages
│   │   └── api
│   │       └── auth
│   │           └── [...nextauth].js
│   └── routes-manifest.json
├── assets
│   ├── BUILD_ID
│   ├── _next
│   │   ├── data
│   │   │   └── VYqAiP3KsZSxRrAqToc5l
│   │   │       ├── contact.json
│   │   │       ├── index.json
│   │   │       └── products.json
│   │   └── static
│   │       ├── VYqAiP3KsZSxRrAqToc5l
│   │       │   ├── _buildManifest.js
│   │       │   └── _ssgManifest.js
│   │       ├── chunks
│   │       │   ├── 1bfc9850.c8a70c1a6d80895144eb.js
│   │       │   ├── 252f366e.cc4df410de28b33613de.js
│   │       │   ├── 68d96d8dea06f46304db45ccefbfe2e77afe930f.11e556bb04dd6b857ec2.js
│   │       │   ├── 9f8cf9a5fe6f78b75cd4f3950f8751eae6026112.e24e05a6d9f0cabdbed6.js
│   │       │   ├── commons.0e4f5fcc136d6b919420.js
│   │       │   ├── framework.8f5c9b5ad9ab0f9b0592.js
│   │       │   ├── main-b98586f123ef58bc77aa.js
│   │       │   ├── pages
│   │       │   │   ├── _app-49a79c217e9d61538824.js
│   │       │   │   ├── _error-78a9006d8ba77f5d833a.js
│   │       │   │   ├── account-f0837073fb5202f16feb.js
│   │       │   │   ├── contact-7525176d3691a02fc984.js
│   │       │   │   ├── index-722e09bd7f2d74d04323.js
│   │       │   │   └── products-85e1e7200514c2407f11.js
│   │       │   ├── polyfills-4f4acd756cef4fe6da1b.js
│   │       │   └── webpack-50bee04d1dc61f8adf5b.js
│   │       ├── css
│   │       │   ├── 91c186b9d9397e7d4bb4.css
│   │       │   └── b32593cd38d5156699c5.css
│   │       └── media
│   │           ├── roboto-all-400-normal.b5a0a0df312d3423493b163e9675488c.woff
│   │           ├── roboto-cyrillic-400-normal.3605d18d8a4ce5bd39c3d32119574854.woff2
│   │           ├── roboto-cyrillic-ext-400-normal.dd55ea0a2e90d5d86e15444a623c385a.woff2
│   │           ├── roboto-greek-400-normal.352cc77aaed5208ce13c79cf9c89892f.woff2
│   │           ├── roboto-greek-ext-400-normal.bc7ace6e50c7d9d4d95923b61b1de59c.woff2
│   │           ├── roboto-latin-400-normal.176f8f5bd5f02b3abfcf894955d7e919.woff2
│   │           ├── roboto-latin-ext-400-normal.dcc07bcfcd475f34f20b2c1265bd1265.woff2
│   │           └── roboto-vietnamese-400-normal.52cebac009baf5592e7d0ee53d8e45ed.woff2
│   ├── public
│   │   ├── android-chrome-192x192.png
│   │   ├── android-chrome-512x512.png
│   │   ├── apple-touch-icon.png
│   │   ├── email
│   │   │   ├── hub_1.png
│   │   │   ├── hub_logo_blue.png
│   │   │   └── logo_white.png
│   │   ├── favicon-16x16.png
│   │   ├── favicon-32x32.png
│   │   ├── favicon.ico
│   │   ├── icons
│   │   │   ├── ask.svg
│   │   │   ├── business.svg
│   │   │   ├── clipboard.svg
│   │   │   ├── contact.svg
│   │   │   ├── developer.svg
│   │   │   ├── status.svg
│   │   │   └── word-balloon.svg
│   │   ├── illustrations
│   │   │   ├── building.svg
│   │   │   ├── home_page_1.png
│   │   │   ├── home_page_2.png
│   │   │   ├── news.svg
│   │   │   └── supplier.svg
│   │   ├── logo-blue.png
│   │   ├── logo.svg
│   │   └── site.webmanifest
│   └── static-pages
│       └── VYqAiP3KsZSxRrAqToc5l
│           ├── 404.html
│           ├── contact.html
│           ├── index.html
│           └── products.html
├── default-lambda
│   ├── GetObjectCommand-4007ef8b.js
│   ├── PutObjectCommand-ee4a4ff3.js
│   ├── S3Client-2ce44c80.js
│   ├── _commonjsHelpers-429052b6.js
│   ├── configurations-e825ea6c.js
│   ├── defaultRoleAssumers-9657d50f.js
│   ├── httpRequest-31970d1c.js
│   ├── index-0737fe4f.js
│   ├── index-084130cb.js
│   ├── index-246324a8.js
│   ├── index-4b0544e0.js
│   ├── index-855e9789.js
│   ├── index-891c56ba.js
│   ├── index-9e574644.js
│   ├── index-c10a6c3c.js
│   ├── index.js
│   ├── manifest.json
│   ├── pages
│   │   ├── _error.js
│   │   ├── account.js
│   │   ├── contact.js
│   │   ├── index.js
│   │   └── products.js
│   ├── prerender-manifest.json
│   ├── routes-manifest.json
│   └── serdePlugin-7f212178.js
├── image-lambda
└── regeneration-lambda
    ├── PutObjectCommand-847f6a70.js
    ├── S3Client-4495fe4b.js
    ├── configurations-e825ea6c.js
    ├── httpRequest-31970d1c.js
    ├── index-0737fe4f.js
    ├── index-8cd8cf24.js
    ├── index.js
    ├── manifest.json
    └── pages
        ├── _error.js
        ├── account.js
        ├── contact.js
        ├── index.js
        └── products.js

Regeneration-lambda manifest:

{
  "buildId": "VYqAiP3KsZSxRrAqToc5l",
  "pages": {
    "dynamic": [],
    "ssr": {
      "dynamic": {},
      "nonDynamic": {
        "/account": "pages/account.js",
        "/contact": "pages/contact.js",
        "/": "pages/index.js",
        "/products": "pages/products.js",
        "/_error": "pages/_error.js"
      }
    },
    "html": { "dynamic": {}, "nonDynamic": { "/404": "pages/404.html" } },
    "ssg": {
      "dynamic": {},
      "nonDynamic": {
        "/contact": { "initialRevalidateSeconds": 10, "srcRoute": null },
        "/products": { "initialRevalidateSeconds": 10, "srcRoute": null },
        "/": { "initialRevalidateSeconds": 10, "srcRoute": null }
      }
    }
  },
  "publicFiles": {
    "/android-chrome-192x192.png": "android-chrome-192x192.png",
    "/android-chrome-512x512.png": "android-chrome-512x512.png",
    "/apple-touch-icon.png": "apple-touch-icon.png",
    "/email/hub_1.png": "email/hub_1.png",
    "/email/hub_logo_blue.png": "email/hub_logo_blue.png",
    "/email/logo_white.png": "email/logo_white.png",
    "/favicon-16x16.png": "favicon-16x16.png",
    "/favicon-32x32.png": "favicon-32x32.png",
    "/favicon.ico": "favicon.ico",
    "/icons/ask.svg": "icons/ask.svg",
    "/icons/business.svg": "icons/business.svg",
    "/icons/clipboard.svg": "icons/clipboard.svg",
    "/icons/contact.svg": "icons/contact.svg",
    "/icons/developer.svg": "icons/developer.svg",
    "/icons/status.svg": "icons/status.svg",
    "/icons/word-balloon.svg": "icons/word-balloon.svg",
    "/illustrations/building.svg": "illustrations/building.svg",
    "/illustrations/home_page_1.png": "illustrations/home_page_1.png",
    "/illustrations/home_page_2.png": "illustrations/home_page_2.png",
    "/illustrations/news.svg": "illustrations/news.svg",
    "/illustrations/supplier.svg": "illustrations/supplier.svg",
    "/logo-blue.png": "logo-blue.png",
    "/logo.svg": "logo.svg",
    "/site.webmanifest": "site.webmanifest"
  },
  "trailingSlash": false,
  "domainRedirects": {},
  "enableHTTPCompression": false,
  "logLambdaExecutionTimes": true
}

Default-lambda manifest:

{
  "buildId": "VYqAiP3KsZSxRrAqToc5l",
  "pages": {
    "dynamic": [],
    "ssr": {
      "dynamic": {},
      "nonDynamic": {
        "/account": "pages/account.js",
        "/contact": "pages/contact.js",
        "/": "pages/index.js",
        "/products": "pages/products.js",
        "/_error": "pages/_error.js"
      }
    },
    "html": { "dynamic": {}, "nonDynamic": { "/404": "pages/404.html" } },
    "ssg": {
      "dynamic": {},
      "nonDynamic": {
        "/contact": { "initialRevalidateSeconds": 10, "srcRoute": null },
        "/products": { "initialRevalidateSeconds": 10, "srcRoute": null },
        "/": { "initialRevalidateSeconds": 10, "srcRoute": null }
      }
    }
  },
  "publicFiles": {
    "/android-chrome-192x192.png": "android-chrome-192x192.png",
    "/android-chrome-512x512.png": "android-chrome-512x512.png",
    "/apple-touch-icon.png": "apple-touch-icon.png",
    "/email/hub_1.png": "email/hub_1.png",
    "/email/hub_logo_blue.png": "email/hub_logo_blue.png",
    "/email/logo_white.png": "email/logo_white.png",
    "/favicon-16x16.png": "favicon-16x16.png",
    "/favicon-32x32.png": "favicon-32x32.png",
    "/favicon.ico": "favicon.ico",
    "/icons/ask.svg": "icons/ask.svg",
    "/icons/business.svg": "icons/business.svg",
    "/icons/clipboard.svg": "icons/clipboard.svg",
    "/icons/contact.svg": "icons/contact.svg",
    "/icons/developer.svg": "icons/developer.svg",
    "/icons/status.svg": "icons/status.svg",
    "/icons/word-balloon.svg": "icons/word-balloon.svg",
    "/illustrations/building.svg": "illustrations/building.svg",
    "/illustrations/home_page_1.png": "illustrations/home_page_1.png",
    "/illustrations/home_page_2.png": "illustrations/home_page_2.png",
    "/illustrations/news.svg": "illustrations/news.svg",
    "/illustrations/supplier.svg": "illustrations/supplier.svg",
    "/logo-blue.png": "logo-blue.png",
    "/logo.svg": "logo.svg",
    "/site.webmanifest": "site.webmanifest"
  },
  "trailingSlash": false,
  "domainRedirects": {},
  "enableHTTPCompression": false,
  "logLambdaExecutionTimes": true
}

[teban3010]

I've made a deployment using this repo https://github.com/dapperlabs/nextjs-isr-testing, just add static props on index and adding the same serverless.yml file that I already have, and it worked. As I'm based on Argentina I have some delay on the time I believe that is due to replication on Cloudfront.

@dphang yes, the error is the same, is it possible that there is something old on the deployment? how can i completely remove the deployment, I don't see any stack created so I'm not sure how to delete everything at one, so I can redeploy it from scratch and check if that is the actual issue. Is npx serverless remove enough?

I am also planning to revamp the development guide + add a debugging guide too to help you all self-diagnose (since I'm the only main contributor right now, hard to do it all myself :)

That's good!!

[dphang]

Thanks, will take a look later this weekend

[hinsxd]

Have already been using admin roles so SQS permissions should not be problem. But never able to get ISR working :(

Can someone please provide a simple list of setups to try ISR? I can only ctrl-f the readme.md for "incremental" and don't know if it needs extra configuration...

Also, is it expected to be flanky right now? I was trying to do ISR but it cant revalidate, then I tried gSSP but it always gives the default locale when route change. Now I am forced to use gSP without revalidate and use a CI/CD to rebuild every hour...

[teban3010]

Just as an update. I've been deploying with the latest version for the last days and also removed the custom policy that I have. Now I see the following behaviour:

• The lambda that updates the content is being executed and get the data as expected
• The page is not being updated
• I don't have any errors on any lambda

[dphang]

@teban3010 sorry I hadn't had much luck when taking a look yet. Are you able to build from source and add some logging statements, it might be helpful to understand where the code is failing.

[telmperez]

I'm using a custom handler defined in my serverless config file that I have in a file called custom.js in the nextjs root directory. Example,

myNextApplication:
component: '@sls-next/[email protected]'
inputs:
handler: custom.handler

However when the "Next.js Regeneration Lambda" deploys it doesn't seem to copy the custom handler code over (which is fine by me). The problem is that the Regeneration Lambda seems to set the custom handler name into the lambda definition instead of "index.handler".

ISR seems to work if I update the Regeneration Lambda handler to "index.handler" instead of the "custom.handler" that I had before.

Not sure if this is a bug but it seems suspicious. Thanks!

[igormartimiano]

@teban3010 sorry I hadn't had much luck when taking a look yet. Are you able to build from source and add some logging statements, it might be helpful to understand where the code is failing.

@dphang I previously got some errors related to permissions but I'm currently stuck at the same point as @teban3010. And I might have some time to build from source and add some logs to it, but I'm not familiar with the project at all. Can you give me some pointers so I can try to help you guys out?

[mikejcooper]

@dphang Firstly, thanks for all the work on this repo, it's working great for our team!

I'm having trouble with ISR, the getStaticProps is acting like getServerSideProps. The example code below is my index.ts file adapted from your original PR with an added sleep for 5s.

Each request to index takes ~5.3s. I was expecting a cached version served back to the client. What am I doing wrong here?

const sleep = (milliseconds: number) => {
  return new Promise((resolve) => setTimeout(resolve, milliseconds))
}

export async function getStaticProps() {
  await sleep(5000)

  return {
    revalidate: 60,
    props: {
      date: new Date().toLocaleString(),
    },
  }
}

const AboutPage = (props: { date: string }) => (
  // ...
  <p>This is the about page, and the time is about {props.date}</p>
  // ...
)

export default AboutPage

Using 3.2.0-alpha.16

[dphang]

@igormartimiano yes, you can edit the code in default-handler (lambda-at-edge package) and look at CONTRIBUTING.md for some help as well. The trigger of the static regeneration from default lambda is here:

serverless-next.js/packages/libs/lambda-at-edge/src/lib/triggerStaticRegeneration.ts

Line 12 in 3f75647

export const triggerStaticRegeneration = async (

. And this is the handler that handles regeneration of the s3 page: https://github.com/serverless-nextjs/serverless-next.js/blob/master/packages/libs/lambda-at-edge/src/regeneration-handler.ts. So you can log those places and see what the exact behavior is like.

@mikejcooper sure I can take a look this week, if you have time you can also try to debug following above steps to see the exact behavior

cc: @kirkness if you have any ideas as well.

[mxmtsk]

I'm encountering similar same issues with pages not being refreshed after the set revalidate time, e.g. this page:
https://www.voteswiper.org/tr/almanya
where State election Mecklenburg-Vorpommern 2021 is still English when it should've been updated to Turkish.

I've checked logs but couldn't find any errors. I've noticed that only the main homepage, so https://www.voteswiper.org/tr is refreshed.

The whole source code is published here: https://github.com/MOVACT/voteswiper-web/

As far as I see, I'm not doing anything out of the ordinary. Any ideas on how to debug this further?

Edit: I've noticed that every request is a Miss from cloudfront and the cache-control header is always cache-control: public, max-age=0, s-maxage=0, must-revalidate, except the main homepage, which has a correct cache-control header - so this looks similar to what @danjiro was/is experiencing. Were you able to fix it?

[mxmtsk]

After debugging further it seems to me that ISR is not working correctly when i specify a bucketRegion - in my case it was set to eu-central-1. When I remove it from my serverless.yml and deploy a fresh new stack ISR works correctly.

I was incorrect. I've debugged so many things that I wrongly assumed this. The actual issue for me was providing paths in getStaticPaths so that it generates some pages at build time. As soon as I returned an empty array, regeneration works.

[J3tto]

Also can confirm I'm also unable to get revalidate working on root / (i.e. homepage) as either a index.tsx or as a [[...path]].tsx file.

But works without issues everywhere else, so its just the root that's broken.

I'm willing to try and have a go at fixing this.

Update: Looks like it gets added to the s3 bucket with the wrong basePath name

Update 2:
After a bit of digging looks like its this line as it doesn't account for the fact the uri might be / which after regex replace results in a empty string and then a .html file which is missing the index. Just needs a explicit check of /. I will update and raise a PR.

serverless-next.js/packages/libs/lambda-at-edge/src/s3/s3StorePage.ts

Line 32 in 494121a

const baseKey = options.uri

[rightones]

I get "NotImplemented" error on Next 11. It is fixed right after I downgraded Next to 10.2.3.
I think it is a same problem with AWS Amplify Console's issue(#).

[WarcraftYax]

I don't have much to add to this, other than ISR stopped working for me on a recent deploy. One of the new features I added was using next.js's i18n localization. On a bit of a whim, I decided to remove it and redeploy, and now ISR is working again.

@dphang - not sure if that information helps to narrow this down at all.

Maybe something to do with the Accept Language header being part of the CloudFront cache key when localization is enabled? Or something to do with the localized subpathing (though I only had a single en localization and I was accessing it without a subpath and still had the issue).

I noticed all my x-cache response headers were misses from CloudFront. Am I right in thinking that means they are hitting the origin and should therefore be invoking the Default Lambda? Because I couldn't see any CloudWatch Logs (and I still can't, even though it's now working)! This makes debugging very hard ... is it hitting the Lambda and error-ing silently? Or is it not hitting it at all?

[yogeshydvme]

Something strange is happening in ISR, sometime it properly follow the revalidate timings and regenerate the page but sometime event after 10-20 refreshes and waiting for 30 sec it doesn't do anythings, it returns the same page.
Revalidate is 10 sec in my case

Sometime it works fine, sometime it doesn't

I am getting always cache-control: public, max-age=0, s-maxage=0, must-revalidate as header when it doesn't work,

Is something wrong with SQS or cloudfront @dphang Can you please help
current serverless.yml looks like
web:
component: "@sls-next/[email protected]"
inputs:
cloudfront:
# if you want to use an existing cloudfront distribution, provide it here
distributionId: E3O2OAXSARFDR
timeout:
defaultLambda: 30
apiLambda: 30
imageLambda: 30
regenerationLambda: 30
memory: 2048
publicDirectoryCache: false
name:
defaultLambda: web_default_dev
apiLambda: web_api_dev
imageLambda: web_image_dev
regenerationLambda: web_re_dev
bucketName: web-ui
bucketRegion: 'ap-south-1'
build:
env:
NEXT_PUBLIC_APP_ENV: dev

[dphang]

@WarcraftYax hm yes there is not good e2e test coverage for i18n locales yet, it might be that the path is not translated correctly to hit the regeneration lambda (and save the new page), are there any errors you can see in the regeneration lambda? Can you maybe open a new issue with the same details so it's easier to track.

@yogeshydvme maybe can you open a new issue with more details like page you are trying to hit, any error logs, etc (both in the default and regeneration lambdas).

I'll close this soon; I think this issue has gotten quite large so opening separate issues will help in tracking it better.

[ddcech]

With the latest v3.3.0-alpha.5 and Nextjs 11.1.0 my ISR issues have been resolved. I removed all the resources created by AWS and removed the directories created by serverless-next, and redeployed. Thank you for the good work on this project.

[danjiro]

ive just tried again on 3.3.0-alpha.5 and nextjs 11.1.1 and am receiving this error on my regeneration lambda:

2021-08-31T06:05:21.849Z	2b02e3cb-c865-5385-ad24-ba6c00b7c2e0	ERROR	Invoke Error 	
{
    "errorType": "TypeError",
    "errorMessage": "result is not a function",
    "stack": [
        "TypeError: result is not a function",
        "    at /var/task/regeneration-handler-fef292c5.js:490:9",
        "    at new Promise (<anonymous>)",
        "    at resultToChunks (/var/task/regeneration-handler-fef292c5.js:488:12)",
        "    at renderPageToHtml (/var/task/regeneration-handler-fef292c5.js:517:47)",
        "    at processTicksAndRejections (internal/process/task_queues.js:95:5)",
        "    at async /var/task/regeneration-handler-fef292c5.js:530:38",
        "    at async Promise.all (index 0)",
        "    at async Runtime.handler (/var/task/regeneration-handler-fef292c5.js:524:5)"
    ]
}

Edit: Looks like nextjs 11.1.1 has changed result to be an actual Observer instead of a mock one

[dphang]

@danjiro yes I just tried to fix here: #1611, hopefully it solves it. I think the problem is Next.js keeps changing the implementation - but I updated it to the latest and also added a fallback to use the page's renderReqToHtml (without renderMode specified, which seems to render HTML string regardless of version).

[dphang]

Closing this issue since I believe most of the initial issues should have been resolved (or if not, have asked to create a new issue), going forward please create new issues so it's easier to separate and track. Thanks!

[danjiro]

@danjiro yes I just tried to fix here: #1611, hopefully it solves it. I think the problem is Next.js keeps changing the implementation - but I updated it to the latest and also added a fallback to use the page's renderReqToHtml (without renderMode specified, which seems to render HTML string regardless of version).

thanks so much for fixing this!

[enzoames]

Hello! quick question. Can anyone talk a little bit about the aws costs of running ISR? I have one page that would need to be regenerated every 10 seconds. I'm not sure how things work under the hood but I assume there are a handful of services that will rerun every 10 seconds and therefore increasing cost? Thanks!