Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't deploy GraphQL to Lambda with CDK #7749

Closed
tekkeon opened this issue Aug 3, 2021 · 5 comments
Closed

Can't deploy GraphQL to Lambda with CDK #7749

tekkeon opened this issue Aug 3, 2021 · 5 comments
Labels
needs triage This issue has not been looked into

Comments

@tekkeon
Copy link

tekkeon commented Aug 3, 2021

Bug Report

Hi there, my organization uses CDK to deploy resources and I'm close to convincing my teams to move most of our service stacks to using NestJS as we rebuild our entire backend.I am trying to build a proof-of-concept using CDK to deploy a NestJS GraphQL API to AWS Lambda behind API Gateway. In order to fit with the constraints of Lambda's code size, my understanding is we'd need to use a bundler of some sort to do tree-shaking and minify the code. However, it seems like bundling is discouraged based on the final response in this thread: #1706. So I'm wondering if you have any guidance on how to deploy to Lambda. It seems Serverless Framework does... something... with their plugins to make it work, but I can't use Serverless Framework unfortunately.

Current behavior

Using the Webpack configuration provided in https://docs.nestjs.com/faq/serverless and running nest build --webpack with a basic GraphQL server yields:

ERROR in ./node_modules/@nestjs/graphql/dist/federation/graphql-gateway.module.js 78:108-134
Module not found: Error: Can't resolve '@apollo/gateway' in '/Users/NA/Desktop/dev/test-nest/node_modules/@nestjs/graphql/dist/federation'
 @ ./node_modules/@nestjs/graphql/dist/federation/index.js 9:21-56
 @ ./node_modules/@nestjs/graphql/dist/index.js 6:21-44
 @ ./node_modules/@nestjs/graphql/index.js 6:9-26
 @ ./src/app.module.ts 11:18-44
 @ ./src/main.ts 9:21-44

ERROR in ./node_modules/@nestjs/graphql/dist/graphql-ast.explorer.js 17:56-75
Module not found: Error: Can't resolve 'ts-morph' in '/Users/NA/Desktop/dev/test-nest/node_modules/@nestjs/graphql/dist'
 @ ./node_modules/@nestjs/graphql/dist/index.js 7:21-54
 @ ./node_modules/@nestjs/graphql/index.js 6:9-26
 @ ./src/app.module.ts 11:18-44
 @ ./src/main.ts 9:21-44

ERROR in ./node_modules/@nestjs/graphql/dist/graphql-schema.builder.js 78:135-180
Module not found: Error: Can't resolve '@apollo/federation/dist/directives' in '/Users/NA/Desktop/dev/test-nest/node_modules/@nestjs/graphql/dist'
 @ ./node_modules/@nestjs/graphql/dist/graphql.factory.js 14:33-68
 @ ./node_modules/@nestjs/graphql/dist/index.js 11:21-49
 @ ./node_modules/@nestjs/graphql/index.js 6:9-26
 @ ./src/app.module.ts 11:18-44
 @ ./src/main.ts 9:21-44

ERROR in ./node_modules/@nestjs/mapped-types/dist/type-helpers.utils.js 69:27-63
Module not found: Error: Can't resolve 'class-transformer/storage' in '/Users/NA/Desktop/dev/test-nest/node_modules/@nestjs/mapped-types/dist'
 @ ./node_modules/@nestjs/mapped-types/dist/index.js 19:27-58
 @ ./node_modules/@nestjs/mapped-types/index.js 6:9-26
 @ ./node_modules/@nestjs/graphql/dist/type-helpers/intersection-type.helper.js 4:23-54
 @ ./node_modules/@nestjs/graphql/dist/type-helpers/index.js 4:21-58
 @ ./node_modules/@nestjs/graphql/dist/index.js 20:21-46
 @ ./node_modules/@nestjs/graphql/index.js 6:9-26
 @ ./src/app.module.ts 11:18-44
 @ ./src/main.ts 9:21-44

ERROR in ./node_modules/fsevents/fsevents.node 1:0
Module parse failed: Unexpected character '�' (1:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
(Source code omitted for this binary file)
 @ ./node_modules/fsevents/fsevents.js 13:15-41
 @ ./node_modules/chokidar/lib/fsevents-handler.js 9:13-32
 @ ./node_modules/chokidar/index.js 15:24-57
 @ ./node_modules/@nestjs/graphql/dist/graphql-definitions.factory.js 8:17-36
 @ ./node_modules/@nestjs/graphql/dist/index.js 8:21-61
 @ ./node_modules/@nestjs/graphql/index.js 6:9-26
 @ ./src/app.module.ts 11:18-44
 @ ./src/main.ts 9:21-44

Trying to manually install these dependencies causes it to need more dependencies so surely that's not the solution.

Input Code

Copied Webpack config from NestJS website:

module.exports = (options, webpack) => {
  const lazyImports = [
    '@nestjs/microservices/microservices-module',
    '@nestjs/websockets/socket-module'
  ];

  return {
    ...options,
    externals: [],
    plugins: [
      ...options.plugins,
      new webpack.IgnorePlugin({
        checkResource(resource) {
          if (lazyImports.includes(resource)) {
            try {
              require.resolve(resource);
            } catch (err) {
              return true;
            }
          }
          return false;
        },
      }),
    ],
  };
};

Copied entry file from NestJS website:

import { NestFactory } from '@nestjs/core';
import serverlessExpress from '@vendia/serverless-express';
import { Callback, Context, Handler } from 'aws-lambda';
import { AppModule } from './app.module';

let server: Handler;

async function bootstrap(): Promise<Handler> {
  const app = await NestFactory.create(AppModule);
  await app.init();

  const expressApp = app.getHttpAdapter().getInstance();
  return serverlessExpress({ app: expressApp });
}

export const handler: Handler = async (
  event: any,
  context: Context,
  callback: Callback,
) => {
  server = server ?? (await bootstrap());
  return server(event, context, callback);
};

Expected behavior

I'm able to build the package with Webpack or package the code up in some other way to be able to deploy a GraphQL API to Lambda with CDK (rather than Serverless Framework).

Environment


Nest version: 8.0.0

 
For Tooling issues:
- Node version: 12.19.0
- Platform:  Mac
@tekkeon tekkeon added the needs triage This issue has not been looked into label Aug 3, 2021
@tekkeon tekkeon changed the title Can't bundle GraphQL API (for deployment to AWS Lambda with CDK/CloudFormation) Can't bundle serverless GraphQL API (for deployment to AWS Lambda with CDK/CloudFormation) Aug 3, 2021
@tekkeon tekkeon changed the title Can't bundle serverless GraphQL API (for deployment to AWS Lambda with CDK/CloudFormation) Can't deploy GraphQL to Lambda with CDK Aug 3, 2021
@kamilmysliwiec
Copy link
Member

Add @apollo/gateway, ts-morph etc. to the lazyImports array

@tekkeon
Copy link
Author

tekkeon commented Aug 3, 2021

So I did try doing this. The one that i was unsure of was the last error which was a bit different:

ERROR in ./node_modules/fsevents/fsevents.node 1:0
Module parse failed: Unexpected character '�' (1:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
(Source code omitted for this binary file)
 @ ./node_modules/fsevents/fsevents.js 13:15-41
 @ ./node_modules/chokidar/lib/fsevents-handler.js 9:13-32
 @ ./node_modules/chokidar/index.js 15:24-57
 @ ./node_modules/@nestjs/graphql/dist/graphql-definitions.factory.js 8:17-36
 @ ./node_modules/@nestjs/graphql/dist/index.js 8:21-61
 @ ./node_modules/@nestjs/graphql/index.js 6:9-26
 @ ./src/app.module.ts 11:18-44
 @ ./src/main.ts 9:21-44

I went ahead and added it to the lazyImports array to get this as my config:

module.exports = (options, webpack) => {
  const lazyImports = [
    '@nestjs/microservices/microservices-module',
    '@nestjs/websockets/socket-module',
    '@apollo/federation',
    '@apollo/gateway',
    'ts-morph',
    'class-transformer/storage',
    '@apollo/federation/dist/directives',
    './fsevents.node'
  ];

  return {
    ...options,
    externals: [],
    plugins: [
      ...options.plugins,
      new webpack.IgnorePlugin({
        checkResource(resource) {
          if (lazyImports.includes(resource)) {
            try {
              require.resolve(resource);
            } catch (err) {
              return true;
            }
          }
          return false;
        },
      }),
    ],
  };
};

I'm now getting a weird error from Lambda claiming my main.handler does not exist or was not exported, even though i verified that it does exist in the bundled file. I guess my main question at this point is, even if I happen to get this working, is this even maintainable/practical? And is there any better way that you know of?

@kamilmysliwiec
Copy link
Member

I guess my main question at this point is, even if I happen to get this working, is this even maintainable/practical

If you want to bundle your code, this is the best way to accomplish it.

I went ahead and added it to the lazyImports array to get this as my config:

fsevents shouldn't be defined as a lazy import.

Lambda claiming my main.handler does not exist or was not exported

Did you set the libraryTarget?

image

@tekkeon
Copy link
Author

tekkeon commented Aug 3, 2021

I hadn't noticed that section, whoops. I added that configuration and added the fsevents to the externals instead to get:

module.exports = (options, webpack) => {
  const lazyImports = [
    '@nestjs/microservices/microservices-module',
    '@nestjs/websockets/socket-module',
    '@apollo/federation',
    '@apollo/gateway',
    'ts-morph',
    'class-transformer/storage',
    '@apollo/federation/dist/directives'
  ];

  return {
    ...options,
    externals: {
      fsevents: "require('fsevents')"
    },
    plugins: [
      ...options.plugins,
      new webpack.IgnorePlugin({
        checkResource(resource) {
          if (lazyImports.includes(resource)) {
            try {
              require.resolve(resource);
            } catch (err) {
              return true;
            }
          }
          return false;
        },
      }),
    ],
    output: {
      libraryTarget: 'commonjs2'
    }
  };
};

This builds and Lambda is able to run it, but it immediately fails after running NestFactory and loading the modules, saying:

"stack": [
        "Error: Cannot use GraphQLSchema \"{ __validationErrors: [], description: undefined, extensions: undefined, astNode: undefined, extensionASTNodes: [], _queryType: Query, _mutationType: Mutation, _subscriptionType: undefined, _directives: [@include, @skip, @deprecated, @specifiedBy], _typeMap: { Offer: Offer, String: String, Int: Int, Query: Query, Mutation: Mutation, CreateOfferInput: CreateOfferInput, UpdateOfferInput: UpdateOfferInput, Boolean: Boolean, __Schema: __Schema, __Type: __Type, __TypeKind: __TypeKind, __Field: __Field, __InputValue: __InputValue, __EnumValue: __EnumValue, __Directive: __Directive, __DirectiveLocation: __DirectiveLocation }, _subTypeMap: {}, _implementationsMap: {} }\" from another module or realm.",
        "",
        "Ensure that there is only one instance of \"graphql\" in the node_modules",
        "directory. If different versions of \"graphql\" are the dependencies of other",
        "relied on modules, use \"resolutions\" to ensure only one version is installed.",
        "",
        "https://yarnpkg.com/en/docs/selective-version-resolutions",
        "",
        "Duplicate \"graphql\" modules cannot be used at the same time since different",
        "versions may have different capabilities and behavior. The data from one",
        "version used in the function from another could produce confusing and",
        "spurious results.",
        "    at instanceOf (/var/task/main.js:101595:13)",
        "    at isSchema (/var/task/main.js:100833:34)",
        "    at assertSchema (/var/task/main.js:100837:8)",
        "    at validateSchema (/var/task/main.js:106444:28)",
        "    at assertValidSchema (/var/task/main.js:106468:16)",
        "    at assertValidExecutionArguments (/var/task/main.js:107413:35)",
        "    at executeImpl (/var/task/main.js:107361:3)",
        "    at Object.execute (/var/task/main.js:107323:63)",
        "    at Object.generateSchemaHash (/var/task/main.js:130507:32)",
        "    at ApolloServer.generateSchemaDerivedData (/var/task/main.js:111314:41)"
]

Running npm ls graphql gives me

npm ls graphql
[email protected] /Users/NA/Desktop/dev/test-nest
└── [email protected] 

So it doesn't appear to be installed multiple times. And I ran npm dedupe as some suggested and tried again, but got the same error.

@kamilmysliwiec
Copy link
Member

Check out these threads:
graphql/graphql-js#2801
apollographql/apollo-server#4637

Please, use our Discord channel (support) for further questions. We are using GitHub to track bugs, feature requests, and potential improvements.

@nestjs nestjs locked and limited conversation to collaborators Aug 4, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
needs triage This issue has not been looked into
Projects
None yet
Development

No branches or pull requests

2 participants