How to use puppeteer?

[srcnalt]

Hi, I'm trying to get puppeteer working on vercel, does anyone have experience getting it working?

I get the following error at the point browser is launched:
puppeteer failed: Error: Could not find browser revision 768783. Run "PUPPETEER_PRODUCT=firefox npm install" or "PUPPETEER_PRODUCT=firefox yarn install" to download a supported Firefox browser binary.

It should be running chromium, but the error suggests using firefox, which I could not find a way how to do.

[srcnalt]

I ended up using Playwright.

[vasco3]

@srcnalt do you use it for pdf generation? I am using puppeteer but for pdfs it splits the pages without respecting page breaks so it slices text and images

[vasco3]

@psikomanjak ah ok. I use the pro plan

[ycjcl868]

browserType.launch: Failed to launch chromium because executable doesn't exist at /vercel/.cache/ms-playwright/chromium-878941/chrome-linux/chrome

[winston0410]

@ycjcl868 I have the same error. How did you solved that?

[ycjcl868]

I gave up playwright and puppeteer.

[diegoguevara]

Hi, this worked for me:

first:

npm add chrome-aws-lambda

then:

// import * as puppeteer from 'puppeteer';
import chromium from 'chrome-aws-lambda';

// const browser = await puppeteer.launch( { args: ['--no-sandbox'] } );

const browser = await chromium.puppeteer.launch({
    args: [...chromium.args, "--hide-scrollbars", "--disable-web-security"],
    defaultViewport: chromium.defaultViewport,
    executablePath: await chromium.executablePath,
    headless: true,
    ignoreHTTPSErrors: true,
  })

[28development]

@ndom91 but the 50mb is not limited to the free accounts right? I mean if that the issue I have no problem to update my plan to pro 😆

[ndom91]

Yeah that limit is only for free accounts. If you have the budget for paid then you can basically do the same thing as in my code in the blog post, but keep the api route in the default location and continue to be able to use next dev

[28development]

@ndom91 do you mind having a chat on discord? I am currently confused about a few things? I am going to write you an email if that is ok

[amccloud]

Thanks for the article. The limit for lambda deployments are 50mb free or paid from my experience.

From my understand this is lambda limitation, even if you’re paying Vercel / AWS.

https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html

[amyegan]

That's right, it is a Lambda limitation. The 50 MB Serverless Function size limit is the same across all account types. But things like Serverless Function Memory and Serverless Function Execution Timeout do increase on Pro.

[munkychop]

@diegoguevara's answer was really helpful (thank you!), but the missing piece for me was that this won't work locally (at least with my basic express app set up), so I thought it didn't work at all, even though it did in fact work when published to Vercel.

My approach to get the best of both worlds (which I partially borrowed from a Japanese article 😄) was to detect whether or not AWS_LAMBDA_FUNCTION_VERSION is available and then load an appropriate version of puppeteer:

let chrome = {};
let puppeteer;

if (process.env.AWS_LAMBDA_FUNCTION_VERSION) {
  // running on the Vercel platform.
  chrome = require('chrome-aws-lambda');
  puppeteer = require('puppeteer-core');
} else {
  // running locally.
  puppeteer = require('puppeteer');
}

const getData = async (url) => {
  try {
    let browser = await puppeteer.launch({
      args: [...chrome.args, '--hide-scrollbars', '--disable-web-security'],
      defaultViewport: chrome.defaultViewport,
      executablePath: await chrome.executablePath,
      headless: true,
      ignoreHTTPSErrors: true,
    });
  } catch (err) {
    console.error(err);
    return null;
  }
}

Note that, because of this, I installed puppeteer-core and chrome-aws-lambda as dependencies, but installed puppeteer as a dev dependency.

[28development]

Did you locally get the error that chrome was not installed aswell? I get the same error deploying this on vercel (chrome not installed). You did not add puppeteer in your packages right, only chrome-aws-lambda and puppeteer-core regarding this feature?

[munkychop]

I added puppeteer as a dev dependency, so that I could load it locally.

[stubbzilla8]

This worked wonderfully!

[mihhail-lapushkin]

This should work:
https://github.com/vercel/virtual-event-starter-kit/blob/main/lib/screenshot.ts

import chrome from 'chrome-aws-lambda';
import puppeteer from 'puppeteer-core';

export default async function screenshot(url: string) {
  const options = process.env.AWS_REGION
    ? {
        args: chrome.args,
        executablePath: await chrome.executablePath,
        headless: chrome.headless
      }
    : {
        args: [],
        executablePath:
          process.platform === 'win32'
            ? 'C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe'
            : process.platform === 'linux'
            ? '/usr/bin/google-chrome'
            : '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'
      };
  const browser = await puppeteer.launch(options);
  const page = await browser.newPage();
  await page.setViewport({ width: 2000, height: 1000 });
  await page.goto(url, { waitUntil: 'networkidle0' });
  return await page.screenshot({ type: 'png' });
}

The only thing you need to do, is to manually set either one of these environment variables to some value:
https://github.com/alixaxel/chrome-aws-lambda/blob/master/source/index.ts#L199

AWS_LAMBDA_FUNCTION_NAME
FUNCTION_NAME
FUNCTION_TARGET
FUNCTIONS_EMULATOR

For some reason, they are not set in Vercel.

h/t @leerob

[fgiuliani]

Thank you for the help @mihhail-lapushkin

[fgiuliani]

@mihhail-lapushkin I'm still having the issue. I can't find what's the problem. Do you have a minute to take a look at my repo, to see If everything is ok for you?

https://github.com/fgiuliani/automatic-social-images

Again, thank you very much

[mihhail-lapushkin]

I think the difference is that I am generating images at build time and you are on every request. The environments for building and running might be different.

I am not very familiar with Next.js, so can't help you there.

[jgalianoz]

@mihhail-lapushkin One question, where should the variables be set? Since AWS_LAMBDA_FUNCTION_NAME is a reserved variable in vercel.

[mihhail-lapushkin]

I set it in my package.json > scripts like so:

"scripts": {
  "build": "NODE_ENV=production AWS_LAMBDA_FUNCTION_NAME=true <your build command>"
}

[bigsee]

I also had an issue with this. This works locally for me absolutely fine using vercel dev. However, when I deploy this to Vercel, I get the following error:

A plugin listed 'puppeteer-extra-plugin-stealth/evasions/chrome.app' as dependency, which is currently missing.
Please install it:  yarn add puppeteer-extra-plugin-stealth          
Note: You don't need to require the plugin yourself, unless you want to modify it's default settings.

This dependency is in my package.json. Any ideas about how to solve this error?

[aditodkar]

@bigsee Did you manage to resolve it? I am facing same issue.

[ndom91]

If anyone comes across this thread, just wanted to point to this closed issue: vercel/vercel#4739

Basically theres still, at the time of writing, a vercel deployment issue - something about sharing code between the api routes and other files which cause the size to balloon. If you move your /pages/api/.. directory to simply /api then when vercel builds your project, it will come in under the 50mb limit for some reason haha

[nk1tz]

@ndom91 This doesn't seem to work.
Vercel doesn't deploy the /api directory the way it does with /pages/api.

I'm using [email protected].

Could you give more detail about your setup where you achieved this hack?

[ndom91]

Hey so as far as i remember thats all I did. You gotta make sure you use playwright-core and chrome-aws-lambda to keep the size down.

You can check out the repo in which I did this here - https://github.com/ndom91/home2021.

Also using next@11+.

[Julian-quintero]

Im getting this error after trying every single solution in this thread, any help please.

error Error: spawn ETXTBSY at ChildProcess.spawn (internal/child_process.js:403:11) at spawn (child_process.js:667:9) at Object.spawnWithSignal (child_process.js:871:17) at BrowserRunner.start (/var/task/node_modules/puppeteer-core/lib/cjs/puppeteer/node/BrowserRunner.js:80:34) at ChromeLauncher.launch (/var/task/node_modules/puppeteer-core/lib/cjs/puppeteer/node/Launcher.js:94:16) at async funt (/var/task/index.js:33:21) { errno: -26, code: 'ETXTBSY', syscall: 'spawn' }

[vasco3]

I had to put it on its own verbal project so that it won't run out of memory space.

[CWarth93]

I tried everything mentioned here but still cant get it working.

My dependencies:

"dependencies": {
    "chrome-aws-lambda": "^10.1.0",
    "dotenv": "^8.2.0",
    "moment": "^2.29.1",
    "next": "10.0.0",
    "prettier": "^2.0.2",
    "puppeteer-core": "^13.5.1",
    "react": "^16.13.1",
    "react-youtube": "^7.14.0",
    "riteway": "^6.1.2"
  },
  "devDependencies": {
    "puppeteer": "^13.5.1"
  }

How i try to start the browser:

import chromium from 'chrome-aws-lambda';

const getBrowser = async () => {
   try { 
   	const executablePath = await chromium.executablePath;
   	if (executablePath === null) {
   		// running locally
   		const puppeteer = require('puppeteer');
   		return puppeteer.launch({
   			headless: true,
   			ignoreHTTPSErrors: true
   		});
   	} else {
   		// running on vercel
   		return chromium.puppeteer.launch({
   			args: chromium.args,
   			executablePath,
   			headless: chromium.headless,
   			ignoreHTTPSErrors: true
   		});
   	}
   } catch (e) {
   	console.log(e);
   }
};

Locally everything works fine.

But on vercel i get:

{
        "level": "error",
        "message": "Error: Could not find expected browser (chrome) locally. Run `npm install` to download the correct Chromium revision (970485).\n    at ChromeLauncher.launch (/var/task/node_modules/puppeteer/lib/cjs/puppeteer/node/Launcher.js:105:23)\n    at async getGroupPostsByKeywords (/var/task/.next/serverless/pages/api/[endpoint].js:153:15)\n    at async handler (/var/task/.next/serverless/pages/api/[endpoint].js:213:10)\n    at async handleRequest (/var/task/.next/serverless/pages/api/[endpoint].js:236:16)\n    at async api (/var/task/.next/serverless/pages/api/[endpoint].js:263:20)\n    at async apiResolver (/var/task/node_modules/next/dist/next-server/server/api-utils.js:8:1)\n    at async module.exports.Gb4g.__webpack_exports__.default (/var/task/.next/serverless/pages/api/[endpoint].js:617:11)",
        "timestamp": "2022-03-31T11:41:01.329Z"
      }

Does anyone know whats going wrong here?
Do i have to set additional env variables on vercel?

[ndom91]

Looks like either the chrome-aws-lambda install or the executablePath thereof isn't resolving corrently or getting picked up correctly. Or maybe [email protected] is expecting a different version compared to what [email protected] is delivering?

[CWarth93]

after having a look at the supported versions i updated to

"dependencies": {
    "chrome-aws-lambda": "10.1.0",
    "dotenv": "^8.2.0",
    "moment": "^2.29.1",
    "next": "10.0.0",
    "prettier": "^2.0.2",
    "puppeteer-core": "10.1.0",
    "react": "^16.13.1",
    "react-youtube": "^7.14.0",
    "riteway": "^6.1.2"
  },
  "devDependencies": {
    "puppeteer": "10.1.0"
  }

now i get

"Error: Could not find expected browser (chrome) locally. Runnpm install to download the correct Chromium revision (884014).\n at ChromeLauncher.launch (/var/task/node_modules/puppeteer/lib/cjs/puppeteer/node/Launcher.js:82:27)\n at async getGroupPostsByKeywords (/var/task/.next/serverless/pages/api/[endpoint].js:153:15)\n at async handler (/var/task/.next/serverless/pages/api/[endpoint].js:213:10)\n at async handleRequest (/var/task/.next/serverless/pages/api/[endpoint].js:236:16)\n at async api (/var/task/.next/serverless/pages/api/[endpoint].js:263:20)\n at async apiResolver (/var/task/node_modules/next/dist/next-server/server/api-utils.js:8:1)\n at async module.exports.ndtY.__webpack_exports__.default (/var/task/.next/serverless/pages/api/[endpoint].js:674:11)"

884014 is the correct version according to https://www.npmjs.com/package/chrome-aws-lambda

[vasco3]

It works if you only use puppeteer + chrome-aws-lambda in that server.
As long as you add more stuff like next.js it won't fit on the instance

[CWarth93]

does this mean we cant run it as a serverless function in our api?

i can put it to another build like here https://github.com/whitep4nth3r/puppeteer-demo but i would at least need a cors package to access

[vasco3]

seems like it because it is too big so it doesn't fit with any other big dependency like Next.
so I deploy it in a static project with a api/pdf route and that works as a server less function

the drawback is that I need to call it using the full url path from my nextJS app https://blalbla.com/api/pdf instead of just /api/pdf

My guess would be that maybe there is a setting in vercel to increase the memory limit for server less functions but I haven't been successful on that.

so if you just want to get going, I would just put puppeteer on its own instance

[CWarth93]

how do you set cors policies in this new static project to access the api? (without installing new packages)

[vasco3]

I don't set it because I open it with window.open

[boyphongsakorn]

id=24","[pid=24][err] /tmp/chromium: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory"

um.... not work for me by using puppeteer or Playwright.
how to install libnss or fix this error?

[ndom91]

Was getting this error too with node16, looks like you have to bump it back down to node14 in the main project settings.

[boyphongsakorn]

Was getting this error too with node16, looks like you have to bump it back down to node14 in the main project settings.

wait what version of chrome-aws-lambda and puppeteer-core are you using?

i use 2.1.1 but only work in node12

[vasco3]

I think that puppeteer doesn't work in node v16

[ndom91]

Also oddly enough, I see this output about my /api/image function being too big (50mb+) in the build logs.

However it still seems to work in that deployment 🤔

This is with the latest puppeteer-core and chrome-aws-lambda btw.

[jonasgroendahl]

Got it working using older versions that were smaller in size

"chrome-aws-lambda": "6.0.0"
"puppeteer-core": "6.0.0"

Not a great long term solution but works :)

[ndom91]

I probably shouldn't have posted this publicly 😅, mine started to fail to build now after a few days. Still ~50.2mb compressed

[ndom91]

Little tip: I downgraded chrome-aws-lambda from 10.1.0 (Chrome 92) to 9.1.0 (Chrome 91) and that shaved a few mb off the final api route's size. Allowing me to deploy again.

[vasco3]

so with a leaner chrome-aws-lambda it should work. Or if we could expand the size limit of the lambda

[ndom91]

Right. The lambda size is enforced upon Vercel by AWS though unfortunately (See: https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html), so theres not much we can do there..

[egroeg92]

I'm getting this issue - Vercel is unable to find the path to the browser. Does anyone know how to fix this??

c3487a7b-7bfe-435e-8c78-6723f2da8a39	ERROR	Unhandled Promise Rejection 
{"errorType":"Runtime.UnhandledPromiseRejection","errorMessage":"Error: Cannot find module '/var/task/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Browser'","reason":
{"errorType":"Error","errorMessage":"Cannot find module '/var/task/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Browser'","code":"MODULE_NOT_FOUND","path":"/var/task/node_modules/puppeteer-core/package.json","stack":["Error: Cannot find module '/var/task/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Browser'","    at createEsmNotFoundErr (node:internal/modules/cjs/loader:960:15)","    at finalizeEsmResolution (node:internal/modules/cjs/loader:953:15)","    at resolveExports (node:internal/modules/cjs/loader:482:14)","    at Function.Module._findPath (node:internal/modules/cjs/loader:522:31)","    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:27)","    at Function.Module._load (node:internal/modules/cjs/loader:778:27)","    at Module.require (node:internal/modules/cjs/loader:1005:19)","    at require (node:internal/modules/cjs/helpers:102:18)","    at Object.<anonymous> (/var/task/node_modules/chrome-aws-lambda/source/puppeteer/lib/Browser.ts:9:11)","    at Module._compile (node:internal/modules/cjs/loader:1105:14)"]},"promise":{},"stack":["Runtime.UnhandledPromiseRejection: Error: Cannot find module '/var/task/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Browser'","    at process.<anonymous> (file:///var/runtime/index.mjs:775:15)","    at process.emit (node:events:539:35)","    at process.emit (/var/task/___vc/__sourcemap_support.js:587:21)","    at emit (node:internal/process/promises:140:20)","    at processPromiseRejections (node:internal/process/promises:274:27)","    at processTicksAndRejections (node:internal/process/task_queues:97:32)"]}
Unknown application error occurred

[ndom91]

Unfortuntaely I'm running into an issue in my project atm where I'm over 50mb on the API route, so I can't deploy and log out my executablePath to double check.

[vasco3]

I use a vercel project exclusively for puppeteer + chrome-aws-lamda and I'm also stuck in Nodejs v14

[ndom91]

@egroeg92 so I ended up going the route of just making a separate Next.js app / Vercel project just for the "screenshot API" portion of my application :/

Yes, kind of a pain in the ass, but at least I have that separated off now into its own thing and can add as much as I want to my normal API routes.

Anyway, on that new project, I initially had the same error as you, couldn't find the chromium binary at /tmp/chromium. However, as soon as I updated the project settings down to use node 14 it worked like a charm 👍

Using [email protected] and [email protected] this time around.

[ryoid]

Managed to get it working by downgrading to node 14 after getting same error as @ndom91

• [email protected]
• [email protected]

I set the node version in my package.json instead of project settings

"engines": {
    "node": "14.x"
 }

[egroeg92]

I ended up switching to EC2 and the code worked :-( . Didn't try having the headless on a seperate vercel instance though. Thanks for your help!

[mityabeldii]

This is an official solution from Vercel and it works fine for me: https://github.com/vercel/virtual-event-starter-kit/blob/main/lib/screenshot.ts

Only the latest versions of chrome-aws-lambda and puppeteer-core were installed.
It's also a workaround for the 50mb limitation.

[ndom91]

Yeah that might work if yuo don't have any other larger API routes. They all get bundled into one lambda apparently (even though they're supposed to get split after 50mb+, but that still didn't seem to help me).

So once yuo start adding dependencies to your other API routes, that'll burst over the limit.

[leerob]

Hey! If anyone is still trying to use Chromium/Puppeteer for taking screenshots inside Serverless Functions, could you send me an email [email protected] with more information about what you're trying to build? Thank you!

[jgalianoz]

I am trying to use puppeteer but my deployment fails. I am using remix.

[Atharv-Chaudhari]

const express = require('express'),
    app = express();
const chromium = require('chrome-aws-lambda');
const playwright = require('playwright-core');

app.get("/", async (request, response) => {
  try {
    const browser = await playwright.chromium.launch({
    args: [...chromium.args, "--hide-scrollbars", "--disable-web-security"],
    defaultViewport: chromium.defaultViewport,
    executablePath: await chromium.executablePath,
    headless: true,
    ignoreHTTPSErrors: true,
  });

Getting this error for playwright on vercel 👀

[Atharv-Chaudhari]

I ended up this using following code..

const browser = await chromium.puppeteer.launch({
args: [...chromium.args, "--hide-scrollbars", "--disable-web-security"],
defaultViewport: chromium.defaultViewport,
executablePath: await chromium.executablePath,
headless: true,
ignoreHTTPSErrors: true,
ignoreDefaultArgs: ['--disable-extensions'],
});

[ndom91]

Are you using node16 in that Vercel project? Seems node14 might be required for puppeteer/playwright

[Atharv-Chaudhari]

yes node 14 works

[ivenuss]

Any updates or possible solutions?

[fahadamjadckj]

Hi, after spending a day and a half I have finally got a working version with "chrome-aws-lambda": "^8.0.2" and "puppeteer-core": "^8.0.0",
you also need to set engines to be node version to 14 like so in package.json.
{
...
"engines": {
"node": "^14"
}
...
}

[ndom91]

I went the route of making my screenshot api route into its own Next.js / Vercel project. That way I had the full 50mb to play with there, unencumbered by any other potential api route(s) and their dependencies.

However, that also got too large with the latest playwright-core and chrome-aws-lambda versions.

So I moved that single api route next.js project to a simple fastify server with 1 endpoint as well. Other than the scaffolding for fastify, I was able to almost 1:1 copy the contents of my playwright next.js api route over.

This also came with the advantage of better performance as well. Went from generating screenshots of URLs in 7-10s (meaning it would completely fail and time out every once in a while as well) with next.js to 2-5s with fastify

[fahadamjadckj]

Where did you host the fastify server? Aws, google cloud?

[ndom91]

Where did you host the fastify server? Aws, google cloud?

In the same Vercel project as previously hosted the next.js app actually haha. The migration was super easy.

You can check out the fastify project at https://github.com/ndom91/briefkasten-screenshot

[fahadamjadckj]

Oh ok ok, I am also facing the same timeout issue becauase data fetching took so long, I will look into it.

[mehedi-sharif]

Hello Guys, I am getting an error on vercel:

Error: Could not find expected browser (chrome) locally. Run `npm install` to download the correct Chromium revision (901912).
--
09:12:17.130 | at ChromeLauncher.launch (/vercel/path0/node_modules/puppeteer-core/lib/cjs/puppeteer/node/Launcher.js:88:27)
09:12:17.130 | at async screenshot (file:///vercel/path0/src/vercel.js:13:19)
09:12:17.138 | Error: Command "npm run build" exited with 1

and this is my code:

import chrome from "chrome-aws-lambda";
import puppeteer from "puppeteer-core";

async function screenshot() {
  const options = {
    args: [...chrome.args, "--hide-scrollbars", "--disable-web-security"],
    defaultViewport: chrome.defaultViewport,
    executablePath: await chrome.executablePath,
    headless: true,
    ignoreHTTPSErrors: true,
  };
  console.log("trying to execute");
  const browser = await puppeteer.launch(options);
  console.log("browser launched");
  const page = await browser.newPage();
  await page.setViewport({ width: 2000, height: 1000 });
  await page.goto("https://demo.themefisher.com/bexer/", {
    waitUntil: "networkidle0",
  });
  await page.screenshot({ path: "bexer.png" });

  await browser.close();
}
screenshot();

can anyone help with that?

[fahadamjadckj]

Use node version 14 and make sure your aws lamda version is compatiable with puppeteer version

[mehedi-sharif]

That doesn't solve the issue. thanks

[kettanaito]

Hi, folks. If you're still struggling to make this work, I wrote a guide about how I achieved Chromium on Vercel serverless today.

• Chromium on Vercel (serverless) in 2022

It lists the exact steps you need to take, emphasizes the right dependency versions to use (yes, those do matter), and includes solutions to the common problems I (and many of you here) faced while trying to make this work, like The Serverless Function "XYZ" is XX.YYmb which exceeds the maximum size limit of 50mb or Error while loading shared libraries: libnss3.so: cannot open shared object file.

Build something awesome.