Issue when hosted on vercel.

[Conqxeror]

The package is working totally fine when I test is locally, but as soon as I host it on the vercel it is not working. Is it the problem with many people or only I am facing this sort of issue after hosting it on vercel?

I also tried puppeteerArgs: ["--no-sandbox", "--disable-setuid-sandbox"]

Error:

SCRAPING NEWS FROM: https://news.google.com/search?hl=en-US&gl=US&ceid=US:en&q=Cricket News when:12h
Error: Could not find expected browser (chrome) locally. Run `npm install` to download the correct Chromium revision (1002410).

Hosted Site:
https://crickethub-pro.vercel.app/

Api:
https://github.com/Conqxeror/cric-api

App:
https://github.com/Conqxeror/crickethub.pro

[anthonyfranc]

Are you on vercel free? If so, they limit the package to 50 mb. For example you can’t use puppeteer, instead you can use puppeteer-core package which is only 16MB.

But that error looks like it can’t find the package? Did you run npm install

[Conqxeror]

Are you on vercel free? If so, they limit the package to 50 mb. For example you can’t use puppeteer, instead you can use puppeteer-core package which is only 16MB.

Thank you for such a quick response, but can I know how can I do it in the package google-news-scrapper it has all that pre-installed?

Yes the vercel automatically does the npm install isn't it?

[anthonyfranc]

Are you on vercel free? If so, they limit the package to 50 mb. For example you can’t use puppeteer, instead you can use puppeteer-core package which is only 16MB.

Thank you for such a quick response, but can I know how can I do it in the package google-news-scrapper it has all that pre-installed?

Yes the vercel automatically does the npm install isn't it?

I’ll try to see if I can get it to work on vercel. I’ll report back shortly.

[Conqxeror]

Are you on vercel free? If so, they limit the package to 50 mb. For example you can’t use puppeteer, instead you can use puppeteer-core package which is only 16MB.

Thank you for such a quick response, but can I know how can I do it in the package google-news-scrapper it has all that pre-installed?

Yes the vercel automatically does the npm install isn't it?

I’ll try to see if I can get it to work on vercel. I’ll report back shortly.

That's great!! Thank you so much. Respect++

[anthonyfranc]

Are you on vercel free? If so, they limit the package to 50 mb. For example you can’t use puppeteer, instead you can use puppeteer-core package which is only 16MB.

Thank you for such a quick response, but can I know how can I do it in the package google-news-scrapper it has all that pre-installed?
Yes the vercel automatically does the npm install isn't it?

I’ll try to see if I can get it to work on vercel. I’ll report back shortly.

That's great!! Thank you so much. Respect++

Okay so from testing the issue is the package requires pupeteer and within that it also requires chromium which isnt supported by vercel, because the package size is over 50 MB. The package pupeteer-core which seperates chromum. The google-news-scraper will need to make some changes to support serverless services like Vercel.

[Conqxeror]

The google-news-scraper will need to make some changes to support serverless services like Vercel.

Ok, so it means that we have to wait for the changes. Meanwhile there is no alternative for it?

Thank you so much by the way, that was really helpful.

[anthonyfranc]

Unfortunately, I don’t believe so because the google-news-scraper package uses puppeteer and not puppeteer-core Sent from Gmail Mobile

…

On Sat, Feb 17, 2024 at 10:40 AM Wali Mohammad Kadri < ***@***.***> wrote: The google-news-scraper will need to make some changes to support serverless services like Vercel. Ok, so it means that we have to wait for the changes. Meanwhile there is no alternative for it? — Reply to this email directly, view it on GitHub <#41 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AELSWGRMULC7OOEIIJNG4Q3YUDFQDAVCNFSM6AAAAABDNJAOHWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJQGIZTSMBQHA> . You are receiving this because you commented.Message ID: ***@***.***>

[lewisdonovan]

Hi all, thanks for looking into this. I'll look at what we'd need in order to either switch to puppeteer-core, or add a config option and conditional import.

I'll leave the issue open until I've got an update.

[lewisdonovan]

@Conqxeror I've spent today trying to solve this issue, I managed to get the bundle size down to below the 50MB limit that the Vercel free tier has.

However, the only way to get the bundle size below 50MB was to remove Chromium from the bundle, then add a remote executable path to install it from a CDN on the server. This installation has to take place in the Vercel function, which has a max timeout limit of 10 seconds on the free tier. This isn't long enough for the executable to install, so any call to google-news-scraper just times out.

You can see my code here. Feel free to fork the other repo and try to get it working, if you do I'll be happy to review a PR and have a look at if it's feasible to add to main, but unfortunately I'm going to have to leave this for now.