Error: Cannot read property 'replace' of undefined

[regexj]

Operating System: Linux
Node version: 14.19.2
google-play-scraper version: 8.1.0

Description:

We are using this scraper on a cron once an hour as part of a version control system. Occasionally the below Http Exception is thrown and we are emailed about the failure.

Example code:

This is the code we're running.

private static async getLatestAppVersionAndroid (): Promise<string> {
  const scrape = await gplay.app({
    appId: config.appIds.android
  });
  return scrape.version;
}

This is the code in the package that appears to be throwing the error:

function descriptionText (description) {
  // preserve the line breaks when converting to text
  const html = cheerio.load('<div>' + description.replace(/<br>/g, '\r\n') + '</div>');
  return cheerio.text(html('div'));
}

It's as if the scraper fails every now and then and the description field is therefore undefined?

Error message:

"stack": [
    "TypeError: Cannot read property 'replace' of undefined",
    "    at Object.descriptionText [as fun] (/var/www/node2/ms-version-control/node_modules/google-play-scraper/lib/mappers/details.js:125:51)",
    "    at /var/www/node2/ms-version-control/node_modules/google-play-scraper/lib/utils/scriptData.js:52:19",
    "    at XWrap.f (/var/www/node2/ms-version-control/node_modules/ramda/dist/ramda.js:5955:28)",
    "    at XWrap.@@transducer/step (/var/www/node2/ms-version-control/node_modules/ramda/dist/ramda.js:392:25)",
    "    at _arrayReduce (/var/www/node2/ms-version-control/node_modules/ramda/dist/ramda.js:5137:46)",
    "    at _reduce (/var/www/node2/ms-version-control/node_modules/ramda/dist/ramda.js:5167:24)",
    "    at map (/var/www/node2/ms-version-control/node_modules/ramda/dist/ramda.js:5954:20)",
    "    at /var/www/node2/ms-version-control/node_modules/ramda/dist/ramda.js:586:23",
    "    at Object.f2 [as map] (/var/www/node2/ms-version-control/node_modules/ramda/dist/ramda.js:473:22)",
    "    at extractFields (/var/www/node2/ms-version-control/node_modules/google-play-scraper/lib/utils/scriptData.js:40:14)",
    "    at runMicrotasks (<anonymous>)",
    etc. drilling into our files
]

[ChayaGel]

Is something changed in api?
I start get the same error in my project, 2 days ago.

"Cannot read property 'replace' of undefined"

It is not clear-
Rarely manages to run
And most of the time fails

is there new version of google play scraper to fix this?

[dave-filion]

Same thing happening to me, I'm guessing something changed with the descriptionText field?

[VitaliyRDev]

Same thing happening to me, I'm guessing something changed with the descriptionText field?

Yeah, i guess it's time to change mappings

[Davies-Owen]

@facundoolano @icarcal Anything you can do about this? I tried taking a look at the mappings myself but I can't make any sense of it.

[CyrilCermak]

I tried to work around the errors with the mapping all way to the end but then ended up with an empty object. So it's not descriptionText only.

This is the main reason why it's failing. I think now the mapping must be completely re-done for the new UI of the Play Store.

[pvomhoff]

Same thing happening to me. I was able to fix the mapping by completely redo it.
Unfortunaly Google doesnt provide all properties as before.

I created a PR with my changes.
Hope this helps.

[Davies-Owen]

@pvomhoff Thanks! Hopefully that will get merged soon. Out of curiosity, how do you generate the mappings? Is there a special tool you use to do it?

[pvomhoff]

So i fetched this big weird json from Play Store for the old version and the new version.
Then i basically searched the value that was outputted by the scraper in the new json of the Play Store and copied the path. 🙂

My biggest problem was actually that Google didnt fully roll out the changes yet so it kept breaking depending which version the scraper got.
Therefore i added a detection if it's the new design or the old design. The scraper automatically switches then between the different mappings.

[adtimizer]

@pvomhoff
I would like to help with the mapping of search and lists but would need some advice from you.
I was able to get the json for search results (ds:4), my question is, are you using a special editor to count the array elements? i lost counting like 1000000 times while trying to figure out the location of the element within the nested arrays.

[ChayaGel]

I had a strange problem:
try search "waze" - get []
"wazee" (Intentional spelling mistake)- works.

I found the reason and want a solution.

in the new UI, when search term is equals exactly to name of an app, it show this specific app big before all results,
then our mapping crashes.
(open https://play.google.com/store/search?q=waze&c=apps to understand me)

that why, search "word", "Pepper – Mobile Banking" or any other exact names of apps, will not get results in scraper.

can someone try fix it?

[ChayaGel]

Small accuracy:
It's probably not a matter of an exact app name but of a too good fit for a specific app,
For any reason.

(Then Google shows it big and we miss out on the mapping)

In short, our mapping should also be tailored to the option of having one primary outcome first and foremost

[xshagg]

@pvomhoff I would like to help with the mapping of search and lists...

I wrote a pair of scripts which helped me to build my mappings
json-inspect.zip

[lau1944]

Any update? It's been a week.

[JDSeiler]

Any more news or potential workarounds?

[sridhard]

any updates?

[facundoolano]

can you check if this still happens with the latest version?

[regexj]

Updated to v9.0.0. Appears to be working! Have rolled out to production immediately. Thank you

[KaKi87]

Now it's 'split' of undefined : https://git.kaki87.net/playsearch.kaki87.net/v2-api/issues/2

[ChangJoo-Park]

It still happen with latest version 9.0.2

[campiador]

Works great on my end, thanks! I tried the search functionality and it seems to be limited to 30 results. Is that intentional? Bests, Behnam

…

On Jun 23, 2022, at 12:31 PM, Facundo Olano ***@***.***> wrote: can you check if this still happens with the latest version? — Reply to this email directly, view it on GitHub <#539 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKS7ZTL6AADU2WVAXQMBPLVQSGNNANCNFSM5WE5IUMQ>. You are receiving this because you are subscribed to this thread.