The `recommendedVideos` function always returns empty result #92

iced-queen · 2023-10-20T20:36:58Z

For some reason, with or without passing a country, it always returns nothing. ({ data: [], paging: { current: 1, maxPage: 1, isEnd: true } }).
I'm guessing PH made some changes, cause it used to work. I think this started occuring like 1-2 weeks ago, not too sure.

The text was updated successfully, but these errors were encountered:

pionxzh · 2023-10-21T05:44:01Z

Can you provide a reproduction code? I just tested and it works ok. Did you see any error messages?

iced-queen · 2023-10-23T21:35:01Z

I'm on the latest version by the way.

await pornhub.recommendedVideos(countryObj).then(res => {
     console.log(res)
     result = res.data[Object.keys(res.data)[Math.floor(Math.random() * Object.keys(res.data).length)]]
})

countryObj was { country: 'uk' }. Even without the countryObj I get the same empty result.

I don't see any error messages.

pionxzh · 2023-10-24T15:33:00Z

Btw, recommendedVideos does not support country parameter, but it's not an issue~
Can you try to enable the debug log with set DEBUG=* & node ./src/index.js, test it locally, and provide the full log?

iced-queen · 2023-10-24T16:19:48Z

It doesn't? Which function is it that does then (if any)? Is it possible for you to add support for this?

Log from startup:

  REQUEST [Header] Set: Host=www.pornhub.com +0ms
  REQUEST [Header] Set: Origin=https://www.pornhub.com +8ms
  REQUEST [Header] Set: Referer=https://www.pornhub.com/ +8ms
  REQUEST [Header] Set: User-Agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36 +6ms
  REQUEST [Cookie] Set: platform=pc +7ms
  REQUEST [Cookie] Set: accessAgeDisclaimerPH=1 +9ms
  REQUEST [Cookie] Set: accessAgeDisclaimerUK=1 +5ms
  REQUEST [Cookie] Set: accessPH=1 +10ms
  REQUEST [Cookie] Set: age_verified=1 +6ms
  REQUEST [Cookie] Set: atatusScript=hide +5ms
  REQUEST [Cookie] Set: cookiesBannerSeen=1 +9ms
  REQUEST [Cookie] Set: hasVisited=1 +3ms

Log when using the function:

  REQUEST [ RQST ] GET https://www.pornhub.com/recommended +2m
  REQUEST [ RESP ] GET https://www.pornhub.com/recommended 200 OK +81ms
  REQUEST [Cookie] Received Set-Cookie: ss=824159885142554267; expires=Wed, 23-Oct-2024 16:17:59 GMT; Max-Age=31536000; path=/; domain=pornhub.com; secure +1ms
  REQUEST [Cookie] Received Set-Cookie: fg_0d2ec4cbd943df07ec161982a603817e=66256.100000; expires=Thu, 23-Nov-2023 16:17:59 GMT; Max-Age=2592000; path=/; domain=pornhub.com; secure +9ms
  REQUEST [Cookie] Received Set-Cookie: hasVisited=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; domain=www.pornhub.com; secure +4ms
  REQUEST [Cookie] Received Set-Cookie: hasVisited=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; secure +3ms
  REQUEST [Cookie] Received Set-Cookie: hasVisited=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; domain=pornhub.com; secure +2ms
  REQUEST [Cookie] Received Set-Cookie: hasVisited=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; secure +3ms
  REQUEST [Cookie] Received Set-Cookie: __s=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; domain=www.pornhub.com; secure +3ms
  REQUEST [Cookie] Received Set-Cookie: __s=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; secure +3ms
  REQUEST [Cookie] Received Set-Cookie: __s=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; domain=pornhub.com; secure +8ms
  REQUEST [Cookie] Received Set-Cookie: __s=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; secure +5ms
  REQUEST [Cookie] Received Set-Cookie: __l=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; domain=www.pornhub.com; secure +3ms
  REQUEST [Cookie] Received Set-Cookie: __l=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; secure +7ms
  REQUEST [Cookie] Received Set-Cookie: __l=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; domain=pornhub.com; secure +4ms
  REQUEST [Cookie] Received Set-Cookie: __l=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; secure +6ms
  REQUEST [Cookie] Received Set-Cookie: fg_9951ce1ac4434b4ac312a1334fa77d82=28113.100000; expires=Thu, 23-Nov-2023 16:17:59 GMT; Max-Age=2592000; path=/; domain=pornhub.com; secure +7ms
  REQUEST [Cookie] Received Set-Cookie: __s=6537EE37-42FE722901BB170122-29F28C0; Secure; Samesite=None +4ms
  REQUEST [Cookie] Received Set-Cookie: __l=6537EE37-42FE722901BB170122-29F28C0; Secure; Samesite=None; Max-Age=31556926 +4ms

And this gave an empty result.

pionxzh · 2023-10-24T17:10:05Z

It's very weird that you received so many deleted cookies.

pionxzh · 2023-10-24T17:19:15Z

Only videoList support country, recommendedVideos page don't even have options for you to choose country 😄

pionxzh · 2023-10-24T17:44:54Z

Can you try to call .video() first and see if recommendedVideos works after that. Note: you need to use the same instance.
I think the default cookie set is no longer valid in some regions (EU).

iced-queen · 2023-10-24T17:52:44Z

Can you try to call .video() first and see if recommendedVideos works after that. Note: you need to use the same instance. I think the default cookie set is no longer valid in some regions (EU).

That still doesn't work. Using the same instance. My server is located in Ireland so that's where the calls are coming from (I guess)

iced-queen · 2023-10-24T17:54:32Z

Only videoList support country, recommendedVideos page don't even have options for you to choose country 😄

Ohh, it's the Hottest section that does. Is there a function for this already or?

pionxzh · 2023-10-24T17:57:30Z

Ohh, it's the Hottest section that does. Is there a function for this already or?

const result = await pornhub.videoList({
    order: 'Hottest',
    country: 'Ireland',
})

iced-queen · 2023-10-24T17:58:51Z

Ohh, it's the Hottest section that does. Is there a function for this already or?
const result = await pornhub.videoList({
    order: 'Hottest',
    country: 'Ireland',
})

Can the country value just be a country code, such as "uk", or does it have to be the country name?
Also does this support Top Rated and Most Viewed as well?

pionxzh · 2023-10-24T18:01:10Z

Can the country value just be a country code, such as "uk", or does it have to be the country name?

Check this Country interce "key", it's the same country list extracted out from PH.

Pornhub.js/src/types/Country.ts

Lines 1 to 44 in a6506b4

    
           export const CountryMapping = { 
        
               'Argentina': 'ar', 
        
               'Australia': 'au', 
        
               'Austria': 'at', 
        
               'Belgium': 'be', 
        
               'Brazil': 'br', 
        
               'Bulgaria': 'bg', 
        
               'Canada': 'ca', 
        
               'Chile': 'cl', 
        
               'Croatia': 'hr', 
        
               'Czech Republic': 'cz', 
        
               'Denmark': 'dk', 
        
               'Egypt': 'eg', 
        
               'Finland': 'fi', 
        
               'France': 'fr', 
        
               'Germany': 'de', 
        
               'Greece': 'gr', 
        
               'Hungary': 'hu', 
        
               'India': 'in', 
        
               'Ireland': 'ie', 
        
               'Israel': 'il', 
        
               'Italy': 'it', 
        
               'Japan': 'jp', 
        
               'Korea': 'kr', 
        
               'Mexico': 'mx', 
        
               'Morocco': 'ma', 
        
               'Netherlands': 'nl', 
        
               'New Zealand': 'nz', 
        
               'Norway': 'no', 
        
               'Pakistan': 'pk', 
        
               'Poland': 'pl', 
        
               'Portugal': 'pt', 
        
               'Romania': 'ro', 
        
               'Russia': 'ru', 
        
               'Serbia': 'rs', 
        
               'Slovakia': 'sk', 
        
               'Spain': 'es', 
        
               'Sweden': 'se', 
        
               'Switzerland': 'ch', 
        
               'United Kingdom': 'gb', 
        
               'Ukraine': 'ua', 
        
               'United States': 'us', 
        
               'World': 'world', 
        
           }

country only valid when you choose order = Hottest

iced-queen · 2023-10-24T18:03:22Z

Can the country value just be a country code, such as "uk", or does it have to be the country name?

Check this Country interce "key", it's the same country list extracted out from PH.

Pornhub.js/src/types/Country.ts

Lines 1 to 44 in a6506b4

export const CountryMapping = {

'Argentina': 'ar',

'Australia': 'au',

'Austria': 'at',

'Belgium': 'be',

'Brazil': 'br',

'Bulgaria': 'bg',

'Canada': 'ca',

'Chile': 'cl',

'Croatia': 'hr',

'Czech Republic': 'cz',

'Denmark': 'dk',

'Egypt': 'eg',

'Finland': 'fi',

'France': 'fr',

'Germany': 'de',

'Greece': 'gr',

'Hungary': 'hu',

'India': 'in',

'Ireland': 'ie',

'Israel': 'il',

'Italy': 'it',

'Japan': 'jp',

'Korea': 'kr',

'Mexico': 'mx',

'Morocco': 'ma',

'Netherlands': 'nl',

'New Zealand': 'nz',

'Norway': 'no',

'Pakistan': 'pk',

'Poland': 'pl',

'Portugal': 'pt',

'Romania': 'ro',

'Russia': 'ru',

'Serbia': 'rs',

'Slovakia': 'sk',

'Spain': 'es',

'Sweden': 'se',

'Switzerland': 'ch',

'United Kingdom': 'gb',

'Ukraine': 'ua',

'United States': 'us',

'World': 'world',

}

So is that a "Yes" or? 😅 Like can it take both the "Key" and the "Value" or do I HAVE to pass the "Key" strings?

pionxzh · 2023-10-24T18:04:17Z

Only "key", for example, Argentina and Australia.
uk is not allowed.

pionxzh · 2023-10-24T18:06:51Z

Back to the issue, I will need to know what page you were seeing when you got the empty result. I guess it might be the age restriction page... But not sure how to give you a way to provide that information. Let me try VPN tmr.

iced-queen · 2023-10-24T19:55:00Z

Back to the issue, I will need to know what page you were seeing when you got the empty result. I guess it might be the age restriction page... But not sure how to give you a way to provide that information. Let me try VPN tmr.

I mean, the other functions work. I'm able to search for videos and get information about the video afterwards, just not recommendedVideo()

iced-queen · 2023-10-24T20:22:59Z

Also a little side-note, on PH's site, the random video url seems to be BASE_URL/video/random and just not BASE_URL/random.
Both seem to work, but that could be what's causing it to fail from time to time. I just resolve that issue with:

while (res.id === 'https://www.pornhub.com/') {
    res = await pornhub.randomVideo()
}

But would be better if it never happened, right? 😄

pionxzh · 2023-10-25T08:54:52Z

nice finding, it will check tonight.

pionxzh · 2023-10-26T17:12:03Z

PH is down... but ye seems both url can go to the random page. I will update it to /video/random in the next version.

pionxzh · 2023-10-26T17:49:50Z

OK. I might found the root cause. PH has a JS Protection gate to stop crawler and downloader. They won't accept your requests unless you have a correct RNKEY cookie if your IP has made too many requests.

reference: ytdl-org/youtube-dl#5930

Actions:

Bypass the JS protection automatically
Provide a option / way to dump the HTML page requested for debugging

pionxzh · 2023-10-26T17:50:57Z

I might be wrong because you can access other pages without a problem, idk 😄

iced-queen · 2023-10-27T07:44:38Z

I might be wrong because you can access other pages without a problem, idk 😄

Yeah, it's just the recommended page. Bit weird. Although do you think it'd be possible to make a download feature to download videos?

pionxzh · 2023-10-27T09:05:22Z

no, but you can try to download from the m3u8 link. See #90

iced-queen · 2023-11-02T21:49:18Z

Do you have an update on this issue? 🙂

pionxzh · 2023-11-03T02:11:17Z

I was so busy this week :(

pionxzh · 2023-11-05T18:24:23Z

Check #97 and try dump the page to see what's wrong.

iced-queen · 2023-11-05T19:11:23Z

Check #97 and try dump the page to see what's wrong.

A little confused what you want me to do? 😄

pionxzh · 2023-11-06T02:24:40Z

My bad. Can you try to start the application with this constructor? You shall see all the page responses in _dump folder.

const pornhub = new PornHub({ dumpPage: false });

iced-queen · 2023-11-17T10:58:33Z

My bad. Can you try to start the application with this constructor? You shall see all the page responses in _dump folder.
const pornhub = new PornHub({ dumpPage: false });

Hey again,
Sorry for the long wait, I actually forgot about all of this...

After updating and adding that, I don't see a _dump folder? But I do still get the empty result.

iced-queen · 2023-11-17T11:02:21Z

Update, changed false to true and got a _dump folder now.
1700218744596__recommended.zip

pionxzh · 2023-11-17T12:48:51Z

lol my bad. should be true.

const pornhub = new PornHub({ dumpPage: true });

iced-queen · 2023-11-17T13:02:15Z

lol my bad. should be true.

const pornhub = new PornHub({ dumpPage: true });

Yeah I figured 😄 Sent the result above

pionxzh · 2023-11-17T16:57:11Z

Based on the response, you actually received an empty recommendation list... Then this should be related to pornhub's robot detection. I have no way to fix the issue based on these limited information.

pionxzh · 2023-11-17T16:58:02Z

You can try VPN or ip pool to get around it.

pionxzh changed the title ~~The recommendedVideos function always returns { data: [], paging: { current: 1, maxPage: 1, isEnd: true } }~~ The recommendedVideos function always returns empty result Oct 24, 2023

pionxzh added the bug Something isn't working label Oct 24, 2023

pionxzh closed this as completed Nov 17, 2023

pionxzh closed this as not planned Won't fix, can't repro, duplicate, stale Nov 17, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The `recommendedVideos` function always returns empty result #92

The `recommendedVideos` function always returns empty result #92

iced-queen commented Oct 20, 2023

pionxzh commented Oct 21, 2023

iced-queen commented Oct 23, 2023

pionxzh commented Oct 24, 2023

iced-queen commented Oct 24, 2023 •

edited

Loading

pionxzh commented Oct 24, 2023

pionxzh commented Oct 24, 2023

pionxzh commented Oct 24, 2023

iced-queen commented Oct 24, 2023

iced-queen commented Oct 24, 2023

pionxzh commented Oct 24, 2023

iced-queen commented Oct 24, 2023

pionxzh commented Oct 24, 2023 •

edited

Loading

iced-queen commented Oct 24, 2023

pionxzh commented Oct 24, 2023

pionxzh commented Oct 24, 2023

iced-queen commented Oct 24, 2023

iced-queen commented Oct 24, 2023

pionxzh commented Oct 25, 2023

pionxzh commented Oct 26, 2023

pionxzh commented Oct 26, 2023

pionxzh commented Oct 26, 2023

iced-queen commented Oct 27, 2023

pionxzh commented Oct 27, 2023

iced-queen commented Nov 2, 2023

pionxzh commented Nov 3, 2023

pionxzh commented Nov 5, 2023

iced-queen commented Nov 5, 2023

pionxzh commented Nov 6, 2023

iced-queen commented Nov 17, 2023

iced-queen commented Nov 17, 2023

pionxzh commented Nov 17, 2023

iced-queen commented Nov 17, 2023

pionxzh commented Nov 17, 2023

pionxzh commented Nov 17, 2023

The recommendedVideos function always returns empty result #92

The recommendedVideos function always returns empty result #92

Comments

iced-queen commented Oct 20, 2023

pionxzh commented Oct 21, 2023

iced-queen commented Oct 23, 2023

pionxzh commented Oct 24, 2023

iced-queen commented Oct 24, 2023 • edited Loading

pionxzh commented Oct 24, 2023

pionxzh commented Oct 24, 2023

pionxzh commented Oct 24, 2023

iced-queen commented Oct 24, 2023

iced-queen commented Oct 24, 2023

pionxzh commented Oct 24, 2023

iced-queen commented Oct 24, 2023

pionxzh commented Oct 24, 2023 • edited Loading

iced-queen commented Oct 24, 2023

pionxzh commented Oct 24, 2023

pionxzh commented Oct 24, 2023

iced-queen commented Oct 24, 2023

iced-queen commented Oct 24, 2023

pionxzh commented Oct 25, 2023

pionxzh commented Oct 26, 2023

pionxzh commented Oct 26, 2023

pionxzh commented Oct 26, 2023

iced-queen commented Oct 27, 2023

pionxzh commented Oct 27, 2023

iced-queen commented Nov 2, 2023

pionxzh commented Nov 3, 2023

pionxzh commented Nov 5, 2023

iced-queen commented Nov 5, 2023

pionxzh commented Nov 6, 2023

iced-queen commented Nov 17, 2023

iced-queen commented Nov 17, 2023

pionxzh commented Nov 17, 2023

iced-queen commented Nov 17, 2023

pionxzh commented Nov 17, 2023

pionxzh commented Nov 17, 2023

The `recommendedVideos` function always returns empty result #92

The `recommendedVideos` function always returns empty result #92

iced-queen commented Oct 24, 2023 •

edited

Loading

pionxzh commented Oct 24, 2023 •

edited

Loading