[nodriver] how to get websocket request or data from cloudflare detected website?

[SSujitX]

Here is my demo code; how can I capture those web sockets?

wss://io.dexscreener.com/dex/screener/pairs/h24/1?rankBy[key]=trendingScoreH6&rankBy[order]=desc

import asyncio
import json
import nodriver as uc
from nodriver.cdp import fetch

async def main():
    
    url = 'https://dexscreener.com/'

    browser = await uc.start()
    page = await browser.get(url)
    await page.sleep(10)

if __name__ == "__main__":
    uc.loop().run_until_complete(main())

[ultrafunkamsterdam]

Why don't you just use the websockets python package instead of nodriver.

[devblack]

because of cloudflare protection.

Cloudflare use TLS validation for some request. You have to use some cypher to validate the CONNECT.

For example:

ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384

[SSujitX]

because of cloudflare protection.

Cloudflare use TLS validation for some request. You have to use some cypher to validate the CONNECT.

For example:

ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384

I am a noob at this. If you have time, can you please help me or provide any documents that can show how to use TLS validation and cipher,?

[devblack]

because of cloudflare protection.

Cloudflare use TLS validation for some request. You have to use some cypher to validate the CONNECT.
For example:
ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384

You are a life saver, but unfortunately, I am a noob at this. If you have time, can you please help me or provide any documents that can show how to use TLS validation and cipher,?

Here is:

import asyncio
import ssl
import websockets

async def connect_with_custom_ciphers():
    # Create an SSL context
    ssl_context = ssl.create_default_context()

    # Set the preferred TLS ciphers (example ciphers; adjust as needed)
    ssl_context.set_ciphers("ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384")

    WS_HEADERS = {
        "Host": "io.dexscreener.com",
        "Origin": "https://dexscreener.com",
        "Pragma": "no-cache",
        "Cache-Control": "no-cache",
        "Accept-Encoding": "gzip, deflate, br, zstd",
        "Accept-Language": "en-US,en;q=0.9",
        "Sec-Websocket-Version": "13",
        "Sec-Websocket-Key": "GrgNmhHdu1u9JGuJf6GnvA==",
        "Sec-Websocket-Extensions": "permessage-deflate; client_max_window_bits",
        "Upgrade": "websocket",
        "Connection": "Upgrade",
        "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36"
    }

    ws_url = 'wss://io.dexscreener.com/dex/screener/pairs/h24/1?rankBy[key]=pairAge&rankBy[order]=asc'

    # Connect to the WebSocket server with the SSL context
    async with websockets.connect(ws_url, extra_headers=WS_HEADERS, ssl=ssl_context) as websocket:
        # Send a message
        while True:
            message = await websocket.recv()
            print(f"Received: {message}")
        

asyncio.run(connect_with_custom_ciphers())

[ultrafunkamsterdam]

I think the answer is much simpler than trying to manually setup ssl context while the package (websockets), supports it out of the box.

You are not sending anything to the websocket. You should at least send something (subscribing etc), before recv()

You must use something like await ws.send(json.dumps(dict(a=1,b=2)))

And leave those headers out. They re http headers (upgrade is a common header to request websocket upgrade from http) but you already using wss://

[SSujitX]

I think the answer is much simpler than trying to manually setup ssl context while the package (websockets), supports it out of the box.

You are not sending anything to the websocket. You should at least send something (subscribing etc), before recv()

You must use something like await ws.send(json.dumps(dict(a=1,b=2)))

And leave those headers out. They re http headers (upgrade is a common header to request websocket upgrade from http) but you already using wss://

well, can you show me pls? i didn't work for me. thanks anyway.