Crypto engagement top x

[StephanAkkerman]

https://lunarcrush.com/categories/cryptocurrencies
Sorted on altrank might be more interesting: https://lunarcrush.com/categories/cryptocurrencies?metric=alt_rank&filter=&network=&rpp=20&page=1&cols=&reverse=0

[StephanAkkerman]

get https://lunarcrush.com/api3/storm/category/cryptocurrencies with token
https://github.com/orai-ailab/airflow_llm_layer/blob/240fd7cda73c1eae024a99cabe51e2516e1b0f32/dags/dag_lunarcrush_api_cryptocurrencies.py#L28

[StephanAkkerman]

Would be better if we can construct the query ourselves, so that we do not need to share deviceID etc.

[StephanAkkerman]

We can use playwright for this:

from playwright.sync_api import sync_playwright


def capture_requests(url, target_url_substring):
    with sync_playwright() as p:
        # Launch a browser (Chromium in this case)
        browser = p.chromium.launch(headless=True)
        page = browser.new_page()

        # Store requests
        captured_requests = []

        # Listen for all network requests
        page.on(
            "request",
            lambda request: (
                captured_requests.append(request)
                if target_url_substring in request.url
                else None
            ),
        )

        # Visit the website
        page.goto(url)

        # Wait for the page to load completely
        page.wait_for_load_state("networkidle")

        # Filter captured requests by the target URL substring
        filtered_requests = [
            r for r in captured_requests if target_url_substring in r.url
        ]

        # Close the browser
        browser.close()

        return filtered_requests


# Example usage
url_to_visit = "https://lunarcrush.com/categories/cryptocurrencies"
target_request = "https://lunarcrush.com/api3/auth/request-access?requestAccess=lunar"  # Part of the URL to match
requests = capture_requests(url_to_visit, target_request)

for req in requests:
    print(f"URL: {req.url}, Method: {req.method}")

[StephanAkkerman]

Would add more libraries to the project and a more difficult installation process