Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

为什么现在需要chrome? #352

Open
Pumpkinhn opened this issue Dec 16, 2024 · 2 comments
Open

为什么现在需要chrome? #352

Pumpkinhn opened this issue Dec 16, 2024 · 2 comments
Assignees
Labels
question Further information is requested

Comments

@Pumpkinhn
Copy link

我在arm平台上使用该项目的基础能力——获取网页的markdown,原本是可以使用的,但我最新的打包docker中报错提示:“Run playwright install chrome”,这在arm上并不能运行,导致我的功能失效,请问这是什么原因导致的?能否修正或避免?

以下是我配置与使用的部分代码

    # 爬虫配置
    crawler_kwargs = {
        'verbose': True,
        'browser_type': "chromium",
        'proxy': settings.PROXY_ADDRESS if settings.PROXY_ADDRESS else None
    }

    try:
        async with AsyncWebCrawler(**crawler_kwargs) as crawler:
            result = await crawler.arun(
                url=url,
                magic=True,
                exclude_external_links=True,  # 去除外部超链接
                excluded_tags=['header', 'footer', 'nav'],  # 去除内容标签
                remove_overlay_elements=True,  # 去除弹窗
                process_iframes=True,  # 加载嵌套页
                exclude_external_images=True  # 去除图片
            )
            log.info("爬虫初始化完成!")

            if result.success:
                # 获取 Markdown 内容
                markdown_content = result.fit_markdown if is_cleared else result.markdown
@Pumpkinhn
Copy link
Author

我原先都是使用chromium的,以下是我的部分dockerfile

# 安装 Playwright 的 Chromium 引擎
RUN python3 -m playwright install chromium

# 安装 Chromium 引擎所需的依赖
RUN python3 -m playwright install-deps chromium

@unclecode
Copy link
Owner

I am trying to understand the issue you are facing. If you have trouble solving the Playwright problem on an ARM CPU, my own machine is Apple Silicon, which is an ARM architecture, and it works fine. Maybe you are using the wrong image for Docker. I have two images: one created for x86 machines and one created for ARM. So please make sure of this part.

@unclecode unclecode self-assigned this Dec 17, 2024
@unclecode unclecode added the question Further information is requested label Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants