Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't access iframe on some websites #548

Closed
ysmood opened this issue Mar 3, 2022 · 57 comments
Closed

Can't access iframe on some websites #548

ysmood opened this issue Mar 3, 2022 · 57 comments
Assignees
Labels
bug When you are sure about it's a bug help wanted We wish someone can help us to work on it

Comments

@ysmood
Copy link
Member

ysmood commented Mar 3, 2022

Rod Version: v0.102.0

The code to demonstrate your question

package main

import (
	"github.com/go-rod/rod"
	"github.com/go-rod/rod/lib/defaults"
)

func main() {
	defaults.Show = true

	page := rod.New().MustConnect().NoDefaultDevice().MustPage("https://captcha.website")
	f := page.MustElement("div:not([style*='display:']) > iframe[data-hcaptcha-widget-id]").MustFrame()
	f.MustElement("#checkbox")
}

The code above will panic. It works fine on headless mode.

@ysmood ysmood added bug When you are sure about it's a bug help wanted We wish someone can help us to work on it labels Mar 3, 2022
@ysmood
Copy link
Member Author

ysmood commented Mar 3, 2022

A temp solution is to use PageFromTargetID to create a standalone page for the iframe:

package main

import (
	"github.com/go-rod/rod"
	"github.com/go-rod/rod/lib/defaults"
	"github.com/go-rod/rod/lib/proto"
	"github.com/go-rod/rod/lib/utils"
)

func main() {
	defaults.Show = true

	page := rod.New().MustConnect().NoDefaultDevice().MustPage("https://captcha.website")
	f := page.MustElement("div:not([style*='display:']) > iframe[data-hcaptcha-widget-id]").MustFrame()
	p := page.Browser().MustPageFromTargetID(proto.TargetTargetID(f.FrameID))
	p.MustElement("#checkbox").MustClick()
	utils.Pause()
}

@ysmood
Copy link
Member Author

ysmood commented Mar 3, 2022

I think it's a bug of devtools, I don't know why Pierce: true can't get the content of the iframe:

node, err := proto.DOMDescribeNode{BackendNodeID: owner.BackendNodeID, Pierce: true}.Call(p)

@KMACEL
Copy link

KMACEL commented Apr 9, 2022

I encountered the same error. When I remove "show" from .rod, it works, but if the browser opens, it gets an error.

@ysmood
Copy link
Member Author

ysmood commented Apr 18, 2022

I can confirm that puppeteer has the same issue, the code below will crash:

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({
    headless: false
  });
  const page = await browser.newPage();
  await page.goto('https://captcha.website')
  await new Promise((r) => setTimeout(r, 5000))
  const el = await page.$("div:not([style*='display:']) > iframe[data-hcaptcha-widget-id]")
  const frame = await el.contentFrame()
  await frame.$('#checkbox').click()
})();

@tlopo
Copy link

tlopo commented Jun 21, 2022

Since this is probably a bug on CDP, does anyone know if it's being tracked somewhere ?

@ysmood
Copy link
Member Author

ysmood commented Jun 21, 2022

@tlopo I have opened an issue on puppeteer, but no update yet

@jhemmmm
Copy link

jhemmmm commented Nov 6, 2022

Any update on this?

Your temp solution didn't work on it me.

@ysmood ysmood added this to Rod Nov 14, 2022
@ysmood ysmood moved this to In Progress in Rod Nov 14, 2022
@ysmood ysmood self-assigned this Dec 6, 2022
@timestee
Copy link

timestee commented Feb 3, 2023

The issue puppeteer/puppeteer#8150 has been confirmed but still no update yet~

@h3isenbug
Copy link

Stuck with the same issue here. FrameID workaround did not help.

@Samler-Lee
Copy link

Samler-Lee commented Apr 3, 2023

A temp solution is to download an older version of Chromium and replace the current version of Chromium. I successfuly tested my code with on version 884014

@alicazi
Copy link

alicazi commented Aug 29, 2023

I'm getting "Command can only be executed on top-level targets" error when PageFromTarget(proto.TargetTargetID(f.FrameID)) which f is what I got from calling el.Frame().
Wondering is this a same issue? then why the error is so different?
BTW, no success with other suggested solutions (not tried old Chromium version though as I don't see it as a sustainable solution).
FYI, I'm trying to create a test that needs filling credit card information in Stripe iframe form.

@ysmood
Copy link
Member Author

ysmood commented Aug 30, 2023

I will take some time to redesign how the js runtime works to completely resolve this issue.

@alicazi
Copy link

alicazi commented Aug 31, 2023

@ysmood Thanks

@hsxsix
Copy link

hsxsix commented Sep 11, 2023

I'm getting "Command can only be executed on top-level targets" error when PageFromTarget(proto.TargetTargetID(f.FrameID)) which f is what I got from calling el.Frame(). Wondering is this a same issue? then why the error is so different? BTW, no success with other suggested solutions (not tried old Chromium version though as I don't see it as a sustainable solution). FYI, I'm trying to create a test that needs filling credit card information in Stripe iframe form.

same problem, any updates?

@zplzpl
Copy link

zplzpl commented Nov 24, 2023

当 PageFromTarget(proto.TargetTargetID(f.FrameID)) 时出现“命令只能在顶级目标上执行”错误,其中 f 是我通过调用 el.Frame() 得到的。 想知道这是同一个问题吗?那么为什么错误如此不同呢? 顺便说一句,其他建议的解决方案没有成功(没有尝试过旧的 Chromium 版本,因为我不认为它是一个可持续的解决方案)。 仅供参考,我正在尝试创建一个需要在 Stripe iframe 表单中填写信用卡信息的测试。

same problem

@zplzpl
Copy link

zplzpl commented Nov 24, 2023

A temp solution is to download an older version of Chromium and replace the current version of Chromium. I successfuly tested my code with on version 884014

worked

@ysmood
Copy link
Member Author

ysmood commented Jun 3, 2024

@alplf123 Thank you for the tips, it works for me for a simple test like below! I will try to create a unit test to reproduce it, if it works as expected, I will make it the default launch option.

The code like below will crash due to not able to access the iframe:

package main

import (
	"fmt"

	"github.com/go-rod/rod"
	"github.com/go-rod/rod/lib/launcher"
)

func main() {
	u := launcher.New().
		Headless(false).
		MustLaunch()

	page := rod.New().ControlURL(u).MustConnect().NoDefaultDevice().MustPage("https://dash.cloudflare.com/sign-up")
	f := page.MustElement(`iframe[src*="https://challenges.cloudflare.com"]`).MustFrame()
	fmt.Println(f.MustElement("#success").MustHTML())
}

But the code like this will work:

package main

import (
	"fmt"

	"github.com/go-rod/rod"
	"github.com/go-rod/rod/lib/launcher"
)

func main() {
	u := launcher.New().
		NoSandbox(true).
		Set("disable-web-security").
		Set("disable-site-isolation-trials").
		MustLaunch()

	page := rod.New().ControlURL(u).MustConnect().NoDefaultDevice().MustPage("https://dash.cloudflare.com/sign-up")
	f := page.MustElement(`iframe[src*="https://challenges.cloudflare.com"]`).MustFrame()
	fmt.Println(f.MustElement("#success").MustHTML())
}

The three flags are both critical:

		NoSandbox(true).
		Set("disable-web-security").
		Set("disable-site-isolation-trials").

@ysmood ysmood closed this as completed in f71a812 Jun 17, 2024
ysmood added a commit that referenced this issue Jun 17, 2024
ysmood added a commit that referenced this issue Jun 17, 2024
@ysmood
Copy link
Member Author

ysmood commented Jun 17, 2024

v0.116.1 has been released, please give it a try.

@Longdexin
Copy link

I've tried v0.116.1, it works as expected, but the cloudflare check progress was looping forever. When I returned to the older version v116.0, the cloudflare checkbox was checked and the page was redirected successfully.

@ysmood
Copy link
Member Author

ysmood commented Jun 18, 2024

@Longdexin It works fine to me, I think it's because of your ip and browser fingerprint.

CleanShot.2024-06-18.at.11.11.47-converted.mp4

@Longdexin
Copy link

@ysmood , yes, you got a point here. Thank you so much for your great work, it helps me a lot.

@Longdexin
Copy link

@ysmood hi, it's me again. I found that you had put flag "disable-site-isolation-trials" into defaultFlags, maybe this is the cause of the cloudflare check loop I mentioned earlier. When I wrote '.Delete("disable-site-isolation-trials")' to delete this flag, the problem was gone.
1
2

@ysmood
Copy link
Member Author

ysmood commented Jun 19, 2024

@Longdexin #1076

@Longdexin
Copy link

@ysmood Hi, my code could no longer access the cloudflare iframe element, and after some digging, I found that the iframe had been using '#shadow-root (closed)' as its parent node recently, as described here https://developer.mozilla.org/en-US/docs/Web/API/ShadowRoot/mode.
I managed to fix the test code somehow as below, hoping to help other users with the same problem.

package main

import (
	"fmt"
	"time"

	"github.com/go-rod/rod"
	"github.com/go-rod/rod/lib/launcher"
)

func main() {
	u := launcher.New().
		Headless(false).
		NoSandbox(true).
		Set("disable-web-security").
		Set("disable-site-isolation-trials").
		MustLaunch()

	page := rod.New().ControlURL(u).MustConnect().NoDefaultDevice().MustPage("https://dash.cloudflare.com/sign-up")
	el, err := page.Timeout(30 * time.Second).Element(`div.cf-turnstile-wrapper`)
	if err != nil {
		panic(err)
	}
	el, err = el.ShadowRoot()
	if err != nil {
		panic(err)
	}
	_, err = el.Eval("()=>this.mode='open'")
	if err != nil {
		panic(err)
	}
	f := el.MustElement(`iframe[src*="https://challenges.cloudflare.com"]`).MustFrame()
	fmt.Println(f.MustElement("#success").MustHTML())
}

@moxcomic
Copy link

@ysmood Hi, my code could no longer access the cloudflare iframe element, and after some digging, I found that the iframe had been using '#shadow-root (closed)' as its parent node recently, as described here https://developer.mozilla.org/en-US/docs/Web/API/ShadowRoot/mode. I managed to fix the test code somehow as below, hoping to help other users with the same problem.

package main

import (
	"fmt"
	"time"

	"github.com/go-rod/rod"
	"github.com/go-rod/rod/lib/launcher"
)

func main() {
	u := launcher.New().
		Headless(false).
		NoSandbox(true).
		Set("disable-web-security").
		Set("disable-site-isolation-trials").
		MustLaunch()

	page := rod.New().ControlURL(u).MustConnect().NoDefaultDevice().MustPage("https://dash.cloudflare.com/sign-up")
	el, err := page.Timeout(30 * time.Second).Element(`div.cf-turnstile-wrapper`)
	if err != nil {
		panic(err)
	}
	el, err = el.ShadowRoot()
	if err != nil {
		panic(err)
	}
	_, err = el.Eval("()=>this.mode='open'")
	if err != nil {
		panic(err)
	}
	f := el.MustElement(`iframe[src*="https://challenges.cloudflare.com"]`).MustFrame()
	fmt.Println(f.MustElement("#success").MustHTML())
}

I tried using your code, but it failed to pass verification. When I manually click the verification checkbox, it spins for a few seconds and then returns to the unchecked state. This happens repeatedly, even after several attempts.

@Longdexin
Copy link

Longdexin commented Jul 20, 2024

@moxcomic Hi, if you look at the html structure of this page (https://dash.cloudflare.com/sign-up) now, you can see it has two #shadow-root in the iframe, while it had only one when I posted the code above.
2
I fixed the example code like below.

package main

import (
	"fmt"
	"time"

	"github.com/go-rod/rod"
	"github.com/go-rod/rod/lib/launcher"
	"github.com/go-rod/rod/lib/utils"
)

func main() {
	u := launcher.New().
		Headless(false).
		NoSandbox(true).
		Set("disable-web-security").
		Set("disable-site-isolation-trials").
		MustLaunch()

	page := rod.New().ControlURL(u).MustConnect().NoDefaultDevice().MustPage("https://dash.cloudflare.com/sign-up")
	el, err := page.Timeout(30 * time.Second).Element(`div.cf-turnstile-wrapper`)
	if err != nil {
		panic(err)
	}
	el, err = el.ShadowRoot()
	if err != nil {
		panic(err)
	}
	_, err = el.Eval("()=>this.mode='open'")
	if err != nil {
		panic(err)
	}
	f := el.MustElement(`iframe[src*="https://challenges.cloudflare.com"]`).MustFrame()
	utils.Sleep(10)
	el, err = f.MustElement("body").ShadowRoot()
	if err != nil {
		panic(err)
	}
	_, err = el.Eval("()=>this.mode='open'")
	if err != nil {
		panic(err)
	}
	fmt.Println(el.MustElement("#success").MustHTML())
}

However, 5 minutes later after I commented last time, I found the same repeating problem as you said and could not find any way to go. Sorry, the cloudflare thing is so disgusting, good luck to both of us.

@moxcomic
Copy link

@moxcomic Hi, if you look at the html structure of this page (https://dash.cloudflare.com/sign-up) now, you can see it has two #shadow-root in the iframe, while it had only one when I posted the code above. 2 I fixed the example code like below.

package main

import (
	"fmt"
	"time"

	"github.com/go-rod/rod"
	"github.com/go-rod/rod/lib/launcher"
	"github.com/go-rod/rod/lib/utils"
)

func main() {
	u := launcher.New().
		Headless(false).
		NoSandbox(true).
		Set("disable-web-security").
		Set("disable-site-isolation-trials").
		MustLaunch()

	page := rod.New().ControlURL(u).MustConnect().NoDefaultDevice().MustPage("https://dash.cloudflare.com/sign-up")
	el, err := page.Timeout(30 * time.Second).Element(`div.cf-turnstile-wrapper`)
	if err != nil {
		panic(err)
	}
	el, err = el.ShadowRoot()
	if err != nil {
		panic(err)
	}
	_, err = el.Eval("()=>this.mode='open'")
	if err != nil {
		panic(err)
	}
	f := el.MustElement(`iframe[src*="https://challenges.cloudflare.com"]`).MustFrame()
	utils.Sleep(10)
	el, err = f.MustElement("body").ShadowRoot()
	if err != nil {
		panic(err)
	}
	_, err = el.Eval("()=>this.mode='open'")
	if err != nil {
		panic(err)
	}
	fmt.Println(el.MustElement("#success").MustHTML())
}

However, 5 minutes later after I commented last time, I found the same repeating problem as you said and could not find any way to go. Sorry, the cloudflare thing is so disgusting, good luck to both of us.

I found the problem. If Set("disable-site-isolation-trials") is added, it triggers this issue. Without it, verification passes normally, but it gives a runtime error: invalid memory address or nil pointer dereference error at el, err = f.MustElement("body").ShadowRoot().

@Longdexin
Copy link

A temp solution is to use PageFromTargetID to create a standalone page for the iframe:

package main

import (
	"github.com/go-rod/rod"
	"github.com/go-rod/rod/lib/defaults"
	"github.com/go-rod/rod/lib/proto"
	"github.com/go-rod/rod/lib/utils"
)

func main() {
	defaults.Show = true

	page := rod.New().MustConnect().NoDefaultDevice().MustPage("https://captcha.website")
	f := page.MustElement("div:not([style*='display:']) > iframe[data-hcaptcha-widget-id]").MustFrame()
	p := page.Browser().MustPageFromTargetID(proto.TargetTargetID(f.FrameID))
	p.MustElement("#checkbox").MustClick()
	utils.Pause()
}

@moxcomic maybe you should take this temp solution into consideration to avoid the runtime error.
I'll be back here to share my code if I figure a way out in several days.

@moxcomic
Copy link

@moxcomic maybe you should take this temp solution into consideration to avoid the runtime error. I'll be back here to share my code if I figure a way out in several days.

@Longdexin Thank you. I was using it normally before, but recently, after some changes in CF, I can no longer access the checkbox, and I haven't found a good solution yet.

@Longdexin
Copy link

@moxcomic "同是天涯沦落人", 没错,就是最近两周出的问题,我现在基本就是手动勾选验证,先凑合着用,后面再研究研究解决办法,万恶的cloudflare!!!

@moxcomic
Copy link

@moxcomic "同是天涯沦落人", 没错,就是最近两周出的问题,我现在基本就是手动勾选验证,先凑合着用,后面再研究研究解决办法,万恶的cloudflare!!!

你这么一说倒是提醒到我了!

page := rod.New().ControlURL(u).MustConnect().NoDefaultDevice().MustPage("https://dash.cloudflare.com/sign-up")
el, err := page.Timeout(30 * time.Second).Element(`div.cf-turnstile-wrapper`)
if err != nil {
    panic(err)
}

<-time.After(time.Second * 10)

res, err := el.Eval(`() => {
const rect = this.getBoundingClientRect();
    return { x: rect.left, y: rect.top };
}`)
if err != nil {
    panic(err)
}

x, y := res.Value.Get("x").Num(), res.Value.Get("y").Num()

fmt.Println(x, y)

page.Mouse.MoveLinear(proto.NewPoint(x+27, y+29), 20)

// 使用JavaScript在鼠标位置添加一个视觉标记
page.Eval(`(x, y) => {
        const marker = document.createElement('div');
        marker.style.position = 'fixed';  // 使用fixed定位确保可见
        marker.style.left = x + 'px';
        marker.style.top = y + 'px';
        marker.style.width = '10px';
        marker.style.height = '10px';
        marker.style.backgroundColor = 'red';
        marker.style.borderRadius = '50%';
        marker.style.zIndex = 9999;  // 确保在最上层
        document.body.appendChild(marker);
    }`, x+27, y+29)

page.Mouse.MustClick(proto.InputMouseButtonLeft)
fmt.Println("click.")

我这么用能过验证(可能比手动好一点?)

@yusufmalikul
Copy link

I'm getting "Command can only be executed on top-level targets" error when PageFromTarget(proto.TargetTargetID(f.FrameID)) which f is what I got from calling el.Frame(). Wondering is this a same issue? then why the error is so different? BTW, no success with other suggested solutions (not tried old Chromium version though as I don't see it as a sustainable solution). FYI, I'm trying to create a test that needs filling credit card information in Stripe iframe form.

@alicazi did you solve the credit card filling issue? I also has the same issue when trying to access the iframe it get panic.

@lsy88
Copy link

lsy88 commented Sep 11, 2024

@moxcomic“同是天涯沦落人”,正确的办法,就是最近提出的问题,我现在基本就是手动勾选验证,先凑合着用,后面再研究研究解决,万恶的cloudflare!

你这么说倒是提醒我了!

page := rod.New().ControlURL(u).MustConnect().NoDefaultDevice().MustPage("https://dash.cloudflare.com/sign-up")
el, err := page.Timeout(30 * time.Second).Element(`div.cf-turnstile-wrapper`)
if err != nil {
    panic(err)
}

<-time.After(time.Second * 10)

res, err := el.Eval(`() => {
const rect = this.getBoundingClientRect();
    return { x: rect.left, y: rect.top };
}`)
if err != nil {
    panic(err)
}

x, y := res.Value.Get("x").Num(), res.Value.Get("y").Num()

fmt.Println(x, y)

page.Mouse.MoveLinear(proto.NewPoint(x+27, y+29), 20)

// 使用JavaScript在鼠标位置添加一个视觉标记
page.Eval(`(x, y) => {
        const marker = document.createElement('div');
        marker.style.position = 'fixed';  // 使用fixed定位确保可见
        marker.style.left = x + 'px';
        marker.style.top = y + 'px';
        marker.style.width = '10px';
        marker.style.height = '10px';
        marker.style.backgroundColor = 'red';
        marker.style.borderRadius = '50%';
        marker.style.zIndex = 9999;  // 确保在最上层
        document.body.appendChild(marker);
    }`, x+27, y+29)

page.Mouse.MustClick(proto.InputMouseButtonLeft)
fmt.Println("click.")

我用能过验证(可能比这么手动好一点?)

有新的方法解决cloudflare吗,目前尝试这种方法也不行了

@moxcomic
Copy link

@moxcomic“同是天涯沦落人”,正确的办法,就是最近提出的问题,我现在基本就是手动勾选验证,先凑合着用,后面再研究研究解决,万恶的cloudflare!

你这么说倒是提醒我了!

page := rod.New().ControlURL(u).MustConnect().NoDefaultDevice().MustPage("https://dash.cloudflare.com/sign-up")

el, err := page.Timeout(30 * time.Second).Element(div.cf-turnstile-wrapper)

if err != nil {

panic(err)

}

<-time.After(time.Second * 10)

res, err := el.Eval(`() => {

const rect = this.getBoundingClientRect();

return { x: rect.left, y: rect.top };

}`)

if err != nil {

panic(err)

}

x, y := res.Value.Get("x").Num(), res.Value.Get("y").Num()

fmt.Println(x, y)

page.Mouse.MoveLinear(proto.NewPoint(x+27, y+29), 20)

// 使用JavaScript在鼠标位置添加一个视觉标记

page.Eval(`(x, y) => {

    const marker = document.createElement('div');
    marker.style.position = 'fixed';  // 使用fixed定位确保可见
    marker.style.left = x + 'px';
    marker.style.top = y + 'px';
    marker.style.width = '10px';
    marker.style.height = '10px';
    marker.style.backgroundColor = 'red';
    marker.style.borderRadius = '50%';
    marker.style.zIndex = 9999;  // 确保在最上层
    document.body.appendChild(marker);
}`, x+27, y+29)

page.Mouse.MustClick(proto.InputMouseButtonLeft)

fmt.Println("click.")

我用能过验证(可能比这么手动好一点?)

有新的方法解决cloudflare吗,目前尝试这种方法也不行了

目前我这里今天还是正常使用,应该是其他问题吧,你网站发出来看看

@lsy88
Copy link

lsy88 commented Sep 11, 2024

@moxcomic“同是天涯沦落人”,正确的办法,就是最近提出的问题,我现在基本就是手动勾选验证,先凑合着用,后面再研究研究解决,万恶的cloudflare!

你这么说倒是提醒我了!

page := rod.New().ControlURL(u).MustConnect().NoDefaultDevice().MustPage("https://dash.cloudflare.com/sign-up")

el, err := page.Timeout(30 * time.Second).Element(div.cf-turnstile-wrapper)

if err != nil {

panic(err)

}

<-time.After(time.Second * 10)

res, err := el.Eval(`() => {

const rect = this.getBoundingClientRect();

return { x: rect.left, y: rect.top };

}`)

if err != nil {

panic(err)

}

x, y := res.Value.Get("x").Num(), res.Value.Get("y").Num()

fmt.Println(x, y)

page.Mouse.MoveLinear(proto.NewPoint(x+27, y+29), 20)

// 使用JavaScript在鼠标位置添加一个视觉标记

page.Eval(`(x, y) => {

    const marker = document.createElement('div');
    marker.style.position = 'fixed';  // 使用fixed定位确保可见
    marker.style.left = x + 'px';
    marker.style.top = y + 'px';
    marker.style.width = '10px';
    marker.style.height = '10px';
    marker.style.backgroundColor = 'red';
    marker.style.borderRadius = '50%';
    marker.style.zIndex = 9999;  // 确保在最上层
    document.body.appendChild(marker);
}`, x+27, y+29)

page.Mouse.MustClick(proto.InputMouseButtonLeft)

fmt.Println("click.")

我用能过验证(可能比这么手动好一点?)

有新的方法解决cloudflare吗,目前尝试这种方法也不行了

目前我这里今天还是正常使用,应该是其他问题吧,你网站发出来看看

https://nulled.to 当点击cloudflare checkbox时发现页面会同步刷新导致失效

@moxcomic
Copy link

@lsy88 这个网站不是reCaptcha的吗? 这也不是cloudflare啊

@lsy88
Copy link

lsy88 commented Sep 11, 2024

@lsy88 这个网站不是reCaptcha的吗? 这也不是cloudflare啊
我这边触发的是cloudflare
image

@moxcomic
Copy link

@lsy88 这个网站不是reCaptcha的吗? 这也不是cloudflare啊

我这边触发的是cloudflare

image

image

@lsy88
Copy link

lsy88 commented Sep 11, 2024

@lsy88 这个网站不是reCaptcha的吗? 这也不是cloudflare啊

我这边触发的是cloudflare
image

image

reCaptcha这是登录的页面,我是在用rod开无头浏览器爬的时候,触发的cloudflare,还没有进入首页

@moxcomic
Copy link

@lsy88 这个网站不是reCaptcha的吗? 这也不是cloudflare啊

我这边触发的是cloudflare

image

image

reCaptcha这是登录的页面,我是在用rod开无头浏览器爬的时候,触发的cloudflare,还没有进入首页

尝试用普通的Chrome浏览器,并删除掉rod自带的flags

@lsy88
Copy link

lsy88 commented Sep 11, 2024

@lsy88 这个网站不是reCaptcha的吗? 这也不是cloudflare啊

我这边触发的是cloudflare

image

image

reCaptcha这是登录的页面,我是在用rod开无头浏览器爬的时候,触发的cloudflare,还没有进入首页

尝试用普通的Chrome浏览器,并删除掉rod自带的flags

Delete("disable-site-isolation-trials"). Delete("rod-working-dir"). Delete("rod-env"). Delete("rod-xvfb"). Delete("rod-preferences"). Delete("rod-leakless"). Delete("rod-keep-user-data-dir"). Delete("rod-bin") 删除掉自带的flags也并没有带来明显的效果改善

@moxcomic
Copy link

@lsy88 这个网站不是reCaptcha的吗? 这也不是cloudflare啊

我这边触发的是cloudflare

image

image

reCaptcha这是登录的页面,我是在用rod开无头浏览器爬的时候,触发的cloudflare,还没有进入首页

尝试用普通的Chrome浏览器,并删除掉rod自带的flags

Delete("disable-site-isolation-trials"). Delete("rod-working-dir"). Delete("rod-env"). Delete("rod-xvfb"). Delete("rod-preferences"). Delete("rod-leakless"). Delete("rod-keep-user-data-dir"). Delete("rod-bin") 删除掉自带的flags也并没有带来明显的效果改善

	l := launcher.NewUserMode()
	for _, v := range l.FormatArgs() {
		l.Delete(flags.Flag(strings.Split(v, "=")[0][2:]))
	}
	l.
		RemoteDebuggingPort(9222).
		Set("no-first-run").
		Set("no-default-browser-check").
		Set("enable-privacy-sandbox-ads-apis", "false").
		Leakless(true).
		Headless(true).
		Bin("/Applications/Google Chrome.app/Contents/MacOS/Google Chrome")

	browser := rod.New().ControlURL(l.MustLaunch()).MustConnect().NoDefaultDevice()
	page := browser.MustPage("https://www.nulled.to")

	<-time.After(time.Second * 5)
	page.MustScreenshotFullPage("./111.png")

至少在我的电脑上测试并没有弹出cloudflare
111

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug When you are sure about it's a bug help wanted We wish someone can help us to work on it
Projects
No open projects
Status: In Progress
Development

No branches or pull requests