-
-
Notifications
You must be signed in to change notification settings - Fork 354
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can't access iframe on some websites #548
Comments
A temp solution is to use package main
import (
"github.com/go-rod/rod"
"github.com/go-rod/rod/lib/defaults"
"github.com/go-rod/rod/lib/proto"
"github.com/go-rod/rod/lib/utils"
)
func main() {
defaults.Show = true
page := rod.New().MustConnect().NoDefaultDevice().MustPage("https://captcha.website")
f := page.MustElement("div:not([style*='display:']) > iframe[data-hcaptcha-widget-id]").MustFrame()
p := page.Browser().MustPageFromTargetID(proto.TargetTargetID(f.FrameID))
p.MustElement("#checkbox").MustClick()
utils.Pause()
} |
I think it's a bug of devtools, I don't know why Line 349 in 14ebb72
|
I encountered the same error. When I remove "show" from .rod, it works, but if the browser opens, it gets an error. |
I can confirm that puppeteer has the same issue, the code below will crash: const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch({
headless: false
});
const page = await browser.newPage();
await page.goto('https://captcha.website')
await new Promise((r) => setTimeout(r, 5000))
const el = await page.$("div:not([style*='display:']) > iframe[data-hcaptcha-widget-id]")
const frame = await el.contentFrame()
await frame.$('#checkbox').click()
})(); |
Signed-off-by: allan716 <[email protected]>
Since this is probably a bug on CDP, does anyone know if it's being tracked somewhere ? |
@tlopo I have opened an issue on puppeteer, but no update yet |
Any update on this? Your temp solution didn't work on it me. |
The issue puppeteer/puppeteer#8150 has been |
Stuck with the same issue here. FrameID workaround did not help. |
A temp solution is to download an older version of Chromium and replace the current version of Chromium. I successfuly tested my code with on version 884014 |
I'm getting "Command can only be executed on top-level targets" error when PageFromTarget(proto.TargetTargetID(f.FrameID)) which f is what I got from calling el.Frame(). |
I will take some time to redesign how the js runtime works to completely resolve this issue. |
@ysmood Thanks |
same problem, any updates? |
same problem |
worked |
@alplf123 Thank you for the tips, it works for me for a simple test like below! I will try to create a unit test to reproduce it, if it works as expected, I will make it the default launch option. The code like below will crash due to not able to access the iframe: package main
import (
"fmt"
"github.com/go-rod/rod"
"github.com/go-rod/rod/lib/launcher"
)
func main() {
u := launcher.New().
Headless(false).
MustLaunch()
page := rod.New().ControlURL(u).MustConnect().NoDefaultDevice().MustPage("https://dash.cloudflare.com/sign-up")
f := page.MustElement(`iframe[src*="https://challenges.cloudflare.com"]`).MustFrame()
fmt.Println(f.MustElement("#success").MustHTML())
} But the code like this will work: package main
import (
"fmt"
"github.com/go-rod/rod"
"github.com/go-rod/rod/lib/launcher"
)
func main() {
u := launcher.New().
NoSandbox(true).
Set("disable-web-security").
Set("disable-site-isolation-trials").
MustLaunch()
page := rod.New().ControlURL(u).MustConnect().NoDefaultDevice().MustPage("https://dash.cloudflare.com/sign-up")
f := page.MustElement(`iframe[src*="https://challenges.cloudflare.com"]`).MustFrame()
fmt.Println(f.MustElement("#success").MustHTML())
} The three flags are both critical: NoSandbox(true).
Set("disable-web-security").
Set("disable-site-isolation-trials"). |
v0.116.1 has been released, please give it a try. |
I've tried v0.116.1, it works as expected, but the cloudflare check progress was looping forever. When I returned to the older version v116.0, the cloudflare checkbox was checked and the page was redirected successfully. |
@Longdexin It works fine to me, I think it's because of your ip and browser fingerprint. CleanShot.2024-06-18.at.11.11.47-converted.mp4 |
@ysmood , yes, you got a point here. Thank you so much for your great work, it helps me a lot. |
@ysmood hi, it's me again. I found that you had put flag "disable-site-isolation-trials" into defaultFlags, maybe this is the cause of the cloudflare check loop I mentioned earlier. When I wrote '.Delete("disable-site-isolation-trials")' to delete this flag, the problem was gone. |
@ysmood Hi, my code could no longer access the cloudflare iframe element, and after some digging, I found that the iframe had been using '#shadow-root (closed)' as its parent node recently, as described here https://developer.mozilla.org/en-US/docs/Web/API/ShadowRoot/mode. package main
import (
"fmt"
"time"
"github.com/go-rod/rod"
"github.com/go-rod/rod/lib/launcher"
)
func main() {
u := launcher.New().
Headless(false).
NoSandbox(true).
Set("disable-web-security").
Set("disable-site-isolation-trials").
MustLaunch()
page := rod.New().ControlURL(u).MustConnect().NoDefaultDevice().MustPage("https://dash.cloudflare.com/sign-up")
el, err := page.Timeout(30 * time.Second).Element(`div.cf-turnstile-wrapper`)
if err != nil {
panic(err)
}
el, err = el.ShadowRoot()
if err != nil {
panic(err)
}
_, err = el.Eval("()=>this.mode='open'")
if err != nil {
panic(err)
}
f := el.MustElement(`iframe[src*="https://challenges.cloudflare.com"]`).MustFrame()
fmt.Println(f.MustElement("#success").MustHTML())
} |
I tried using your code, but it failed to pass verification. When I manually click the verification checkbox, it spins for a few seconds and then returns to the unchecked state. This happens repeatedly, even after several attempts. |
@moxcomic Hi, if you look at the html structure of this page (https://dash.cloudflare.com/sign-up) now, you can see it has two #shadow-root in the iframe, while it had only one when I posted the code above. package main
import (
"fmt"
"time"
"github.com/go-rod/rod"
"github.com/go-rod/rod/lib/launcher"
"github.com/go-rod/rod/lib/utils"
)
func main() {
u := launcher.New().
Headless(false).
NoSandbox(true).
Set("disable-web-security").
Set("disable-site-isolation-trials").
MustLaunch()
page := rod.New().ControlURL(u).MustConnect().NoDefaultDevice().MustPage("https://dash.cloudflare.com/sign-up")
el, err := page.Timeout(30 * time.Second).Element(`div.cf-turnstile-wrapper`)
if err != nil {
panic(err)
}
el, err = el.ShadowRoot()
if err != nil {
panic(err)
}
_, err = el.Eval("()=>this.mode='open'")
if err != nil {
panic(err)
}
f := el.MustElement(`iframe[src*="https://challenges.cloudflare.com"]`).MustFrame()
utils.Sleep(10)
el, err = f.MustElement("body").ShadowRoot()
if err != nil {
panic(err)
}
_, err = el.Eval("()=>this.mode='open'")
if err != nil {
panic(err)
}
fmt.Println(el.MustElement("#success").MustHTML())
} However, 5 minutes later after I commented last time, I found the same repeating problem as you said and could not find any way to go. Sorry, the cloudflare thing is so disgusting, good luck to both of us. |
I found the problem. If |
@moxcomic maybe you should take this temp solution into consideration to avoid the runtime error. |
@Longdexin Thank you. I was using it normally before, but recently, after some changes in CF, I can no longer access the checkbox, and I haven't found a good solution yet. |
@moxcomic "同是天涯沦落人", 没错,就是最近两周出的问题,我现在基本就是手动勾选验证,先凑合着用,后面再研究研究解决办法,万恶的cloudflare!!! |
你这么一说倒是提醒到我了! page := rod.New().ControlURL(u).MustConnect().NoDefaultDevice().MustPage("https://dash.cloudflare.com/sign-up")
el, err := page.Timeout(30 * time.Second).Element(`div.cf-turnstile-wrapper`)
if err != nil {
panic(err)
}
<-time.After(time.Second * 10)
res, err := el.Eval(`() => {
const rect = this.getBoundingClientRect();
return { x: rect.left, y: rect.top };
}`)
if err != nil {
panic(err)
}
x, y := res.Value.Get("x").Num(), res.Value.Get("y").Num()
fmt.Println(x, y)
page.Mouse.MoveLinear(proto.NewPoint(x+27, y+29), 20)
// 使用JavaScript在鼠标位置添加一个视觉标记
page.Eval(`(x, y) => {
const marker = document.createElement('div');
marker.style.position = 'fixed'; // 使用fixed定位确保可见
marker.style.left = x + 'px';
marker.style.top = y + 'px';
marker.style.width = '10px';
marker.style.height = '10px';
marker.style.backgroundColor = 'red';
marker.style.borderRadius = '50%';
marker.style.zIndex = 9999; // 确保在最上层
document.body.appendChild(marker);
}`, x+27, y+29)
page.Mouse.MustClick(proto.InputMouseButtonLeft)
fmt.Println("click.") 我这么用能过验证(可能比手动好一点?) |
@alicazi did you solve the credit card filling issue? I also has the same issue when trying to access the iframe it get panic. |
有新的方法解决cloudflare吗,目前尝试这种方法也不行了 |
目前我这里今天还是正常使用,应该是其他问题吧,你网站发出来看看 |
https://nulled.to 当点击cloudflare checkbox时发现页面会同步刷新导致失效 |
@lsy88 这个网站不是reCaptcha的吗? 这也不是cloudflare啊 |
|
|
reCaptcha这是登录的页面,我是在用rod开无头浏览器爬的时候,触发的cloudflare,还没有进入首页 |
尝试用普通的Chrome浏览器,并删除掉rod自带的flags |
|
l := launcher.NewUserMode()
for _, v := range l.FormatArgs() {
l.Delete(flags.Flag(strings.Split(v, "=")[0][2:]))
}
l.
RemoteDebuggingPort(9222).
Set("no-first-run").
Set("no-default-browser-check").
Set("enable-privacy-sandbox-ads-apis", "false").
Leakless(true).
Headless(true).
Bin("/Applications/Google Chrome.app/Contents/MacOS/Google Chrome")
browser := rod.New().ControlURL(l.MustLaunch()).MustConnect().NoDefaultDevice()
page := browser.MustPage("https://www.nulled.to")
<-time.After(time.Second * 5)
page.MustScreenshotFullPage("./111.png") |
Rod Version: v0.102.0
The code to demonstrate your question
The code above will panic. It works fine on headless mode.
The text was updated successfully, but these errors were encountered: