Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance enhancement #87

Open
2516455367 opened this issue Nov 18, 2024 · 9 comments
Open

Performance enhancement #87

2516455367 opened this issue Nov 18, 2024 · 9 comments
Labels
enhancement New feature or request

Comments

@2516455367
Copy link

1: Starting the browser every time is much slower than the normal playwright tool.2. I hope to speed up the startup.3: The memory usage is quite large, three or four hundred, and the headless mode is two or three hundred. The normal browser is only over one hundred.

@2516455367 2516455367 added the enhancement New feature or request label Nov 18, 2024
@FayzulSaimun
Copy link

Yeah, the memory use is huge while starting the browser. Smaller VPS instances sometimes hang.

@daijro
Copy link
Owner

daijro commented Nov 19, 2024

Thanks for reporting. I ran a few tests in Docker with Playwright under the following websites:

URL Firefox Camoufox Camoufox (uBO)
about:blank 288 MB 339 MB 354 MB
https://google.com 303 MB 373 MB 362 MB
https://yahoo.com 485 MB 497 MB 456 MB

There does seem to be a memory difference. This could either be caused by some of Playwright's config tweaks being undone to prevent detection (for example, fission and web content isolation is enabled in Camoufox to prevent WAF suspicion). uBlock Origin also seems to cause a bit of a memory issue, but it helps alleviate the memory usage in some cases like yahoo.com.

However, Camoufox is also configured with memory-saving tweaks like dropping a page's memory cache as soon as the page is left or tab is discarded, which should prevent the memory from going up exponentially when navigating through several pages.

I will look into what else that can be killed/stripped from Firefox to help the memory that won't be noticeable through JS inspection.

@oryondark
Copy link

I �faced the same issue.

The problem I'm facing is that when performing JS based scroll down on pages with many image resources,
but memory usage increases excessively.

While monitoring,
I figured out that camoufox-bin has a very high oom kill score.
Currently, I'm using an AWS t3.large instance with 8GB of memory, and it often exceeds this limit.

@daijro
Copy link
Owner

daijro commented Nov 19, 2024

I �faced the same issue.

The problem I'm facing is that when performing JS based scroll down on pages with many image resources, but memory usage increases excessively.

While monitoring, I figured out that camoufox-bin has a very high oom kill score. Currently, I'm using an AWS t3.large instance with 8GB of memory, and it often exceeds this limit.

I think this setting imported from Fastfox is causing the issue:

defaultPref("image.mem.decode_bytes_at_a_time", 32768)

Try setting it to 8192 or 4096. This fix will be included in the latest release.

@daijro daijro closed this as completed in 1e8e667 Nov 19, 2024
@daijro daijro reopened this Nov 19, 2024
@daijro
Copy link
Owner

daijro commented Nov 19, 2024

Didn't mean to close this issue, GitHub saw "fixes #87" in the commit message as a signal to close

@2516455367
Copy link
Author

我遇到了同样的问题。
我面临的问题是,在具有许多图像资源的页面上执行基于 JS 的向下滚动时,内存使用量会过度增加。
在监控过程中,我发现 camoufox-bin 的 oom kill 分数非常高。目前,我正在使用具有 8GB 内存的 AWS t3.large 实例,它经常超出此限制。

我认为从 Fastfox 导入的这个设置导致了此问题:

defaultPref("image.mem.decode_bytes_at_a_time", 32768)

尝试将其设置为81924096。此修复将包含在最新版本中。

Has the latest version been improved?

@daijro
Copy link
Owner

daijro commented Nov 19, 2024

Has the latest version been improved?

The latest version should be out within the next day. 👍

@NCGSolutions
Copy link

Thanks for reporting. I ran a few tests in Docker with Playwright under the following websites:

URL Firefox Camoufox Camoufox (uBO)
about:blank 288 MB 339 MB 354 MB
https://google.com 303 MB 373 MB 362 MB
https://yahoo.com 485 MB 497 MB 456 MB
There does seem to be a memory difference. This could either be caused by some of Playwright's config tweaks being undone to prevent detection (for example, fission and web content isolation is enabled in Camoufox to prevent WAF suspicion). uBlock Origin also seems to cause a bit of a memory issue, but it helps alleviate the memory usage in some cases like yahoo.com.

However, Camoufox is also configured with memory-saving tweaks like dropping a page's memory cache as soon as the page is left or tab is discarded, which should prevent the memory from going up exponentially when navigating through several pages.

I will look into what else that can be killed/stripped from Firefox to help the memory that won't be noticeable through JS inspection.

Out of curiosity how are you measuring the memory usage. And do you have a script available? I seem to be getting much higher results.

daijro added a commit that referenced this issue Dec 1, 2024
@daijro
Copy link
Owner

daijro commented Dec 1, 2024

Out of curiosity how are you measuring the memory usage. And do you have a script available? I seem to be getting much higher results.

@NCGSolutions I have made my memory benchmark available under the scripts/benchmark directory.

It takes the average memory usage over 10 seconds on about:black, Google, and Yahoo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

5 participants