Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow Downloads, could torrents be supported? #38

Open
MikeS159 opened this issue Apr 6, 2023 · 5 comments
Open

Slow Downloads, could torrents be supported? #38

MikeS159 opened this issue Apr 6, 2023 · 5 comments

Comments

@MikeS159
Copy link

MikeS159 commented Apr 6, 2023

Possibly a Cloudflare limitation rather than the tool. It seems to be capping out at about 60Mb/s.
Are torrent files still made available and if so could the tool provide a magnet link (or similar) to the most up to date torrent?

@troyhunt
Copy link
Contributor

troyhunt commented Apr 6, 2023

@stebet, have you observed this? I don't think we've had reports of bandwidth throttling before and I'm pretty sure that's not happening at Cloudflare.

Torrents are permanently gone, they became way too out of date as the monolithic downloads aren't being updated any more.

@filimonic
Copy link

filimonic commented Jan 31, 2024

Same here, it takes around 1 day to download ntlm database in 32 threads. Bandwidth drops from 60 Mbps to 3 Mbps. ISP does not limit anything

@filimonic
Copy link

IMO it will be better to make wider ranges supported by API, ex, making 4 chars instead of 5

@GGG-KILLER
Copy link

GGG-KILLER commented Mar 17, 2024

Don't have an entire day's ETA but I do have an entire hour's ETA:
image
I've compiled this locally using dotnet publish -r linux-x64 -c Release -f net8.0 -o publish.

I have a 1 Gbps down 500 Mbps up connection and am not being limited by neither disk (NVMe), CPU nor network
image
Unsure if this is expected at all as there's no comment on download times anywhere.

Final result of the download:

Finished downloading all hash ranges in 3,873,525ms (270.70 hashes per second).
We made 1,048,576 Cloudflare requests (avg response time: 38.91ms). Of those, Cloudflare had already cached 1,048,576 requests, and made 0 requests to the Have I Been Pwned origin server.

Took a total of 65 minutes and 26.3 seconds.

@stebet
Copy link
Contributor

stebet commented Mar 18, 2024

You can try with more threads using the -p switch, for example -p 64. That can help if you have plenty of bandwidth.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants