Getting ratelimited #3560

Koyamie · 2019-11-01T00:24:46Z

Please describe the problem you are having in as much detail as possible:
I am getting ratelimited by discord for an unknown reason
As said here it should never happen but I do not know why this is happening for me

discord.js/src/rest/RequestHandler.js

Line 149 in 6be5051

// A ratelimit was hit - this should never happen

For more information, it's happening frequently at 00:00 GMT

I am also having a lot of 500 errors when this is happening.

Include a reproducible code sample here, if possible:

429 hit on route /gateway/bot

Code : 500
AbortError: The user aborted a request.
at RequestHandler.execute (/home/koyamie/Koya/node_modules/discord.js/src/rest/RequestHandler.js:106:9)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (internal/process/task_queues.js:93:5)

Further details:

discord.js version: lastest master
Node.js version: 12.13.0
Operating system: debian 9
Priority this issue should have – please be realistic and elaborate if possible:

I have also tested the issue on latest master, commit hash:

Koyamie · 2019-11-03T00:20:02Z

Hello, kinda high issue, happening everyday since I updated the repo (was on a september commit)

Fyko · 2019-11-03T01:45:47Z

Is it possible you hit the max of 1000 logins per day? Have you tried listening on the debug event?

Koyamie · 2019-11-03T01:51:05Z

Is it possible you hit the max of 1000 logins per day? Have you tried listening on the debug event?

Nope, when I am hitting 1k logins a day its different, I can't login for 24 hours and debugging tells how many time I have to wait, here I am getting a 429 "Too many requests" and debugging only shows this :
"429 hit on route /gateway/bot"

monbrey · 2019-11-03T06:45:51Z

If you need help with discord.js installation or usage, please go to the discord.js Discord server instead: https://discord.gg/bRCvFy9
This issue tracker is only for bug reports and enhancement suggestions.

Since this is happening at exactly 0:00 it would seem there's some scheduled task running at this time causing it. We can investigate further on Discord.

Koyamie · 2019-11-03T12:25:37Z

If you need help with discord.js installation or usage, please go to the discord.js Discord server instead: https://discord.gg/bRCvFy9
This issue tracker is only for bug reports and enhancement suggestions.

Since this is happening at exactly 0:00 it would seem there's some scheduled task running at this time causing it. We can investigate further on Discord.

Yeah but, as the line is saying "This should never happen" I guess this is related to the lib.

DevYukine · 2019-11-03T12:59:55Z

i could imagine that this might be related to multiple Shards/Clients fetching the /bot/gateway endpoint at the same time and since they dont know of each other they do too many requests and hit the ratelimit, are you sure you dont fetch the /bot/gateway endpoint on your own? because d.js should only ever fetch that on startup and i can't imagine that your bot restarts everytime on 0:00 GMT

Koyamie · 2019-11-03T13:05:27Z

i could imagine that this might be related to multiple Shards/Clients fetching the /bot/gateway endpoint at the same time and since they dont know of each other they do too many requests and hit the ratelimit, are you sure you dont fetch the /bot/gateway endpoint on your own? because d.js should only ever fetch that on startup and i can't imagine that your bot restarts everytime on 0:00 GMT

I think too but I have no reason fetching this endpoint on my own.
Yup my bot isn't restarting everytime but at 0:00 GMT my game timestamps reset (like the tatsumaki daily and reputation) but there is no reason that it fetch this endpoint 🤔
This is weird

DevYukine · 2019-11-03T13:24:29Z

Okay i just looked in the src and saw that d.js actuall does call the /bot/gateway endpoint everytime it reconnects (this shouldn't be like that) and since you mentioned that you get these every day at 0:00 GMT this probaly means that your bot reconnects every day at that time (maybe related to your host?) sadly d.js does always fetch the session endpoint even when resuming a session but vlad actuall fixed that in #3393 so you could give this pr a try since it should fix your issue (he changed it so it only fetches /gateway/bot if no session_id id present).

Also you should probaly check why the bot reconnects everytime at 00:00 GMT, do you maybe change your ip or does your isp/host do that? i would recommend looking into that.

Even tho #3393 probaly fixes this issue i will open another one because the current implementation is very poor designed for bots using any ShardingManager and this needs to be changed since we currently do not handle ratelimits for /gateway/bot across shards.

Koyamie · 2019-11-03T13:34:20Z

Okay i just looked in the src and saw that d.js actuall does call the /bot/gateway endpoint everytime it reconnects (this shouldn't be like that) and since you mentioned that you get these every day at 0:00 GMT this probaly means that your bot reconnects every day at that time (maybe related to your host?) sadly d.js does always fetch the session endpoint even when resuming a session but vlad actuall fixed that in #3393 so you could give this pr a try since it should fix your issue (he changed it so it only fetches /gateway/bot if no session_id id present).

Also you should probaly check why the bot reconnects everytime at 00:00 GMT, do you maybe change your ip or does your isp/host do that? i would recommend looking into that.

Even tho #3393 probaly fixes this issue i will open another one because the current implementation is very poor designed for bots using any ShardingManager and this needs to be changed since we currently do not handle ratelimits for /gateway/bot across shards.

This is was I thought, I will give a try to this PR and check tonight if it is happening again and will update the issue. My bot is not reconnecting at 0:00 this is why I do not understand 😯

DevYukine · 2019-11-03T13:37:32Z

Okay i just looked in the src and saw that d.js actuall does call the /bot/gateway endpoint everytime it reconnects (this shouldn't be like that) and since you mentioned that you get these every day at 0:00 GMT this probaly means that your bot reconnects every day at that time (maybe related to your host?) sadly d.js does always fetch the session endpoint even when resuming a session but vlad actuall fixed that in #3393 so you could give this pr a try since it should fix your issue (he changed it so it only fetches /gateway/bot if no session_id id present).
Also you should probaly check why the bot reconnects everytime at 00:00 GMT, do you maybe change your ip or does your isp/host do that? i would recommend looking into that.
Even tho #3393 probaly fixes this issue i will open another one because the current implementation is very poor designed for bots using any ShardingManager and this needs to be changed since we currently do not handle ratelimits for /gateway/bot across shards.

This is was I thought, I will give a try to this PR and check tonight if it is happening again and will update the issue. My bot is not reconnecting at 0:00 this is why I do not understand 😯

Are you sure about this? because it very much looks like it (reconnecting does not only include re-identifying but also resuming a session) can you make sure and listen to the shardReconnecting event (e.g log it to the console or somehow notify yourself) and update this post once you tested it?

Koyamie · 2019-11-04T08:40:11Z

Update : I started debugging tonight to see what happen at 0:00 GMT, I found what in my bot was getting me ratelimited for 1h.

I was hitting 429 very very many times but with different enpoints, the most I saw was /channels/ID/messages this one should be fixed in my code but still hitting 429 even if it should never happen.

I havn't tried the PR because I got no time yesterday.

Anyway, I'll keep this issue open until we get a fix on master.

iCrawl · 2019-11-04T09:34:35Z

You are being very vague about this.

/channels/Id/messages, doesn't mean a lot. This needs more information on your part, maybe even some code samples of what your usage is and why you are hitting it.

It would be nice if this is getting a bit more context.

DevYukine · 2019-11-04T10:34:14Z

If you get ratelimited on other routes e.g /channels/ID/messages you might hit the Global Ratelimit and then hitting a 429 is normal but then there is still the question why you hit /gateway/bot aswell at that time 🤔

Koyamie · 2019-11-04T10:36:13Z

You are being very vague about this.

/channels/Id/messages, doesn't mean a lot. This needs more information on your part, maybe even some code samples of what your usage is and why you are hitting it.

It would be nice if this is getting a bit more context.

Yes very sorry will explain better as soon as I can

If you get ratelimited on other routes e.g /channels/ID/messages you might hit the Global Ratelimit and then hitting a 429 is normal but then there is still the question why you hit /gateway/bot aswell at that time 🤔

Maybe hitting it too much result to a gateway ban for 1 hour (Can't connect for 1 hour everytime it happens)

Koyamie · 2019-12-01T20:29:59Z

Hello, this is happening today but in different way, restarted my bot after making changes and now bot is having very difficulties to connect and I am hitting 429 way too much (I should never hit 429 remember)
Any help ?

Could be a possible API issue (https://status.discordapp.com/) but still, I should not hit 429

Koyamie · 2020-01-02T23:14:40Z

Hello, it happened again tonight with discord increased API response time. Any news for this ?

Koyamie · 2020-03-03T23:16:05Z

UP

ghost · 2020-05-14T00:04:41Z

Having the same issue here now.
429 hit on route /gateway/bot

Have 0 idea this has never happened before.

very frustrating

vladfrangu · 2020-05-14T00:15:50Z

If it happens once or a few times and you're sharded across multiple processes, it happens. If it's spammed, it's an issue.

d.js version, node.js version please!

ghost · 2020-05-14T01:12:10Z

Discord.js ^12.2.0
Node.js 11.x

It isnt spammed, just happens every minute or so.

monbrey · 2020-05-14T01:35:12Z

Discord.js ^12.2.0
Node.js 11.x

Discord.js v12 has a minimum Node version of v12 also. This doesn't line up

ghost · 2020-05-14T02:38:11Z

it worked for 3 months originally?

SudhanPlayz · 2020-05-14T15:25:39Z

Even me bot the problem. But i was just connected like 10 times

429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot

Node Engine: 12.x
Discord.js: ^12.2.0

SudhanPlayz · 2020-05-19T08:13:21Z

My project is not working on Any Bot Tokens Pls help everybots getting strucked at

Provided token: NzEwODE2MTI3MTA5OTU1NjA1.XsOUQw.***************************
Preparing to connect to the gateway...
429 hit on route /gateway/bot

I Don't what I want to do. also I just logging one time in my project

Koyamie · 2020-05-19T08:14:50Z

My project is not working on Any Bot Tokens Pls help everybots getting strucked at
Provided token: NzEwODE2MTI3MTA5OTU1NjA1.XsOUQw.***************************
Preparing to connect to the gateway...
429 hit on route /gateway/bot
I Don't what I want to do. also I just logging one time in my project

You are IP banned for 1 hour, just wait and retry.

DjDeveloperr · 2020-09-15T04:49:09Z

I'm pretty late here - I had found the cause lol. It was because of webhooks, which sent "logs" so fast and their rate limits were not handled.

For @DjDeveloperr, since I noticed you modified the code, can you on 429's log the via header from res.headers? Discord replies will always have that header (usually with the content 1.1 google; Cloudflare bans don't have that).. We currently don't handle the difference between Clouflare and Discord bans, which may cause this issue

It doesn't happen anymore.

Unless you changed the REST api version to v8 yourself via client options, Discord returns the header as ms instead of seconds right now on the API version we support and use (v7)..

I didn't change, but I had to multiply the retryAfter with 1000 for avoiding the additional spam. :/

vladfrangu · 2020-09-15T10:41:28Z

I didn't change, but I had to multiply the retryAfter with 1000 for avoiding the additional spam. :/

That's most likely because the API ban comes from Cloudflare (or so I assume)... Hence why the multiply by 1000 worked; this is why I asked if you can log the Via header 😃

Koyamie · 2020-09-15T16:56:45Z

Yes, the API ban is coming from Cloudflare, we also call it a CF ban.

For my case, I will explain how it was happening:

My bot was hitting the 50req/s limit set by Discord, this was causing a Global Ratelimit. The shard that was receiving the ratelimit was correclty handling it but other ones could not, this is how I was getting massive 429's (with 320 shards means a lot) because they were still sending request while being ratelimited.

When this was happening it could solve itself after few seconds and was hitting around 300-400 429's but sometimes it was reaching the 10k limit in few seconds.

The last few weeks, it was happening 1 time a day and sometimes 2.

DjDeveloperr · 2020-09-16T06:19:49Z

I asked if you can log the Via header 😃

Sorry, but since I removed the webhook logs Rate limit didn't get hit since then, I assume its fixed now. It was a problem on my end.

Koyamie · 2020-09-16T11:26:40Z

I can confirm that using webhooks to logs creates high rate of 429's.

Im reaching around 100 ratelimits in past 10 minutes when I enable them.

Deivu · 2020-12-24T12:07:45Z

I got banned 2x this day thanks to this 429 at gateway/bot. I disabled webhook logs, still getting cloudflare banned.
To solve this, I tried the code snippet by @DjDeveloperr like so

const body = await res.json();
this.manager.client.emit('debug', `429 hit on route ${request.route} | ${this.retryAfter} idk if this ms or seconds`);
await Util.delayFor(this.retryAfter * (body?.message?.includes("You are being blocked") ? 1000 : 1));

I've been down for a bit of time, around 4 hours already due to this issue. This issue could potentially prevent big bots from logging in 👀

WilcoSp · 2021-02-07T23:44:56Z

I also got a rate limit and only discovered it via the debug eventemitter, it would be nice if the "ratelimit" event would really emit when 429 was given as a response

atomheartother · 2021-02-10T23:50:05Z

My bot is down for hours at a time because of this issue, I'm running 7 shards and getting rate-limited almost immediately on boot. I don't understand why it suddenly started getting rate-limited a week ago, and again today, is this really on Discord's end?

Also very unhelpful, ShardingManager appears to just throw an unhandledexception when this occurs, it does not display any sort of 429 error, which made this extremely confusing to debug...

polypixeldev · 2021-08-01T20:56:15Z

My bot is down for hours at a time because of this issue, I'm running 7 shards and getting rate-limited almost immediately on boot. I don't understand why it suddenly started getting rate-limited a week ago, and again today, is this really on Discord's end?

Also very unhelpful, ShardingManager appears to just throw an unhandledexception when this occurs, it does not display any sort of 429 error, which made this extremely confusing to debug...

Is this still an issue? If so, what discord.js/node.js version?

Muh9049 · 2021-08-06T19:51:57Z

Is this still an issue? If so, what discord.js/node.js version?

Recently just got the 429 error. I have no idea what caused it. It could be because of the webhooks for errors my bot uses, or maybe it was using the .fetch method too often when fetching a channel (?).

I'm going to try and restart the bot in about an hour to see if it goes away or not by then. Also going to listen to the rateLimit and probably the debug event to see what initially causes this to avoid this in the future.

Jiralite · 2022-11-02T11:30:25Z

Closing this as it is a stale issue. Feel free to make a new issue if this problem still persists.

Koyamie added s: unverified labels Nov 1, 2019

SpaceEEC added the t: sharding label Feb 29, 2020

MattIPv4 mentioned this issue Mar 3, 2020

client.login has no ratelimit backoff #3890

Closed

iCrawl mentioned this issue Mar 25, 2020

My bot can't start because he is getting ratelimited #3991

Closed

1 task

vladfrangu added t: ratelimits and removed t: sharding labels Sep 15, 2020

DjDeveloperr mentioned this issue Oct 10, 2020

rateLimit event not fired for ratelimit received during login #3891

Closed

atomheartother mentioned this issue Feb 10, 2021

Bot unable to login for 24hrs #5315

Closed

github-actions bot removed t: REST labels Aug 7, 2021

iCrawl added bug ratelimits sharding REST labels Aug 7, 2021

kyranet mentioned this issue Dec 3, 2021

feat: added @discordjs/sharder module discordjs/discord.js-modules#87

Draft

13 tasks

sebasptsch mentioned this issue Dec 19, 2021

Sharding DynamicaBot/Dynamica#20

Closed

iCrawl added the packages:discord.js label Jan 7, 2022

kyranet mentioned this issue Jan 8, 2022

feat: added @discordjs/sharder package #7204

Closed

13 tasks

Jiralite closed this as not planned Won't fix, can't repro, duplicate, stale Nov 2, 2022

github-actions bot locked as resolved and limited conversation to collaborators Nov 2, 2023

Getting ratelimited #3560

Getting ratelimited #3560

Comments

Koyamie commented Nov 1, 2019

Koyamie commented Nov 3, 2019

Fyko commented Nov 3, 2019

Koyamie commented Nov 3, 2019

monbrey commented Nov 3, 2019 • edited Loading

Koyamie commented Nov 3, 2019

DevYukine commented Nov 3, 2019

Koyamie commented Nov 3, 2019

DevYukine commented Nov 3, 2019

Koyamie commented Nov 3, 2019

DevYukine commented Nov 3, 2019

Koyamie commented Nov 4, 2019

iCrawl commented Nov 4, 2019

DevYukine commented Nov 4, 2019

Koyamie commented Nov 4, 2019

Koyamie commented Dec 1, 2019 • edited Loading

Koyamie commented Jan 2, 2020

Koyamie commented Mar 3, 2020

ghost commented May 14, 2020

vladfrangu commented May 14, 2020

ghost commented May 14, 2020

monbrey commented May 14, 2020

ghost commented May 14, 2020

SudhanPlayz commented May 14, 2020 • edited Loading

SudhanPlayz commented May 19, 2020

Koyamie commented May 19, 2020

DjDeveloperr commented Sep 15, 2020 • edited Loading

vladfrangu commented Sep 15, 2020 • edited Loading

Koyamie commented Sep 15, 2020

DjDeveloperr commented Sep 16, 2020

Koyamie commented Sep 16, 2020

Deivu commented Dec 24, 2020

WilcoSp commented Feb 7, 2021

atomheartother commented Feb 10, 2021

polypixeldev commented Aug 1, 2021

Muh9049 commented Aug 6, 2021 • edited Loading

Jiralite commented Nov 2, 2022

monbrey commented Nov 3, 2019 •

edited

Loading

Koyamie commented Dec 1, 2019 •

edited

Loading

SudhanPlayz commented May 14, 2020 •

edited

Loading

DjDeveloperr commented Sep 15, 2020 •

edited

Loading

vladfrangu commented Sep 15, 2020 •

edited

Loading

Muh9049 commented Aug 6, 2021 •

edited

Loading