Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run several feed fetches concurrently #703

Closed
Half-Shot opened this issue Apr 5, 2023 · 1 comment · Fixed by #779
Closed

Run several feed fetches concurrently #703

Half-Shot opened this issue Apr 5, 2023 · 1 comment · Fixed by #779
Labels
Feeds Involves the RSS feed service. T-Enhancement New features, changes in functionality, performance boosts, user-facing improvements.

Comments

@Half-Shot
Copy link
Contributor

We currently fetch feeds in a simple FIFO pattern but it's fairly dumb and so if you've got 10000 feeds, it's going to take a while. Also, a feed that just times out will clog up the whole process. We should instead run multiple feed checks concurrently.

A smarter solution would be to actually rank the responsiveness of feeds and the slow ones can be put into a slow queue, so as to not slow down the reasonably speedy ones.

@Half-Shot Half-Shot added T-Enhancement New features, changes in functionality, performance boosts, user-facing improvements. Feeds Involves the RSS feed service. labels Apr 5, 2023
@bkil
Copy link

bkil commented May 23, 2023

Related #765 #766

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feeds Involves the RSS feed service. T-Enhancement New features, changes in functionality, performance boosts, user-facing improvements.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants