Run several feed fetches concurrently #703
Labels
Feeds
Involves the RSS feed service.
T-Enhancement
New features, changes in functionality, performance boosts, user-facing improvements.
We currently fetch feeds in a simple FIFO pattern but it's fairly dumb and so if you've got 10000 feeds, it's going to take a while. Also, a feed that just times out will clog up the whole process. We should instead run multiple feed checks concurrently.
A smarter solution would be to actually rank the responsiveness of feeds and the slow ones can be put into a slow queue, so as to not slow down the reasonably speedy ones.
The text was updated successfully, but these errors were encountered: