Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download: Move extra channel metadata import to background #650

Closed
wants to merge 1 commit into from

Conversation

dbnicholson
Copy link
Member

Like the unavailable content thumbnails, move the download of the extra channel metadata to the background instead of blocking the initial starter pack download on it.

I'm marking this as a draft for now since we need some frontend work so it can be notified when new channels appear.

Helps: #592

@dbnicholson dbnicholson marked this pull request as draft June 20, 2023 22:32
Like the unavailable content thumbnails, move the download of the extra
channel metadata to the background instead of blocking the initial
starter pack download on it.

Helps: #592
@manuq manuq force-pushed the all-channels-background branch from 44f9934 to 0e3d699 Compare June 21, 2023 11:58
@dbnicholson
Copy link
Member Author

As discussed on Slack, this isn't going to work. The way this PR works is that the extra channel and thumbnail tasks are all created at once and put into a background queue. However, the thumbnail tasks can't be created until the channel metadata exists. There are 3 ways I can think to handle this:

  1. Leave the extra channel download in the foreground. I.e., do nothing.
  2. Download all the channels in the foreground immediately. This is the same as 1 except the channel metadata downloads aren't split into 2 stages.
  3. Have the frontend keep driving the stages even after it's left the download page. This would kinda weird since every page would need to keep polling the download manager.
  4. Have the backend drive the background stages by using a StorageHook that's notified when tasks complete. That would allow it to do something similar to how the foreground tasks are handled. I.e., a background_tasks_pending list that needs to be exhausted before moving on to the next background stage. Or you could go all the way and use this hook to drive all the stages and then the frontend polling would purely return status without also trying to manage the state machine.

@dbnicholson
Copy link
Member Author

I'm closing this since it's not going to ever work this way.

@dbnicholson dbnicholson deleted the all-channels-background branch August 16, 2023 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant