Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce Franchise-level crawlers #112

Open
fcamblor opened this issue Nov 10, 2024 · 0 comments
Open

Introduce Franchise-level crawlers #112

fcamblor opened this issue Nov 10, 2024 · 0 comments

Comments

@fcamblor
Copy link
Collaborator

Franchises like Devoxx/Voxxed which have a lot of yearly recurring conferences can't be autonomous when it comes to declaring their event in Voxxrin Crawlers (for each event, they need to rely on a Voxxrin Administrators to declare a new crawler entry)

Idea of this is to introduce a new family-crawlers-provider firestore collection which is declared once for all per franchise by Voxxrin Administrators.

Having an entry on this collection will allow to call a new POST /api/family-crawlers-providers/{{familyCrawlersProviderId}}/requestFamilyCrawlersRefresh endpoint (secured by a familyOrganizer-type token)
(a daily CRON might also automatically call this endpoint)

This endpoint will:

  • Rely on /family-crawlers-provider/:familyCrawlersProviderId descriptor firestore entry
  • This firestore document will provide an implementation type allowing to tell which family crawler provider implementation to take
  • Devoxx/Voxxed implementation will look for events declared on https://www.devoxxians.com/api/public/events/future/devoxx endpoint and, for each of them, will create (if not already exists) a new /crawlers/{crawlerId}
  • /crawlers/{crawlerId} entries should now have a new metadata field, allowing to pass some specific data to the crawler (typically, to avoid having to create a new JSON descriptor for every events if we consider composable descriptors Introduce Crawler descriptors composition #111)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant