Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace Argo Events stack with Provider Service component #391

Draft
wants to merge 15 commits into
base: master
Choose a base branch
from

Conversation

alexgeorgousis
Copy link
Contributor

@alexgeorgousis alexgeorgousis commented Nov 5, 2024

Closes #359

Tasks

  • Remove old VAI eventing server code (argo/providers/vai/eventing_server.go and argo/providers/vai/provider.go)
  • Remove old KFP eventing server code
  • Remove EventingServer interface layer along with CLI layer for eventsource-server from argo module
  • Integrated tests (possibly with docker-compose spinning up pubsub for VAI and the equivalent for KFP)
  • Implement streaming solution for both VAI and KFP to consume from their source and push run completion event data to the HTTP webhook in the operator
  • Documentation updated

@peter-hazell peter-hazell force-pushed the event-processor branch 4 times, most recently from c6b732b to 2b3d6a3 Compare November 7, 2024 16:06
@alexgeorgousis alexgeorgousis changed the title Event processor Replace Argo Events stack with Event Processor component Nov 7, 2024
@alexgeorgousis alexgeorgousis changed the title Replace Argo Events stack with Event Processor component Replace Argo Events stack with Provider Service component Nov 7, 2024
Co-Authored-By: Peter Hazell <[email protected]>
Co-Authored-By: Alex Georgousis <[email protected]>
Co-Authored-By: Toby Pinfold <[email protected]>
Co-Authored-By: Ian <[email protected]>
@alexgeorgousis
Copy link
Contributor Author

I have a question about the way we've implemented the event processing for providers.

Up until this point, the steps to introduce a new provider (I think) were:

  1. Build a publish a docker image that handles CRUD for resources in the new provider and handles events coming out of the new provider
  2. Deploy a new Provider resource with that provider image specified

What we've basically done in this PR is move the event processing logic for the existing providers to a separate service.

So as far as I can tell, the steps are now:

  1. Build a publish a docker image that handles CRUD for resources in the new provider
  2. Deploy a new Provider resource with that provider image specified
  3. Raise a PR in the KFP Operator repo to extend the provider-service module with eventing logic for the new provider

Questions:

  1. Am I missing something?
  2. If not, do we think this is a problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Event Processor (remove argo events from provider) (Step 4)
3 participants