Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UDP support (/process_graphs) #90

Closed
soxofaan opened this issue Jan 16, 2023 · 4 comments
Closed

UDP support (/process_graphs) #90

soxofaan opened this issue Jan 16, 2023 · 4 comments

Comments

@soxofaan
Copy link
Member

soxofaan commented Jan 16, 2023

Add support for UDPs (GET /process_graphs and related).

Questions:

  • where to store? at level of aggregator, or "highest prio" upstream back-end, or all backends?
  • what to list? only UDPs defined at aggregator or combine multiple upstream back-ends?
  • how to handle job with UDP usage if UDPs are stored only at level of aggregator? Automatically store UDP in upstream back-end first, or rewrite "UDP calls" in original process graph with actual UDP implementation?
@m-mohr
Copy link
Member

m-mohr commented Jan 16, 2023

The details have to be discussed, but the most important thing is that files and user-defined processes must be accessible by all back-ends in the federation because otherwise, users can't re-use them properly in other processes. User shouldn't need to care about where something is available and where it's not available.

@Patrick1G
Copy link

@m-mohr @soxofaan @jdries
UPD support by the aggregator is important for the platform part!
Could we add a task as part of SAP09 Operationalization on this?

soxofaan added a commit that referenced this issue Sep 13, 2023
Just forward to "first" upstream backend
soxofaan added a commit to Open-EO/openeo-python-driver that referenced this issue Sep 13, 2023
and only include non-standard "public" field when True

related to Open-EO/openeo-aggregator#90
soxofaan added a commit that referenced this issue Sep 13, 2023
Just forward to "first" upstream backend
@soxofaan
Copy link
Member Author

soxofaan commented Sep 13, 2023

We just added very basic UDP support to the aggregator (just on dev instance openeocloud-dev.vito.be at the moment):
UDP interactions are just proxied directly to VITO back-end: creating, getting, listing UDPs on the aggregator goes directly to VITO UDP storage.

This is not ideal because there is no federation aspect to this, but at least the basics are working for majority of processing requests (which go to VITO anyway).

We just decided this was an easy first step to have basic functionality in place

@soxofaan
Copy link
Member Author

basic implementation is available, will close this ticket.

Follow-up is for https://github.com/openEOPlatform/architecture-docs/issues/368

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants