Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

850 avoid search by id before every server mode download #916

Conversation

jlahovnik
Copy link
Collaborator

@jlahovnik jlahovnik commented Nov 8, 2023

  • download info is stored in instance of EODataAccessGateway with key composed of product type, provider and id
  • if download is attempted for product where id is not in download info and search by id is available for provider and product type, it will be executed; if search by id is not possible an error will be raised to do a search

@jlahovnik jlahovnik linked an issue Nov 8, 2023 that may be closed by this pull request
@jlahovnik jlahovnik self-assigned this Nov 8, 2023
Copy link
Contributor

github-actions bot commented Nov 8, 2023

Test Results

    4 files  ±0      4 suites  ±0   4m 3s ⏱️ ±0s
  442 tests +1    439 ✅ +1   3 💤 ±0  0 ❌ ±0 
1 768 runs  +4  1 688 ✅ +2  80 💤 +2  0 ❌ ±0 

Results for commit 49974c0. ± Comparison against base commit 9b95224.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Nov 8, 2023

Code Coverage (Ubuntu)

File Coverage
All files 82%

Minimum allowed coverage is 70%

Generated by 🐒 cobertura-action against 49974c0

Copy link
Contributor

github-actions bot commented Nov 8, 2023

Code Coverage (Windows)

File Coverage
All files 76%

Minimum allowed coverage is 70%

Generated by 🐒 cobertura-action against 49974c0

@alambare alambare marked this pull request as draft November 16, 2023 16:36
@jlahovnik jlahovnik force-pushed the 850-avoid-search-by-id-before-every-server-mode-download branch 2 times, most recently from 83d5e2c to 0e981c2 Compare December 6, 2023 12:42
@jlahovnik
Copy link
Collaborator Author

updated based on input from @alambare-csgroup

@jlahovnik jlahovnik marked this pull request as ready for review December 6, 2023 15:46
@@ -185,6 +185,7 @@ def __init__(
os.path.join(self.conf_dir, "shp"),
)
self.set_locations_conf(locations_conf_path)
self.download_info = {}
Copy link
Collaborator

@sbrunato sbrunato Dec 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please store full SearchResult objects in a _search_results_cache collections.dequeue() having maxlen default to 100 (stored in a new constant) and customizable using an environment variable like EODAG_SEARCH_RESULTS_CACHE_MAXLEN.

Please also put _search_results_cache in rest.utils as it should only concern server-mode downloads.

To find a matching product, loop over the cache starting with the most recent result. (check at first only provider and product type of the first product of each search result)

@sbrunato sbrunato marked this pull request as draft December 21, 2023 08:01
@jlahovnik jlahovnik force-pushed the 850-avoid-search-by-id-before-every-server-mode-download branch from c771dec to f59180a Compare January 4, 2024 08:40
@jlahovnik jlahovnik closed this Feb 6, 2024
@jlahovnik
Copy link
Collaborator Author

closed because change requests were implemented on new branch -> new PR: #1012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

avoid search by id before every server mode download
2 participants