Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

archive_depth only set for PEPS and sobloo #266

Closed
maximlt opened this issue Apr 27, 2021 · 3 comments · Fixed by #746
Closed

archive_depth only set for PEPS and sobloo #266

maximlt opened this issue Apr 27, 2021 · 3 comments · Fixed by #746
Assignees
Labels
enhancement New feature or request

Comments

@maximlt
Copy link
Collaborator

maximlt commented Apr 27, 2021

plugins.download.base.Download._finalize takes care of extracting a product if required and returns the path to the product. There is a mechanism to inspect

There is a mechanism to handle which path to return when the product is extracted:

# Handle depth levels in the product archive. For example, if the downloaded archive was
# extracted to: /top_level/product_base_dir and archive_depth was configured to 2, the product
# location will be /top_level/product_base_dir.
# WARNING: A strong assumption is made here: there is only one subdirectory per level
archive_depth = getattr(self.config, "archive_depth", 1)
count = 1
while count < archive_depth:
product_path = os.path.join(product_path, os.listdir(product_path)[0])
count += 1
return product_path

PEPS and sobloo have archive_depth: 2 in their config, which means that the path returned after extraction is outputs_prefix/product_title/subdirectory.

I guess it needs to be set for some other providers, e.g. onda, theia, creodias (those that make use of the HTTPDownload plugin).

(I saw that while updating eodag-sentinelsat.)

@maximlt maximlt added the bug Something isn't working label Apr 27, 2021
@maximlt
Copy link
Collaborator Author

maximlt commented Apr 27, 2021

As the comment says # WARNING: A strong assumption is made here: there is only one subdirectory per level , I'm not sure whether this mechanism is robust to all the products made available by these providers.

@sbrunato sbrunato added enhancement New feature or request and removed bug Something isn't working labels Apr 28, 2021
@aoyono
Copy link
Contributor

aoyono commented Jul 9, 2021

See the following comment

@sbrunato
Copy link
Collaborator

See also the poll on #446 to vote for the output folder pattern that you prefer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants