Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nested files .DIM and .TIF for SPOT constellations #145

Closed
jteulade opened this issue May 15, 2024 · 1 comment · Fixed by #146
Closed

Nested files .DIM and .TIF for SPOT constellations #145

jteulade opened this issue May 15, 2024 · 1 comment · Fixed by #146
Labels
bug Something isn't working

Comments

@jteulade
Copy link
Contributor

jteulade commented May 15, 2024

Files METADATA.DIM and IMAGERY.TIF can be found at any level according to this
But here, we are looking only at the root of the product.
That's why, we get the following error:

Traceback (most recent call last):
  File "e:\jteulade\repositories\eoreader\eoreader\products\optical\vhr_product.py", line 516, in _get_path
    prod_path = next(self.path.glob(f"{filename}*.{extension}"))
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\JTeulade\repositories\stac\scripts\add_items.py", line 28, in <module>
    add_items(S3_TARGET_PATH, utils.get_ids())
  File "E:\JTeulade\repositories\stac\libs\sertit-utils\sertit\s3.py", line 99, in s3_env_wrapper
    return function(*_args, **_kwargs)
  File "E:\JTeulade\repositories\stac\stac\stac.py", line 255, in add_items
    path_products = create_items(valid_path_products, catalog, s3_target_path)
  File "E:\JTeulade\repositories\stac\stac\stac.py", line 174, in create_items
    item: pystac.Item = prod.stac.create_item()
  File "e:\jteulade\repositories\eoreader\eoreader\products\product.py", line 1424, in stac
    self._stac = StacItem(self)
  File "e:\jteulade\repositories\eoreader\eoreader\stac\stac_item.py", line 69, in __init__
    self.proj = ProjExt(prod, **kwargs)
  File "e:\jteulade\repositories\eoreader\eoreader\stac\stac_extensions.py", line 212, in __init__
    self.geometry = gdf_to_geometry(self.geometry_fct())
  File "C:\Users\jteulade\AppData\Local\anaconda3\envs\arcgispro-eo\lib\site-packages\methodtools.py", line 72, in __call__
    return self.__call__(*args, **kwargs)
  File "e:\jteulade\repositories\eoreader\eoreader\__init__.py", line 34, in wrapper
    return func(*args, **kwargs)
  File "e:\jteulade\repositories\eoreader\eoreader\stac\stac_extensions.py", line 243, in geometry_fct
    return self._prod.footprint().to_crs(self.crs())
  File "C:\Users\jteulade\AppData\Local\anaconda3\envs\arcgispro-eo\lib\site-packages\methodtools.py", line 72, in __call__
    return self.__call__(*args, **kwargs)
  File "e:\jteulade\repositories\eoreader\eoreader\__init__.py", line 34, in wrapper
    return func(*args, **kwargs)
  File "e:\jteulade\repositories\eoreader\eoreader\utils.py", line 364, in simplify_wrapper
    footprint = footprint_fct(self)
  File "e:\jteulade\repositories\eoreader\eoreader\products\optical\dimap_v1_product.py", line 151, in footprint
    self.get_default_band_path(),
  File "e:\jteulade\repositories\eoreader\eoreader\products\optical\vhr_product.py", line 139, in get_default_band_path
    return self._get_default_utm_band(self.pixel_size, **kwargs)
  File "e:\jteulade\repositories\eoreader\eoreader\products\optical\vhr_product.py", line 629, in _get_default_utm_band
    def_path = self.get_band_paths([default_band], pixel_size=def_res)[default_band]
  File "e:\jteulade\repositories\eoreader\eoreader\products\optical\vhr_product.py", line 222, in get_band_paths
    self.ortho_path = self._get_ortho_path(**kwargs)
  File "e:\jteulade\repositories\eoreader\eoreader\products\optical\vhr_product.py", line 189, in _get_ortho_path
    ortho_path = self._get_tile_path()
  File "e:\jteulade\repositories\eoreader\eoreader\products\optical\spot45_product.py", line 616, in _get_tile_path
    return self._get_path("IMAGERY", "TIF")
  File "e:\jteulade\repositories\eoreader\eoreader\products\optical\vhr_product.py", line 519, in _get_path
    raise InvalidProductError(
eoreader.exceptions.InvalidProductError: No file corresponding to **IMAGERY*.TIF found in C:\Users\jteulade\AppData\Local\Temp\tmp162cml_f\SP05_HRG_THX__3_20120729T112957_20120729T113005_TOU_1234_f1ec.DIMA

You can test the following product: s3://sertit-projects-archives/projects/2019/PREFER/Traitements/TEST_SITES/PORTUGAL/Images/SPOT/SP05_HRG_THX__3_20120729T112957_20120729T113005_TOU_1234_f1ec.DIMA.tar

@jteulade jteulade added the bug Something isn't working label May 15, 2024
jteulade added a commit that referenced this issue May 15, 2024
@jteulade jteulade linked a pull request May 15, 2024 that will close this issue
2 tasks
@remi-braun
Copy link
Member

The right thing is indeed to only look at the root of the product 👍

jteulade added a commit that referenced this issue May 21, 2024
jteulade added a commit that referenced this issue May 21, 2024
jteulade added a commit that referenced this issue May 21, 2024
jteulade added a commit that referenced this issue May 21, 2024
pixalytics added a commit to pixalytics-ltd/eoreader that referenced this issue Aug 12, 2024
* Fix sertit#140: replace QB satellite ID with QB02 satellite ID

* Fix sertit#142: catch StopIteration and return InvalidProductError

* Fix sertit#142: catch StopIteration in several products

* Fix sertit#142: raise InvalidProductError instead of log

* Fix sertit#145: remove nested=-1 for SPOT constellations, use the default nested=0 instead

* Fix sertit#145: update CHANGES.md

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix sertit#142: catch StopIteration and return InvalidProductError

* Avoid KeyError if a product metadata doesn't have a namespace

* sertit#142: catch StopIteration and return InvalidProductError

* sertit#156: catch InvalidProductError when we try to get cloud_coverage property

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* sertit#156: catch all InvalidProductError due to properties not found in metadata

* sertit#161: return an error if SPECTRAL_PROCESSING not found in metadata

* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/PyCQA/flake8: 7.0.0 → 7.1.0](PyCQA/flake8@7.0.0...7.1.0)

* FIX: Handle RCM and RS2 products that doesn't bundle their extent in a KML file sertit#155

* OPTIM: Search correctly nested metadata in the Reader (without accidentally using a recursive glob)

* sertit#152: add SGF in RS2 product type

* FIX: Fix error in looking for bands in Sentinel-2 L1C archived products sertit#168

* FIX: Handle wrongly recognized Planet products because of the recursive nested mtd in the Reader sertit#169

* FIX: Force the loading of `DimapV1` bands in `float32`

* FIX: Handle the case where fiona isn't installed anymore (with `geopandas 1.0`)

* FIX: Don't make `pystac` a mandatory requirement

* Use sertit.types.is_iterable

* Use sertit.types.make_iterable

* Correct typo

* FIX: Fix an unknown Planet bug that just appeared (`'...Path' has no len()`)

* CI: Remove for now end-to-end tests with Python 3.11 and 3.10.

* v0.21.2

* INSTALL: Remove `pystac[validation]` (as it is an optional dependency) from setup.py, and create a `stac` extra feature.

* Updates in CHANGES.md

* v0.21.2

* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/PyCQA/flake8: 7.1.0 → 7.1.1](PyCQA/flake8@7.1.0...7.1.1)
- [github.com/psf/black: 24.4.2 → 24.8.0](psf/black@24.4.2...24.8.0)

* FIX: Get better window name (if available) when writing bands on disk (in tmp folder) + update sertit's version to benefit to the vector's name

---------

Co-authored-by: jteulade <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Rémi Braun <[email protected]>
Co-authored-by: BRAUN REMI <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants