Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xcube server cannot serve nested datasets #579

Closed
AliceBalfanz opened this issue Jan 12, 2022 · 1 comment · Fixed by #588
Closed

xcube server cannot serve nested datasets #579

AliceBalfanz opened this issue Jan 12, 2022 · 1 comment · Fixed by #588
Labels
bug Something isn't working xcube serve This is related to server component, CLI "xcube serve"

Comments

@AliceBalfanz
Copy link
Contributor

Describe the bug
I have a s3 cloud bucket, where user can store their cubes. These cubes are served by xcube server in order to be visualised with xcube viewer. I don't know how many cubes each user has or who the user are. The bucket structure is: s3://<bucket>/cubes/<user_name>/*.zarr

therefore my configuration of the server looks like this:

DataStores:
  - Identifier: <precise_identifyer>
    StoreId: s3
    StoreParams:
      root: <bucket>/cubes
      max_depth: 3
      storage_options:
        anon: false # I have a .aws file, which 
    Datasets:
      - Identifier: "*.zarr"
        Style: "default"

The viewer launches without problem, but does not show the data layer of first data cube of the list. It is the correct location within the bucket though, shown by the id in the info panel. I know where the data of the cube is, therefore create a time series for a point where I know data should be. This leads to a Cannot reach server message:

image

image

the server logs don't show any errors:

(xcube) alicja@alicja:~/Desktop/projects/service-cyanoalert$ xcube serve -c /home/alicja/Desktop/projects/service-cyanoalert/bigfe/1_development/xcube-viewer-app/stage/api/server-config_for_testing.yml -v 
xcube Server: WMTS, catalogue, data access, tile, feature, time-series services for xarray-enabled data cubes, version 0.9.3.dev0
[I 220112 08:33:31 service:239] configuration file '/home/alicja/Desktop/projects/service-cyanoalert/bigfe/1_development/xcube-viewer-app/stage/api/server-config_for_testing.yml' successfully loaded
[I 220112 08:33:31 service:168] service running, listening on localhost:8080, try http://localhost:8080/datasets
[I 220112 08:33:31 service:169] press CTRL+C to stop service
[I 220112 08:34:03 perf:79] get granted scopes: took 0.03ms
[I 220112 08:34:03 context:283] scanning store 'bigfe'
[I 220112 08:34:03 credentials:1224] Found credentials in shared credentials file: ~/.aws/credentials

For testing reasons I do know the user prefix in the bucket, and if I test them separately they work as expected:

DataStores:
  - Identifier: <precise_identifyer>
    StoreId: s3
    StoreParams:
      root: <bucket>/cubes/<user_name>
      max_depth: 3
      storage_options:
        anon: false # I have a .aws file, which 
    Datasets:
      - Identifier: "*.zarr"
        Style: "default"

image

@AliceBalfanz AliceBalfanz added xcube serve This is related to server component, CLI "xcube serve" bug Something isn't working labels Jan 12, 2022
@forman
Copy link
Member

forman commented Jan 12, 2022

Ok, the problem is that datasets identified in any sub-paths of root will have identifiers that contain forward slashes. And these are currently not URL-encoded.

@forman forman changed the title xcube server not working with certain DataStores configuration xcube server cannot serve nested datasets Jan 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working xcube serve This is related to server component, CLI "xcube serve"
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants