Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] use starrocks[Paimon Catalog] query Paimon[FilesystemCatalog-S3] failed #4688

Open
1 of 2 tasks
lukeyan2023 opened this issue Dec 11, 2024 · 0 comments
Open
1 of 2 tasks
Labels
bug Something isn't working

Comments

@lukeyan2023
Copy link

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

paimon 0.9.0

Compute Engine

starrocks 3.2.12/3.3.7

Minimal reproduce step

  1. Building Paimon Table Based on AWS S3 Compatibility Object Storage, Paimon Catalog DDL is as follows:
    image
  2. Create Paimon Catalog in Starrocks:
    image
  3. Querying Paimon Catalog Data Table in Starrocks
    image
  4. Query failed with the following error message
    image

What doesn't meet your expectations?

fragment_executor.cpp:183] Prepare(): query_id=cd5b43d6-b7c7-11ef-910f-564ab30c5f0c fragment_instance_id=cd5b43d6-b7c7-11ef-910f-564ab30c5f0e is_stream_pipeline=0 backend_num=1
E1211 21:57:10.449843 441 scan_operator.cpp:434] scan fragment cd5b43d6-b7c7-11ef-910f-564ab30c5f0e driver 0 Scan tasks error: IO error: BE access S3 file failed, SdkResponseCode=403, SdkErrorType=22, SdkErrorMessage=The request signature we calculated does not match the signature you provided. Check your key and signing method.: file = s3://paimons3/datalake/ods.db/table/dt=20240927/hh=18/bucket-0/data-3af7b53b-99c5-4c4a-b259-fceba56caee0-1.parquet
be/src/connector/hive_connector.cpp:129 _init_scanner(state)
be/src/exec/pipeline/scan/connector_scan_operator.cpp:758 _data_source->open(state)
be/src/exec/pipeline/scan/connector_scan_operator.cpp:785 _open_data_source(state, &mem_alloc_failed)
E1211 21:57:10.450788 449 scan_operator.cpp:434] scan fragment cd5b43d6-b7c7-11ef-910f-564ab30c5f0e driver 1 Scan tasks error: IO error: BE access S3 file failed, SdkResponseCode=403, SdkErrorType=22, SdkErrorMessage=The request signature we calculated does not match the signature you provided. Check your key and signing method.: file = s3://paimons3/datalake/ods.db/table/dt=20240927/hh=18/bucket-0/data-3af7b53b-99c5-4c4a-b259-fceba56caee0-0.parquet
be/src/connector/hive_connector.cpp:129 _init_scanner(state)

Anything else?

  1. Paimon table storage uses AWS S3 compatible object storage,and it has been confirmed that AK/SK is correct
  2. According to current testing, starrocks 3.1.15 + Paimon 0.9.0 does not have this issue
  3. Using the same set of AWS S3 compatible object storage, Starrocks can query its own internal table (using this object storage) normally, but when Starrocks queries the Paimon table (also stored using this object storage) through the Paimon catalog, it will fail

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@lukeyan2023 lukeyan2023 added the bug Something isn't working label Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant