Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow for passing list of globs to files #21

Open
0xcaff opened this issue Sep 7, 2024 · 1 comment
Open

allow for passing list of globs to files #21

0xcaff opened this issue Sep 7, 2024 · 1 comment

Comments

@0xcaff
Copy link
Owner

0xcaff commented Sep 7, 2024

I'd like to be able to pass lists of values to files argument.

SELECT *
FROM protobuf(
    descriptors = './descriptor.pb',
    files = ['./scrape/data/SceneVersion/**/*.bin', './other/location/data/SceneVersion/**/*.bin'],
    message_type = 'test_server.v1.GetUserSceneVersionResponse',
    delimiter = 'BigEndianFixed'
)
LIMIT 10;

I would need to change this value here

"files".to_string(),
LogicalType::new(LogicalTypeId::Varchar),
),

It seems like the CSV plugin does this same thing here

https://github.com/duckdb/duckdb/blob/3550708a8456e193e5d8ccc21a3dd6563e8a0c21/src/execution/operator/csv_scanner/util/csv_reader_options.cpp#L302-L304

using this value

https://github.com/duckdb/duckdb/blob/3ed5d833dd95556c0a4f88175f4bf9722f978b9e/src/function/table/read_csv.cpp#L254

unfortunately, it seems like the C-API only exposes a subset of types

https://github.com/duckdb/duckdb/blob/05176cd88e42c33a2a039bdfa5da798e0f5266ff/src/main/capi/logical_types-c.cpp#L25-L27

not including any

https://github.com/duckdb/duckdb/blob/c1e3416ad904855c925b0ecc8a16b2f958e9bf77/src/main/capi/helper-c.cpp#L5-L61

https://github.com/duckdb/duckdb/blob/4aa0fb8a4968c0a5da7addda7cd2784f7420485f/src/include/duckdb/common/types.hpp#L189

I asked in discord to see whether we can get this added

https://discord.com/channels/909674491309850675/1148659944669851849/1282054644096434177

@0xcaff
Copy link
Owner Author

0xcaff commented Sep 11, 2024

looks like libduckdb-sys needs to be upgraded for this to happen. the new any type is in the v1.1.0 release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant