You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When depending on datafusion-common with default-features = false, there are compilation errors related to parquet functionality.
To Reproduce
In the datafusion-expr crate, add default-features = false to the datafusion-common dependency. Then try to build the datafusion-expr crate.
There are compilation errors such as:
error[E0433]: failed to resolve: use of undeclared crate or module `parquet`
--> datafusion/common/src/file_options/parse_utils.rs:20:5
|
20 | use parquet::{
| ^^^^^^^ use of undeclared crate or module `parquet`
error[E0433]: failed to resolve: use of undeclared crate or module `parquet`
--> datafusion/common/src/file_options/parquet_writer.rs:20:5
|
20 | use parquet::file::properties::{WriterProperties, WriterPropertiesBuilder};
| ^^^^^^^ use of undeclared crate or module `parquet`
error[E0433]: failed to resolve: use of undeclared crate or module `parquet`
--> datafusion/common/src/file_options/parse_utils.rs:43:13
|
43 | ) -> Result<parquet::basic::Encoding> {
| ^^^^^^^ use of undeclared crate or module `parquet`
error[E0433]: failed to resolve: use of undeclared crate or module `parquet`
--> datafusion/common/src/file_options/parse_utils.rs:46:23
|
46 | "plain" => Ok(parquet::basic::Encoding::PLAIN),
| ^^^^^^^ use of undeclared crate or module `parquet`
|
...
Expected behavior
It would be possible to compile datafusion-common with the default parquet feature disabled.
Additional context
In VegaFusion I compile datafusion-common into WASM, and the parquet crate is not (easily?) compatible with WASM due to some non-rust compression dependencies.
I'll have a PR with suggested fix up shortly
The text was updated successfully, but these errors were encountered:
In VegaFusion I compile datafusion-common into WASM, and the parquet crate is not (easily?) compatible with WASM due to some non-rust compression dependencies.
@jonmmease in order to avoid such errors in the future, perhaps we can encode the usecase into a CI test
Can you share the command / commands you use to compile for wasm (specifically what target)? Here is what we use for sqlparser-rs
That's a good idea. I made a PR in #7633 that adds a minimal wasm library crate that depends on the various DataFusion crates that are currently wasm compatible.
Describe the bug
When depending on datafusion-common with
default-features = false
, there are compilation errors related to parquet functionality.To Reproduce
In the
datafusion-expr
crate, adddefault-features = false
to thedatafusion-common
dependency. Then try to build thedatafusion-expr
crate.There are compilation errors such as:
Expected behavior
It would be possible to compile
datafusion-common
with the defaultparquet
feature disabled.Additional context
In VegaFusion I compile datafusion-common into WASM, and the
parquet
crate is not (easily?) compatible with WASM due to some non-rust compression dependencies.I'll have a PR with suggested fix up shortly
The text was updated successfully, but these errors were encountered: