You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
The is a significant amount of logic converting from DataFusionError to ArrowError and back, a large amount of which originates from the fact SendableRecordBatchStream returns ArrowError instead of DataFusionError
Describe the solution you'd like
It seems strange that the DataFusion stream abstraction would not return DataFusion's error type. I would like to change SendableRecordBatchStream to
pub trait RecordBatchStream: Stream<Item = Result<RecordBatch, DataFusionError>> {
/// Returns the schema of this `RecordBatchStream`.
///
/// Implementation of this trait should guarantee that all `RecordBatch`'s returned by this
/// stream should have the same schema as returned from this method.
fn schema(&self) -> SchemaRef;
}
Describe alternatives you've considered
We could not do this
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered:
That sounds good, DatafusionError and ArrowError are not 1 to 1 match, but we can expand it if needed or still use DatafusionError::ArrowError to wrap Arrow errors. Hope I can crunch it
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
The is a significant amount of logic converting from DataFusionError to ArrowError and back, a large amount of which originates from the fact
SendableRecordBatchStream
returnsArrowError
instead ofDataFusionError
Describe the solution you'd like
It seems strange that the DataFusion stream abstraction would not return DataFusion's error type. I would like to change
SendableRecordBatchStream
toDescribe alternatives you've considered
We could not do this
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered: