Improve optimizer performance by not using Errors in the happy path

[alamb]

Is your feature request related to a problem or challenge?

We have had a few issues / reports where the creation of DataFusionError during query planning has contributed significantly to the overall planning time. Specifically, #5309 and more recently #7522

At the core of the issue is that creating a DataFusionError is a fairly slow operation (as it requires allocating a new String to hold the error message, and sometimes, as in #5309, the construction of the message itself is significant.

Describe the solution you'd like

I would like to update the DataFusion codebase so it does not use Errors on the "happy path"

Tasks:

• Make a faster way to check column existence in optimizer (not is_err()) #5309
• Optimize FilterExec::statistics / don't ignore errors #7553

Describe alternatives you've considered

We could potentially make it faster / cheaper to create DataFusion errors

Additional context

No response

[alamb]

Here is an example of using Errors in the happy path (aka ignoring an error rather than fixing the underlying issue)

https://github.com/apache/arrow-datafusion/blob/c47ae0d13dbc9474897609f3dea4f80a93cf4de0/datafusion/physical-plan/src/filter.rs#L206-L209

[comphead]

I wanna check that if no other volunteers

[comphead]

I would also close this as a lot of job was done for planner to improve the performance

[comphead]

@alamb I'm closing this, please feel free to reopen if needed

[alamb]

I agree -- thanks for cleaning these up @comphead -- we can file new tickets if/when we have actionable changes to make