-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ARROW-12402: [Rust] [DataFusion] Implement SQL metrics example #10049
Conversation
@alamb @jorgecarleitao @Dandandan @returnString Let me know what you think. It is really hard to debug distributed queries without any metrics 😉 |
@edrevo fyi |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks great! Indeed great to have for debugging / optimizing DataFusion and ballista!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me, nice simple API and no constraints imposed on consumers :)
One potential user-friendliness thing: would it be worth having SQLMetric
use atomics internally to avoid the Mutex<SQLMetric>
that'll probably be required for most usage? Almost certainly makes no difference from a perf perspective, the contention will be pretty much non-existent, just thinking about the API.
I think this would be a good idea @returnString FYI. Perhaps part of a subsequent PR |
This introduces a new method on `ExecutionPlan` to be able to access generic metrics from any physical operator. One metric is implemented to demonstrate usage. Closes apache#10049 from andygrove/ARROW-12402 Authored-by: Andy Grove <[email protected]> Signed-off-by: Krisztián Szűcs <[email protected]>
This introduces a new method on `ExecutionPlan` to be able to access generic metrics from any physical operator. One metric is implemented to demonstrate usage. Closes apache#10049 from andygrove/ARROW-12402 Authored-by: Andy Grove <[email protected]> Signed-off-by: Krisztián Szűcs <[email protected]>
This introduces a new method on `ExecutionPlan` to be able to access generic metrics from any physical operator. One metric is implemented to demonstrate usage. Closes apache#10049 from andygrove/ARROW-12402 Authored-by: Andy Grove <[email protected]> Signed-off-by: Krisztián Szűcs <[email protected]>
This introduces a new method on
ExecutionPlan
to be able to access generic metrics from any physical operator.One metric is implemented to demonstrate usage.