-
Notifications
You must be signed in to change notification settings - Fork 185
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add tiledb_stats_is_enabled to C and C++ APIs #5395
Conversation
bc5d939
to
3050a7e
Compare
3050a7e
to
eb352cb
Compare
Co-authored-by: Ypatia Tsavliri <[email protected]>
Co-authored-by: Ypatia Tsavliri <[email protected]>
Co-authored-by: Ypatia Tsavliri <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would prefer any nontrivial library or application to avoid using the global stats APIs and instead get stats scoped to a context or query with the tiledb_(ctx|query)_get_stats
functions (from a quick read these are always enabled).
If we want to double down on global stats, we should instead add a tiledb_stats_enable_v2(uint8_t* previous);
function that also returns whether stats were already enabled, which can be implemented atomically.
If we don't want to support these existing APIs then they should be deprecated. But if we don't want to deprecate them then this change must be added for completeness. Global stats may not be the best tool for an application developer but it is a useful tool for a system monitor so in my opinion deprecation does not make sense. We certainly will use the context/query stats APIs in tables eventually for finer-grained profiling but my short-term goal is to enable collection and introspection into any degree of metrics. |
Co-authored-by: Theodore Tsirpanis <[email protected]>
Co-authored-by: Theodore Tsirpanis <[email protected]>
I'm not arguing to deprecate the global stats APIs, but they should be toggled only "globally" in the top level of an application or script. Quoting the PR's description (emphasis mine):
If an application is toggling global stats deep inside its logic, where it cannot be certain whether they are previously enabled, it should not be using global stats in the first place.
Can't the application enable them in the very beginning of its lifetime? And if it wants to skip collecting stats for some operations it can disable them temporarily. My previous sentence does not apply in this case; our imaginary application could track on its own whether stats are enabled at any time, and does not need a new C API. |
This is specifically what I want to avoid and the reason I am adding this API. This is for the tables CLI. Users should be able to turn stats on and off, and check whether they are enabled already. This can be done by tracking the state separately but I consider duplicated state such as that to be the wrong thing to do when there is this alternative. |
Okay I understand now the use case, but still think that tracking this state in the app is the best way to go. I am concerned that adding this API would encourage people to use global stats in places they shouldn't, but am fine if other people agree with adding it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. With regards to the concerns on encouraging the "wrong" way to enable/disable stats with the addition of this API, my only idea would be to remove the example (that I suggested to add in the first place 😅) and fix the example as it is today to show the "correct" way of getting stats for a query.
The C API has functions
tiledb_stats_enable() and
tiledb_stats_disable()` which affect global process state.A library or application running on top of tiledb might want to enable statistics for a span and then restore the previous state. This is not possible without a way of asking whether stats gathering is currently enabled.
This pull request adds
tiledb_stats_is_enabled
so that users can check whether stats are currently enabled. Use of this API is demonstrated in tests viaScopedStats
.TYPE: C_API | CPP_API
DESC: add tiledb_stats_is_enabled to C and C++ APIs