-
Notifications
You must be signed in to change notification settings - Fork 5.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use data size for directory listing cache #23176
Conversation
presto-hdfs-core/src/main/java/com/facebook/presto/hive/BlockLocation.java
Outdated
Show resolved
Hide resolved
presto-hive/src/main/java/com/facebook/presto/hive/HiveClientConfig.java
Outdated
Show resolved
Hide resolved
presto-hdfs-core/src/main/java/com/facebook/presto/hive/HiveFileInfo.java
Outdated
Show resolved
Hide resolved
de750ac
to
67e3cc7
Compare
67e3cc7
to
981de86
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The deprecated configuration property `hive.file-status-cache-size does not appear to be documented, so no doc change is needed for it.
Suggest add documentation for hive.file-status-cache.max-retained-size
configuration property - maybe to https://github.com/prestodb/presto/blob/master/presto-docs/src/main/sphinx/connector/hive.rst#hive-configuration-properties
981de86
to
54304a1
Compare
54304a1
to
0da1bea
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! (docs)
Pull branch, new local doc build, looks good. Thanks!
Description
Use data size for directory listing cache
Motivation and Context
We currently only have a size control on the number of values but not on the size in bytes. Given this is in memory cache, having a max size in bytes for the cache will be helpful
Impact
hive.file-status-cache-size
is deprecated andhive.file-status-cache.max-retained-size
now represents the max size in bytes for the directory listing cache.Test Plan
existing tests
Contributor checklist
Release Notes
Please follow release notes guidelines and fill in the release notes below.