Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(secrets): Warn if OS limit for locked memory is too low #12993

Merged
merged 6 commits into from
Apr 3, 2023

Conversation

srebhan
Copy link
Member

@srebhan srebhan commented Mar 30, 2023

relates to #12980

Plugins using secrets rely on locked memory in Telegraf. Due to limitations in the underlying library, each secret will use two pages of (locked) memory for the access time. Some operating systems or operators set a low limit on the maximum allowed amount of locked memory, making Telegraf hang (see #12924 or #12980) or panic of too many secrets are accessed at the same time. To be on the safe-side, warn the user if the limit set by the OS is too low for the number of secrets managed.

@telegraf-tiger telegraf-tiger bot added the fix pr to fix corresponding bug label Mar 30, 2023
cmd/telegraf/telegraf.go Outdated Show resolved Hide resolved
cmd/telegraf/telegraf.go Show resolved Hide resolved
cmd/telegraf/telegraf_posix.go Outdated Show resolved Hide resolved
@srebhan srebhan force-pushed the secret_store_issue_12980 branch from c5799cf to 160d800 Compare March 31, 2023 08:41
Copy link
Contributor

@powersj powersj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok played with this a bit an created a config with 10x influxdb_v2 outputs each using a secret and reduced the ulimit to 64K:

2023-03-31T14:10:25Z I! Found 10 secrets...
2023-03-31T14:10:25Z W! Insufficient lockable memory 64kb when 80kb is required. Please increase the limit for Telegraf in your Operating System!

and after increasing to 80k it did not show up. I had Windows up and all I tested that the binary continued to work.

@srebhan srebhan added ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. plugin/secretstores labels Apr 3, 2023
@srebhan srebhan merged commit d8adb1e into influxdata:master Apr 3, 2023
@srebhan srebhan deleted the secret_store_issue_12980 branch April 3, 2023 08:49
srebhan added a commit that referenced this pull request Apr 3, 2023
@srebhan srebhan added this to the v1.26.1 milestone Jun 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fix pr to fix corresponding bug plugin/secretstores ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants