Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delay initial prometheus status metric #2874

Merged
merged 1 commit into from
Jul 30, 2018

Conversation

aledbf
Copy link
Member

@aledbf aledbf commented Jul 30, 2018

Which issue this PR fixes:

W0730 11:32:25.892550      31 nginx_status.go:207] unexpected error obtaining nginx status info: unexpected error scraping nginx status page: unexpected error scraping nginx : Get http://0.0.0.0:18080/nginx_status: dial tcp 0.0.0.0:18080: connect: connection refused
W0730 11:32:25.893659      31 nginx_status.go:207] unexpected error obtaining nginx status info: unexpected error scraping nginx status page: unexpected error scraping nginx : Get http://0.0.0.0:18080/nginx_status: dial tcp 0.0.0.0:18080: connect: connection refused

This confuses users and they think there's an error when the issue here is just timing.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jul 30, 2018
// the default nginx.conf does not contains
// a server section with the status port
go func() {
time.Sleep(10 * time.Second)
Copy link
Member

@ElvinEfendi ElvinEfendi Jul 30, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO 10s is a bit too much, potentially we can loose stats for thousands of requests. Maybe 5s? For the first sync of dynamic configuration we sleep only 1s and it has been fine so far.

But ideally we would need a mechanism to deterministically detect that the port is ready.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just in case, this delay only applies to the basic nginx connection count.

@codecov-io
Copy link

codecov-io commented Jul 30, 2018

Codecov Report

Merging #2874 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #2874   +/-   ##
=======================================
  Coverage   47.56%   47.56%           
=======================================
  Files          76       76           
  Lines        5483     5483           
=======================================
  Hits         2608     2608           
  Misses       2540     2540           
  Partials      335      335

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5e4137c...5e13de4. Read the comment docs.

@ElvinEfendi
Copy link
Member

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 30, 2018
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aledbf, ElvinEfendi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit c921a00 into kubernetes:master Jul 30, 2018
@aledbf aledbf deleted the fix-status branch July 30, 2018 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants