Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes for nginx test when running against a fresh project #364

Merged
merged 6 commits into from
Feb 11, 2022

Conversation

martijnvans
Copy link
Contributor

@martijnvans martijnvans commented Jan 26, 2022

See #363.

This PR was tested with the following command:

PROJECT=[REDACTED]
TRANSFERS_BUCKET=[REDACTED]
GOOGLE_APPLICATION_CREDENTIALS="${HOME}/credentials.json"
ZONE=us-central1-b
PLATFORMS=debian-10,centos-7
SCRIPTS_DIR=third_party_apps_data
go test -v third_party_apps_test.go -test.parallel=1000 -tags=integration_test -timeout=4

And, it didn't pass, for various reasons. Mostly that my test file is at head but it's testing the stable agent, which is substantially behind head. But I believe the failures were unrelated to this change. (and nginx passed.) The interesting part of this change was changing the retry logic for metrics queries, and none of the failures that I saw were due to a metrics query not being retried when it was supposed to.

@martijnvans martijnvans marked this pull request as ready for review January 26, 2022 20:13
Copy link
Contributor

@qingling128 qingling128 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

strings.Contains(err.Error(), "failed to look up metric workload.googleapis.com")
// The error doesn't always look the same, hopefully looking for "code = NotFound" will catch all variations.
strings.Contains(err.Error(), "failed to look up metric workload.googleapis.com") ||
strings.Contains(err.Error(), "code = NotFound")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to check the code directly instead of checking the string version?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(with grouchiness) this api doesn't seem to document what kind of error object is being returned, and looking at the code didn't help either: https://github.com/googleapis/google-cloud-go/blob/monitoring/v1.2.0/monitoring/apiv3/metric_client.go#L362

it could be some kind of gax-ian error object, or it could be decided by c.metricClient.ListTimeSeries, which just links back to itself, very unhelpful.

I agree that matching on the error string is gross but don't have a better solution at the moment. suggestions welcome.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect it returns an error compatible with status.FromError, which returns a Status object that has a Code() method that you can compare against codes.NotFound.

I would also expect "failed to look up metric" to be paired with a code (perhaps even NotFound), though maybe you want to explicitly match on workload.googleapis.com for some reason?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I changed it to just look for NotFound (and Internal). Much cleaner and hopefully more robust to tiny changes in the error message. It seems to work.

@martijnvans martijnvans force-pushed the martijnvans-fix-nginx branch from 26a7c71 to e2e351e Compare February 10, 2022 15:49
@martijnvans martijnvans merged commit d38f922 into master Feb 11, 2022
@martijnvans martijnvans deleted the martijnvans-fix-nginx branch February 23, 2022 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants