Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky test - TestExporter_PushEvent/publish_with_ecs_encoding #32910

Closed
jpkrohling opened this issue May 7, 2024 · 8 comments · Fixed by #32917
Closed

Flaky test - TestExporter_PushEvent/publish_with_ecs_encoding #32910

jpkrohling opened this issue May 7, 2024 · 8 comments · Fixed by #32917
Labels
bug Something isn't working ci-cd CI, CD, testing, build issues exporter/elasticsearch flaky test a test is flaky

Comments

@jpkrohling
Copy link
Member

Component(s)

exporter/elasticsearch

What happened?

Description

On changes to unrelated components, the following test is failing intermittently:

=== FAIL: . TestExporter_PushEvent/publish_with_ecs_encoding (re-run 1) (0.01s)
    logger.go:146: 2024-05-07T14:43:31.587Z	DEBUG	Request roundtrip completed.	{"path": "/", "method": "GET", "duration": "936.23µs", "status": "200 OK"}
    logs_exporter_test.go:178: 
        	Error Trace:	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/elasticsearchexporter/logs_exporter_test.go:178
        	            				/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/elasticsearchexporter/utils_test.go:162
        	            				/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/elasticsearchexporter/utils_test.go:182
        	            				/opt/hostedtoolcache/go/1.21.9/x64/src/net/http/server.go:2136
        	            				/opt/hostedtoolcache/go/1.21.9/x64/src/net/http/server.go:2514
        	            				/opt/hostedtoolcache/go/1.21.9/x64/src/net/http/server.go:2938
        	            				/opt/hostedtoolcache/go/1.21.9/x64/src/net/http/server.go:2009
        	            				/opt/hostedtoolcache/go/1.21.9/x64/src/runtime/asm_amd64.s:1650
        	Error:      	Not equal: 
        	            	expected: "{\"attrKey1\":\"abc\",\"attrKey2\":\"def\",\"application\":\"myapp\",\"service\":{\"name\":\"myservice\"},\"error\":{\"stacktrace\":\"no no no no\"},\"agent\":{\"name\":\"otlp\"},\"@timestamp\":\"1970-01-01T00:00:00.000000000Z\",\"message\":\"hello world\"}"
        	            	actual  : "{\"attrKey1\":\"abc\",\"attrKey2\":\"def\",\"service\":{\"name\":\"myservice\"},\"error\":{\"stacktrace\":\"no no no no\"},\"application\":\"myapp\",\"agent\":{\"name\":\"otlp\"},\"@timestamp\":\"1970-01-01T00:00:00.000000000Z\",\"message\":\"hello world\"}"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1 +1 @@
        	            	-{"attrKey1":"abc","attrKey2":"def","application":"myapp","service":{"name":"myservice"},"error":{"stacktrace":"no no no no"},"agent":{"name":"otlp"},"@timestamp":"1970-01-01T00:00:00.000000000Z","message":"hello world"}
        	            	+{"attrKey1":"abc","attrKey2":"def","service":{"name":"myservice"},"error":{"stacktrace":"no no no no"},"application":"myapp","agent":{"name":"otlp"},"@timestamp":"1970-01-01T00:00:00.000000000Z","message":"hello world"}
        	Test:       	TestExporter_PushEvent/publish_with_ecs_encoding
    logger.go:146: 2024-05-07T14:43:31.589Z	DEBUG	Request roundtrip completed.	{"path": "/_bulk", "method": "POST", "duration": "1.346576ms", "status": "200 OK"}

Example:
https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/8987287280/job/24685476376?pr=32509#step:5:107

Steps to Reproduce

Expected Result

Actual Result

Collector version

main

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

No response

@jpkrohling jpkrohling added bug Something isn't working needs triage New item requiring triage ci-cd CI, CD, testing, build issues flaky test a test is flaky exporter/elasticsearch and removed needs triage New item requiring triage labels May 7, 2024
Copy link
Contributor

github-actions bot commented May 7, 2024

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Copy link
Contributor

github-actions bot commented May 7, 2024

Pinging code owners for exporter/elasticsearch: @JaredTan95 @ycombinator @carsonip. See Adding Labels via Comments if you do not have permissions to add labels yourself.

@crobert-1
Copy link
Member

crobert-1 commented May 7, 2024

Looks like this is a result of #31694.

From what I can tell the results are identical, just a different order. The assert.Equal comparison needs to be modified to ignore order of arguments.

(The challenge here is that the type of arguments being compared is string, rather than map.)

@carsonip
Copy link
Contributor

carsonip commented May 7, 2024

Ack. Will publish a fix ASAP.

TylerHelmuth pushed a commit that referenced this issue May 7, 2024
…#32917)

**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
Fix a flaky test from #31694 due to non-deterministic JSON key order.

**Link to tracking Issue:** Closes #32910
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working ci-cd CI, CD, testing, build issues exporter/elasticsearch flaky test a test is flaky
Projects
None yet
4 participants