[test-failed]: X-Pack API Integration Tests.x-pack/test/api_integration/apis/telemetry/opt_in·ts - apis Telemetry /api/telemetry/v2/optIn API should support sending false #51522

liza-mae · 2019-11-22T22:42:40Z

Version: 7.5
Class: X-Pack API Integration Tests.x-pack/test/api_integration/apis/telemetry/opt_in·ts
Test Failures: 2
Other Tests:
X-Pack API Integration Tests.x-pack/test/api_integration/apis/telemetry/opt_in·ts.apis Telemetry /api/telemetry/v2/optIn API should support sending true

Stack Trace:
Error: expected 200 "OK", got 400 "Bad Request"
at Test._assertStatus (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/node/linux-immutable/ci/cloud/build/kibana/node_modules/supertest/lib/test.js:268:12)
at Test._assertFunction (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/node/linux-immutable/ci/cloud/build/kibana/node_modules/supertest/lib/test.js:283:11)
at Test.assert (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/node/linux-immutable/ci/cloud/build/kibana/node_modules/supertest/lib/test.js:173:18)
at assert (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/node/linux-immutable/ci/cloud/build/kibana/node_modules/supertest/lib/test.js:131:12)
at /var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/node/linux-immutable/ci/cloud/build/kibana/node_modules/supertest/lib/test.js:128:5
at Test.Request.callback (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/node/linux-immutable/ci/cloud/build/kibana/node_modules/superagent/lib/node/index.js:718:3)
at parser (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/node/linux-immutable/ci/cloud/build/kibana/node_modules/superagent/lib/node/index.js:906:18)
at IncomingMessage.res.on (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/node/linux-immutable/ci/cloud/build/kibana/node_modules/superagent/lib/node/parsers/json.js:19:7)
at endReadableNT (_stream_readable.js:1103:12)
at process._tickCallback (internal/process/next_tick.js:63:19)

Platform: cloud
Build Num: 24

elasticmachine · 2019-11-22T22:42:42Z

Pinging @elastic/kibana-test-triage (failed-test)

elasticmachine · 2019-11-23T01:39:33Z

Pinging @elastic/kibana-stack-services (Team:Stack Services)

elasticmachine · 2019-12-12T16:19:10Z

Pinging @elastic/pulse (Team:Pulse)

Bamieh · 2019-12-13T18:26:56Z

This will fail on cloud since we do not allow changing opt in status on cloud (configured via telemetry.allowChangingOptInStatus: false).

I believe at the moment this is only skipped for cloud, how can i double check? If so I believe we can close this.

liza-mae · 2019-12-16T23:18:24Z

@Bamieh , thanks when you say we don't allowing changing opt in status on cloud, who is we? Would you say this is not something supported for a customer to do on cloud?

Bamieh · 2020-01-09T00:28:29Z

@liza-mae
this config is set by the cloud team on deployed kibana instances: telemetry.allowChangingOptInStatus: false

Cloud users cannot change this kibana config, hence cloud users will not be able to set opt_in to false via the API.

liza-mae · 2020-01-09T00:43:06Z

Thanks @Bamieh ! We should create a PR to skip this test for cloud then and reference this issue as the explanation as to why it is skipped.

Bamieh · 2020-01-09T00:44:55Z

@liza-mae awesome! where do we do that?

liza-mae · 2020-01-09T19:23:37Z

@Bamieh, in file for the following tests:

kibana/x-pack/test/api_integration/apis/telemetry/opt_in.ts

Line 26 in dfac5d8

it('should support sending false', async () => {

kibana/x-pack/test/api_integration/apis/telemetry/opt_in.ts

Line 33 in dfac5d8

it('should support sending true', async () => {

I was thinking instead of skipping them on cloud, we can verify these commands get rejected on cloud (gives us a better test scenario). We recently added an isCloud() method, so we can use something as follows:

if ((await PageObjects.common.isCloud())) {
await postTelemetryV2Optin(supertest, false, 400);
}
else ... do the rest of the test

liza-mae · 2020-01-24T04:53:07Z

Version: 7.6
Class: X-Pack API Integration Tests.x-pack/test/api_integration/apis/telemetry/opt_in·ts
Stack Trace:
Error: expected 200 "OK", got 400 "Bad Request"
at Test._assertStatus (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/supertest/lib/test.js:268:12)
at Test._assertFunction (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/supertest/lib/test.js:283:11)
at Test.assert (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/supertest/lib/test.js:173:18)
at assert (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/supertest/lib/test.js:131:12)
at /var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/supertest/lib/test.js:128:5
at Test.Request.callback (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/superagent/lib/node/index.js:718:3)
at parser (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/superagent/lib/node/index.js:906:18)
at IncomingMessage.res.on (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/superagent/lib/node/parsers/json.js:19:7)
at endReadableNT (_stream_readable.js:1143:12)
at process._tickCallback (internal/process/next_tick.js:63:19)
Platform: cloud
Build Num: 42

liza-mae · 2020-01-24T17:13:32Z

I had a case on staging (not the above) where this command POST /api/telemetry/v2/optIn actually succeeded with 200 response code, which is incorrect. The correct response is 400 {"error":"Not allowed to change Opt-in Status."}.
So making these test changes will help catch this type of failure and shows the value of having these checks.
I did verify the released version on production - did not have the problem. It also shows the value of running these tests on production.

Reference Results: elastic+estf-cloud-kibana-tests/41/testReport

Bamieh · 2020-02-04T11:27:04Z

@liza-mae added to fix in 7.7

liza-mae · 2020-02-27T23:43:26Z

Version: 7.6.1
Class: X-Pack API Integration Tests.x-pack/test/api_integration/apis/telemetry/opt_in·ts
Stack Trace:
Error: expected 200 "OK", got 400 "Bad Request"
at Test._assertStatus (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/supertest/lib/test.js:268:12)
at Test._assertFunction (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/supertest/lib/test.js:283:11)
at Test.assert (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/supertest/lib/test.js:173:18)
at assert (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/supertest/lib/test.js:131:12)
at /var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/supertest/lib/test.js:128:5
at Test.Request.callback (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/superagent/lib/node/index.js:718:3)
at parser (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/superagent/lib/node/index.js:906:18)
at IncomingMessage.res.on (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackExt/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/superagent/lib/node/parsers/json.js:19:7)
at endReadableNT (_stream_readable.js:1145:12)
at process._tickCallback (internal/process/next_tick.js:63:19)
Platform: cloud
Build Num: 50

elasticmachine · 2021-10-01T21:46:10Z

Pinging @elastic/kibana-core (Team:Core)

liza-mae added failed-test A test failure on a tracked branch, potentially flaky-test test-cloud labels Nov 22, 2019

liza-mae added the Team:Stack Services label Nov 23, 2019

liza-mae added the Feature:Telemetry label Nov 23, 2019

bmcconaghy added Team:Pulse and removed Team:Stack Services labels Dec 12, 2019

liza-mae mentioned this issue Jan 24, 2020

Navigating to apps when welcome screen is present #39689

Closed

Bamieh added the v7.7.0 label Feb 4, 2020

liza-mae mentioned this issue Feb 6, 2020

[test-failed]: X-Pack API Integration Tests.x-pack/test/api_integration/apis/telemetry/opt_in·ts - apis Telemetry /api/telemetry/v2/optIn API should support sending true #55786

Closed

Bamieh mentioned this issue Mar 2, 2020

[Telemetry] fix opt in tests #59040

Closed

afharo mentioned this issue Mar 25, 2020

[Telemetry] Fix cloud tests (NP version of #59040) #61271

Merged

2 tasks

afharo closed this as completed in #61271 Mar 25, 2020

lukeelmers added the Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc label Oct 1, 2021

lukeelmers removed the Team:KibanaTelemetry label Oct 1, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[test-failed]: X-Pack API Integration Tests.x-pack/test/api_integration/apis/telemetry/opt_in·ts - apis Telemetry /api/telemetry/v2/optIn API should support sending false #51522

[test-failed]: X-Pack API Integration Tests.x-pack/test/api_integration/apis/telemetry/opt_in·ts - apis Telemetry /api/telemetry/v2/optIn API should support sending false #51522

liza-mae commented Nov 22, 2019 •

edited

Loading

elasticmachine commented Nov 22, 2019

elasticmachine commented Nov 23, 2019

elasticmachine commented Dec 12, 2019

Bamieh commented Dec 13, 2019

liza-mae commented Dec 16, 2019

Bamieh commented Jan 9, 2020

liza-mae commented Jan 9, 2020

Bamieh commented Jan 9, 2020

liza-mae commented Jan 9, 2020

liza-mae commented Jan 24, 2020

liza-mae commented Jan 24, 2020 •

edited

Loading

Bamieh commented Feb 4, 2020

liza-mae commented Feb 27, 2020

elasticmachine commented Oct 1, 2021

[test-failed]: X-Pack API Integration Tests.x-pack/test/api_integration/apis/telemetry/opt_in·ts - apis Telemetry /api/telemetry/v2/optIn API should support sending false #51522

[test-failed]: X-Pack API Integration Tests.x-pack/test/api_integration/apis/telemetry/opt_in·ts - apis Telemetry /api/telemetry/v2/optIn API should support sending false #51522

Comments

liza-mae commented Nov 22, 2019 • edited Loading

elasticmachine commented Nov 22, 2019

elasticmachine commented Nov 23, 2019

elasticmachine commented Dec 12, 2019

Bamieh commented Dec 13, 2019

liza-mae commented Dec 16, 2019

Bamieh commented Jan 9, 2020

liza-mae commented Jan 9, 2020

Bamieh commented Jan 9, 2020

liza-mae commented Jan 9, 2020

liza-mae commented Jan 24, 2020

liza-mae commented Jan 24, 2020 • edited Loading

Bamieh commented Feb 4, 2020

liza-mae commented Feb 27, 2020

elasticmachine commented Oct 1, 2021

liza-mae commented Nov 22, 2019 •

edited

Loading

liza-mae commented Jan 24, 2020 •

edited

Loading