Failing test: X-Pack Fleet API Integration Tests.x-pack/test/fleet_api_integration/apis/integrations/inputs_with_standalone_docker_agent·ts - Fleet Endpoints Integrations inputs_with_standalone_docker_agent "before all" hook for "generate a valid config for standalone agents"

[kibanamachine]

A test failed on a tracked branch

AxiosError: Request failed with status code 502
    at settle (node_modules/axios/lib/core/settle.js:19:12)
    at IncomingMessage.handleStreamEnd (node_modules/axios/lib/adapters/http.js:589:11)
    at IncomingMessage.emit (node:events:531:35)
    at endReadableNT (node:internal/streams/readable:1696:12)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)
    at Axios.request (node_modules/axios/lib/core/Axios.js:45:41)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at getLatestVersion (artifact_manager.ts:12:25)
    at Context.<anonymous> (inputs_with_standalone_docker_agent.ts:43:61)
    at Object.apply (wrap_function.js:73:16) {
  code: 'ERR_BAD_RESPONSE',
  config: {
    transitional: {
      silentJSONParsing: true,
      forcedJSONParsing: true,
      clarifyTimeoutError: false
    },
    adapter: [ 'xhr', 'http' ],
    transformRequest: [ [Function: transformRequest] ],
    transformResponse: [ [Function: transformResponse] ],
    timeout: 0,
    xsrfCookieName: 'XSRF-TOKEN',
    xsrfHeaderName: 'X-XSRF-TOKEN',
    maxContentLength: -1,
    maxBodyLength: -1,
    env: { FormData: [Function], Blob: [class Blob] },
    validateStatus: [Function: validateStatus],
    headers: Object [AxiosHeaders] {
      Accept: 'application/json, text/plain, */*',
      'Content-Type': undefined,
      'User-Agent': 'axios/1.6.8',
      'Accept-Encoding': 'gzip, compress, deflate, br'
    },
    url: 'https://artifacts-api.elastic.co/v1/versions',
    method: 'get',
    data: undefined
  },
  request: <ref *1> ClientRequest {
    _events: [Object: null prototype] {
      abort: [Function (anonymous)],
      aborted: [Function (anonymous)],
      connect: [Function (anonymous)],
      error: [Function (anonymous)],
      socket: [Function (anonymous)],
      timeout: [Function (anonymous)],
      finish: [Function: requestOnFinish]
    },
    _eventsCount: 7,
    _maxListeners: undefined,
    outputData: [],
    outputSize: 0,
    writable: true,
    destroyed: true,
    _last: true,
    chunkedEncoding: false,
    shouldKeepAlive: true,
    maxRequestsOnConnectionReached: false,
    _defaultKeepAlive: true,
    useChunkedEncodingByDefault: false,
    sendDate: false,
    _removedConnection: false,
    _removedContLen: false,
    _removedTE: false,
    strictContentLength: false,
    _contentLength: 0,
    _hasBody: true,
    _trailer: '',
    finished: true,
    _headerSent: true,
    _closed: true,
    socket: TLSSocket {
      _tlsOptions: [Object],
      _secureEstablished: true,
      _securePending: false,
      _newSessionPending: false,
      _controlReleased: true,
      secureConnecting: false,
      _SNICallback: null,
      servername: 'artifacts-api.elastic.co',
      alpnProtocol: false,
      authorized: true,
      authorizationError: null,
      encrypted: true,
      _events: [Object: null prototype],
      _eventsCount: 9,
      connecting: false,
      _hadError: false,
      _parent: null,
      _host: 'artifacts-api.elastic.co',
      _closeAfterHandlingError: false,
      _readableState: [ReadableState],
      _writableState: [WritableState],
      allowHalfOpen: false,
      _maxListeners: undefined,
      _sockname: null,
      _pendingData: null,
      _pendingEncoding: '',
      server: undefined,
      _server: null,
      ssl: [TLSWrap],
      _requestCert: true,
      _rejectUnauthorized: true,
      timeout: 5000,
      parser: null,
      _httpMessage: null,
      [Symbol(alpncallback)]: null,
      [Symbol(res)]: [TLSWrap],
      [Symbol(verified)]: true,
      [Symbol(pendingSession)]: null,
      [Symbol(async_id_symbol)]: -1,
      [Symbol(kHandle)]: [TLSWrap],
      [Symbol(lastWriteQueueSize)]: 0,
      [Symbol(timeout)]: Timeout {
        _idleTimeout: 5000,
        _idlePrev: [TimersList],
        _idleNext: [TimersList],
        _idleStart: 939979,
        _onTimeout: [Function: bound ],
        _timerArgs: undefined,
        _repeat: null,
        _destroyed: false,
        [Symbol(refed)]: false,
        [Symbol(kHasPrimitive)]: false,
        [Symbol(asyncId)]: 116219,
        [Symbol(triggerId)]: 116217
      },
      [Symbol(kBuffer)]: null,
      [Symbol(kBufferCb)]: null,
      [Symbol(kBufferGen)]: null,
      [Symbol(shapeMode)]: true,
      [Symbol(kCapture)]: false,
      [Symbol(kSetNoDelay)]: false,
      [Symbol(kSetKeepAlive)]: true,
      [Symbol(kSetKeepAliveInitialDelay)]: 1,
      [Symbol(kBytesRead)]: 0,
      [Symbol(kBytesWritten)]: 0,
      [Symbol(connect-options)]: [Object]
    },
    _header: 'GET /v1/versions HTTP/1.1\r\n' +
      'Accept: application/json, text/plain, */*\r\n' +
      'User-Agent: axios/1.6.8\r\n' +
      'Accept-Encoding: gzip, compress, deflate, br\r\n' +
      'Host: artifacts-api.elastic.co\r\n' +
      'Connection: keep-alive\r\n' +
      '\r\n',
    _keepAliveTimeout: 0,
    _onPendingData: [Function: nop],
    agent: Agent {
      _events: [Object: null prototype],
      _eventsCount: 2,
      _maxListeners: undefined,
      defaultPort: 443,
      protocol: 'https:',
      options: [Object: null prototype],
      requests: [Object: null prototype] {},
      sockets: [Object: null prototype] {},
      freeSockets: [Object: null prototype],
      keepAliveMsecs: 1000,
      keepAlive: true,
      maxSockets: Infinity,
      maxFreeSockets: 256,
      scheduling: 'lifo',
      maxTotalSockets: Infinity,
      totalSocketCount: 1,
      maxCachedSessions: 100,
      _sessionCache: [Object],
      [Symbol(shapeMode)]: false,
      [Symbol(kCapture)]: false
    },
    socketPath: undefined,
    method: 'GET',
    maxHeaderSize: undefined,
    insecureHTTPParser: undefined,
    joinDuplicateHeaders: undefined,
    path: '/v1/versions',
    _ended: true,
    res: { method: null, url: '', complete: true, aborted: false },
    aborted: false,
    timeoutCb: null,
    upgradeOrConnect: false,
    parser: null,
    maxHeadersCount: null,
    reusedSocket: false,
    host: 'artifacts-api.elastic.co',
    protocol: 'https:',
    _redirectable: Writable {
      _events: [Object],
      _writableState: [WritableState],
      _maxListeners: undefined,
      _options: [Object],
      _ended: true,
      _ending: true,
      _redirectCount: 0,
      _redirects: [],
      _requestBodyLength: 0,
      _requestBodyBuffers: [],
      _eventsCount: 3,
      _onNativeResponse: [Function (anonymous)],
      _currentRequest: [Circular *1],
      _currentUrl: 'https://artifacts-api.elastic.co/v1/versions',
      [Symbol(shapeMode)]: true,
      [Symbol(kCapture)]: false
    },
    [Symbol(shapeMode)]: false,
    [Symbol(kCapture)]: false,
    [Symbol(kBytesWritten)]: 0,
    [Symbol(kNeedDrain)]: false,
    [Symbol(corked)]: 0,
    [Symbol(kOutHeaders)]: [Object: null prototype] {
      accept: [Array],
      'user-agent': [Array],
      'accept-encoding': [Array],
      host: [Array]
    },
    [Symbol(errored)]: null,
    [Symbol(kHighWaterMark)]: 16384,
    [Symbol(kRejectNonStandardBodyWrites)]: false,
    [Symbol(kUniqueHeaders)]: null
  },
  response: {
    status: 502,
    statusText: 'Bad Gateway',
    headers: Object [AxiosHeaders] {
      date: 'Mon, 03 Jun 2024 18:13:56 GMT',
      'content-type': 'text/html',
      'content-length': '150',
      connection: 'keep-alive',
      'strict-transport-security': 'max-age=15724800; includeSubDomains'
    },
    config: {
      transitional: [Object],
      adapter: [Array],
      transformRequest: [Array],
      transformResponse: [Array],
      timeout: 0,
      xsrfCookieName: 'XSRF-TOKEN',
      xsrfHeaderName: 'X-XSRF-TOKEN',
      maxContentLength: -1,
      maxBodyLength: -1,
      env: [Object],
      validateStatus: [Function: validateStatus],
      headers: [Object [AxiosHeaders]],
      url: 'https://artifacts-api.elastic.co/v1/versions',
      method: 'get',
      data: undefined
    },
    request: <ref *1> ClientRequest {
      _events: [Object: null prototype],
      _eventsCount: 7,
      _maxListeners: undefined,
      outputData: [],
      outputSize: 0,
      writable: true,
      destroyed: true,
      _last: true,
      chunkedEncoding: false,
      shouldK
[report_failure] output truncated to 8192 characters

First failure: kibana-on-merge - main

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[kpollich]

@nchaulet - I think maybe we need retries for spinning up this docker container? Sometimes the elastic docker registry 502's. We have the same problem in FTR tests.

[nchaulet]

@nchaulet - I think maybe we need retries for spinning up this docker container? Sometimes the elastic docker registry 502's. We have the same problem in FTR tests.

Looks like the flakyness was in calling the artificat API (retrieving elastic-agent docker image version) we probably can add some retry to that request to start. Or maybe have a fixed version

[kibanamachine]

New failure: kibana-on-merge - main

[kibanamachine]

New failure: kibana-on-merge - main

[kibanamachine]

New failure: kibana-on-merge - main

[kibanamachine]

New failure: kibana-on-merge - main

[kibanamachine]

New failure: kibana-on-merge - main

[mistic]

Skipped.

main: 2194f85

[jlind23]

@nchaulet @kpollich weren't we talking about retries for https://artifacts-api.elastic.co/v1/versions
? Those tests are useful but brought up a lot of flakiness.

[nchaulet]

@jlind23 Yes just did a PR that add those retries and a fallback value #186459

[kibanamachine]

New failure: kibana-on-merge - main

[kibanamachine]

New failure: kibana-on-merge - main

[kibanamachine]

New failure: kibana-on-merge - main

[kibanamachine]

New failure: kibana-on-merge - main

[kibanamachine]

New failure: kibana-on-merge - main

[kibanamachine]

New failure: kibana-on-merge - main

[delanni]

/skip

[kibanamachine]

Skipped

main: 3f3a842

[delanni]

Missing docker image for elastic-agent: docker.elastic.co/beats/elastic-agent:8.15.0

[kibanamachine]

New failure: kibana-on-merge - main

[kibanamachine]

New failure: kibana-on-merge - main

[jbudz]

/skip

edit: missed the skip above, whoops.

[kibanamachine]

Skipped

main: 7968b14

[kibanamachine]

New failure: kibana-elasticsearch-snapshot-verify - main

[kpollich]

No failures since July. I think this is fixed by the Linked PR we just had a single failure after the fact.