Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing test: X-Pack API Integration Tests.x-pack/test/api_integration/apis/entity_manager/search·ts - Entity Manager _search API includes source and additional metadata fields #203982

Closed
kibanamachine opened this issue Dec 12, 2024 · 19 comments · Fixed by #204183
Closed

Failing test: X-Pack API Integration Tests.x-pack/test/api_integration/apis/entity_manager/search·ts - Entity Manager _search API includes source and additional metadata fields #203982

kibanamachine opened this issue Dec 12, 2024 · 19 comments · Fixed by #204183
Assignees
@klacabane
Labels
failed-test A test failure on a tracked branch, potentially flaky-test needs-team Issues missing a team label

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Dec 12, 2024 •
edited
Loading

A test failed on a tracked branch

JestAssertionError: expect(received).toEqual(expected) // deep equality

- Expected  - 1
+ Received  + 4

  Array [
    Object {
      "agent.name": ArrayContaining [
        "agent-one",
        "agent-ten",
      ],
      "entity.display_name": "service-one",
      "entity.id": "service-one",
      "entity.last_seen_timestamp": "2024-12-12T10:20:02.558Z",
      "entity.type": "services-with-hosts",
      "host.name": ArrayContaining [
        "host-one",
        "host-two",
        "host-ten",
      ],
      "service.name": "service-one",
    },
    Object {
-     "agent.name": "agent-nine",
+     "agent.name": Array [
+       "agent-nine",
+       null,
+     ],
      "entity.display_name": "service-two",
      "entity.id": "service-two",
      "entity.last_seen_timestamp": "2024-12-12T10:19:02.558Z",
      "entity.type": "services-with-hosts",
      "host.name": ArrayContaining [
        "host-three",
        "host-nine",
      ],
      "service.name": "service-two",
    },
  ]
    at Context.<anonymous> (search.ts:395:24)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Object.apply (wrap_function.js:74:16) {
  matcherResult: {
    actual: [ [Object], [Object] ],
    expected: [ [Object], [Object] ],
    message: '\x1B[2mexpect(\x1B[22m\x1B[31mreceived\x1B[39m\x1B[2m).\x1B[22mtoEqual\x1B[2m(\x1B[22m\x1B[32mexpected\x1B[39m\x1B[2m) // deep equality\x1B[22m\n' +
      '\n' +
      '\x1B[32m- Expected  - 1\x1B[39m\n' +
      '\x1B[31m+ Received  + 4\x1B[39m\n' +
      '\n' +
      '\x1B[2m  Array [\x1B[22m\n' +
      '\x1B[2m    Object {\x1B[22m\n' +
      '\x1B[2m      "agent.name": ArrayContaining [\x1B[22m\n' +
      '\x1B[2m        "agent-one",\x1B[22m\n' +
      '\x1B[2m        "agent-ten",\x1B[22m\n' +
      '\x1B[2m      ],\x1B[22m\n' +
      '\x1B[2m      "entity.display_name": "service-one",\x1B[22m\n' +
      '\x1B[2m      "entity.id": "service-one",\x1B[22m\n' +
      '\x1B[2m      "entity.last_seen_timestamp": "2024-12-12T10:20:02.558Z",\x1B[22m\n' +
      '\x1B[2m      "entity.type": "services-with-hosts",\x1B[22m\n' +
      '\x1B[2m      "host.name": ArrayContaining [\x1B[22m\n' +
      '\x1B[2m        "host-one",\x1B[22m\n' +
      '\x1B[2m        "host-two",\x1B[22m\n' +
      '\x1B[2m        "host-ten",\x1B[22m\n' +
      '\x1B[2m      ],\x1B[22m\n' +
      '\x1B[2m      "service.name": "service-one",\x1B[22m\n' +
      '\x1B[2m    },\x1B[22m\n' +
      '\x1B[2m    Object {\x1B[22m\n' +
      '\x1B[32m-     "agent.name": "agent-nine",\x1B[39m\n' +
      '\x1B[31m+     "agent.name": Array [\x1B[39m\n' +
      '\x1B[31m+       "agent-nine",\x1B[39m\n' +
      '\x1B[31m+       null,\x1B[39m\n' +
      '\x1B[31m+     ],\x1B[39m\n' +
      '\x1B[2m      "entity.display_name": "service-two",\x1B[22m\n' +
      '\x1B[2m      "entity.id": "service-two",\x1B[22m\n' +
      '\x1B[2m      "entity.last_seen_timestamp": "2024-12-12T10:19:02.558Z",\x1B[22m\n' +
      '\x1B[2m      "entity.type": "services-with-hosts",\x1B[22m\n' +
      '\x1B[2m      "host.name": ArrayContaining [\x1B[22m\n' +
      '\x1B[2m        "host-three",\x1B[22m\n' +
      '\x1B[2m        "host-nine",\x1B[22m\n' +
      '\x1B[2m      ],\x1B[22m\n' +
      '\x1B[2m      "service.name": "service-two",\x1B[22m\n' +
      '\x1B[2m    },\x1B[22m\n' +
      '\x1B[2m  ]\x1B[22m',
    name: 'toEqual',
    pass: false
  }
}

First failure: kibana-on-merge - main
@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Dec 12, 2024
@botelastic botelastic bot added the needs-team Issues missing a team label label Dec 12, 2024
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@delanni
Copy link
Contributor

delanni commented Dec 12, 2024

/skip

kibanamachine added a commit that referenced this issue Dec 12, 2024
@kibanamachine
skip failing test suite (#203982)
6154ddf
@kibanamachine
Copy link
Contributor Author

Skipped

main: 6154ddf
8.x: 224e36e

kibanamachine added a commit that referenced this issue Dec 12, 2024
@kibanamachine
skip failing test suite (#203982)
224e36e
@delanni
Copy link
Contributor

delanni commented Dec 12, 2024

@klacabane - I can see this is a new test. It seems to be quite flaky on CI, so I skipped it for now

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@klacabane klacabane self-assigned this Dec 13, 2024
@klacabane
Copy link
Contributor

The problem lies in the merging logic, I reproduced it with this unit test[1] which fails in a similary way. Now the flakiness comes from the random arrival time of the queries. since we execute two queries in parallel, if the first one has the null value for a field the merging logic will ignore it and test succeeds, if the null value arrives second it fails. I'll send a fix

+    it('ignores null values when merging', () => {
+      const entities = [
+        {
+          'entity.id': 'foo',
+          'entity.type': 'service',
+          'entity.display_name': 'foo',
+          'service.name': 'foo',
+          'host.name': null,
+        },
+        {
+          'entity.id': 'foo',
+          'entity.type': 'service',
+          'entity.display_name': 'foo',
+          'service.name': 'foo',
+          'host.name': 'host-2',
+        },
+        {
+          'entity.id': 'foo',
+          'entity.type': 'service',
+          'entity.display_name': 'foo',
+          'service.name': 'foo',
+          'host.name': null,
+        },
+      ];
+
+      const mergedEntities = mergeEntitiesList({
+        entities,
+        metadataFields: ['host.name'],
+        sources: [
+          { identity_fields: ['service.name'], metadata_fields: [] as string[] },
+          { identity_fields: ['service.name'], metadata_fields: [] as string[] },
+        ] as EntitySourceDefinition[],
+      });
+      expect(mergedEntities.length).toEqual(1);
+      expect(mergedEntities[0]).toEqual({
+        'entity.id': 'foo',
+        'entity.type': 'service',
+        'entity.display_name': 'foo',
+        'service.name': 'foo',
+        'host.name': 'host-2',
+      });
+    });

---->
   expect(received).toEqual(expected) // deep equality

    - Expected  - 1
    + Received  + 4

      Object {
        "entity.display_name": "foo",
        "entity.id": "foo",
        "entity.type": "service",
    -   "host.name": "host-2",
    +   "host.name": Array [
    +     "host-2",
    +     null,
    +   ],
        "service.name": "foo",
      }

@klacabane klacabane mentioned this issue Dec 13, 2024
Merged
@klacabane
Copy link
Contributor

flaky test build https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7595

kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Dec 13, 2024
@klacabane
[eem] merging nulls fix (elastic#204183)
e1f9a11 
Closes elastic#203982

Ignore nulls when merging metadata fields.

The flakiness comes from the random arrival time of the queries. since
we execute two queries in parallel, if the first one has the null value
for a field the merging logic will ignore it and test succeeds, if the
null value arrives second it fails.

(cherry picked from commit d89153d)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@klacabane klacabane

Labels
failed-test A test failure on a tracked branch, potentially flaky-test needs-team Issues missing a team label
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[eem] merging nulls fix klacabane/kibana
3 participants
@delanni @klacabane @kibanamachine