/api/kibana/management/saved_objects/scroll/counts very slow when lot of documents

[pgayvallet]

When testing on a 500k objects cluster, I saw that the call to /api/kibana/management/saved_objects/scroll/counts takes approximatively 2minutes. During this time, the counts are not properly surfaced in the UI.

We can probably improve that by increasing the number of documents per page:

kibana/src/plugins/saved_objects_management/server/routes/scroll_count.ts

Lines 45 to 48 in a02c00b

	const findOptions: SavedObjectsCreatePointInTimeFinderOptions = {
	type: typesToInclude,
	perPage: 1000,
	};

Plus, we're currently storing all the results in memory, just to count the number of document per type

kibana/src/plugins/saved_objects_management/server/routes/scroll_count.ts

Lines 58 to 60 in a02c00b

	const objects = await findAll(client, findOptions);
	const counts = objects.reduce((accum, result) => {

We should increment the counter between each batch instead of keeping all documents in memory.

Also, we can probably improve the search request by asking to not return any attribute fields (See SavedObjectsFindOptions.fields) which should also increase the overall performances.

As an alternative, we could also see if we can use an aggregation instead of a full scroll.

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[rahuldimri]

@pgayvallet Can I work on this ?

[pgayvallet]

@rahuldimri Sure, please feel free to take a shot at it!

[rahuldimri]

Thanks @pgayvallet can you assign this one to me.

[pgayvallet]

Fixed by #168918