Skip to content

Commit

Permalink
[core.savedObjects] Add helper for using find with pit and search_aft…
Browse files Browse the repository at this point in the history
…er. (#92981) (#95374)

Co-authored-by: Luke Elmers <[email protected]>
  • Loading branch information
kibanamachine and lukeelmers authored Mar 25, 2021
1 parent 6993f20 commit 077caf4
Show file tree
Hide file tree
Showing 41 changed files with 904 additions and 167 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [ISavedObjectsPointInTimeFinder](./kibana-plugin-core-server.isavedobjectspointintimefinder.md) &gt; [close](./kibana-plugin-core-server.isavedobjectspointintimefinder.close.md)

## ISavedObjectsPointInTimeFinder.close property

Closes the Point-In-Time associated with this finder instance.

Once you have retrieved all of the results you need, it is recommended to call `close()` to clean up the PIT and prevent Elasticsearch from consuming resources unnecessarily. This is only required if you are done iterating and have not yet paged through all of the results: the PIT will automatically be closed for you once you reach the last page of results, or if the underlying call to `find` fails for any reason.

<b>Signature:</b>

```typescript
close: () => Promise<void>;
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [ISavedObjectsPointInTimeFinder](./kibana-plugin-core-server.isavedobjectspointintimefinder.md) &gt; [find](./kibana-plugin-core-server.isavedobjectspointintimefinder.find.md)

## ISavedObjectsPointInTimeFinder.find property

An async generator which wraps calls to `savedObjectsClient.find` and iterates over multiple pages of results using `_pit` and `search_after`<!-- -->. This will open a new Point-In-Time (PIT), and continue paging until a set of results is received that's smaller than the designated `perPage` size.

<b>Signature:</b>

```typescript
find: () => AsyncGenerator<SavedObjectsFindResponse>;
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [ISavedObjectsPointInTimeFinder](./kibana-plugin-core-server.isavedobjectspointintimefinder.md)

## ISavedObjectsPointInTimeFinder interface


<b>Signature:</b>

```typescript
export interface ISavedObjectsPointInTimeFinder
```

## Properties

| Property | Type | Description |
| --- | --- | --- |
| [close](./kibana-plugin-core-server.isavedobjectspointintimefinder.close.md) | <code>() =&gt; Promise&lt;void&gt;</code> | Closes the Point-In-Time associated with this finder instance.<!-- -->Once you have retrieved all of the results you need, it is recommended to call <code>close()</code> to clean up the PIT and prevent Elasticsearch from consuming resources unnecessarily. This is only required if you are done iterating and have not yet paged through all of the results: the PIT will automatically be closed for you once you reach the last page of results, or if the underlying call to <code>find</code> fails for any reason. |
| [find](./kibana-plugin-core-server.isavedobjectspointintimefinder.find.md) | <code>() =&gt; AsyncGenerator&lt;SavedObjectsFindResponse&gt;</code> | An async generator which wraps calls to <code>savedObjectsClient.find</code> and iterates over multiple pages of results using <code>_pit</code> and <code>search_after</code>. This will open a new Point-In-Time (PIT), and continue paging until a set of results is received that's smaller than the designated <code>perPage</code> size. |

3 changes: 3 additions & 0 deletions docs/development/core/server/kibana-plugin-core-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ The plugin integrates with the core system via lifecycle events: `setup`<!-- -->
| [IndexSettingsDeprecationInfo](./kibana-plugin-core-server.indexsettingsdeprecationinfo.md) | |
| [IRenderOptions](./kibana-plugin-core-server.irenderoptions.md) | |
| [IRouter](./kibana-plugin-core-server.irouter.md) | Registers route handlers for specified resource path and method. See [RouteConfig](./kibana-plugin-core-server.routeconfig.md) and [RequestHandler](./kibana-plugin-core-server.requesthandler.md) for more information about arguments to route registrations. |
| [ISavedObjectsPointInTimeFinder](./kibana-plugin-core-server.isavedobjectspointintimefinder.md) | |
| [IScopedClusterClient](./kibana-plugin-core-server.iscopedclusterclient.md) | Serves the same purpose as the normal [cluster client](./kibana-plugin-core-server.iclusterclient.md) but exposes an additional <code>asCurrentUser</code> method that doesn't use credentials of the Kibana internal user (as <code>asInternalUser</code> does) to request Elasticsearch API, but rather passes HTTP headers extracted from the current user request to the API instead. |
| [IUiSettingsClient](./kibana-plugin-core-server.iuisettingsclient.md) | Server-side client that provides access to the advanced settings stored in elasticsearch. The settings provide control over the behavior of the Kibana application. For example, a user can specify how to display numeric or date fields. Users can adjust the settings via Management UI. |
| [KibanaRequestEvents](./kibana-plugin-core-server.kibanarequestevents.md) | Request events. |
Expand Down Expand Up @@ -158,6 +159,7 @@ The plugin integrates with the core system via lifecycle events: `setup`<!-- -->
| [SavedObjectsComplexFieldMapping](./kibana-plugin-core-server.savedobjectscomplexfieldmapping.md) | See [SavedObjectsFieldMapping](./kibana-plugin-core-server.savedobjectsfieldmapping.md) for documentation. |
| [SavedObjectsCoreFieldMapping](./kibana-plugin-core-server.savedobjectscorefieldmapping.md) | See [SavedObjectsFieldMapping](./kibana-plugin-core-server.savedobjectsfieldmapping.md) for documentation. |
| [SavedObjectsCreateOptions](./kibana-plugin-core-server.savedobjectscreateoptions.md) | |
| [SavedObjectsCreatePointInTimeFinderDependencies](./kibana-plugin-core-server.savedobjectscreatepointintimefinderdependencies.md) | |
| [SavedObjectsDeleteByNamespaceOptions](./kibana-plugin-core-server.savedobjectsdeletebynamespaceoptions.md) | |
| [SavedObjectsDeleteFromNamespacesOptions](./kibana-plugin-core-server.savedobjectsdeletefromnamespacesoptions.md) | |
| [SavedObjectsDeleteFromNamespacesResponse](./kibana-plugin-core-server.savedobjectsdeletefromnamespacesresponse.md) | |
Expand Down Expand Up @@ -305,6 +307,7 @@ The plugin integrates with the core system via lifecycle events: `setup`<!-- -->
| [SavedObjectsClientFactoryProvider](./kibana-plugin-core-server.savedobjectsclientfactoryprovider.md) | Provider to invoke to retrieve a [SavedObjectsClientFactory](./kibana-plugin-core-server.savedobjectsclientfactory.md)<!-- -->. |
| [SavedObjectsClientWrapperFactory](./kibana-plugin-core-server.savedobjectsclientwrapperfactory.md) | Describes the factory used to create instances of Saved Objects Client Wrappers. |
| [SavedObjectsClosePointInTimeOptions](./kibana-plugin-core-server.savedobjectsclosepointintimeoptions.md) | |
| [SavedObjectsCreatePointInTimeFinderOptions](./kibana-plugin-core-server.savedobjectscreatepointintimefinderoptions.md) | |
| [SavedObjectsExportTransform](./kibana-plugin-core-server.savedobjectsexporttransform.md) | Transformation function used to mutate the exported objects of the associated type.<!-- -->A type's export transform function will be executed once per user-initiated export, for all objects of that type. |
| [SavedObjectsFieldMapping](./kibana-plugin-core-server.savedobjectsfieldmapping.md) | Describe a [saved object type mapping](./kibana-plugin-core-server.savedobjectstypemappingdefinition.md) field.<!-- -->Please refer to [elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html) For the mapping documentation |
| [SavedObjectsImportHook](./kibana-plugin-core-server.savedobjectsimporthook.md) | A hook associated with a specific saved object type, that will be invoked during the import process. The hook will have access to the objects of the registered type.<!-- -->Currently, the only supported feature for import hooks is to return warnings to be displayed in the UI when the import succeeds. The only interactions the hook can have with the import process is via the hook's response. Mutating the objects inside the hook's code will have no effect. |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@

Closes a Point In Time (PIT) by ID. This simply proxies the request to ES via the Elasticsearch client, and is included in the Saved Objects Client as a convenience for consumers who are using [SavedObjectsClient.openPointInTimeForType()](./kibana-plugin-core-server.savedobjectsclient.openpointintimefortype.md)<!-- -->.

Only use this API if you have an advanced use case that's not solved by the [SavedObjectsClient.createPointInTimeFinder()](./kibana-plugin-core-server.savedobjectsclient.createpointintimefinder.md) method.

<b>Signature:</b>

```typescript
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [SavedObjectsClient](./kibana-plugin-core-server.savedobjectsclient.md) &gt; [createPointInTimeFinder](./kibana-plugin-core-server.savedobjectsclient.createpointintimefinder.md)

## SavedObjectsClient.createPointInTimeFinder() method

Returns a [ISavedObjectsPointInTimeFinder](./kibana-plugin-core-server.isavedobjectspointintimefinder.md) to help page through large sets of saved objects. We strongly recommend using this API for any `find` queries that might return more than 1000 saved objects, however this API is only intended for use in server-side "batch" processing of objects where you are collecting all objects in memory or streaming them back to the client.

Do NOT use this API in a route handler to facilitate paging through saved objects on the client-side unless you are streaming all of the results back to the client at once. Because the returned generator is stateful, you cannot rely on subsequent http requests retrieving new pages from the same Kibana server in multi-instance deployments.

The generator wraps calls to [SavedObjectsClient.find()](./kibana-plugin-core-server.savedobjectsclient.find.md) and iterates over multiple pages of results using `_pit` and `search_after`<!-- -->. This will open a new Point-In-Time (PIT), and continue paging until a set of results is received that's smaller than the designated `perPage`<!-- -->.

Once you have retrieved all of the results you need, it is recommended to call `close()` to clean up the PIT and prevent Elasticsearch from consuming resources unnecessarily. This is only required if you are done iterating and have not yet paged through all of the results: the PIT will automatically be closed for you once you reach the last page of results, or if the underlying call to `find` fails for any reason.

<b>Signature:</b>

```typescript
createPointInTimeFinder(findOptions: SavedObjectsCreatePointInTimeFinderOptions, dependencies?: SavedObjectsCreatePointInTimeFinderDependencies): ISavedObjectsPointInTimeFinder;
```

## Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| findOptions | <code>SavedObjectsCreatePointInTimeFinderOptions</code> | |
| dependencies | <code>SavedObjectsCreatePointInTimeFinderDependencies</code> | |

<b>Returns:</b>

`ISavedObjectsPointInTimeFinder`

## Example


```ts
const findOptions: SavedObjectsCreatePointInTimeFinderOptions = {
type: 'visualization',
search: 'foo*',
perPage: 100,
};

const finder = savedObjectsClient.createPointInTimeFinder(findOptions);

const responses: SavedObjectFindResponse[] = [];
for await (const response of finder.find()) {
responses.push(...response);
if (doneSearching) {
await finder.close();
}
}

```

Original file line number Diff line number Diff line change
Expand Up @@ -30,13 +30,14 @@ The constructor for this class is marked as internal. Third-party code should no
| [bulkGet(objects, options)](./kibana-plugin-core-server.savedobjectsclient.bulkget.md) | | Returns an array of objects by id |
| [bulkUpdate(objects, options)](./kibana-plugin-core-server.savedobjectsclient.bulkupdate.md) | | Bulk Updates multiple SavedObject at once |
| [checkConflicts(objects, options)](./kibana-plugin-core-server.savedobjectsclient.checkconflicts.md) | | Check what conflicts will result when creating a given array of saved objects. This includes "unresolvable conflicts", which are multi-namespace objects that exist in a different namespace; such conflicts cannot be resolved/overwritten. |
| [closePointInTime(id, options)](./kibana-plugin-core-server.savedobjectsclient.closepointintime.md) | | Closes a Point In Time (PIT) by ID. This simply proxies the request to ES via the Elasticsearch client, and is included in the Saved Objects Client as a convenience for consumers who are using [SavedObjectsClient.openPointInTimeForType()](./kibana-plugin-core-server.savedobjectsclient.openpointintimefortype.md)<!-- -->. |
| [closePointInTime(id, options)](./kibana-plugin-core-server.savedobjectsclient.closepointintime.md) | | Closes a Point In Time (PIT) by ID. This simply proxies the request to ES via the Elasticsearch client, and is included in the Saved Objects Client as a convenience for consumers who are using [SavedObjectsClient.openPointInTimeForType()](./kibana-plugin-core-server.savedobjectsclient.openpointintimefortype.md)<!-- -->.<!-- -->Only use this API if you have an advanced use case that's not solved by the [SavedObjectsClient.createPointInTimeFinder()](./kibana-plugin-core-server.savedobjectsclient.createpointintimefinder.md) method. |
| [create(type, attributes, options)](./kibana-plugin-core-server.savedobjectsclient.create.md) | | Persists a SavedObject |
| [createPointInTimeFinder(findOptions, dependencies)](./kibana-plugin-core-server.savedobjectsclient.createpointintimefinder.md) | | Returns a [ISavedObjectsPointInTimeFinder](./kibana-plugin-core-server.isavedobjectspointintimefinder.md) to help page through large sets of saved objects. We strongly recommend using this API for any <code>find</code> queries that might return more than 1000 saved objects, however this API is only intended for use in server-side "batch" processing of objects where you are collecting all objects in memory or streaming them back to the client.<!-- -->Do NOT use this API in a route handler to facilitate paging through saved objects on the client-side unless you are streaming all of the results back to the client at once. Because the returned generator is stateful, you cannot rely on subsequent http requests retrieving new pages from the same Kibana server in multi-instance deployments.<!-- -->The generator wraps calls to [SavedObjectsClient.find()](./kibana-plugin-core-server.savedobjectsclient.find.md) and iterates over multiple pages of results using <code>_pit</code> and <code>search_after</code>. This will open a new Point-In-Time (PIT), and continue paging until a set of results is received that's smaller than the designated <code>perPage</code>.<!-- -->Once you have retrieved all of the results you need, it is recommended to call <code>close()</code> to clean up the PIT and prevent Elasticsearch from consuming resources unnecessarily. This is only required if you are done iterating and have not yet paged through all of the results: the PIT will automatically be closed for you once you reach the last page of results, or if the underlying call to <code>find</code> fails for any reason. |
| [delete(type, id, options)](./kibana-plugin-core-server.savedobjectsclient.delete.md) | | Deletes a SavedObject |
| [deleteFromNamespaces(type, id, namespaces, options)](./kibana-plugin-core-server.savedobjectsclient.deletefromnamespaces.md) | | Removes namespaces from a SavedObject |
| [find(options)](./kibana-plugin-core-server.savedobjectsclient.find.md) | | Find all SavedObjects matching the search query |
| [get(type, id, options)](./kibana-plugin-core-server.savedobjectsclient.get.md) | | Retrieves a single object |
| [openPointInTimeForType(type, options)](./kibana-plugin-core-server.savedobjectsclient.openpointintimefortype.md) | | Opens a Point In Time (PIT) against the indices for the specified Saved Object types. The returned <code>id</code> can then be passed to [SavedObjectsClient.find()](./kibana-plugin-core-server.savedobjectsclient.find.md) to search against that PIT. |
| [openPointInTimeForType(type, options)](./kibana-plugin-core-server.savedobjectsclient.openpointintimefortype.md) | | Opens a Point In Time (PIT) against the indices for the specified Saved Object types. The returned <code>id</code> can then be passed to [SavedObjectsClient.find()](./kibana-plugin-core-server.savedobjectsclient.find.md) to search against that PIT.<!-- -->Only use this API if you have an advanced use case that's not solved by the [SavedObjectsClient.createPointInTimeFinder()](./kibana-plugin-core-server.savedobjectsclient.createpointintimefinder.md) method. |
| [removeReferencesTo(type, id, options)](./kibana-plugin-core-server.savedobjectsclient.removereferencesto.md) | | Updates all objects containing a reference to the given {<!-- -->type, id<!-- -->} tuple to remove the said reference. |
| [resolve(type, id, options)](./kibana-plugin-core-server.savedobjectsclient.resolve.md) | | Resolves a single object, using any legacy URL alias if it exists |
| [update(type, id, attributes, options)](./kibana-plugin-core-server.savedobjectsclient.update.md) | | Updates an SavedObject |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@

Opens a Point In Time (PIT) against the indices for the specified Saved Object types. The returned `id` can then be passed to [SavedObjectsClient.find()](./kibana-plugin-core-server.savedobjectsclient.find.md) to search against that PIT.

Only use this API if you have an advanced use case that's not solved by the [SavedObjectsClient.createPointInTimeFinder()](./kibana-plugin-core-server.savedobjectsclient.createpointintimefinder.md) method.

<b>Signature:</b>

```typescript
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [SavedObjectsCreatePointInTimeFinderDependencies](./kibana-plugin-core-server.savedobjectscreatepointintimefinderdependencies.md) &gt; [client](./kibana-plugin-core-server.savedobjectscreatepointintimefinderdependencies.client.md)

## SavedObjectsCreatePointInTimeFinderDependencies.client property

<b>Signature:</b>

```typescript
client: Pick<SavedObjectsClientContract, 'find' | 'openPointInTimeForType' | 'closePointInTime'>;
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [SavedObjectsCreatePointInTimeFinderDependencies](./kibana-plugin-core-server.savedobjectscreatepointintimefinderdependencies.md)

## SavedObjectsCreatePointInTimeFinderDependencies interface


<b>Signature:</b>

```typescript
export interface SavedObjectsCreatePointInTimeFinderDependencies
```

## Properties

| Property | Type | Description |
| --- | --- | --- |
| [client](./kibana-plugin-core-server.savedobjectscreatepointintimefinderdependencies.client.md) | <code>Pick&lt;SavedObjectsClientContract, 'find' &#124; 'openPointInTimeForType' &#124; 'closePointInTime'&gt;</code> | |

Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [SavedObjectsCreatePointInTimeFinderOptions](./kibana-plugin-core-server.savedobjectscreatepointintimefinderoptions.md)

## SavedObjectsCreatePointInTimeFinderOptions type


<b>Signature:</b>

```typescript
export declare type SavedObjectsCreatePointInTimeFinderOptions = Omit<SavedObjectsFindOptions, 'page' | 'pit' | 'searchAfter'>;
```
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@

Closes a Point In Time (PIT) by ID. This simply proxies the request to ES via the Elasticsearch client, and is included in the Saved Objects Client as a convenience for consumers who are using `openPointInTimeForType`<!-- -->.

Only use this API if you have an advanced use case that's not solved by the [SavedObjectsRepository.createPointInTimeFinder()](./kibana-plugin-core-server.savedobjectsrepository.createpointintimefinder.md) method.

<b>Signature:</b>

```typescript
Expand Down
Loading

0 comments on commit 077caf4

Please sign in to comment.