Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ingest Manager] Provide for conflict detection/resolution on Datasource update #69992

Closed
madirey opened this issue Jun 25, 2020 · 3 comments · Fixed by #70680
Closed

[Ingest Manager] Provide for conflict detection/resolution on Datasource update #69992

madirey opened this issue Jun 25, 2020 · 3 comments · Fixed by #70680
Assignees
Labels
Feature:Endpoint Elastic Endpoint feature Team:Fleet Team label for Observability Data Collection Fleet team

Comments

@madirey
Copy link
Contributor

madirey commented Jun 25, 2020

Describe the feature:
The Datasource service does not currently provide for a way to detect conflicts when multiple updates are performed at once. This opens us up to potentially nasty race conditions, as we have several places within the app where non-atomic datasource updates are performed.

Describe a specific use case for the feature:
One potential race condition is the following scenario:

  1. A user manually updates a policy from the Endpoint UI.
  2. At the same time, another user adds an allowlist entry (exception).
  3. In both cases, the Endpoint data sources will be retrieved, modified, and then updated... neither of the updates is atomic, so we could end up in a bad state.

Proposed solution:
Return a new version parameter when retrieving a Datasource which is equivalent to the version returned at the top level of the saved object. This version can be echoed back during Datasource update and used in the SavedObjectsClient.update() call. If multiple modifications are attempted on same version, a 409 will be returned and the client will have to try again.

Example:

//
// datasourceVersion is returned at the top level of SO response when SavedObjectsClient.get() (or create()
// or anything that returns a SavedObject) is called.
//
// It would need to be returned from DatasourceService.get() / .create() / and any of the find() functions
// and passed into DatasourceService.update() to ensure conflict resolution on updates.
//
await soClient.update<DatasourceSOAttributes>(
  SAVED_OBJECT_TYPE,
  id, {
      ...datasource,
      revision: oldDatasource.revision + 1,
      updated_at: new Date().toISOString(),
      updated_by: options?.user?.username ?? 'system',
  },
  { version: datasourceVersion }
);
@madirey madirey added Feature:Endpoint Elastic Endpoint feature Team:Fleet Team label for Observability Data Collection Fleet team Ingest Management:beta1 labels Jun 25, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/endpoint-app-team (Feature:Endpoint)

@elasticmachine
Copy link
Contributor

Pinging @elastic/ingest-management (Team:Ingest Management)

@jen-huang
Copy link
Contributor

jen-huang commented Jun 26, 2020

I'm unassigning myself for now as the current task I'm on will take some time, so opening this up for anyone else who has the bandwidth before I'm freed up again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Endpoint Elastic Endpoint feature Team:Fleet Team label for Observability Data Collection Fleet team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants