Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement few operator handlers #88097

Merged
merged 8 commits into from
Jul 4, 2022

Conversation

grcevski
Copy link
Contributor

Implement operator handlers for cluster state and ILM.

These operator handlers are the first of the operator handlers we'll be implementing, one specific to cluster state settings and another dealing with cluster state entities (ILM).

Relates to #86224.

Implement operator handlers for cluster state and ILM
@grcevski grcevski added >non-issue :Core/Infra/Core Core issues without another label Team:Core/Infra Meta label for core/infra team v8.4.0 labels Jun 27, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

protected Set<String> modifiedKeys(Request request) {
return Collections.emptySet();
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's an example of how these two methods operatorHandlerName and modifiedKeys could be used in doExecute below to verify that an operation is allowed to execute:

https://github.com/elastic/elasticsearch/pull/86224/files#diff-e7c445981d60f65376f04699ec430437d9b1c78fac6b5ef582f39f84e43de9ceR175

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding the validateForOperatorState(...) validation method in the referenced link,
I do think the logic looks good, but I do think this also needs to be validated as part
of a cluster state update. Right now the validation there only occurs on the action/transport
level and there is no guarantee that what is validated at that point in time is also
true when the update if really performed. I think it is a good pre-check, that is likely
to catch almost all cases when resources are updates that are being managed operator state.

For example in order to guarantee that no operator state managed ilm policies are modified
via API. The UpdateLifecyclePolicyTask/DeleteLifecyclePolicyTask should do this validation
check as well as part of their respective execute(...) method.

@grcevski grcevski added the :Data Management/ILM+SLM Index and Snapshot lifecycle management label Jun 27, 2022
@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label Jun 27, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ILM parts look good, I left some minor comments around it.
I also commented on your comment, which is a bit more involved,
but I think falls a bit outside of this PR, but I think is important.

* Internally it uses {@link TransportPutLifecycleAction} and
* {@link TransportDeleteLifecycleAction} to add, update and delete ILM policies.
*/
public class ImmutableLifecycleAction implements ImmutableClusterStateHandler<LifecyclePolicy> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe move this class to the org.elasticsearch.xpack.ilm.action package?
That way the newly added constructors to UpdateLifecyclePolicyTask and DeleteLifecyclePolicyTask can have package level visibility? And should avoid accidental use outside this package.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea!

/**
* Used by the {@link org.elasticsearch.immutablestate.ImmutableClusterStateHandler} for ILM
* {@link ImmutableLifecycleAction}
* @param policyName
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe remove @param line here? Since it doesn't add documentation.

* <p>
* It disables verbose logging and has no filtered headers.
*
* @param request
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove empty @param lines here?

/**
* ILM Provider implementation for the {@link ImmutableClusterStateHandlerProvider} service interface
*/
public class ILMImmutableStateHandlerProvider implements ImmutableClusterStateHandlerProvider {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe move this class to org.elasticsearch.xpack.ilm.action or org.elasticsearch.xpack.ilm package?
Since this package only contains a single class.


public UpdateLifecyclePolicyTask(
Request request,
ActionListener<AcknowledgedResponse> listener,
XPackLicenseState licenseState,
Map<String, String> filteredHeaders,
NamedXContentRegistry xContentRegistry,
Client client
Client client,
boolean verboseLogging
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe remove the verboseLogging param here?
and just set the verboseLogging to true?
Then the other constructor can't reuse this constructor, but
then for regular usage of this class, verbose logging is guaranteed to be enabled.

protected Set<String> modifiedKeys(Request request) {
return Collections.emptySet();
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding the validateForOperatorState(...) validation method in the referenced link,
I do think the logic looks good, but I do think this also needs to be validated as part
of a cluster state update. Right now the validation there only occurs on the action/transport
level and there is no guarantee that what is validated at that point in time is also
true when the update if really performed. I think it is a good pre-check, that is likely
to catch almost all cases when resources are updates that are being managed operator state.

For example in order to guarantee that no operator state managed ilm policies are modified
via API. The UpdateLifecyclePolicyTask/DeleteLifecyclePolicyTask should do this validation
check as well as part of their respective execute(...) method.

return handlers;
}

public static void handler(ImmutableClusterStateHandler<?> handler) {
Copy link
Member

@martijnvg martijnvg Jun 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think does need to be invoked and should be provided a ImmutableLifecycleAction instance, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


private final ClusterSettings clusterSettings;

public ImmutableClusterSettingsAction(ClusterSettings clusterSettings) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, 100%. I left that part to happen in my follow-up draft PR I have now based on this one here: https://github.com/elastic/elasticsearch/pull/88224/files#diff-b56bed42f5b0025886eb243ecb48678ac048bfba0ef047545dbb6504e357edf2R905

After I wire them up, I call the initialize on the controller, which I haven't brought in yet from the initial WIP PR. Once we merge this in, that's the next one.

Copy link
Contributor

@ChrisHegarty ChrisHegarty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ILM side LGTM 👍

@grcevski grcevski merged commit fc93f77 into elastic:master Jul 4, 2022
@grcevski
Copy link
Contributor Author

grcevski commented Jul 4, 2022

Thanks @ChrisHegarty and @martijnvg !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Core/Infra/Core Core issues without another label :Data Management/ILM+SLM Index and Snapshot lifecycle management >non-issue Team:Core/Infra Meta label for core/infra team Team:Data Management Meta label for data/management team v8.4.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants