Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ILM] Data tier notices should reflect tier preferences #78398

Merged
28 changes: 26 additions & 2 deletions x-pack/plugins/index_lifecycle_management/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Index Lifecycle Management

## Quick steps for testing ILM in Index Management
## Testing

### Quick steps for testing ILM in Index Management

You can test that the `Frozen` badge, phase filtering, and lifecycle information is surfaced in
Index Management by running this series of requests in Console:
Expand Down Expand Up @@ -92,4 +94,26 @@ After about a minute, there should be an error on this index. When you click the
ILM information in the detail panel as well as an error. You can dismiss the error by clicking
`Manage > Retry lifecycle step`.

![image](https://user-images.githubusercontent.com/1238659/78087984-a6811000-7377-11ea-880e-1a7b182c14f1.png)
![image](https://user-images.githubusercontent.com/1238659/78087984-a6811000-7377-11ea-880e-1a7b182c14f1.png)

### Data tier notifications

When creating or editing an ILM policy the UI should notify users that under certain conditions their data will not be
moved to a tier corresponding to a phase. For instance, when a cluster only has hot-tier nodes. We test the UI
with this cluster state by starting an ES node with the `data_hot` role. Using this command:

```bash
yarn es snapshot --license=trial -E node.roles=data_hot,master,data_content
```

This will create a cluster where we have a single node that belongs to the hot-tier. In the data allocation section of
both the warm and cold phase you should see notice like the following:

![image](https://user-images.githubusercontent.com/8155004/94132944-4b306600-fe60-11ea-9c3d-02229e3055b8.png)

Default configuration for a node is that it belongs to all tiers, in which case you should not see this notice. Test
this by running:

```bash
yarn es snapshot --license=trial
```
Original file line number Diff line number Diff line change
Expand Up @@ -503,6 +503,30 @@ describe('edit policy', () => {
expect(rendered.find('.euiLoadingSpinner').exists()).toBeFalsy();
expect(findTestSubject(rendered, 'defaultAllocationWarning').exists()).toBeTruthy();
});
test('should show default allocation notice when hot tier exists, but not warm tier', async () => {
http.setupNodeListResponse({
nodesByAttributes: {},
nodesByRoles: { data_hot: ['test'], data_cold: ['test'] },
});
const rendered = mountWithIntl(component);
noRollover(rendered);
setPolicyName(rendered, 'mypolicy');
await activatePhase(rendered, 'warm');
expect(rendered.find('.euiLoadingSpinner').exists()).toBeFalsy();
expect(findTestSubject(rendered, 'defaultAllocationNotice').exists()).toBeTruthy();
});
test('should not show default allocation notice when node with "data" role exists', async () => {
http.setupNodeListResponse({
nodesByAttributes: {},
nodesByRoles: { data: ['test'] },
});
const rendered = mountWithIntl(component);
noRollover(rendered);
setPolicyName(rendered, 'mypolicy');
await activatePhase(rendered, 'warm');
expect(rendered.find('.euiLoadingSpinner').exists()).toBeFalsy();
expect(findTestSubject(rendered, 'defaultAllocationNotice').exists()).toBeFalsy();
});
});
describe('cold phase', () => {
beforeEach(() => {
Expand Down Expand Up @@ -610,6 +634,30 @@ describe('edit policy', () => {
expect(rendered.find('.euiLoadingSpinner').exists()).toBeFalsy();
expect(findTestSubject(rendered, 'defaultAllocationWarning').exists()).toBeTruthy();
});
test('should show default allocation notice when warm or hot tiers exists, but not cold tier', async () => {
http.setupNodeListResponse({
nodesByAttributes: {},
nodesByRoles: { data_hot: ['test'], data_warm: ['test'] },
});
const rendered = mountWithIntl(component);
noRollover(rendered);
setPolicyName(rendered, 'mypolicy');
await activatePhase(rendered, 'cold');
expect(rendered.find('.euiLoadingSpinner').exists()).toBeFalsy();
expect(findTestSubject(rendered, 'defaultAllocationNotice').exists()).toBeTruthy();
});
test('should not show default allocation notice when node with "data" role exists', async () => {
http.setupNodeListResponse({
nodesByAttributes: {},
nodesByRoles: { data: ['test'] },
});
const rendered = mountWithIntl(component);
noRollover(rendered);
setPolicyName(rendered, 'mypolicy');
await activatePhase(rendered, 'cold');
expect(rendered.find('.euiLoadingSpinner').exists()).toBeFalsy();
expect(findTestSubject(rendered, 'defaultAllocationNotice').exists()).toBeFalsy();
});
});
describe('delete phase', () => {
test('should allow 0 for phase timing', async () => {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

// Order of node roles matters here, the warm phase prefers allocating data
// to the data_warm role.
import { NodeDataRole, PhaseWithAllocation } from '../types';

const WARM_PHASE_NODE_PREFERENCE: NodeDataRole[] = ['data_warm', 'data_hot'];

const COLD_PHASE_NODE_PREFERENCE: NodeDataRole[] = ['data_cold', 'data_warm', 'data_hot'];

export const phaseToNodePreferenceMap: Record<PhaseWithAllocation, NodeDataRole[]> = Object.freeze({
warm: WARM_PHASE_NODE_PREFERENCE,
cold: COLD_PHASE_NODE_PREFERENCE,
});
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@
import { i18n } from '@kbn/i18n';
import { LicenseType } from '../../../licensing/common/types';

export { phaseToNodePreferenceMap } from './data_tiers';

const basicLicense: LicenseType = 'basic';

export const PLUGIN = {
Expand Down
4 changes: 2 additions & 2 deletions x-pack/plugins/index_lifecycle_management/common/types/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
* you may not use this file except in compliance with the Elastic License.
*/

export type NodeDataRole = 'data' | 'data_hot' | 'data_warm' | 'data_cold' | 'data_frozen';
import { NodeDataRoleWithCatchAll } from '.';

export interface ListNodesRouteResponse {
nodesByAttributes: { [attributePair: string]: string[] };
nodesByRoles: { [role in NodeDataRole]?: string[] };
nodesByRoles: { [role in NodeDataRoleWithCatchAll]?: string[] };
}
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,10 @@
export * from './api';

export * from './policies';

/**
* These roles reflect how nodes are stratified into different data tiers. The "data" role
* is a catch-all that can be used to store data in any phase.
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love these comments!

export type NodeDataRole = 'data_hot' | 'data_warm' | 'data_cold';
export type NodeDataRoleWithCatchAll = 'data' | NodeDataRole;

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import {
NodeDataRole,
ListNodesRouteResponse,
PhaseWithAllocation,
} from '../../../../common/types';

import { phaseToNodePreferenceMap } from '../../../../common/constants';

export type AllocationNodeRole = NodeDataRole | 'none';

/**
* Given a phase and current cluster node roles, determine which nodes the phase
* will allocate data to. For instance, for the warm phase, with warm
* tier nodes, we would expect "data_warm".
*
* If no nodes can be identified for allocation (very special case) then
* we return "none".
*/
export const getAvailableNodeRoleForPhase = (
phase: PhaseWithAllocation,
nodesByRoles: ListNodesRouteResponse['nodesByRoles']
): AllocationNodeRole => {
const preferredNodeRoles = phaseToNodePreferenceMap[phase];

// The 'data' role covers all node roles, so if we have at least one node with the data role
// we can allocate to our first preference.
if (nodesByRoles.data?.length) {
return preferredNodeRoles[0];
}

return preferredNodeRoles.find((role) => Boolean(nodesByRoles[role]?.length)) ?? 'none';
};
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@

export * from './determine_allocation_type';

export * from './check_phase_compatibility';
export * from './get_available_node_roles_for_phase';
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import { NodeDataRole, PhaseWithAllocation } from '../../../../common/types';
import { phaseToNodePreferenceMap } from '../../../../common/constants';

export const isNodeRoleFirstPreference = (phase: PhaseWithAllocation, nodeRole: NodeDataRole) => {
return phaseToNodePreferenceMap[phase][0] === nodeRole;
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import { i18n } from '@kbn/i18n';
import React, { FunctionComponent } from 'react';
import { EuiCallOut, EuiSpacer } from '@elastic/eui';

import { PhaseWithAllocation, NodeDataRole } from '../../../../../../common/types';

import { AllocationNodeRole } from '../../../../lib';

const i18nTextsNodeRoleToDataTier: Record<NodeDataRole, string> = {
data_hot: i18n.translate('xpack.indexLifecycleMgmt.editPolicy.dataTierHotLabel', {
defaultMessage: 'hot',
}),
data_warm: i18n.translate('xpack.indexLifecycleMgmt.editPolicy.dataTierWarmLabel', {
defaultMessage: 'warm',
}),
data_cold: i18n.translate('xpack.indexLifecycleMgmt.editPolicy.dataTierColdLabel', {
defaultMessage: 'cold',
}),
};

const i18nTexts = {
notice: {
warm: {
title: i18n.translate(
'xpack.indexLifecycleMgmt.warmPhase.dataTier.defaultAllocationNotice.warm.title',
{ defaultMessage: 'No nodes assigned to the warm tier' }
),
body: (nodeRole: NodeDataRole) =>
i18n.translate('xpack.indexLifecycleMgmt.warmPhase.dataTier.defaultAllocationNotice.warm', {
defaultMessage:
'This policy will move data in the warm phase to {tier} tier nodes instead.',
values: { tier: i18nTextsNodeRoleToDataTier[nodeRole] },
}),
},
cold: {
title: i18n.translate(
'xpack.indexLifecycleMgmt.warmPhase.dataTier.defaultAllocationNotice.cold.title',
{ defaultMessage: 'No nodes assigned to the cold tier' }
),
body: (nodeRole: NodeDataRole) =>
i18n.translate('xpack.indexLifecycleMgmt.warmPhase.dataTier.defaultAllocationNotice.cold', {
defaultMessage:
'This policy will move data in the cold phase to {tier} tier nodes instead.',
values: { tier: i18nTextsNodeRoleToDataTier[nodeRole] },
}),
},
},
warning: {
warm: {
title: i18n.translate(
'xpack.indexLifecycleMgmt.warmPhase.dataTier.defaultAllocationNotAvailableTitle',
{ defaultMessage: 'No nodes assigned to the warm tier' }
),
body: i18n.translate(
'xpack.indexLifecycleMgmt.warmPhase.dataTier.defaultAllocationNotAvailableBody',
{
defaultMessage:
'Assign at least one node to the warm or hot tier to use role-based allocation. The policy will fail to complete allocation if there are no available nodes.',
}
),
},
cold: {
title: i18n.translate(
'xpack.indexLifecycleMgmt.coldPhase.dataTier.defaultAllocationNotAvailableTitle',
{ defaultMessage: 'No nodes assigned to the cold tier' }
),
body: i18n.translate(
'xpack.indexLifecycleMgmt.coldPhase.dataTier.defaultAllocationNotAvailableBody',
{
defaultMessage:
'Assign at least one node to the cold, warm, or hot tier to use role-based allocation. The policy will fail to complete allocation if there are no available nodes.',
}
),
},
},
};

interface Props {
phase: PhaseWithAllocation;
targetNodeRole: AllocationNodeRole;
}

export const DefaultAllocationNotice: FunctionComponent<Props> = ({ phase, targetNodeRole }) => {
const content =
targetNodeRole === 'none' ? (
<EuiCallOut
data-test-subj="defaultAllocationWarning"
title={i18nTexts.warning[phase].title}
color="warning"
>
{i18nTexts.warning[phase].body}
</EuiCallOut>
) : (
<EuiCallOut data-test-subj="defaultAllocationNotice" title={i18nTexts.notice[phase].title}>
{i18nTexts.notice[phase].body(targetNodeRole)}
</EuiCallOut>
);

return (
<>
<EuiSpacer size="s" />
{content}
</>
);
};
Loading