Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce "Feature States" for managing snapshots of system indices #63513

Merged
merged 157 commits into from
Feb 11, 2021
Merged
Changes from 1 commit
Commits
Show all changes
157 commits
Select commit Hold shift + click to select a range
65903ab
Add plugin name to SystemIndexPlugin
gwbrown Oct 8, 2020
6d32a65
Use plugin name for system index descriptor map
gwbrown Oct 8, 2020
37e4f4b
Add pluginStates to create and restore snapshot requests
gwbrown Oct 8, 2020
1fefb26
Plumb system index descriptor map into SnapshotsService
gwbrown Oct 8, 2020
1869e91
Remove unnecessary comment
gwbrown Oct 8, 2020
ac2961f
Merge branch 'master' into si/snapshot-and-restore
gwbrown Oct 12, 2020
febcdd7
Only include pluginStates in request XContent if non-empty
gwbrown Oct 12, 2020
6561940
Merge branch 'master' into si/snapshot-and-restore
gwbrown Oct 13, 2020
c16e472
Add plugin states to SnapshotInfo
gwbrown Oct 13, 2020
0c53a22
pluginStates->featureStates
gwbrown Oct 14, 2020
f0ef426
More plugin->feature
gwbrown Oct 14, 2020
1f4ecfe
Break out SnapshotFeatureInfo, add tests and equals/hashcode
gwbrown Oct 14, 2020
f824abe
Merge branch 'master' into si/snapshot-and-restore
gwbrown Oct 14, 2020
040fda6
Null guard for `featureStates` in `toXContent`
gwbrown Oct 15, 2020
013b764
Merge branch 'master' into si/snapshot-and-restore
gwbrown Oct 15, 2020
9120ffa
Another null guard for `featureStates` in `toXContent`
gwbrown Oct 15, 2020
26e94f6
writeTo version guard
gwbrown Oct 15, 2020
df3deff
ACTUAL writeTo version guard
gwbrown Oct 15, 2020
ce24b74
SnapshotInfo featureStates now non-nullable
gwbrown Oct 19, 2020
9382885
Test featureStates in CreateSnapshotResponseTests & fix parser
gwbrown Oct 19, 2020
261f5c0
NOCOMMIT -> GWB todos
gwbrown Oct 19, 2020
89d8e19
Line length
gwbrown Oct 19, 2020
c207993
Merge branch 'master' into si/snapshot-and-restore
gwbrown Oct 19, 2020
08406bd
Add `feature_states` to expected responses in the docs
gwbrown Oct 19, 2020
d2b517b
Merge branch 'master' into si/snapshot-and-restore
gwbrown Oct 20, 2020
93af221
Add system indices during snapshot creation (first draft)
gwbrown Oct 20, 2020
b6392af
Merge branch 'master' into si/snapshot-and-restore
gwbrown Oct 21, 2020
e4a4580
Address various TODOs
gwbrown Oct 21, 2020
b7ef98e
Refactor giant pile of streams
gwbrown Oct 21, 2020
9197bb8
Spotless
gwbrown Oct 21, 2020
8d6cbcb
Line length
gwbrown Oct 21, 2020
27df301
Merge branch 'master' into si/snapshot-and-restore
gwbrown Oct 28, 2020
a344a65
Fix accidental change to startTime and add featureStates to Entry ser…
gwbrown Oct 28, 2020
2a5c42a
Merge branch 'master' into si/snapshot-and-restore
gwbrown Oct 29, 2020
2ff22b1
Merge branch 'master' into si/snapshot-and-restore
gwbrown Oct 30, 2020
7ecabdc
Fix NPE in SnapshotResiliencyTests
gwbrown Oct 30, 2020
f069cf1
Remove null guards for SnapshotInfo featureStates as featureStates ca…
gwbrown Oct 30, 2020
93c96df
Omit empty featureStates when creating the snapshots
gwbrown Oct 30, 2020
9a1ee0f
Ensure built-in features (i.e. tasks) are propagated to SnapshotsService
gwbrown Nov 3, 2020
06898ba
Merge branch 'master' into si/snapshot-and-restore
gwbrown Nov 3, 2020
b86dcf6
Don't include global state in test snapshot (for now) to prevent conf…
gwbrown Nov 5, 2020
4b5cb08
Merge branch 'master' into si/snapshot-and-restore
gwbrown Nov 5, 2020
1966575
Merge branch 'master' into si/snapshot-and-restore
gwbrown Nov 9, 2020
e3efb29
Very basic first cut of snapshottable features API
gwbrown Nov 10, 2020
c8ed77a
Add tests for snapshot and restore of system indices
williamrandolph Nov 12, 2020
e9b25ac
Add feature description to system index plugins
gwbrown Nov 13, 2020
a8398fa
Fix typo in test name
gwbrown Nov 13, 2020
87fde96
Merge branch 'master' into si/snapshot-and-restore
gwbrown Nov 13, 2020
8ccbba6
Merge branch 'si/snapshot-and-restore' into si/snapshot-tests
williamrandolph Nov 17, 2020
410b8bb
Delete indices by UUID rather than name
williamrandolph Nov 17, 2020
c262c0b
Merge branch 'master' into si/snapshot-and-restore
gwbrown Nov 17, 2020
952ffb0
Plumbing through descriptions
gwbrown Nov 17, 2020
aa32eb5
Merge branch 'master' into si/snapshot-and-restore
gwbrown Nov 18, 2020
7ba77bd
Allow restoring by feature state
williamrandolph Nov 18, 2020
d544f1e
Code review fixes
williamrandolph Nov 19, 2020
d2adae1
Merge pull request #1 from williamrandolph/si/snapshot-tests
gwbrown Nov 23, 2020
88c6c41
Merge branch 'master' into si/snapshot-and-restore
gwbrown Nov 25, 2020
c3f8813
Merge branch 'master' into si/snapshot-and-restore
gwbrown Nov 30, 2020
64f60d0
Merge branch 'master' into si/snapshot-and-restore
gwbrown Dec 1, 2020
ab4106a
Treat a `null` and empty list of feature states differently
gwbrown Dec 2, 2020
031e86e
Don't rename system indices
gwbrown Dec 2, 2020
9b6d1ff
Refactoring/cleanup. No functionality changes
gwbrown Dec 3, 2020
f24ac78
Merge branch 'master' into si/snapshot-and-restore
gwbrown Dec 4, 2020
c8ca4a1
Name and description for Fleet
gwbrown Dec 4, 2020
c16d60f
Add deprecation warning for directly restoring system indices
williamrandolph Dec 4, 2020
3b0fb7f
Add test for bad feature state request
williamrandolph Dec 4, 2020
cfbfbcb
Make HashSet variable final
williamrandolph Dec 4, 2020
d0b3a44
Deprecate direct system index restoration
gwbrown Dec 4, 2020
091ca4c
Only warn when explicitly requesting system indices if the snapshot i…
gwbrown Dec 8, 2020
05fd6d7
Some missed s/plugin/feature/
gwbrown Dec 8, 2020
4af0a24
Address a couple minor TODO comments
gwbrown Dec 8, 2020
439679d
Address TODO by generating random feature infos in test
gwbrown Dec 8, 2020
376e264
Merge branch 'master' into si/snapshot-and-restore
gwbrown Dec 8, 2020
3c3b1b8
One missed s/plugin/feature/
gwbrown Dec 8, 2020
d36f6c8
Basic associated indices implementation
gwbrown Dec 8, 2020
ef88902
Remove unused method
gwbrown Dec 9, 2020
10f2802
Don't include associated indices in the feature state (but snapshot t…
gwbrown Dec 9, 2020
e9b0f80
Delete all system indices for features that are being restored
gwbrown Dec 10, 2020
2e54c94
Always restore aliases for system indices
gwbrown Dec 10, 2020
7300c0c
Merge branch 'master' into si/snapshot-and-restore
gwbrown Dec 10, 2020
2b9945b
Merge branch 'master' into si/snapshot-and-restore
gwbrown Dec 10, 2020
cdc3bd1
Don't delete all indices if there aren't any feature states to restore
gwbrown Dec 10, 2020
9520873
Null guards for FeatureStates
gwbrown Dec 10, 2020
a0baac2
Make Get Snapshottable Features non-operator only
gwbrown Dec 11, 2020
262760f
Fix test now that system indices aren't always deleted
gwbrown Dec 11, 2020
b416193
Merge branch 'master' into si/snapshot-and-restore
gwbrown Dec 14, 2020
2dde730
Improve system index snapshot IT
williamrandolph Dec 17, 2020
6831671
Merge branch 'master' into si/snapshot-and-restore
gwbrown Dec 17, 2020
510ab18
Name/description for test plugins
gwbrown Dec 16, 2020
20c6b98
Add a test to ensure system indices in a feature state are removed wh…
gwbrown Dec 18, 2020
15b4903
Add test to ensure system index aliases are always restored
gwbrown Dec 18, 2020
792d735
Missed a necessary refresh
gwbrown Dec 18, 2020
b375b4b
Merge branch 'master' into si/snapshot-and-restore
gwbrown Jan 4, 2021
85b90e8
Fix rest test and add test coverage elsewhere
williamrandolph Jan 5, 2021
3909cfc
Ignore global state in searchable snapshot test
williamrandolph Jan 5, 2021
86ed638
Nest features array under an object key in Get Features API to better…
gwbrown Jan 6, 2021
0a70ae1
Get Snapshottable Features API spec & yaml spec/smoke test
gwbrown Jan 6, 2021
9f16920
Update docs for feature states
gwbrown Jan 6, 2021
2e652fa
Merge branch 'master' into si/snapshot-and-restore
gwbrown Jan 6, 2021
178e941
Merge remote-tracking branch 'gwbrown/si/snapshot-and-restore' into s…
gwbrown Jan 6, 2021
afdf54c
Add version-based skip to Get Features YAML test
gwbrown Jan 6, 2021
8c324fd
Add HLRC support for Get Features API
gwbrown Jan 7, 2021
eec5785
Merge branch 'master' into si/snapshot-and-restore
gwbrown Jan 7, 2021
1b99941
Checkstyle
gwbrown Jan 7, 2021
055adf2
Merge branch 'master' into si/snapshot-and-restore
gwbrown Jan 12, 2021
290b203
Cluster state -> global state in docs where it makes sense
gwbrown Jan 15, 2021
f7a0dc2
Fix Client javadoc
gwbrown Jan 15, 2021
774f11c
Typo/wording fixes per review
gwbrown Jan 15, 2021
82cfdac
Remove excess line
gwbrown Jan 15, 2021
46f5eed
new HashMap<>() -> Collections.emptyMap()
gwbrown Jan 15, 2021
8be4e74
Throw an error when attempting to restore feature states that go with…
gwbrown Jan 15, 2021
1bec52b
Merge branch 'master' into si/snapshot-and-restore
gwbrown Jan 15, 2021
59ab240
Clean up error message
gwbrown Jan 15, 2021
b938323
Flip incorrect boolean logic (d'oh)
gwbrown Jan 15, 2021
201eec6
Merge branch 'master' into si/snapshot-and-restore
gwbrown Jan 19, 2021
7ceecae
Add category for deprecation warning
gwbrown Jan 19, 2021
6c4669f
Handle null & empty feature states the same in Create/Restore requests
gwbrown Jan 19, 2021
596c9ac
Add "none" special case for including no feature states
gwbrown Jan 20, 2021
d677aeb
Don't include feature_states in restore request JSON if it's empty
gwbrown Jan 20, 2021
8946c94
Merge branch 'master' into si/snapshot-and-restore
gwbrown Jan 20, 2021
1476d8d
Exercise feature_states in HLRC tests
gwbrown Jan 20, 2021
884983c
Add integration test for feature states and security index
williamrandolph Jan 21, 2021
1dc1a80
Add a test to ensure system indices can be renamed if restored by name
gwbrown Jan 21, 2021
1e37dc6
Add UUID comparison to security index state change listeners
williamrandolph Jan 21, 2021
8f038ee
Exclude all feature states in `createSnapshot`
gwbrown Jan 25, 2021
898a3b3
Merge branch 'master' into si/snapshot-and-restore
gwbrown Jan 25, 2021
fe203f6
Merge remote-tracking branch 'gwbrown/si/snapshot-and-restore' into s…
gwbrown Jan 25, 2021
dc2ac03
Merge branch 'master' into si/snapshot-and-restore
gwbrown Jan 28, 2021
1e65d12
Undo import changes (thanks IntelliJ)
gwbrown Feb 2, 2021
fda3a9c
! -> == false
gwbrown Feb 2, 2021
455f762
DRY up test per review
gwbrown Feb 2, 2021
ba8aba3
Don't pre-size arrays per review
gwbrown Feb 2, 2021
4c5ce56
Rearrange `startedEntry` params per review
gwbrown Feb 2, 2021
eeda4e1
Move `FEATURE_STATES_VERSION` to `SnapshotsService` per review
gwbrown Feb 2, 2021
b78fbcd
Merge branch 'master' into si/snapshot-and-restore
gwbrown Feb 2, 2021
facf8fa
"Partial" fix in SnapshotsService#finalizeSnapshotEntry
gwbrown Feb 2, 2021
4bc6bae
Merge branch 'master' into si/snapshot-and-restore
gwbrown Feb 3, 2021
56fbb20
New license headers that didn't get auto-updated
gwbrown Feb 3, 2021
6b1cf06
Merge branch 'master' into si/snapshot-and-restore
gwbrown Feb 4, 2021
12d1b25
Don't put Descriptor patterns through IndexNameExpressionResolver
gwbrown Feb 4, 2021
699c2ae
Unused imports
gwbrown Feb 5, 2021
d503868
Only use feature_states if all nodes are on a version that can handle…
gwbrown Feb 5, 2021
f2d7bbe
If any feature state indices are partial, drop associated feature state
gwbrown Feb 6, 2021
ef0dda4
Merge branch 'master' into si/snapshot-and-restore
gwbrown Feb 6, 2021
6d9edd8
Unused import
gwbrown Feb 6, 2021
2b52037
Merge branch 'master' into si/snapshot-and-restore
gwbrown Feb 8, 2021
38022dd
Check indices in snapshot as well as failed shards
gwbrown Feb 8, 2021
c5d68ea
Merge branch 'master' into si/snapshot-and-restore
gwbrown Feb 8, 2021
873de63
Remove extraneous line break
gwbrown Feb 8, 2021
6dd31f1
`!` -> `== false`
gwbrown Feb 8, 2021
66bc5cd
Merge branch 'master' into si/snapshot-and-restore
gwbrown Feb 9, 2021
90bf18d
Merge branch 'master' into si/snapshot-and-restore
gwbrown Feb 11, 2021
dd8889f
Add test - but this test doesn't fail if logic is removed
gwbrown Feb 11, 2021
d300586
Remove commented lines
gwbrown Feb 11, 2021
b636420
Merge branch 'master' into si/snapshot-and-restore
gwbrown Feb 11, 2021
6930e00
Fix test and use real final indices list
gwbrown Feb 11, 2021
d4267d4
Merge branch 'master' into si/snapshot-and-restore
gwbrown Feb 11, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Name and description for Fleet
gwbrown committed Dec 4, 2020
commit c8ca4a16998163dc073a2855149fe3432371241f
Original file line number Diff line number Diff line change
@@ -30,4 +30,14 @@ public Collection<SystemIndexDescriptor> getSystemIndexDescriptors(Settings sett
new SystemIndexDescriptor(".fleet-actions*", "Fleet actions")
);
}

@Override
public String getFeatureName() {
return "fleet";
}

@Override
public String getFeatureDescription() {
return "Manages configuration for Fleet";
}
}