Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate security implicitly disabled on trial/basic #72339

Merged
merged 16 commits into from
Jul 15, 2021
Merged

Deprecate security implicitly disabled on trial/basic #72339

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
baf768a
Deprecate security implicitly disabled on trial/basic
jkakavas Apr 27, 2021
ad2b84c
fix tests
jkakavas Apr 28, 2021
eed4977
Merge remote-tracking branch 'origin/7.x' into deprecate-implicit-sec…
jkakavas Apr 29, 2021
3238835
typo
jkakavas May 4, 2021
787aa00
Merge remote-tracking branch 'origin/7.x' into deprecate-implicit-sec…
jkakavas May 4, 2021
e5e9e2a
Address feedback
jkakavas May 4, 2021
11c5dc2
Adust deprecation criticality and message
jkakavas May 4, 2021
59b85d7
Address feedback
jkakavas May 5, 2021
aa2df0f
fix string
jkakavas May 5, 2021
b559931
update text matcher
jkakavas May 5, 2021
7be6983
Merge remote-tracking branch 'origin/7.x' into deprecate-implicit-sec…
jkakavas May 10, 2021
50570d5
address feedback
jkakavas May 10, 2021
eca4496
Merge remote-tracking branch 'origin/7.x' into deprecate-implicit-sec…
jkakavas Jun 8, 2021
9207bac
fix test
jkakavas Jun 8, 2021
2358a8d
Merge remote-tracking branch 'origin/7.x' into deprecate-implicit-sec…
jkakavas Jul 15, 2021
1ea2f73
Finalize wording for deprecation message
jkakavas Jul 15, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/reference/migration/index.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ For more information about {minor-version},
see the <<release-highlights>> and <<es-release-notes>>.
For information about how to upgrade your cluster, see <<setup-upgrade>>.

* <<breaking-changes-7.14,Migrating to 7.14>>
* <<breaking-changes-7.13,Migrating to 7.13>>
* <<breaking-changes-7.12,Migrating to 7.12>>
* <<breaking-changes-7.11,Migrating to 7.11>>
Expand All @@ -45,6 +46,7 @@ For information about how to upgrade your cluster, see <<setup-upgrade>>.

--

include::migrate_7_14.asciidoc[]
include::migrate_7_13.asciidoc[]
include::migrate_7_12.asciidoc[]
include::migrate_7_11.asciidoc[]
Expand Down
50 changes: 50 additions & 0 deletions docs/reference/migration/migrate_7_14.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
[[migrating-7.14]]
== Migrating to 7.14
++++
<titleabbrev>7.14</titleabbrev>
++++

This section discusses the changes that you need to be aware of when migrating
your application to {es} 7.14.

See also <<release-highlights>> and <<es-release-notes>>.

// * <<breaking_714_blah_changes>>
// * <<breaking_714_blah_changes>>

//NOTE: The notable-breaking-changes tagged regions are re-used in the
//Installation and Upgrade Guide

//tag::notable-breaking-changes[]

[discrete]
[[breaking-changes-7.14]]
=== Breaking changes

The following changes in {es} 7.14 might affect your applications
and prevent them from operating normally.
Before upgrading to 7.14, review these changes and take the described steps
to mitigate the impact.

NOTE: Breaking changes introduced in minor versions are
normally limited to security and bug fixes.
Significant changes in behavior are deprecated in a minor release and
the old behavior is supported until the next major release.
To find out if you are using any deprecated functionality,
enable <<deprecation-logging, deprecation logging>>.

[discrete]
[[breaking_714_security_changes]]
==== Security deprecations

[[implicitly-disabled-security]]
.The default behavior of disabling security on basic and trial licenses is deprecated
[%collapsible]
====
*Details* +
Currently, security features are disabled when operating on a basic or trial
license when `xpack.security.enabled` has not been explicitly set to `true`.
This behavior is now deprecated. In version 8.0.0, security features will be
enabled by default for all licenses, unless explicitly disabled (by setting
`xpack.security.enabled` to `false`).
====
39 changes: 23 additions & 16 deletions ...ugin/deprecation/src/main/java/org/elasticsearch/xpack/deprecation/DeprecationChecks.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
import org.elasticsearch.cluster.metadata.IndexMetadata;
import org.elasticsearch.cluster.node.DiscoveryNode;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.license.XPackLicenseState;
import org.elasticsearch.xpack.core.XPackSettings;
import org.elasticsearch.xpack.core.deprecation.DeprecationInfoAction;
import org.elasticsearch.xpack.core.deprecation.DeprecationIssue;
Expand All @@ -19,7 +20,6 @@
import java.util.Collections;
import java.util.List;
import java.util.Objects;
import java.util.function.BiFunction;
import java.util.function.Function;
import java.util.stream.Collectors;
import java.util.stream.Stream;
Expand All @@ -42,13 +42,14 @@ private DeprecationChecks() {
ClusterDeprecationChecks::checkTemplatesWithMultipleTypes
));

static final List<BiFunction<Settings, PluginsAndModules, DeprecationIssue>> NODE_SETTINGS_CHECKS;
static final List<NodeDeprecationCheck<Settings, PluginsAndModules, XPackLicenseState, DeprecationIssue>> NODE_SETTINGS_CHECKS;

static {
final Stream<BiFunction<Settings, PluginsAndModules, DeprecationIssue>> legacyRoleSettings = DiscoveryNode.getPossibleRoles()
final Stream<NodeDeprecationCheck<Settings, PluginsAndModules, XPackLicenseState, DeprecationIssue>> legacyRoleSettings =
DiscoveryNode.getPossibleRoles()
.stream()
.filter(r -> r.legacySetting() != null)
.map(r -> (s, p) -> NodeDeprecationChecks.checkLegacyRoleSettings(r.legacySetting(), s, p));
.map(r -> (s, p, t) -> NodeDeprecationChecks.checkLegacyRoleSettings(r.legacySetting(), s, p));
NODE_SETTINGS_CHECKS = Stream.concat(
legacyRoleSettings,
Stream.of(
Expand All @@ -58,33 +59,34 @@ private DeprecationChecks() {
NodeDeprecationChecks::checkMissingRealmOrders,
NodeDeprecationChecks::checkUniqueRealmOrders,
NodeDeprecationChecks::checkImplicitlyDisabledBasicRealms,
(settings, pluginsAndModules) -> NodeDeprecationChecks.checkThreadPoolListenerQueueSize(settings),
(settings, pluginsAndModules) -> NodeDeprecationChecks.checkThreadPoolListenerSize(settings),
(settings, pluginsAndModules, state) -> NodeDeprecationChecks.checkThreadPoolListenerQueueSize(settings),
(settings, pluginsAndModules, state) -> NodeDeprecationChecks.checkThreadPoolListenerSize(settings),
NodeDeprecationChecks::checkClusterRemoteConnectSetting,
NodeDeprecationChecks::checkNodeLocalStorageSetting,
NodeDeprecationChecks::checkGeneralScriptSizeSetting,
NodeDeprecationChecks::checkGeneralScriptExpireSetting,
NodeDeprecationChecks::checkGeneralScriptCompileSettings,
(settings, pluginsAndModules) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
(settings, pluginsAndModules, state) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
XPackSettings.ENRICH_ENABLED_SETTING),
(settings, pluginsAndModules) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
(settings, pluginsAndModules, state) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
XPackSettings.FLATTENED_ENABLED),
(settings, pluginsAndModules) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
(settings, pluginsAndModules, state) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
XPackSettings.INDEX_LIFECYCLE_ENABLED),
(settings, pluginsAndModules) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
(settings, pluginsAndModules, state) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
XPackSettings.MONITORING_ENABLED),
(settings, pluginsAndModules) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
(settings, pluginsAndModules, state) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
XPackSettings.ROLLUP_ENABLED),
(settings, pluginsAndModules) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
(settings, pluginsAndModules, state) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
XPackSettings.SNAPSHOT_LIFECYCLE_ENABLED),
(settings, pluginsAndModules) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
(settings, pluginsAndModules, state) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
XPackSettings.SQL_ENABLED),
(settings, pluginsAndModules) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
(settings, pluginsAndModules, state) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
XPackSettings.TRANSFORM_ENABLED),
(settings, pluginsAndModules) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
(settings, pluginsAndModules, state) -> NodeDeprecationChecks.checkNodeBasicLicenseFeatureEnabledSetting(settings,
XPackSettings.VECTORS_ENABLED),
NodeDeprecationChecks::checkMultipleDataPaths,
NodeDeprecationChecks::checkDataPathsList,
NodeDeprecationChecks::checkImplicitlyDisabledSecurityOnBasicAndTrial,
NodeDeprecationChecks::checkBootstrapSystemCallFilterSetting
)
).collect(Collectors.toList());
Expand All @@ -106,10 +108,15 @@ private DeprecationChecks() {
*
* @param checks The functional checks to execute using the mapper function
* @param mapper The function that executes the lambda check with the appropriate arguments
* @param <T> The signature of the check (BiFunction, Function, including the appropriate arguments)
* @param <T> The signature of the check (TriFunction, BiFunction, Function, including the appropriate arguments)
* @return The list of {@link DeprecationIssue} that were found in the cluster
*/
static <T> List<DeprecationIssue> filterChecks(List<T> checks, Function<T, DeprecationIssue> mapper) {
return checks.stream().map(mapper).filter(Objects::nonNull).collect(Collectors.toList());
}

@FunctionalInterface
public interface NodeDeprecationCheck<F, S, T, R> {
R apply(F first, S second, T third);
}
}
63 changes: 50 additions & 13 deletions .../deprecation/src/main/java/org/elasticsearch/xpack/deprecation/NodeDeprecationChecks.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

package org.elasticsearch.xpack.deprecation;

import org.elasticsearch.Version;
import org.elasticsearch.action.admin.cluster.node.info.PluginsAndModules;
import org.elasticsearch.bootstrap.JavaVersion;
import org.elasticsearch.cluster.node.DiscoveryNode;
Expand All @@ -19,11 +20,14 @@
import org.elasticsearch.common.util.concurrent.EsExecutors;
import org.elasticsearch.common.util.set.Sets;
import org.elasticsearch.env.Environment;
import org.elasticsearch.license.License;
import org.elasticsearch.license.XPackLicenseState;
import org.elasticsearch.node.Node;
import org.elasticsearch.node.NodeRoleSettings;
import org.elasticsearch.script.ScriptService;
import org.elasticsearch.threadpool.FixedExecutorBuilder;
import org.elasticsearch.transport.RemoteClusterService;
import org.elasticsearch.xpack.core.XPackSettings;
import org.elasticsearch.xpack.core.deprecation.DeprecationIssue;
import org.elasticsearch.xpack.core.security.authc.RealmConfig;
import org.elasticsearch.xpack.core.security.authc.RealmSettings;
Expand All @@ -41,7 +45,8 @@

class NodeDeprecationChecks {

static DeprecationIssue checkPidfile(final Settings settings, final PluginsAndModules pluginsAndModules) {
static DeprecationIssue checkPidfile(final Settings settings, final PluginsAndModules pluginsAndModules,
XPackLicenseState licenseState) {
return checkDeprecatedSetting(
settings,
pluginsAndModules,
Expand All @@ -50,7 +55,8 @@ static DeprecationIssue checkPidfile(final Settings settings, final PluginsAndMo
"https://www.elastic.co/guide/en/elasticsearch/reference/7.4/breaking-changes-7.4.html#deprecate-pidfile");
}

static DeprecationIssue checkProcessors(final Settings settings , final PluginsAndModules pluginsAndModules) {
static DeprecationIssue checkProcessors(final Settings settings , final PluginsAndModules pluginsAndModules,
final XPackLicenseState licenseState) {
return checkDeprecatedSetting(
settings,
pluginsAndModules,
Expand All @@ -59,7 +65,8 @@ static DeprecationIssue checkProcessors(final Settings settings , final PluginsA
"https://www.elastic.co/guide/en/elasticsearch/reference/7.4/breaking-changes-7.4.html#deprecate-processors");
}

static DeprecationIssue checkMissingRealmOrders(final Settings settings, final PluginsAndModules pluginsAndModules) {
static DeprecationIssue checkMissingRealmOrders(final Settings settings, final PluginsAndModules pluginsAndModules,
final XPackLicenseState licenseState) {
final Set<String> orderNotConfiguredRealms = RealmSettings.getRealmSettings(settings).entrySet()
.stream()
.filter(e -> false == e.getValue().hasValue(RealmSettings.ORDER_SETTING_KEY))
Expand All @@ -83,7 +90,8 @@ static DeprecationIssue checkMissingRealmOrders(final Settings settings, final P
);
}

static DeprecationIssue checkUniqueRealmOrders(final Settings settings, final PluginsAndModules pluginsAndModules) {
static DeprecationIssue checkUniqueRealmOrders(final Settings settings, final PluginsAndModules pluginsAndModules,
final XPackLicenseState licenseState) {
final Map<String, List<String>> orderToRealmSettings =
RealmSettings.getRealmSettings(settings).entrySet()
.stream()
Expand Down Expand Up @@ -116,7 +124,29 @@ static DeprecationIssue checkUniqueRealmOrders(final Settings settings, final Pl
);
}

static DeprecationIssue checkImplicitlyDisabledBasicRealms(final Settings settings, final PluginsAndModules pluginsAndModules) {
static DeprecationIssue checkImplicitlyDisabledSecurityOnBasicAndTrial(final Settings settings,
final PluginsAndModules pluginsAndModules,
final XPackLicenseState licenseState) {
if ( XPackSettings.SECURITY_ENABLED.exists(settings) == false
&& (licenseState.getOperationMode().equals(License.OperationMode.BASIC)
|| licenseState.getOperationMode().equals(License.OperationMode.TRIAL))) {
String details = "The default behavior of disabling security on " + licenseState.getOperationMode().description()
+ " licenses is deprecated. A later version of Elasticsearch will set [xpack.security.enabled] to \"true\" "
Copy link
Member

@rjernst rjernst May 10, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be better phrased as "will change the default of [xpack.security.enabled] to 'true' regardless of the license level."?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion Ryan. I don't have a strong opinion here, I think both versions convey the meaning equally clearly and as a non-native speaker, I can't say I see an impactful difference. Your version is shorter and I do like that. I'll ping @lockewritesdocs for an opinion and if he's also +1, I'll make the change

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that "will change the default of [xpack.security.enabled] to 'true'" is a little ambiguous. It almost sounds like we're moving away from the default, which is currently [xpack.security.enabled] to true. I'd suggest the following revision, which incorporates Ryan's shorter version:

A later version of Elasticsearch will set [xpack.security.enabled] to true by default, regardless of the license level.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was trying to move away from "will set" because it implies the value is actually being set (for example in elasticsearch.yml) when it is not. It is a default value that we will now be applied regardless of other factors.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good callout @rjernst. The behavior will operate as if [xpack.security.enabled] is true. Perhaps we can explain that nuance with something like:

A later version of Elasticsearch will enable security by default and behave as if [xpack.security.enabled] is true, regardless of the license level.

Copy link
Member Author

@jkakavas jkakavas Jun 8, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we're using phrases such as "The behavior will operate as if X is Y" for settings that have default value set to Y anywhere else. I find this confusing to be honest. May I suggest

A later version of Elasticsearch will set [xpack.security.enabled] to \"true\"

-->

In a later version of Elasticsearch, the value of [xpack.security.enabled] will default to \"true\" ?

+ "by default, regardless of the license level."
+ "See https://www.elastic.co/guide/en/elasticsearch/reference/" + Version.CURRENT.major + "."
+ Version.CURRENT.minor + "/security-minimal-setup.html to enable security, or explicitly disable security by "
+ "setting [xpack.security.enabled] to \"false\" in elasticsearch.yml";
return new DeprecationIssue(
DeprecationIssue.Level.CRITICAL,
"Security is enabled by default for all licenses in the next major version.",
"https://www.elastic.co/guide/en/elasticsearch/reference/7.14/migrating-7.14.html#implicitly-disabled-security",
details);
}
return null;
}

static DeprecationIssue checkImplicitlyDisabledBasicRealms(final Settings settings, final PluginsAndModules pluginsAndModules,
final XPackLicenseState licenseState) {
final Map<RealmConfig.RealmIdentifier, Settings> realmSettings = RealmSettings.getRealmSettings(settings);
if (realmSettings.isEmpty()) {
return null;
Expand Down Expand Up @@ -186,7 +216,8 @@ private static DeprecationIssue checkThreadPoolListenerSetting(final String name
"https://www.elastic.co/guide/en/elasticsearch/reference/7.x/breaking-changes-7.7.html#deprecate-listener-thread-pool");
}

public static DeprecationIssue checkClusterRemoteConnectSetting(final Settings settings, final PluginsAndModules pluginsAndModules) {
public static DeprecationIssue checkClusterRemoteConnectSetting(final Settings settings, final PluginsAndModules pluginsAndModules,
final XPackLicenseState licenseState) {
return checkDeprecatedSetting(
settings,
pluginsAndModules,
Expand All @@ -200,7 +231,8 @@ public static DeprecationIssue checkClusterRemoteConnectSetting(final Settings s
);
}

public static DeprecationIssue checkNodeLocalStorageSetting(final Settings settings, final PluginsAndModules pluginsAndModules) {
public static DeprecationIssue checkNodeLocalStorageSetting(final Settings settings, final PluginsAndModules pluginsAndModules,
final XPackLicenseState licenseState) {
return checkRemovedSetting(
settings,
Node.NODE_LOCAL_STORAGE_SETTING,
Expand All @@ -216,7 +248,8 @@ public static DeprecationIssue checkNodeBasicLicenseFeatureEnabledSetting(final
);
}

public static DeprecationIssue checkGeneralScriptSizeSetting(final Settings settings, final PluginsAndModules pluginsAndModules) {
public static DeprecationIssue checkGeneralScriptSizeSetting(final Settings settings, final PluginsAndModules pluginsAndModules,
final XPackLicenseState licenseState) {
return checkDeprecatedSetting(
settings,
pluginsAndModules,
Expand All @@ -227,7 +260,8 @@ public static DeprecationIssue checkGeneralScriptSizeSetting(final Settings sett
);
}

public static DeprecationIssue checkGeneralScriptExpireSetting(final Settings settings, final PluginsAndModules pluginsAndModules) {
public static DeprecationIssue checkGeneralScriptExpireSetting(final Settings settings, final PluginsAndModules pluginsAndModules,
final XPackLicenseState licenseState) {
return checkDeprecatedSetting(
settings,
pluginsAndModules,
Expand All @@ -238,7 +272,8 @@ public static DeprecationIssue checkGeneralScriptExpireSetting(final Settings se
);
}

public static DeprecationIssue checkGeneralScriptCompileSettings(final Settings settings, final PluginsAndModules pluginsAndModules) {
public static DeprecationIssue checkGeneralScriptCompileSettings(final Settings settings, final PluginsAndModules pluginsAndModules,
final XPackLicenseState licenseState) {
return checkDeprecatedSetting(
settings,
pluginsAndModules,
Expand Down Expand Up @@ -374,7 +409,7 @@ static DeprecationIssue checkRemovedSetting(final Settings settings, final Setti
return new DeprecationIssue(DeprecationIssue.Level.CRITICAL, message, url, details);
}

static DeprecationIssue javaVersionCheck(Settings nodeSettings, PluginsAndModules plugins) {
static DeprecationIssue javaVersionCheck(Settings nodeSettings, PluginsAndModules plugins, final XPackLicenseState licenseState) {
final JavaVersion javaVersion = JavaVersion.current();

if (javaVersion.compareTo(JavaVersion.parse("11")) < 0) {
Expand All @@ -388,7 +423,8 @@ static DeprecationIssue javaVersionCheck(Settings nodeSettings, PluginsAndModule
return null;
}

static DeprecationIssue checkMultipleDataPaths(Settings nodeSettings, PluginsAndModules plugins) {
static DeprecationIssue checkMultipleDataPaths(final Settings nodeSettings, final PluginsAndModules pluginsAndModules,
final XPackLicenseState licenseState) {
List<String> dataPaths = Environment.PATH_DATA_SETTING.get(nodeSettings);
if (dataPaths.size() > 1) {
return new DeprecationIssue(DeprecationIssue.Level.CRITICAL,
Expand All @@ -399,7 +435,8 @@ static DeprecationIssue checkMultipleDataPaths(Settings nodeSettings, PluginsAnd
return null;
}

static DeprecationIssue checkDataPathsList(Settings nodeSettings, PluginsAndModules plugins) {
static DeprecationIssue checkDataPathsList(final Settings nodeSettings, final PluginsAndModules pluginsAndModules,
final XPackLicenseState licenseState) {
if (Environment.dataPathUsesList(nodeSettings)) {
return new DeprecationIssue(DeprecationIssue.Level.CRITICAL,
"[path.data] in a list is deprecated, use a string value",
Expand Down
Loading