Script: Restore the scripting general cache

Deprecate the script context cache in favor of the general cache.

Users should use the following settings:
`script.max_compilations_rate` to set the max compilation rate
  for user scripts such as filter scripts.  Certain script contexts
  that submit scripts outside of the control of the user are
  exempted from this rate limit.  Examples include runtime fields,
  ingest and watcher.

`script.cache.max_size` to set the max size of the cache.

`script.cache.expire` to set the expiration time for entries in
the cache.

Whats deprecated?
`script.max_compilations_rate: use-context`.  This special
setting value was used to turn on the script context-specific caches.

`script.context.$CONTEXT.cache_max_size`, use `script.cache.max_size`
instead.

`script.context.$CONTEXT.cache_expire`, use `script.cache.expire`
instead.

`script.context.$CONTEXT.max_compilations_rate`, use
`script.max_compilations_rate` instead.

The default cache size was increased from `100` to `3000`, which
was approximately the max cache size when using context-specific caches.

The default compilation rate limit was increased from `75/5m` to
`150/5m` to account for increasing uses of scripts.

System script contexts can now opt-out of compilation rate limiting
using a flag rather than a sentinel rate limit value.

7.16: Script: Deprecate script context cache elastic#79508
Refs: elastic#62899

7.16: Script: Opt-out system contexts from script compilation rate limit elastic#79459
Refs: elastic#62899

docs/reference/modules/indices/circuit_breaker.asciidoc

@@ -126,11 +126,11 @@ within a period of time.
 See the "prefer-parameters" section of the <<modules-scripting-using,scripting>>
 documentation for more information.
 
-`script.context.$CONTEXT.max_compilations_rate`::
+`script.max_compilations_rate`::
     (<<dynamic-cluster-setting,Dynamic>>)
     Limit for the number of unique dynamic scripts within a certain interval
-    that are allowed to be compiled for a given context. Defaults to `75/5m`,
-    meaning 75 every 5 minutes.
+    that are allowed to be compiled. Defaults to `150/5m`,
+    meaning 150 every 5 minutes.
 
 [[regex-circuit-breaker]]
 [discrete]

docs/reference/scripting/using.asciidoc

@@ -120,12 +120,8 @@ the `multiplier` parameter without {es} recompiling the script.
 }
 ----
 
-For most contexts, you can compile up to 75 scripts per 5 minutes by default.
-For ingest contexts, the default script compilation rate is unlimited. You
-can change these settings dynamically by setting
-`script.context.$CONTEXT.max_compilations_rate`. For example, the following
-setting limits script compilation to 100 scripts every 10 minutes for the
-{painless}/painless-field-context.html[field context]:
+You can compile up to 150 scripts per 5 minutes by default.
+For ingest contexts, the default script compilation rate is unlimited.
 
 [source,js]
 ----
@@ -406,8 +402,8 @@ small.
 
 All scripts are cached by default so that they only need to be recompiled
 when updates occur. By default, scripts do not have a time-based expiration.
-You can change this behavior by using the `script.context.$CONTEXT.cache_expire` setting.
-Use the `script.context.$CONTEXT.cache_max_size` setting to configure the size of the cache.
+You can change this behavior by using the `script.cache.expire` setting.
+Use the `script.cache.max_size` setting to configure the size of the cache.
 
 NOTE: The size of scripts is limited to 65,535 bytes. Set the value of `script.max_size_in_bytes` to increase that soft limit. If your scripts are
 really large, then consider using a

server/src/main/java/org/elasticsearch/script/AbstractFieldScript.java

@@ -19,6 +19,8 @@
 import java.util.Map;
 import java.util.function.Function;
 
+import static org.elasticsearch.core.TimeValue.timeValueMillis;
+
 /**
  * Abstract base for scripts to execute to build scripted fields. Inspired by
  * {@link AggregationScript} but hopefully with less historical baggage.
@@ -30,21 +32,33 @@ public abstract class AbstractFieldScript extends DocBasedScript {
     public static final int MAX_VALUES = 100;
 
     static <F> ScriptContext<F> newContext(String name, Class<F> factoryClass) {
-        /*
-         * We rely on the script cache in two ways:
-         * 1. It caches the "heavy" part of mappings generated at runtime.
-         * 2. Mapping updates tend to try to compile the script twice. Not
-         *    for any good reason. They just do.
-         * Thus we use the default cache size.
-         *
-         * We also disable compilation rate limits for runtime fields so we
-         * don't prevent mapping updates because we've performed too
-         * many recently. That'd just be lame. We also compile these
-         * scripts during search requests so this could totally be a
-         * source of runaway script compilations. We think folks will
-         * mostly reuse scripts though.
-         */
-        return new ScriptContext<>(name, factoryClass, false);
+        return new ScriptContext<>(
+            name,
+            factoryClass,
+            /*
+             * We rely on the script cache in two ways:
+             * 1. It caches the "heavy" part of mappings generated at runtime.
+             * 2. Mapping updates tend to try to compile the script twice. Not
+             *    for any good reason. They just do.
+             * Thus we use the default 100.
+             */
+            100,
+            timeValueMillis(0),
+            /*
+             * Disable compilation rate limits for runtime fields so we
+             * don't prevent mapping updates because we've performed too
+             * many recently. That'd just be lame. We also compile these
+             * scripts during search requests so this could totally be a
+             * source of runaway script compilations. We think folks will
+             * mostly reuse scripts though.
+             */
+            false,
+            /*
+             * Disable runtime fields scripts from being allowed
+             * to be stored as part of the script meta data.
+             */
+            false
+        );
     }
 
     private static final Map<String, Function<Object, Object>> PARAMS_FUNCTIONS = Map.of(

server/src/main/java/org/elasticsearch/script/IngestConditionalScript.java

@@ -8,6 +8,8 @@
 
 package org.elasticsearch.script;
 
+import org.elasticsearch.core.TimeValue;
+
 import java.util.Map;
 
 /**
@@ -18,7 +20,8 @@ public abstract class IngestConditionalScript {
     public static final String[] PARAMETERS = { "ctx" };
 
     /** The context used to compile {@link IngestConditionalScript} factories. */
-    public static final ScriptContext<Factory> CONTEXT = new ScriptContext<>("processor_conditional", Factory.class, true);
+    public static final ScriptContext<Factory> CONTEXT = new ScriptContext<>("processor_conditional", Factory.class,
+        200, TimeValue.timeValueMillis(0), false, true);
 
     /** The generic runtime parameters for the script. */
     private final Map<String, Object> params;

server/src/main/java/org/elasticsearch/script/IngestScript.java

@@ -9,6 +9,8 @@
 
 package org.elasticsearch.script;
 
+import org.elasticsearch.core.TimeValue;
+
 import java.util.Map;
 
 /**
@@ -19,7 +21,8 @@ public abstract class IngestScript {
     public static final String[] PARAMETERS = { "ctx" };
 
     /** The context used to compile {@link IngestScript} factories. */
-    public static final ScriptContext<Factory> CONTEXT = new ScriptContext<>("ingest", Factory.class, true);
+    public static final ScriptContext<Factory> CONTEXT = new ScriptContext<>("ingest", Factory.class,
+        200, TimeValue.timeValueMillis(0), false, true);
 
     /** The generic runtime parameters for the script. */
     private final Map<String, Object> params;

server/src/main/java/org/elasticsearch/script/ScriptContext.java

@@ -75,8 +75,8 @@ public final class ScriptContext<FactoryType> {
     public final boolean allowStoredScript;
 
     /** Construct a context with the related instance and compiled classes with caller provided cache defaults */
-    ScriptContext(String name, Class<FactoryType> factoryClazz, int cacheSizeDefault, TimeValue cacheExpireDefault,
-                  boolean compilationRateLimited, boolean allowStoredScript) {
+    public ScriptContext(String name, Class<FactoryType> factoryClazz, int cacheSizeDefault, TimeValue cacheExpireDefault,
+                         boolean compilationRateLimited, boolean allowStoredScript) {
         this.name = name;
         this.factoryClazz = factoryClazz;
         Method newInstanceMethod = findMethod("FactoryType", factoryClazz, "newInstance");
@@ -105,22 +105,12 @@ public final class ScriptContext<FactoryType> {
     }
 
     /** Construct a context with the related instance and compiled classes with defaults for cacheSizeDefault, cacheExpireDefault and
-     *  maxCompilationRateDefault and allow scripts of this context to be stored scripts */
+     *  compilationRateLimited and allow scripts of this context to be stored scripts */
     public ScriptContext(String name, Class<FactoryType> factoryClazz) {
         // cache size default, cache expire default, max compilation rate are defaults from ScriptService.
         this(name, factoryClazz, 100, TimeValue.timeValueMillis(0), true, true);
     }
 
-    /**
-     * Construct a context for a system script usage, using the:
-     *  - default cache size (only relevant when context caching enabled): 200
-     *  - default cache expiration: no expiration
-     *  - unlimited compilation rate
-     */
-    public ScriptContext(String name, Class<FactoryType> factoryClazz, boolean allowStoredScript) {
-        this(name, factoryClazz, 200, TimeValue.timeValueMillis(0), false, allowStoredScript);
-    }
-
     /** Returns a method with the given name, or throws an exception if multiple are found. */
     private Method findMethod(String type, Class<?> clazz, String methodName) {
         Method foundMethod = null;

server/src/main/java/org/elasticsearch/script/ScriptService.java

@@ -23,6 +23,8 @@
 import org.elasticsearch.cluster.metadata.Metadata;
 import org.elasticsearch.cluster.service.ClusterService;
 import org.elasticsearch.common.Strings;
+import org.elasticsearch.common.logging.DeprecationCategory;
+import org.elasticsearch.common.logging.DeprecationLogger;
 import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.common.settings.Setting;
 import org.elasticsearch.common.settings.Setting.Property;
@@ -50,6 +52,7 @@
 public class ScriptService implements Closeable, ClusterStateApplier, ScriptCompiler {
 
     private static final Logger logger = LogManager.getLogger(ScriptService.class);
+    private static final DeprecationLogger deprecationLogger = DeprecationLogger.getLogger(ScriptService.class);
 
     static final String DISABLE_DYNAMIC_SCRIPTING_SETTING = "script.disable_dynamic";
 
@@ -59,16 +62,22 @@ public class ScriptService implements Closeable, ClusterStateApplier, ScriptComp
     static final String USE_CONTEXT_RATE_KEY = "use-context";
 
     public static final Setting<Integer> SCRIPT_GENERAL_CACHE_SIZE_SETTING =
-        Setting.intSetting("script.cache.max_size", 100, 0, Property.NodeScope);
+        Setting.intSetting("script.cache.max_size", 3000, 0, Property.Dynamic, Property.NodeScope);
     public static final Setting<TimeValue> SCRIPT_GENERAL_CACHE_EXPIRE_SETTING =
-        Setting.positiveTimeSetting("script.cache.expire", TimeValue.timeValueMillis(0), Property.NodeScope);
+        Setting.positiveTimeSetting("script.cache.expire", TimeValue.timeValueMillis(0), Property.Dynamic, Property.NodeScope);
     public static final Setting<Integer> SCRIPT_MAX_SIZE_IN_BYTES =
         Setting.intSetting("script.max_size_in_bytes", 65535, 0, Property.Dynamic, Property.NodeScope);
     public static final Setting<ScriptCache.CompilationRate> SCRIPT_GENERAL_MAX_COMPILATIONS_RATE_SETTING =
-        new Setting<>("script.max_compilations_rate", USE_CONTEXT_RATE_KEY,
+        new Setting<>("script.max_compilations_rate", "150/5m",
             (String value) -> value.equals(USE_CONTEXT_RATE_KEY) ? USE_CONTEXT_RATE_VALUE: new ScriptCache.CompilationRate(value),
             Property.Dynamic, Property.NodeScope);
 
+    public static final String USE_CONTEXT_RATE_KEY_DEPRECATION_MESSAGE = "[" + USE_CONTEXT_RATE_KEY + "] is deprecated for the setting [" +
+        SCRIPT_GENERAL_MAX_COMPILATIONS_RATE_SETTING.getKey() + "] as system scripts are now exempt from the rate limit. " +
+        "Set to a value such as [150/5m] (a rate of 150 compilations per five minutes) to rate limit user scripts in case the " +
+        "script cache [" + SCRIPT_GENERAL_CACHE_SIZE_SETTING.getKey() + "] is undersized causing script compilation thrashing.";
+
+
     // Per-context settings
     static final String CONTEXT_PREFIX = "script.context.";
 
@@ -77,13 +86,14 @@ public class ScriptService implements Closeable, ClusterStateApplier, ScriptComp
     public static final Setting.AffixSetting<Integer> SCRIPT_CACHE_SIZE_SETTING =
         Setting.affixKeySetting(CONTEXT_PREFIX,
             "cache_max_size",
-            key -> Setting.intSetting(key, SCRIPT_GENERAL_CACHE_SIZE_SETTING, 0, Property.NodeScope, Property.Dynamic));
+            key -> Setting.intSetting(key, SCRIPT_GENERAL_CACHE_SIZE_SETTING, 0,
+                                      Property.NodeScope, Property.Dynamic, Property.Deprecated));
 
     public static final Setting.AffixSetting<TimeValue> SCRIPT_CACHE_EXPIRE_SETTING =
         Setting.affixKeySetting(CONTEXT_PREFIX,
             "cache_expire",
             key -> Setting.positiveTimeSetting(key, SCRIPT_GENERAL_CACHE_EXPIRE_SETTING, TimeValue.timeValueMillis(0),
-                                               Property.NodeScope, Property.Dynamic));
+                                               Property.NodeScope, Property.Dynamic, Property.Deprecated));
 
     // Unlimited compilation rate for context-specific script caches
     static final String UNLIMITED_COMPILATION_RATE_KEY = "unlimited";
@@ -94,7 +104,7 @@ public class ScriptService implements Closeable, ClusterStateApplier, ScriptComp
             key -> new Setting<ScriptCache.CompilationRate>(key, "75/5m",
                 (String value) -> value.equals(UNLIMITED_COMPILATION_RATE_KEY) ? ScriptCache.UNLIMITED_COMPILATION_RATE:
                                                                                  new ScriptCache.CompilationRate(value),
-                Property.NodeScope, Property.Dynamic));
+                Property.NodeScope, Property.Dynamic, Property.Deprecated));
 
     private static final ScriptCache.CompilationRate SCRIPT_COMPILATION_RATE_ZERO = new ScriptCache.CompilationRate(0, TimeValue.ZERO);
 
@@ -205,6 +215,10 @@ public ScriptService(Settings settings, Map<String, ScriptEngine> engines, Map<S
         this.setCacheHolder(settings);
     }
 
+    public static boolean isUseContextCacheSet(Settings settings) {
+        return SCRIPT_GENERAL_MAX_COMPILATIONS_RATE_SETTING.get(settings).equals(USE_CONTEXT_RATE_VALUE);
+    }
+
     /**
      * This is overridden in tests to disable compilation rate limiting.
      */
@@ -249,6 +263,10 @@ void registerClusterSettingsListeners(ClusterSettings clusterSettings) {
      */
     void validateCacheSettings(Settings settings) {
         boolean useContext = SCRIPT_GENERAL_MAX_COMPILATIONS_RATE_SETTING.get(settings).equals(USE_CONTEXT_RATE_VALUE);
+        if (useContext) {
+            deprecationLogger.warn(DeprecationCategory.SCRIPTING, "scripting-context-cache",
+                    USE_CONTEXT_RATE_KEY_DEPRECATION_MESSAGE);
+        }
         List<Setting.AffixSetting<?>> affixes = Arrays.asList(SCRIPT_MAX_COMPILATIONS_RATE_SETTING, SCRIPT_CACHE_EXPIRE_SETTING,
                                                               SCRIPT_CACHE_SIZE_SETTING);
         List<String> customRates = new ArrayList<>();
@@ -277,7 +295,7 @@ void validateCacheSettings(Settings settings) {
                     String.join(", ", customRates) + "] if compile rates disabled via [" +
                     SCRIPT_DISABLE_MAX_COMPILATIONS_RATE_SETTING.getKey() + "]");
             }
-            if (useContext == false) {
+            if (useContext == false && SCRIPT_GENERAL_MAX_COMPILATIONS_RATE_SETTING.exists(settings)) {
                 throw new IllegalArgumentException("Cannot set custom general compilation rates [" +
                     SCRIPT_GENERAL_MAX_COMPILATIONS_RATE_SETTING.getKey() + "] to [" +
                     SCRIPT_GENERAL_MAX_COMPILATIONS_RATE_SETTING.get(settings) + "] if compile rates disabled via [" +
@@ -568,6 +586,7 @@ void setCacheHolder(Settings settings) {
         } else if (current.general.rate.equals(SCRIPT_GENERAL_MAX_COMPILATIONS_RATE_SETTING.get(settings)) == false ||
                    current.general.cacheExpire.equals(SCRIPT_GENERAL_CACHE_EXPIRE_SETTING.get(settings)) == false ||
                    current.general.cacheSize != SCRIPT_GENERAL_CACHE_SIZE_SETTING.get(settings)) {
+            // General compilation rate, cache expiration or cache size changed
             cacheHolder.set(generalCacheHolder(settings));
         }
     }

server/src/main/java/org/elasticsearch/script/TemplateScript.java

@@ -8,6 +8,8 @@
 
 package org.elasticsearch.script;
 
+import org.elasticsearch.core.TimeValue;
+
 import java.util.Map;
 
 /**
@@ -39,5 +41,6 @@ public interface Factory {
     // Remove compilation rate limit for ingest.  Ingest pipelines may use many mustache templates, triggering compilation
     // rate limiting.  MustacheScriptEngine explicitly checks for TemplateScript.  Rather than complicating the implementation there by
     // creating a new Script class (as would be customary), this context is used to avoid the default rate limit.
-    public static final ScriptContext<Factory> INGEST_CONTEXT = new ScriptContext<>("ingest_template", Factory.class, true);
+    public static final ScriptContext<Factory> INGEST_CONTEXT = new ScriptContext<>("ingest_template", Factory.class,
+            200, TimeValue.timeValueMillis(0), false, true);
 }