You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In this context, k is something like type and the associated v is an ArrayList straight out of the params object. But the params object is 'static' (allocated once per script processor, not once per script processor execution) so it's a shared object. Later on in the same pipeline, there are some append processors that operate on the resulting 'type', like this one:
Which means that if multiple Elasticsearch indexing threads are pushing documents through this pipeline simultaneously, then it's possible (albeit rare!) for one of them to throw a ConcurrentModificationException. As a very subtle point, note that (arguably) it's acceptable to leak the shared reference as long as there is no later modification of the leaked list.
Here's a commit that I think will fix the issue, I tried to model it off the similarly-purposed code in #3492:
elastic/elasticsearch#91977 is the upstream issue in Elasticsearch -- it shouldn't be necessary to make a defensive copy like this, but at least in the context of the current behavior of Elasticsearch, the defensive copy is necessary.
This code leaks a shared mutable reference:
integrations/packages/suricata/data_stream/eve/elasticsearch/ingest_pipeline/default.yml
Line 235 in e6fb971
In this context,
k
is something liketype
and the associatedv
is anArrayList
straight out of theparams
object. But the params object is 'static' (allocated once per script processor, not once per script processor execution) so it's a shared object. Later on in the same pipeline, there are some append processors that operate on the resulting 'type', like this one:integrations/packages/suricata/data_stream/eve/elasticsearch/ingest_pipeline/default.yml
Lines 377 to 380 in e6fb971
Which means that if multiple Elasticsearch indexing threads are pushing documents through this pipeline simultaneously, then it's possible (albeit rare!) for one of them to throw a
ConcurrentModificationException
. As a very subtle point, note that (arguably) it's acceptable to leak the shared reference as long as there is no later modification of the leaked list.Here's a commit that I think will fix the issue, I tried to model it off the similarly-purposed code in #3492:
main...joegallo:integrations:suricata-copy-param-lists
elastic/elasticsearch#91977 is the upstream issue in Elasticsearch -- it shouldn't be necessary to make a defensive copy like this, but at least in the context of the current behavior of Elasticsearch, the defensive copy is necessary.
/cc @andrewkroh @P1llus @masseyke
The text was updated successfully, but these errors were encountered: