OpenSearch compatibility - include_type_name deprecated

[maximus321]

Hi there, we are moving from ES to OS. We are migrating elastalert also to use OpenSearch.

While starting up, I get this error, that seems to be related to a deprecated query param:

/7/2024, 2:26:52 PM	elastalert[2068]	[Status: Downloaded newer image for prod.tmhub.io/prod/docker.io/jertel/elastalert2:2.17.0](http://10.188.81.250:19531/browse#entry)
5/7/2024, 2:26:52 PM	elastalert[2068]	[prod.tmhub.io/prod/docker.io/jertel/elastalert2:2.17.0](http://10.188.81.250:19531/browse#entry)
5/7/2024, 2:26:52 PM	elastalert[2226]	[Error: No such container: elastalert](http://10.188.81.250:19531/browse#entry)
5/7/2024, 2:26:56 PM	elastalert[2232]	[WARNING:elasticsearch:PUT REMOVED:443/prd2138_elastalert_status/_doc/_mapping?include_type_name=true [status:400 request:0.009s]](http://10.188.81.250:19531/browse#entry)
5/7/2024, 2:26:56 PM	elastalert[2232]	[Traceback (most recent call last):](http://10.188.81.250:19531/browse#entry)
5/7/2024, 2:26:56 PM	elastalert[2232]	[File "/usr/local/bin/elastalert-create-index", line 8, in <module>](http://10.188.81.250:19531/browse#entry)
5/7/2024, 2:26:56 PM	elastalert[2232]	[sys.exit(main())](http://10.188.81.250:19531/browse#entry)
5/7/2024, 2:26:56 PM	elastalert[2232]	[^^^^^^](http://10.188.81.250:19531/browse#entry)
5/7/2024, 2:26:56 PM	elastalert[2232]	[File "/usr/local/lib/python3.12/site-packages/elastalert/create_index.py", line 242, in main](http://10.188.81.250:19531/browse#entry)
5/7/2024, 2:26:56 PM	elastalert[2232]	[create_index_mappings(es_client=es, ea_index=index, recreate=args.recreate, old_ea_index=old_index)](http://10.188.81.250:19531/browse#entry)
5/7/2024, 2:26:56 PM	elastalert[2232]	[File "/usr/local/lib/python3.12/site-packages/elastalert/create_index.py", line 78, in create_index_mappings](http://10.188.81.250:19531/browse#entry)
5/7/2024, 2:26:56 PM	elastalert[2232]	[es_client.indices.put_mapping(index=ea_index, doc_type='_doc',](http://10.188.81.250:19531/browse#entry)

We are on this version on OpenSearch:

{
  "version" : {
    "number" : "7.10.2",
    "build_type" : "tar",
    "build_hash" : "unknown",
    "build_date" : "2024-02-09T12:35:41.689803723Z",
    "build_snapshot" : false,
    "lucene_version" : "9.7.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

This check seems to be in cause, we are sending the deprecated attribute, but we would like to not send it

https://github.com/jertel/elastalert2/blob/master/elastalert/create_index.py

I can confirm with a simple get here that gives a BAD REQUEST
https://REMOVED/prd2138_elastalert_status/_doc/_mapping?include_type_name=true
returns

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"request [/prd2138_elastalert_status/_doc/_mapping] contains unrecognized parameter: [include_type_name]"}],"type":"illegal_argument_exception","reason":"request [/prd2138_elastalert_status/_doc/_mapping] contains unrecognized parameter: [include_type_name]"},"status":400}

With https://REMOVED/prd2138_elastalert_status/_doc/_mapping, no BAD_REQUEST

[maximus321]

After a deploy, I still received my alerts, so not sure if this error is a false warning? The index looks to be created correctly

{"prd2138_elastalert_status":{"aliases":{},"mappings":{"properties":{"@timestamp":{"type":"date"},"alert_info":{"properties":{"slack_username_override":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"type":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}}},"alert_sent":{"type":"boolean"},"alert_time":{"type":"date"},"match_body":{"properties":{"datetime":{"type":"date"},"num_hits":{"type":"long"},"num_matches":{"type":"long"}}},"match_time":{"type":"date"},"rule_name":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}}},"settings":{"index":{"replication":{"type":"DOCUMENT"},"mapping":{"ignore_malformed":"true"},"number_of_shards":"1","provided_name":"prd2138_elastalert_status","creation_date":"1715106413813","number_of_replicas":"1","uuid":"vsW3J66PTyKE12gCqku-Jw","version":{"created":"136327827"}}}}}

[jertel]

I've seen others mention a similar error in discussions. It's not a false warning, as it's giving a legitimate error. But it should be able to continue since that parameter isn't critical. In other words, the error is not fatal.

[maximus321]

thanks for your answer jertel, as long as we receive our alerts, it works for us. thanks for the project