Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ElasticSearch 7.x support #2202

Open
kago-dk opened this issue Apr 11, 2019 · 39 comments
Open

ElasticSearch 7.x support #2202

kago-dk opened this issue Apr 11, 2019 · 39 comments

Comments

@kago-dk
Copy link

kago-dk commented Apr 11, 2019

I upgraded my Elasticsearch installation to 7.0.0 and since then elastalert is giving errors related to "[_source_include] -> did you mean any of [_source_includes, _source_excludes]", which I assume is due to "Remove deprecated url parameters _source_include and _source_exclude #35097 (issues: #22792, #33475)" (https://www.elastic.co/guide/en/elasticsearch/reference/7.0/release-notes-7.0.0.html)

Is there in plan to make Elastalert compatible with ElasticSearch 7.0.0 anytime soon?

@matsgoran
Copy link
Contributor

I've currently got a pull request trying to resolve this issue #2194 . We are currently running this branch against our ES7.0-beta1 server. If the change is accepted into the main project is unknown

@pipari666
Copy link

Same problem here. Worked nicely before. I hope it will be merged soon, else i have to downgrade or change component for alerts.

best regards.

@moetemad
Copy link

Are there any updates on using elastalert on elastic 7.0?

@Yv-o-rL
Copy link

Yv-o-rL commented Apr 30, 2019

Hi!
Is there any type of workaround? I couldn't postpone the upgrade, but the alert is pretty important for me :(

@Qmando
Copy link
Member

Qmando commented May 1, 2019

Try installing elastalert==0.2.0b2. I've just merged in a branch for ES7 support, but it's still under the beta releases as I've not extensively tested it.

@jbecker94
Copy link

I've tested the version 0.2.0b2 but unfortunatly. I'm still getting the error contains unrecognized parameter: [_source_include] -> did you mean any of [_source_includes, _source_excludes]?')"

I have been using the bitsensor/elastalert dockerfile to build an new image. Any other ideas?

@matsgoran
Copy link
Contributor

@jbecker94 Can you provide more information? In order to track down a possible bug we need to know which version of Elasticsearch you're running against so we can replicate this. Just to make sure can you compare the elastalert.py file in the docker image against the same file in tag 0.2.0b2? The complete stack trace would also be of benefit to us since we get more information of the origins of this error.

@jbecker94
Copy link

jbecker94 commented May 3, 2019

@matsgoran I have done a little bit more troubleshooting and it is now working using the follwing commands according to bitsensor:

git clone https://github.com/bitsensor/elastalert.git && cd elastalert make build v=v0.2.0b2

the only thing which isn't working is the elastalert-test-rule command. I'm not sure if this is an issue with the docker-container or the version. I used the following commands to test my rule which works under 0.1.39 but not using 0.2.0b2.

sudo docker exec -it elastalert sh cd .. cd elastalert elastalert-test-rule rules/my-super-rule.yaml

This respond in the following stackstrace:

Traceback (most recent call last): File "/usr/bin/elastalert-test-rule", line 11, in <module> load_entry_point('elastalert==0.2.0b2', 'console_scripts', 'elastalert-test-rule')() File "/usr/lib/python2.7/site-packages/elastalert-0.2.0b2-py2.7.egg/elastalert/test_rule.py", line 441, in main test_instance.run_rule_test() File "/usr/lib/python2.7/site-packages/elastalert-0.2.0b2-py2.7.egg/elastalert/test_rule.py", line 408, in run_rule_test conf = load_conf(args, defaults, overwrites) File "/usr/lib/python2.7/site-packages/elastalert-0.2.0b2-py2.7.egg/elastalert/config.py", line 44, in load_conf conf = yaml_loader(filename) File "/usr/lib/python2.7/site-packages/PyStaticConfiguration-0.10.4-py2.7.egg/staticconf/loader.py", line 167, in yaml_loader with open(filename) as fh: TypeError: coercing to Unicode: need string or buffer, NoneType found
Elasticsearch-Version: 7.0.0

@matsgoran
Copy link
Contributor

@jbecker94 from what i can figure the test-rule command expects a —config parameter pointing to the elastalert configuration, there is no default for this option as far as I can tell in elastalert. Therefore my first reaction would be that this option needs to be provided by bitsensor/elastalert. FYI I may be wrong in this assesment since I'm away from the computer atm

@jbecker94
Copy link

@matsgoran for my understanding if it runs under 0.1.39 it should also be running under 0.2.0b2. The config should be available since the exact same folders are mounted into both of the containers. I have used the exact same Dockerbuildfile as the original bitsensor-docker-image, so there should be no difference except the different elastalert-version.

Hopefully in a couple of days there will be an offical bitsensor-version which fixes this issue. Most likely I have done something wrong in the build process or can anybody confirm that the test-rule command doesn't work?

@matsgoran
Copy link
Contributor

matsgoran commented May 3, 2019

@jbecker94 Let's hope so:)The changes related to the ES7 compatibility in the beta did not touch any part of the test-rule bootstrap process

@Qmando
Copy link
Member

Qmando commented May 3, 2019

[_source_include] -> did you mean any of [_source_includes..

Man, this is the most annoying kind of breaking change. Just grepping through the code I can see that these don't always appear together in a search/deprecated_search pairs. I'll go through the code and double check this gets used correctly. More context from THAT error would be useful.

Elastalert-test-rule is kind of a mess, the error handling there is not great, and there's all sorts of special snowflake config being done. The TypeError: coercing to Unicode: need string or buffer, NoneType found does seem clear cut like the config parameter was missing. Again, terrible error handling, sorry about that.

@RiteshKuchukulla
Copy link

RiteshKuchukulla commented May 7, 2019

I tried with 0.2.0b version of ElastAlert (I'm using Elastic stack 7.0 version)
I'm still getting this error -
"message": "Error running query: TransportError(400, u\u0027illegal_argument_exception\u0027, u\u0027request [/auditbeat-/_search] contains unrecognized parameter: [_source_include] -> did you mean any of [_source_includes, _source_excludes]?\u0027)",
"traceback": [
"Traceback (most recent call last):",
" File "/etc/elastalert/elastalert/elastalert.py", line 390, in get_hits",
" **extra_args",
" File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/utils.py", line 76, in _wrapped",
" v = kwargs.pop(p)",
" File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/init.py", line 655, in search",
" to the total number of copies for the shard (number of replicas + 1)",
" File "/usr/local/lib/python2.7/dist-packages/elasticsearch/transport.py", line 314, in perform_request",
" # add a delay before attempting the next retry",
" File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_requests.py", line 90, in perform_request",
" self.log_request_fail(method, url, response.request.path_url, body, duration, response.status_code, raw_data)",
" File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/base.py", line 125, in _raise_error",
" error_message = additional_info.get(\u0027error\u0027, error_message)",
"RequestError: TransportError(400, u\u0027illegal_argument_exception\u0027, u\u0027request [/auditbeat-
/_search] contains unrecognized parameter: [_source_include] -> did you mean any of [_source_includes, _source_excludes]?\u0027)"

@matsgoran
Copy link
Contributor

@RiteshKuchukulla Are you running 0.2.0b1 or 0.2.0b2? ES7 support was added in 0.2.0b2

@RiteshKuchukulla
Copy link

RiteshKuchukulla commented May 7, 2019

@matsgoran
I was running 0.2.0b1, which gave the above error.
Now I started to run 0.2.0b2, which doesn't give me this error, thank you

@fpompermaier
Copy link

When I try to build the beta version I get this error on Centos 7:

error: urllib3 1.25.3 is installed but urllib3<1.25,>=1.20 is required by set(['botocore'])

@moix
Copy link

moix commented Jun 7, 2019

hi here, I read different comments about beta version working or not in 7.x, Can anyone confirm if 0.2.0b2 works? I needed to upgrade to 7.1 and since then we dont have alerting so would like to give it a try soon.
I'll start with 0.2.0b2 but if you are using it already in 7.x please can confirm?
Thanks! great work

@shahid-dgs
Copy link

I Just installed latest version of ELK 7.1 but ElastAlert is not working i am getting. any one got any fixed for ES7.1

@Qmando
Copy link
Member

Qmando commented Jun 24, 2019

@fpompermaier
Did you try updating botocore?

@Qmando
Copy link
Member

Qmando commented Jun 24, 2019

@shahid-dgs
You need to share more info if you want help

@fpompermaier
Copy link

@Qmando what should I do? I just followed the build instructions

@bangejsans
Copy link

Anyone bold enough to see if this works with 7.2 yet?

@kago-dk
Copy link
Author

kago-dk commented Jun 26, 2019

@bangejsans No new issues seen using v0.2.0b2 and running for 12 hours.

@zozo6015
Copy link

Hello,

I have installed elastalert beta from pip, when I am trying to create index I am getting the following error message:

Traceback (most recent call last):
  File "/usr/local/bin/elastalert-create-index", line 7, in <module>
    from elastalert.create_index import main
  File "/usr/local/lib/python3.5/dist-packages/elastalert/create_index.py", line 13, in <module>
    from auth import Auth
ImportError: cannot import name 'Auth'

Note I am running elasticsearch 7.2.0 with security enabled.

@gerbdla
Copy link

gerbdla commented Jul 1, 2019

Getting the same errors as above. Is there a resolution yet to the source_includes error?

@Qmando
Copy link
Member

Qmando commented Jul 1, 2019

Run python 2, use v0.2.0b2

@zozo6015
Copy link

zozo6015 commented Jul 1, 2019

Interesting. python2 is end of life in less than 6 months and we still have it as a dependency

@rickywu
Copy link

rickywu commented Jul 26, 2019

velocidi/elastalert this docker image works well for es 7.2

@gdoctor
Copy link

gdoctor commented Feb 15, 2021

Has anyone run this on elasticsearch 7.10.x or 7.11.x?

@nsano-rururu
Copy link
Contributor

@gdoctor

Has anyone run this on elasticsearch 7.10.x or 7.11.x?

It should work.

@NOULeENGINEER
Copy link

@nsano-rururu
it seems that the elastalert version, that've pulled is not compatible with my elasticsearch version 7.9.1. when i run "npm run start", I got this error.
image

@nsano-rururu
Copy link
Contributor

@NOULeENGINEER

It's not Esper, so that kind of listening is frustrating.

elastalert-server?
docker?
manual install?
What is the OS?
What command was executed and what was installed before execution?

@nsano-rururu
Copy link
Contributor

The latest docker image should be python3.8 environment?
https://hub.docker.com/r/praecoapp/elastalert-server

@nsano-rururu
Copy link
Contributor

@NOULeENGINEER
This repository is not a good place to ask ElastAlertServer questions.

@nsano-rururu
Copy link
Contributor

@NOULeENGINEER

It works with the latest 7.11.1, but what?

キャプチャ

@NOULeENGINEER
Copy link

I'm going to try your latest version on the docker hub

@NOULeENGINEER

It's not Esper, so that kind of listening is frustrating.

elastalert-server?
docker?
manual install?
What is the OS?
What command was executed and what was installed before execution?

the version of the elasticsearch is 7.9.1
the OS is ubuntu 18.04.5
the installation is manual, i followed steps that i've found in a medium blog : Deploying a SOCaaS infrastructure
the latest command that i've ran is npm run start

@nsano-rururu
Copy link
Contributor

@NOULeENGINEER

See the official website for the manual installation procedure. I wrote. Because I'm the maintainer.
https://github.com/johnsusek/praeco#manualdev-installation

@udaykumar-tivo
Copy link

I have installed elasticsearch 7.0.1 and elastalert plugin. I am facing the same _source_include issue. Any fix is provided?

Would have written the following documents to writeback index (default is elastalert_status):

elastalert_error - {'message': "Error running query: RequestError(400, u'illegal_argument_exception', u'request [/elastic_log_qa1_*/_search] contains unrecognized parameter: [_source_include] -> did you mean any of [_source_includes, _source_excludes]?')", 'traceback': ['Traceback (most recent call last):', ' File "elastalert/elastalert.py", line 370, in get_hits',

@nsano-rururu
Copy link
Contributor

nsano-rururu commented Apr 5, 2021

@udaykumar-tivo

Please create a new issue. Also, it is difficult to answer without more detailed information about the environment.
Note that the bitsensor elastalert-server and elastalert-kibana-plugin are not maintained.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests