search() got an unexpected keyword argument 'doc_type' #2204
Comments
|
try |
|
this is because of #2203 |
|
I am using the 6.7.0 elastic stack. Cannot migrate to 7.0.0 since elastalert kibana plugin is not available. I am seeing elastalert | File "/opt/elastalert/elastalert/elastalert.py", line 1528, in send_pending_alerts Cannot go to the 5.5.3. What are my options? |
|
For me doing the following fixed it: |
Until Yelp/elastalert#2204 is fixed
|
From the elasticsearch pypi page:
If you install an incompatible version it won't work, so make sure you pick the right one. |
|
|
Hi, I am getting this as well using a containerized version 0.1.38. I see there is 0.1.39 from 3/19. Is this fixed there too? |
|
Hello, Thanks in advance |
|
I was able to fix my issue for v0.1.39. I use this dockerfile: FROM alpine:latest as py-ea
ARG ELASTALERT_VERSION=v0.1.39
ENV ELASTALERT_VERSION=${ELASTALERT_VERSION}
# URL from which to download Elastalert.
ARG ELASTALERT_URL=https://github.com/Yelp/elastalert/archive/$ELASTALERT_VERSION.zip
ENV ELASTALERT_URL=${ELASTALERT_URL}
# Elastalert home directory full path.
ENV ELASTALERT_HOME /opt/elastalert
RUN apk update && \
apk add ca-certificates
WORKDIR /opt
RUN apk add --update --no-cache ca-certificates openssl-dev openssl python2-dev python2 py2-pip py2-yaml libffi-dev gcc musl-dev wget && \
# Download and unpack Elastalert.
wget -O elastalert.zip "${ELASTALERT_URL}" && \
unzip elastalert.zip && \
rm elastalert.zip && \
mv e* "${ELASTALERT_HOME}"
WORKDIR "${ELASTALERT_HOME}"
# Copy requirements.txt with specific elasticsearch version
RUN rm requirements.txt
COPY requirements.txt requirements.txt
# Install Elastalert.
# see: https://github.com/Yelp/elastalert/issues/1654
RUN sed -i 's/jira>=1.0.10/jira>=1.0.10,<1.0.15/g' setup.py && \
python setup.py install && \
pip install -r requirements.txt
FROM node:alpine
LABEL maintainer="BitSensor <[email protected]>"
# Set timezone for this container
ENV TZ Etc/UTC
RUN apk add --update --no-cache curl tzdata python2 make libmagic
COPY --from=py-ea /usr/lib/python2.7/site-packages /usr/lib/python2.7/site-packages
COPY --from=py-ea /opt/elastalert /opt/elastalert
COPY --from=py-ea /usr/bin/elastalert* /usr/bin/
WORKDIR /opt/elastalert-server
COPY . /opt/elastalert-server
RUN npm install --production --quiet
COPY config/elastalert.yaml /opt/elastalert/config.yaml
COPY config/elastalert-test.yaml /opt/elastalert/config-test.yaml
COPY config/smtp_auth.yaml /opt/elastalert/smtp_auth.yaml
COPY config/config.json config/config.json
COPY rule_templates/ /opt/elastalert/rule_templates
COPY elastalert_modules/ /opt/elastalert/elastalert_modules
EXPOSE 3030
ENTRYPOINT ["npm", "start"]As you can see above, after downloading the release I desire, I remove the packaged requirements.txt that specified the latest version of elasticsearch and committed a version of my own to my repo that specified the exact version I wanted. In my case:
|
|
Hi John, thanks for reply. So i am currently using the following versions: elastalert 0.1.39 What I did to install elastalert was the following:
Unfortunately I dont know anymore what to do else now.... |
|
@MaximilianKaltner Have you tried using 0.2.0b2? |
|
Hi @matsgoran, thanks for your advice. Is it necessary to upgrade elasticsearch to 7.0.0 for elastalert 0.2.0b2? |
|
@MaximilianKaltner No it should not. 0.2.0b2 has some specific fixes related to API changes starting with Elasticsearch 6.6. (#2194) |
|
Perfect, i will check this out and give feedback to you! Thanks a lot |
|
Hi @matsgoran, installation and creating indices was no problem with elastalert 0.2.0b2. Unfortunately I am still getting errors during command elastalert-test-rule. It looks like that he is expecting 'elastalert==0.1.39' distribution, which was not found: Running elastalert with this rule look like that it is working fine, found some hits and matches, but gives another error message, that it is not able to connect or sent alert to ms_teams via webhook: Thanks for your support again! |
|
@MaximilianKaltner Did you uninstall elastalert 0.1.39 using pip uninstall? To me it seems like you are executing a dangling 0.1.39 console script. You could always try to execute the elastalert-test rule in a similar fashion as you did with elastalert thus circumventing the script entirely: FYI: The test-rule in 0.2.0b2 contains some bugs that seems fixed in the master branch. Especially an assertion error caused by buggy argument parsing. |
|
Hi @matsgoran Correct, I did uninstall using pip. Trying to execute the testrule-script directly with python is not working, i dont understand why :( elasticsearch-py 7.0.0 is correct to use with the latest elastalert version and also compatible to elasticsearch 6.6.2 on ELK? |
|
@MaximilianKaltner Seems like you have two typos in your command (elastalert-test-rule and --rule). The command should be |
|
@matsgoran Correct, sorry - now I get the mentioned issue. How can I get the master branch for fixed version? |
|
@MaximilianKaltner you could simply do a checkout of the git repository. Run the code directly from your local copy or build the python package via setup-tools and install it using pip |
|
@matsgoran |
|
@MaximilianKaltner The changes in 0.2.0b2 have been merged into the master branch. |
|
Cool - thank you so much @matsgoran ! Testing the rule is working fine now :) Do you also have an idea for my (hopefully) last problem? It looks like that i am not able to sent alerts to ms_teams My rule looks like: |
|
@matsgoran I fixed the problem with ms-teams. I missed the proxy configuration in my rule! ms_teams_proxy: By default ElastAlert will not use a network proxy to send notifications to MS Teams. Set this option using hostname:port if you need to use a proxy. Now everything works - thanks again! |
|
Thanks to @johnjbateman for mentioning a version of elasticsearch that worked with 0.1.38. I have a docker image that pulls down the elastalert source and builds it, so I used sed to replace elasticsearch in setup.py with a specific version (the same could be done in requirements.txt): |
|
@ryanpersaud Nice! I am changing my dockerfile to use sed instead of what I did with committing the requirements.txt. Thanks for the suggestion! |
|
Hi! RUN sed -i 's/jira>=1.0.10/jira>=1.0.10,<1.0.15/g' setup.py && RUN pip install urllib3==1.24.3 It's working now! |
|
@pietrolatorre Nice call out there on urllib3. I also struggled through that. Couple notes:
# Install Elastalert.
RUN sed -i 's/elasticsearch/elasticsearch==6.3.1/' /opt/elastalert/setup.py && \
sed -i 's/aws-requests-auth>=0.3.0/aws-requests-auth==0.3.0/' /opt/elastalert/setup.py && \
sed -i 's/boto3>=1.4.4/boto3==1.4.4/' /opt/elastalert/setup.py && \
python setup.py install && \
pip install -r requirements.txtI did this because I don't use aws, but might not work for others. |
|
@pietrolatorre I had to do the same thing as well (manually install urllib3). I was able to use 1.22. |
|
Had same issue, fixed: |
|
hey, I'm still getting this issue... |
|
+1 |
|
getting this issue also on 7.3.1 |
|
@stavgayer @amundra2016 @mali30 Do you all have elastalert >0.2.0 ? |
|
@Qmando I'm having the same issue on elastisearch 7.3.2. I am using these images in docker: That bitsensor is also the |
|
Here is the full log output from the elastalert container: |
|
And one more to help with debugging. I've updated my image to Deleting the indexes fixed this. It appears that the latest official Docker HUB image (2.0.1) needs an update as it is not running |
|
Please post this in github.com/bitsensor/elastalert if the issue is with that. |
|
I ran into this issue as well. Perhaps we should remove the reference to the docker image on the README if it's not functional? Or at least mention the circumstances in which it's not compatible. |
The text was updated successfully, but these errors were encountered: