Getting TypeError: unsupported operand type(s) for -: 'datetime.datetime' and 'dict' when running my custom rule

[scotbab]

Hi
The custom rule runs fine using elastalert-test-rule but when running using python -m elastalert.elastalert it gives the below error -
Getting TypeError: unsupported operand type(s) for -: 'datetime.datetime' and 'dict'

Below is the custom timestamp used:

timestamp_field: Timestamp
timestamp_format: '%Y-%m-%dT%H:%M:%S.%fZ'

Here is the add_data function my_rule python script..

def add_data(self, data):

    for document in data:

        # To access config options, use self.rules
        # print document['RunId']
        if isinstance(document['Timestamp'], dict):
            document['Timestamp'] = parser.parse(document['Timestamp'])
            if document['RunId'] in self.rules['RunIds']:
               buildno = self.rules['RunIds']
               build = str(buildno).replace("['","")
               build = build.replace("']","")
               act_build = str(document['RunId'])
               if document['RunId'] == build:
                 print "Runid matched"
                 if document['Name'] in self.rules['Names']:
                   stepno = self.rules['Names']
                   step = str(stepno).replace("['", "")
                   step = step.replace("']", "")
                   act_step = str(document['Name'])
                   if document['Name'] == step:
                    print "Name matched"
                    Actual = document['Total']
                    Base_value = self.rules['Base_Line']
                    if int(Actual) > int(Base_value):
                        # To add a match, use self.add_match
                        self.add_match(document)
                        break

Thanks in advance guys..

[Qmando]

if isinstance(document['Timestamp'], dict):
            document['Timestamp'] = parser.parse(document['Timestamp'])

parser.parse works for strings only, assuming that's dateutil.parser. Though, you didn't post the traceback so I don't even know where the error occurred. The timestamp should have been converted to a datetime object before anyway.

Also, you could remove all the string replacement stuff and just do if document['RunId'] in document['RunIds']

[scotbab]

Thanks Qmando, I tried this solution but it was not working, might be I was putting the parsing code in wrong place. Not sure.

[scotbab]

I think I was able to solve it. The attribute 'query_delay' was missing in the rule.yaml file, after placing it the issue was resolved. Can you please describe regarding 'query_delay'. Thanks. I have one more question the rule that I am running using elastalert is continuously running for hours. Is there a way for the rule to execute/check once and then stop ?

[Qmando]

You can give a --end argument to elastalert and it will stop once it hits that timestamp. Scheduling it for certain hours of the day is not yet available, but look for it in a future release.

query_delay simple moves all queries back in time by that amount. For example, a five minute delay would cause queries at 12:00 to end at 11:55 instead of going up to the present time.