Yara does not accept Regex having negative lookahead

[chintanhshah]

Hi ,

Just wanted to point out that Yara 3.4 does not accept the regex having negative lookahead . It throws exceptions on invalid characters . Following is the regex that I am trying to use :

([^\=]*)0x[a-f0-9]{1,3}\s0x[a-f0-9]{8}\s(?!nt|win)

Above regex I was apparently attempting to match the line not having either "nt" or "win" after the space character . While compiling the rule, it throws following error .
error: invalid regular expression "$nt_ssdt": syntax error, unexpected '?'

Please suggest any alternative way of writing negative lookahead regex or fix the issue.

[maxvonhippel]

I was able to reproduce the error, and moreover, I checked the docs and did not find anything about look-aheads or look-behinds in the regex section. I think this feature would be worth adding, but lack the coding acumen to do so myself. That said, I think you could sort of hack it like this:

RuleOne

rule one {
    strings:
        $yay = /baloney[0-9a-z]*/
    condition:
        $yay
}

RuleTwo

rule two {
    strings:
        $nay = /\s(ns|win){1}/
    condition:
        $nay
}

tst.txt

baloney ns
toodlesns
 win
0011uuii win
9090jjuu
baloney
baloneywin
baloney win
baloney092083

And then in Bash:

yara -s RuleOne tst.txt > outOne
yara -s -n RuleTwo tst.txt > outTwo
comm outOne outTwo

Produces:

0x0:$yay: baloney
0x30:$yay: baloney
0x38:$yay: baloneywin
0x43:$yay: baloney
0x4f:$yay: baloney092083

(I'm not much of a REGEX whiz and found yours vaguely confusing, hence the stupid baloney example, but I think the idea is more or less the same?)

[chintanhshah]

Appreciate the response .

We are basically building the bigger Yara rules file with several hunderds of rules in just a single file and scanning all of them at one shot . Doing a Yara -n for a single rule wont workaround and produce lot of False positives as we are matching on the multiple GBs of Memory dump files ..

Image attached here perhaps might give you a better idea.

[maxvonhippel]

Gotcha. As I said, my solution was definitely a "hack". In the docs I see that "not" is a supported keyword, but I'm not really sure how to use it. I tried doing:

condition:
    $yay and not $nay

... but it didn't seem to work. That said, if you can figure out how to use not in the conditional statement, that could possibly solve the problem? I mean it'd be a bit slow, but wouldn't have the memory issues of my previous hack-y sudo-solution.

Edit - for some reason not $nay seems to work for me, but $yay and not $nay does not. It's quite late where I am so I'm going to go sleep, but I think that if you can figure out the negation thing using not, that really could be the solution. Good luck!

[chintanhshah]

Ah ok..I will have to spend some time trying a workaround to this. It would solve quite a bit of FP problems if that works ..

Thanks for your quick suggestion .

[maxvonhippel]

Please comment if you figure it out, I would also like to know the answer to this. If not, I'll try and work on it later this week.
EDIT: school started. I still find this problem super interesting, but cannot truthfully say I will likely get to it "later this week" :(

[chintanhshah]

Stilll not figured out ..Let me know if anybody had luck with this .

[plusvic]

@chintanhshah I'm doing some spring clean up and came across this issue, sorry for not answering before.

Look-around features are not implemented in YARA because its regexp engine is not recursive and doesn't do backtracking. The advantage of this approach is that the running time doesn't blow up exponentially with certain regexps, but the drawback is that some features usually implemented by recursive regexp engines can't be implemented or are more complicated to implement.

Regarding your use case I believe you want to detect dumps where some of the ssdt hooks point to a function not starting with "nt" or "win".

What about this?

rule test
{
    strings:
        $a = /(ssdt hooks)([^\=]*)0x[a-f0-9]{1,3}\s0x[a-f0-9]{8}\s/
        $b = /(ssdt hooks)([^\=]*)0x[a-f0-9]{1,3}\s0x[a-f0-9]{8}\s(nt|win)/

    condition:
        for any i in (1..#a) : ( @a[i] != @b[i])
}

This rule tries to find the regexp with and without the "nt" or "win" strings, and make sure that both regexp match at the same offsets, if this doesn't happens the rule matches. It solves your problem?

[chintanhshah]

Oh that sounds like a good logic . I will definitely check this out today . Thanks a lot for your response and help ..

[ca-schue]

@plusvic 's solution does not work for me.
I'm trying to look for a url without the '.com' suffix.

rule keyword {
	strings:
		$a=/some-url/ wide ascii nocase
		$b=/some-url(\.com)/ wide ascii nocase	
	condition:
		for any i in (1..#a) : ( @a[i] != @b[i])
}