Attempt to add a new YARA rule files during runtime causes panic

[teebow1e]

Hi, I'm trying your go-yara library and testing whether it suits my use-case. Here is the ideas I'm trying to do:

• The binary constantly checking data from network packets to see whether they violate the existing rules.
• If I see a packets has bypassed the existing rules, I'll try to add a new rules to the rules folder.

So, before trying any goroutine to automate the process of detecting new YARA files being added, I tried to live-load a new rule during runtime inside the code like this:

// the default examples - loads and compiles rules when the program starts
for _, rule := range rules {
                f, err := os.Open(rule.filename)
                if err != nil {
                        log.Fatalf("Could not open rule file %s: %s", rule.filename, err)
                }
                err = c.AddFile(f, rule.namespace)
                f.Close()
                if err != nil {
                        log.Fatalf("Could not parse rule file %s: %s", rule.filename, err)
                }
        } 
        r, err := c.GetRules()
        if err != nil {
                log.Fatalf("Failed to compile rules: %s", err)
        }
// my implementation - live load after the initial compilation has finished
        f1, err := os.Open("./samp1e.yara")
        err = c.AddFile(f1, "")
        f1.Close()
        r, err = c.GetRules()
        if err != nil {
                log.Fatalf("Failed to compile rules: %s", err)
        }

My implementation is causing the program to panic. Here is the panic message:

hihi.com: libyara/compiler.c:608: yr_compiler_add_fd: Assertion `compiler->rules == NULL' failed.
SIGABRT: abort
PC=0x7f911eea49fc m=0 sigcode=18446744073709551610
signal arrived during cgo execution

goroutine 1 gp=0xc0000061c0 m=0 mp=0x59a4a0 [syscall]:
runtime.cgocall(0x4b40f0, 0xc000052b20)
        /usr/local/go/src/runtime/cgocall.go:157 +0x4b fp=0xc000052af8 sp=0xc000052ac0 pc=0x40806b
github.com/hillu/go-yara/v4._Cfunc__yr_compiler_add_fd(0x152d050, 0x3, 0x0, 0x1564770)
        _cgo_gotypes.go:1009 +0x4b fp=0xc000052b20 sp=0xc000052af8 pc=0x4a294b
github.com/hillu/go-yara/v4.(*Compiler).AddFile.func4(0x4d33a0?, 0xc00009e038?, 0x0, 0x1564770)
        /root/go/pkg/mod/github.com/hillu/go-yara/[email protected]/compiler.go:127 +0xa5 fp=0xc000052b68 sp=0xc000052b20 pc=0x4a4b05
github.com/hillu/go-yara/v4.(*Compiler).AddFile(0xc0000a4140, 0xc0000a0048, {0x0?, 0x0?})
        /root/go/pkg/mod/github.com/hillu/go-yara/[email protected]/compiler.go:127 +0x1b0 fp=0xc000052c20 sp=0xc000052b68 pc=0x4a4830
main.main()
        /root/hihi/go-yara/_examples/simple-yara/simple-yara.go:98 +0x709 fp=0xc000052f50 sp=0xc000052c20 pc=0x4b28c9
runtime.main()
        /usr/local/go/src/runtime/proc.go:271 +0x29d fp=0xc000052fe0 sp=0xc000052f50 pc=0x43a79d
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000052fe8 sp=0xc000052fe0 pc=0x467dc1

goroutine 2 gp=0xc000006c40 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000042fa8 sp=0xc000042f88 pc=0x43abce
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.forcegchelper()
        /usr/local/go/src/runtime/proc.go:326 +0xb3 fp=0xc000042fe0 sp=0xc000042fa8 pc=0x43aa53
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000042fe8 sp=0xc000042fe0 pc=0x467dc1
created by runtime.init.6 in goroutine 1
        /usr/local/go/src/runtime/proc.go:314 +0x1a

goroutine 3 gp=0xc000007180 m=nil [GC sweep wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000043780 sp=0xc000043760 pc=0x43abce
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.bgsweep(0xc00006a000)
        /usr/local/go/src/runtime/mgcsweep.go:278 +0x94 fp=0xc0000437c8 sp=0xc000043780 pc=0x426e54
runtime.gcenable.gowrap1()
        /usr/local/go/src/runtime/mgc.go:203 +0x25 fp=0xc0000437e0 sp=0xc0000437c8 pc=0x41b9a5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000437e8 sp=0xc0000437e0 pc=0x467dc1
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:203 +0x66

goroutine 4 gp=0xc000007340 m=nil [GC scavenge wait]:
runtime.gopark(0xc00006a000?, 0x503fa8?, 0x1?, 0x0?, 0xc000007340?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000043f78 sp=0xc000043f58 pc=0x43abce
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.(*scavengerState).park(0x599e40)
        /usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000043fa8 sp=0xc000043f78 pc=0x424849
runtime.bgscavenge(0xc00006a000)
        /usr/local/go/src/runtime/mgcscavenge.go:653 +0x3c fp=0xc000043fc8 sp=0xc000043fa8 pc=0x424ddc
runtime.gcenable.gowrap2()
        /usr/local/go/src/runtime/mgc.go:204 +0x25 fp=0xc000043fe0 sp=0xc000043fc8 pc=0x41b945
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000043fe8 sp=0xc000043fe0 pc=0x467dc1
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:204 +0xa5

goroutine 18 gp=0xc000082700 m=nil [finalizer wait]:
runtime.gopark(0xc000042648?, 0x411825?, 0xa8?, 0x1?, 0xc0000061c0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000042620 sp=0xc000042600 pc=0x43abce
runtime.runfinq()
        /usr/local/go/src/runtime/mfinal.go:194 +0x107 fp=0xc0000427e0 sp=0xc000042620 pc=0x41a9e7
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000427e8 sp=0xc0000427e0 pc=0x467dc1
created by runtime.createfing in goroutine 1
        /usr/local/go/src/runtime/mfinal.go:164 +0x3d

rax    0x0
rbx    0x7f911ed24740
rcx    0x7f911eea49fc
rdx    0x6
rdi    0x7ee56
rsi    0x7ee56
rbp    0x7ee56
rsp    0x7ffd4327ece0
r8     0x7ffd4327edb0
r9     0x0
r10    0x8
r11    0x246
r12    0x6
r13    0x16
r14    0x7f911f09541b
r15    0x0
rip    0x7f911eea49fc
rflags 0x246
cs     0x33
fs     0x0
gs     0x0

By removing one c.GetRules(), the program works fine again.

Can you point out where I did things wrong? I assume that compiler.GetRules() should only be called once; but how can I achieve my goal? Thanks in advance.

[hillu]

The yr_compiler object which is wrapped by Compiler can't be re-used after producing a compiled rule set. This is not strictly a bug in go-yara (although we could guard against this).

If you want to update your ruleset at run-time, you'll have to create a fresh Compiler and pass all rule files to it.