Flaky results: Same rule produces different results (Root cause: regexp not getting parsed)

[krishna-birla]

My rules:

func _(m fluent.Matcher) {
	m.Match(`$s != $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t != $s`)
	m.Match(`$s == $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t == $s`)
	m.Match(`$s >= $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t <= $s`)
	m.Match(`$s <= $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t >= $s`)
	m.Match(`$s > $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t < $s`)
	m.Match(`$s < $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t > $s`)
	m.Match(`nil == $s`).Where(!m["s"].Const).Suggest(`$s == nil`)
	m.Match(`nil != $s`).Where(!m["s"].Const).Suggest(`$s != nil`)
	m.Match(`log.Infof($s)`).Where(m["s"].Const && m["s"].Type.Is(`string`)).Suggest(`log.Info($s)`)
	m.Match(`log.Warnf($s)`).Where(m["s"].Const && m["s"].Type.Is(`string`)).Suggest(`log.Warn($s)`)
	m.Match(`log.Errorf($s)`).Where(m["s"].Const && m["s"].Type.Is(`string`)).Suggest(`log.Error($s)`)
}

Results:

[quasilyte]

I'm trying to find a minimal repro. It's not easily reproduced in isolation.

Could you explain what do you mean by "regexp not getting parsed"?

Flaky usually means that you get different results from time to time. Does this issue happen only once in a while?

Based on the screenshot provided, I'm seeing that $s is inserted instead of the matched string.

[krishna-birla]

Could you explain what do you mean by "regexp not getting parsed"?

Basically, $s is not getting replaced back with original string.

Flaky usually means that you get different results from time to time. Does this issue happen only once in a while?

Yes, sometimes it gets parsed properly but then other lines show error.

Main question is, why is $s coming up?

[quasilyte]

In order to understand why it happens, I need to reproduce it. I'll try finding the minimal repro.

[krishna-birla]

Thanks! Please tell me if any more info/help required from me regarding this.

[quasilyte]

1. Is there anything special about the files your running rules on?
   For example: can they be compiled by the Go command or they miss some dependencies?

2. Can you still reproduce this issue if there is only 1 pattern? e.g.:

m.Match(`log.Infof($s)`).Where(m["s"].Const && m["s"].Type.Is(`string`)).Suggest(`log.Info($s)`)

[krishna-birla]

1. Nothing special about my go projects, it is a go module which works perfectly fine with go vet/fmt/build/install.

2. Tried this rules.go:

package gorules

import "github.com/quasilyte/go-ruleguard/dsl/fluent"

func _(m fluent.Matcher) {
	//m.Match(`$s != $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t != $s`)
	//m.Match(`$s == $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t == $s`)
	//m.Match(`$s >= $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t <= $s`)
	//m.Match(`$s <= $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t >= $s`)
	//m.Match(`$s > $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t < $s`)
	//m.Match(`$s < $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t > $s`)
	//m.Match(`nil == $s`).Where(!m["s"].Const).Suggest(`$s == nil`)
	//m.Match(`nil != $s`).Where(!m["s"].Const).Suggest(`$s != nil`)
	m.Match(`log.Infof($s)`).Where(m["s"].Const && m["s"].Type.Is(`string`)).Suggest(`log.Info($s)`)
	//m.Match(`log.Warnf($s)`).Where(m["s"].Const && m["s"].Type.Is(`string`)).Suggest(`log.Warn($s)`)
	//m.Match(`log.Errorf($s)`).Where(m["s"].Const && m["s"].Type.Is(`string`)).Suggest(`log.Error($s)`)
}

Results still same:

[quasilyte]

Thank you! This is helpful indeed.
It will ease the issue pinpointing significantly.

[krishna-birla]

Seeing this issue in other cases also. Example,

My rules:

func _(m fluent.Matcher) {
	m.Match(`$s != $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t != $s`)
	m.Match(`$s == $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t == $s`)
	m.Match(`$s >= $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t <= $s`)
	m.Match(`$s <= $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t >= $s`)
	m.Match(`$s > $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t < $s`)
	m.Match(`$s < $t`).Where(m["s"].Const && !m["t"].Const).Suggest(`$t > $s`)
	m.Match(`nil == $s`).Where(!m["s"].Const).Suggest(`$s == nil`)
	m.Match(`nil != $s`).Where(!m["s"].Const).Suggest(`$s != nil`)
	//m.Match(`log.Infof($s)`).Where(m["s"].Const && m["s"].Type.Is(`string`)).Suggest(`log.Info($s)`)
	//m.Match(`log.Warnf($s)`).Where(m["s"].Const && m["s"].Type.Is(`string`)).Suggest(`log.Warn($s)`)
	//m.Match(`log.Errorf($s)`).Where(m["s"].Const && m["s"].Type.Is(`string`)).Suggest(`log.Error($s)`)
}

Results:

[krishna-birla]

Please feel free to modify the issue title if you think the root cause is something else.

Although, I wonder why someone else did not report this error. This seems to be a big blocker. Is it because I am using [email protected] ?

[krishna-birla]

Another thing is error happens on the exact same lines, so that is deterministic. Even if I edit the file, it happens in that line of code again.

[krishna-birla]

Yet another speculation is it happens with same string (in this case). For example, in current run I faced error in two files:

First file with: clusterContext.PostInstallData.CoreDNSUpdateFunction
Second file with: len(recursiveCreateMessage.RecursiveCreateResults)

This specific string got replaced by $s in whole file. Hope this helps.

[quasilyte]

Sorry for not fixing this for quite a long time.

I'm having busy weeks on my work. :(
Don't be afraid though. It's on my radar.

Thank you for your collaboration and patience. ❤️

[krishna-birla]

No issues. <3

[quasilyte]

I see what's the problem there.

Before -fix was introduced, all warnings were printed to the stdout and in order to keep things fit a single line nicely, there was a limit of what will be interpolated (basically, everything longer than 40 chars was not replaced).

I believe it still makes sense to keep messages somewhat limited (you can match a huge statement with a pattern), but do a full replacement for the replacement suggestions that are used in -fix mode.

I'm planning this fix:

1. For the command-line output, increase the width limit to 60 (instead of 40).
2. When applying -fix suggestions, ignore the width limit.

[krishna-birla]

Wow, great catch! Thanks a lot!! @quasilyte

[krishna-birla]

@quasilyte Will this fix be available in v0.1.2, or just in master?

[quasilyte]

I released v0.1.3 which includes this fix.

[krishna-birla]

Thanks!