Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CFE-4244: Fix RlistEqual comparison on lists of different lengths #5313

Merged
merged 1 commit into from
Sep 13, 2023

Conversation

amousset
Copy link
Contributor

@amousset amousset commented Aug 31, 2023

The original issue that leads to this is the fact that the caches for execresult and execresult_as_data were mixed (CFE-4244). This is caused by two separate issues:

  • The function cache uses the args list as key and discards the function itself. This means different functions with the same args are considered identical, and cache is reused.

Rval *rval = FuncCacheMapGet(ctx->function_cache, args);

  • The function args are passed as an Rlist, and the comparison used ignores the additional items of the longest list when comparing two lists of different lengths, leading to treating execresult and execresult_as_data as identical when using the same command and shell args.

This PR only fixes the specific case of execresult, but leaves other function cache issues. For example, host2ip vs. ip2host could be confused:

bundle agent main {
vars:
  "res1" string => host2ip("cfengine.com");
  "res2" string => ip2host("cfengine.com");
  "res3" string => ip2host("northerntech.atlassian.net");

reports:
  "res1: ${res1}";
  "res2: ${res2}";
  "res3: ${res3}";
}

still returns:

R: res1: 34.107.174.45
R: res2: 34.107.174.45
R: res3: northerntech.atlassian.net

I'll try to add the function name to the Rlist used as cache key in a subsequent PR.

Note: Even if the function name "Equal" suggests lists of different lengths should be different, it might break other stuff using Rlists, I haven't tried to investigate other use cases.

@nickanderson nickanderson changed the title Fix RlistEqual comparison on lists of different lengths CFE-4244: Fix RlistEqual comparison on lists of different lengths Aug 31, 2023
Copy link
Contributor

@larsewi larsewi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice 🚀 I have only a few nitpicks.

Thank you for providing both an acceptance- and unit tests. I see that your acceptance test fails on Windows, and I'm not quite sure why. I will request a build from @cf-bottom in Jenkins, so that I can investigate this further. Feel free to address the suggested changes in the meanwhile.

libpromises/rlist.c Outdated Show resolved Hide resolved
tests/unit/rlist_test.c Outdated Show resolved Hide resolved
@amousset
Copy link
Contributor Author

amousset commented Sep 1, 2023

I see that your acceptance test fails on Windows, and I'm not quite sure why.

I you think it would be acceptable I could skip this test on Windows and macOS.

@larsewi
Copy link
Contributor

larsewi commented Sep 6, 2023

If you think it would be acceptable I could skip this test on Windows and macOS.

Let's not do that. BTW, the macOS failure is due to an unrelated known issue, so you can ignore it for now.

Started a build Build Status. I will investigate and then come back to you :)

@larsewi
Copy link
Contributor

larsewi commented Sep 7, 2023

Hi @amousset, I just tested this manually on Windows, and you can fix it my using "powershell" instead of "useshell" as the second argument of execresult() and execresult_as_data() on Windows. You can achieve this by for example using the windows:: classguard.

@larsewi
Copy link
Contributor

larsewi commented Sep 7, 2023

Awesome, thanks 🚀 Let's run this through CI again, just to make sure everything is good Build Status

@amousset
Copy link
Contributor Author

amousset commented Sep 7, 2023

Second part addressed in #5317

@larsewi
Copy link
Contributor

larsewi commented Sep 8, 2023

Could you please squash the commits as described in https://github.com/cfengine/core/blob/master/CONTRIBUTING.md#interactive-rebase?

@larsewi
Copy link
Contributor

larsewi commented Sep 12, 2023

Still failing on Windows 🤔 I will create a new build, investigate further, and then get back to you. Build Status

The original issue that leads to this is the fact that the cache
for execresult and execresult_as_data were mixed. This is caused by two
separate issues:

* The function cache uses the args list as key and discards the function
  itself. This means different function with the same args are considered
  identical, and cache is reused.
* The args are passed as an Rlist, and the used Rlist comparison ignores the
  additional items of the longest list when comparing two lists of
  different lengths, leading to treating execresult and execresult_as_data
  as identical when using the same command and shell args.

This PR only fixes the specific case of execresult, but leaves other
function cache issues (e.g. host2ip vs. ip2host could be confused).
@larsewi
Copy link
Contributor

larsewi commented Sep 12, 2023

Build Status 🤞

Copy link
Contributor

@larsewi larsewi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution 🚀

@larsewi larsewi merged commit d1df034 into cfengine:master Sep 13, 2023
12 of 13 checks passed
@larsewi
Copy link
Contributor

larsewi commented Sep 13, 2023

Seems that the GitHub Action workflow for windows acceptance tests, tests with master nightly build, instead of testing it with the changes you did to core. Thus I merged this, and will see if it passes when we have a new master nightly build.

@larsewi larsewi added the cherry-pick? Fixes which may need to be cherry-picked to LTS branches label Sep 13, 2023
@larsewi larsewi removed the cherry-pick? Fixes which may need to be cherry-picked to LTS branches label Sep 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants