Swap windows jobs to docker (as much as possible)

[scotthain]

Signed-off-by: Scott Hain [email protected]

There are a few jobs that still fail.

[chef-expeditor]

Hello scotthain! Thanks for the pull request!

Here is what will happen next:

1. Your PR will be reviewed by the maintainers.
2. If everything looks good, one of them will approve it, and your PR will be merged.

Thank you for contributing!

[mwrock]

this looks good to me. just have 1 thing that needs to change so this can still pass locally

[mwrock]

I've gone and made a few changes to allow tests to succeed in a windows container environment. Here is a summary:

• Remove use of canonicalize in fs test code. Its a known issue that this method is not friendly to certain Windows file systems including docker mounts.
• Interactive promote test should not try to use cat as a UI editor. I do not know how this could have ever worked and fails for me locally outside of docker.
• Use copy and remove_file instead of rename when generating the protocols. Rename will not work over a mount.
• set the default rustup host to x86_64-pc-windows-msvc. It was x86_64-pc-windows-gnu which was breaking some butterfly test compilations that was trying to find gcc.exe.
• Make sure the studio tests have needed prereqs (habitat and nuget provider)

[scotthain]

This looks great @mwrock - I think you need to reapprove it and then I can merge it if you're ok with it. Thanks for picking up the ball on this!!

[baumanj]

This function modifies global state (via env::set_var), but will be run by multiple tests concurrently. We should use something like locked_env_var to make this thread-safe or restructure the tests with dependency injection to not depend on global state. I think the latter would be preferable since it doesn't limit the concurrency potential for the tests.

[mwrock]

locked_env_var is defined in common so we would have to move it to core and adjust all callers which feels like it belongs in a separate PR.

[baumanj]

Yeah, it's overdue to be moved into core (probably common/core should be merged entirely), but I think it's worth doing now since these tests are prone to non-deterministic failures without it. A separate PR is good, but it could merge before this one so we can base the fix on it.

How about taking the dependency-injection approach? That's the preferable solution in my view and is appropriate to this PR scope.

[scotthain]

Including that in here feels to me kinda like scope creep for this PR but I'll defer to you both on it!

[baumanj]

I agree that moving locked_env_var deserves its own PR if we go that way. If we use dependency injection instead, we can make those changes in this PR and leave moving locked_env_var for later, but either way I think we should not leave this in a subtly broken state which may crop up some unknown time in the future.

[mwrock]

In this case, I think even if multiple tests concurrently change this, its harmless. worse that will happen id the fixtures will get added to the path multiple times.

[baumanj]

I'm not so sure we can make that strong an assumption about the way env::set_var works. The documentation says "concurrent access to environment variables is safe in Rust", but that doesn't mean that it will do what you expect. I'm not sure we're guaranteed one thread accessing this in the midst of another thread setting it couldn't result in an empty result.

In any case, it's not a huge deal, but it's an error-prone pattern to rely on global state like this, so when we can switch to something more robust (such as injecting global dependencies as arguments) without a huge hassle, I think we'd be well served to do so.

[mwrock]

Looking at this I still think it is out of scope. These tests while not ideal has been in place for 3 years without raising stability issues. I'd just rather not go into refactoring non-test code unrelated to the intent of this PR and get it merged.

[baumanj]

In test code, I'm equally or more worried about false positives, but you're right that we can probably let this one go. The value of getting this merged is quite high.

[scotthain]

I'm going to defer to @mwrock to answer questions on the rust changes.