Put Temp on RAM Disk for Azure Pipelines tests

[chrahunt]

Profiling on Azure Pipelines indicates that the majority of our time is
spent waiting for filesystem operations to complete. As a quick way to
improve our test speed in this area, we can do operations on a RAM disk
instead of the default SSDs.

Related to #4497, #6791.

[chrahunt]

Nice, this saves about 10 minutes for integration tests, 30 seconds for unit tests, and only takes 1 minute extra for setup. It may be worth putting the tox environment in the same filesystem.

Compare:

1. this build
2. previous CI build 1
3. previous CI build 2

[xavfernandez]

Nice speedup !
It would be even better if it was built-in in the pipelines ^^
You might want to create an issue in https://github.com/Microsoft/azure-pipelines-image-generation/issues (not sure it is the right place) to ask for an easier integration ?

[chrahunt]

Not sure what could be improved there, maybe FS-iSCSITarget-Server could be installed by default?

I can clean up our config at least by moving it to a separate script and then invoke it from the yml file.

[xavfernandez]

maybe FS-iSCSITarget-Server could be installed by default?

Ideally a RAM disk would be available by default ? 😛

[xavfernandez]

cc @brcrista (as author of #5785) since he might have more familiarity on what could possibly be suggested/improved :)

[brcrista]

this saves about 10 minutes for integration tests, 30 seconds for unit tests, and only takes 1 minute extra for setup

Well done!

From looking at the logs, it looks like about 30 seconds of that setup is spent on Install-WindowsFeature -Name FS-iSCSITarget-Server.

cc @vtbassmatt -- something to consider pre-installing on the Windows image, depending on how much storage it eats up.

[pradyunsg]

Hurray!

[chrahunt]

Thanks for reviewing @pradyunsg and @xavfernandez, and @brcrista for initiating the agent image update request.

I tested and Install-WindowsFeature will succeed regardless of whether the feature is already installed, so I'll merge this and we can remove the install step if it gets added by default.

[pradyunsg]

@chrahunt if you haven't already, it might be worth investigating if this is a good idea on Appveyor as well, since that's our bottleneck CI service currently.

[chrahunt]

We could also remove it entirely. Given 1 PR is submitted, currently we would have 4 idle runners on Azure Pipelines which could handle the 3 builds we do on Appveyor in parallel.

[pradyunsg]

I followed up on Azure Pipelines + GitHub Actions installing FS-iSCSITarget-Server in their images at actions/runner-images#708.

[pradyunsg]

They've added this to their VM images now.

[chrahunt]

👍