roachprod: RAID0 only network disks if both local and network are present

[nameisbhaskar]

Today, if a machine has both local and network disks, both the disks are selected for RAID'ing.

But, RAID'ing different types of disks causes performance differences.

To address this, local disks are ignored for RAID'ing only if network disks are present.

Fixes: #98783
Epic: none

[cockroach-teamcity]

This change is 

[srosenberg]

We should resolve this TODO [1] and do a full (test) run in both GCE and AWS. Have you thought how to test this definitively? Note that the startup script is captured by the journalctl log (see FetchJournalctl). However, we don't save these unless a test failed (see collectArtifacts). Maybe we change that behavior, at least for the test runs? This way we can post-analyze all the startup scripts to ensure there were no "funny" RAID`ing. What do you think?

[1]

cockroach/pkg/cmd/roachtest/tests/restore.go

Lines 537 to 544 in 41da9ec

	// Work around an issue that RAID0s local NVMe and GP3 storage together:
	// https://github.com/cockroachdb/cockroach/issues/98783.
	//
	// TODO(srosenberg): Remove this workaround when 98783 is addressed.
	// TODO(miral): This now returns an error instead of panicking, so even though
	// we haven't panicked here before, we should handle the error. Moot if this is
	// removed as per TODO above.
	s.AWS.MachineType, _, _ = spec.SelectAWSMachineType(s.CPUs, s.Mem, false /* shouldSupportLocalSSD */, vm.ArchAMD64)

[nameisbhaskar]

We should resolve this TODO [1] and do a full (test) run in both GCE and AWS. Have you thought how to test this definitively? Note that the startup script is captured by the journalctl log (see FetchJournalctl). However, we don't save these unless a test failed (see collectArtifacts). Maybe we change that behavior, at least for the test runs? This way we can post-analyze all the startup scripts to ensure there were no "funny" RAID`ing. What do you think?

[1]

cockroach/pkg/cmd/roachtest/tests/restore.go

Lines 537 to 544 in 41da9ec

	// Work around an issue that RAID0s local NVMe and GP3 storage together:
	// https://github.com/cockroachdb/cockroach/issues/98783.
	//
	// TODO(srosenberg): Remove this workaround when 98783 is addressed.
	// TODO(miral): This now returns an error instead of panicking, so even though
	// we haven't panicked here before, we should handle the error. Moot if this is
	// removed as per TODO above.
	s.AWS.MachineType, _, _ = spec.SelectAWSMachineType(s.CPUs, s.Mem, false /* shouldSupportLocalSSD */, vm.ArchAMD64)

Regarding testing, I have tested roachprod deployment with different disk configuration. Here are the scenarios and the expected behaviour as per my understanding:

1. (local SSD = 0, Network Disk - 1) - no RAID'ing
2. (local SSD = 1, Network Disk - 0) - no RAID'ing
3. (local SSD >= 1, Network Disk = 1) - no RAID'ing
4. (local SSD > 1, Network Disk = 0) - local SSDs selected for RAID'ing
5. (local SSD >= 0, Network Disk > 1) - network disks selected for RAID'ing
   Please let me know if this is the correct assumption. Also, if there are a combination of SSDs and 1 network disk, all the disks will be mounted. Is that also the expected behaviour?

[nameisbhaskar]

made some changes to only use network disk if present:

(local SSD = 0, Network Disk - 1) - no RAID'ing and mount network disk

(local SSD = 1, Network Disk - 0) - no RAID'ing and mount local SSD

(local SSD >= 1, Network Disk = 1) - no RAID'ing and mount network disk

(local SSD > 1, Network Disk = 0) - local SSDs selected for RAID'ing

(local SSD >= 0, Network Disk > 1) - network disks selected for RAID'ing

[srosenberg]

Looks good, thanks!

[nameisbhaskar]

Thanks for reviewing and approving the change @srosenberg !

[nameisbhaskar]

bors r=srosenberg

[craig]

Build succeeded:

• Bazel Essential CI (Cockroach)
• macos_amd64_build
• macos_arm64_build
• windows_build

[blathers-crl]

Encountered an error creating backports. Some common things that can go wrong:

1. The backport branch might have already existed.
2. There was a merge conflict.
3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.

---

error creating merge commit from 6fd7ff8 to blathers/backport-release-23.1-119906: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 23.1.x failed. See errors above.

---

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

[renatolabs]

Nice work!

Sorry if I missed something, but should we do something for Azure as well?

[nameisbhaskar]

Nice work!

Sorry if I missed something, but should we do something for Azure as well?

Azure does not have the RAID'ing implemented.