Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

roachtest: AWS provider sets up local SSD when not requested #106058

Closed
erikgrinaker opened this issue Jul 3, 2023 · 2 comments
Closed

roachtest: AWS provider sets up local SSD when not requested #106058

erikgrinaker opened this issue Jul 3, 2023 · 2 comments
Labels
A-testing Testing tools and infrastructure C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-testeng TestEng Team

Comments

@erikgrinaker
Copy link
Contributor

erikgrinaker commented Jul 3, 2023

In disk-stalled/dmsetup and other tests (see e.g. #106013), we see that a test with UseLocalSSD: false still gets a VM with a local SSD on AWS, and this local SSD is RAID0'ed together with the EBS volume that the test did request. This is related to #98783.

When UseLocalSSD: false, the VM should not use a local SSD at all.

To reproduce:

$ roachtest run --cloud aws --debug-always disk-stalled/dmsetup
[...]
(cluster.go:2247).Run: output in run_094944.124037951_n1-10_echo-0-sudo-blockdev: echo "0 $(sudo blockdev --getsz /dev/nvme1n1) linear /dev/nvme1n1 0" | sudo dmsetup create data1 returned: COMMAND_PROBLEM: exit status 1
[...]
device-mapper: reload ioctl on data1  failed: Device or resource busy

Relevant log entries (from 1.journalctl.txt), note the c6id.xlarge machine type:

Jul 03 11:02:20 ubuntu kernel: DMI: Amazon EC2 c6id.xlarge/, BIOS 1.0 10/16/2017
Jul 03 11:02:43 ip-10-12-4-210 cloud-init[1375]: Disk /dev/nvme1n1 not mounted, need to mount...
Jul 03 11:02:43 ip-10-12-4-210 cloud-init[1375]: Disk /dev/nvme2n1 not mounted, need to mount...
Jul 03 11:02:43 ip-10-12-4-210 cloud-init[1375]: 2 disks mounted, creating /mnt/data1 using RAID 0
Jul 03 11:02:43 ip-10-12-4-210 cloud-init[1375]: + mdadm --create /dev/md0 --level=0 --raid-devices=2 /dev/nvme1n1 /dev/nvme2n1
Jul 03 11:02:46 ip-10-12-4-210 cloud-init[1375]: + lsblk
Jul 03 11:02:46 ip-10-12-4-210 cloud-init[1375]: NAME         MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
Jul 03 11:02:46 ip-10-12-4-210 cloud-init[1375]: nvme2n1      259:0    0 220.7G  0 disk
Jul 03 11:02:46 ip-10-12-4-210 cloud-init[1375]: └─md0          9:0    0 720.5G  0 raid0 /mnt/data1
Jul 03 11:02:46 ip-10-12-4-210 cloud-init[1375]: nvme0n1      259:1    0    10G  0 disk
Jul 03 11:02:46 ip-10-12-4-210 cloud-init[1375]: ├─nvme0n1p1  259:3    0   9.9G  0 part  /
Jul 03 11:02:46 ip-10-12-4-210 cloud-init[1375]: ├─nvme0n1p14 259:4    0     4M  0 part
Jul 03 11:02:46 ip-10-12-4-210 cloud-init[1375]: └─nvme0n1p15 259:5    0   106M  0 part  /boot/efi
Jul 03 11:02:46 ip-10-12-4-210 cloud-init[1375]: nvme1n1      259:2    0   500G  0 disk
Jul 03 11:02:46 ip-10-12-4-210 cloud-init[1375]: └─md0          9:0    0 720.5G  0 raid0 /mnt/data1

Jira issue: CRDB-29374

@erikgrinaker erikgrinaker added C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. A-testing Testing tools and infrastructure T-testeng TestEng Team labels Jul 3, 2023
@blathers-crl
Copy link

blathers-crl bot commented Jul 3, 2023

cc @cockroachdb/test-eng

@srosenberg
Copy link
Member

Resolved by #119906

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testing Testing tools and infrastructure C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-testeng TestEng Team
Projects
None yet
Development

No branches or pull requests

2 participants