-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CI: revamp rootfs artifacts #4858
base: main
Are you sure you want to change the base?
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #4858 +/- ##
=======================================
Coverage 84.07% 84.07%
=======================================
Files 251 251
Lines 28052 28052
=======================================
Hits 23586 23586
Misses 4466 4466
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
b392ac5
to
39f7388
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm, just a few minor comments.
It would be nice to run shellcheck
on all these bash files to catch some common issues (quoting paths, unused vars, etc).
"sudo chown -Rc $USER: snapshot_artifacts", | ||
"mv -v snapshot_artifacts/* snapshots/{instance}_{kv}", | ||
"./tools/devtool -y test --no-build -- -m nonci -n4 integration_tests/functional/test_snapshot_phase1.py", | ||
# punch holes in mem snapshot tiles and tar them so they are preserved in S3 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
curiosity: doesn't firecracker support writing sparse memory snapshots? I see everybody has implemented their own way of punching holes >.<
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we don't do it, but it could be a nice improvement and we probably wouldn't have to gate it under an option.
|
||
kernel_dbg_dir = args.kernel.parent / "debug" | ||
kernel_dbg = kernel_dbg_dir / args.kernel.name | ||
print(f"uvm2 with kernel {kernel_dbg} ...") | ||
uvm2 = vmfcty.build(kernel_dbg, args.rootfs) | ||
uvm2.spawn() | ||
uvm2.add_net_iface() | ||
uvm2.basic_config(vcpu_count=args.vcpus, mem_size_mib=args.guest_mem_size // 2**20) | ||
uvm2.start() | ||
# trace-cmd needs this (DNS resolution?) | ||
uvm2.help.enable_ip_forwarding() | ||
files = uvm2.help.trace_cmd_guest(["-l", "read_msr"], cmd="sleep 5") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't really follow this change. what is this tool used for? do we need 2 vms?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sandbox
is a tool to quickly get a microvm and an IPython shell. It is great for quick checks and to bootstrap new tests.
Just type ./tools/devtool sandbox
in a working copy.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And the 2nd VM just uses a debug kernel so we can try it out.
@@ -0,0 +1,6 @@ | |||
CONFIG_IKCONFIG=y |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if for some reason we need to disable a config option, would it work with this concat approach?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, both these styles work, and are normalized to the is not set
style:
# CONFIG_SQUASHFS_ZSTD is not set
CONFIG_SQUASHFS_ZSTD=n
docs/getting-started.md
Outdated
|
||
# Set user read permission on the ssh key | ||
chmod 400 ./ubuntu-24.04.id_rsa | ||
wget -O ubuntu-24.04.squashfs.orig "https://s3.amazonaws.com/spec.ccfc.min/firecracker-ci/v1.10/${ARCH}/ubuntu-24.04.squashfs" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: could we choose another name than an *.orig? Maybe it's just me, but I tend to use this suffix when creating custom files and backing up the originals.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about upstream
?
39f7388
to
b4c7ed5
Compare
This avoids the need to store and download the image from S3. Signed-off-by: Pablo Barbáchano <[email protected]>
Enabling ftrace in our kernels changed the performance of several tests, so it was reverted. Make a new set of kernels that will not be used for performance tests. While doing this, simplify our guest kernel config customization that relied on patches and use file concatenation instead. Turns out `make olddefconfig` produces the same result and we avoid the complexity of dealing with patches. Signed-off-by: Pablo Barbáchano <[email protected]>
Compress squashfs with zstd since that now we have CONFIG_SQUASHFS_ZSTD=y in all our guest kernels. In my tests it is 78MB vs 85MB (an 8.2% reduction) Signed-off-by: Pablo Barbáchano <[email protected]>
socat v1.8.0 in Ubuntu 24.04 has a bug when using `UDP-LISTEN` without specifying the address family. It looks like: E xioopen_ipdgram_listen(): unknown address family 0 We can work-around it by specifying IPv4. See http://www.dest-unreach.org/socat/CHANGES v1.8.0.1 Signed-off-by: Pablo Barbáchano <[email protected]>
Update guest rootfs to Ubuntu 24.04 Signed-off-by: Pablo Barbáchano <[email protected]>
Generate SSH key after downloading artifacts, and add it to the rootfs. This avoids having an SSH key hardcoded in the rootfs. Downside is that we have to rebuild the rootfs, but that is fast. Signed-off-by: Pablo Barbáchano <[email protected]>
This returns a Popen object instead of waiting for the command to finish. It may be useful when we need to incrementally read the output of a long running process in the guest, without having to use screen. Signed-off-by: Pablo Barbáchano <[email protected]>
So that we don't have to install it in the future. Signed-off-by: Pablo Barbáchano <[email protected]>
For now it's a very simple one, but we can use it as a base to provide more complicated ones in the future. Signed-off-by: Pablo Barbáchano <[email protected]>
Signed-off-by: Pablo Barbáchano <[email protected]>
It's more hassle to keep this as a separate tool than including it in the tests, and we avoid having to treat it specially. Also this way we can run it in parallel easily. Signed-off-by: Pablo Barbáchano <[email protected]>
This is so we use less IO overall sending snapshot to/from S3. - Punch holes in the memory snapshots - Decrease guest memory from 1GB to 512MB as it's not important to the test. This decreases around 10x: Before: 27GB * 22 runs ~ 594GB After: 2.7GB * 22 runs ~ 59.4GB Signed-off-by: Pablo Barbáchano <[email protected]>
Also fix for when the patch kernel version is <100. Signed-off-by: Pablo Barbáchano <[email protected]>
It is unlikely that we will ever support more than one rootfs. Signed-off-by: Pablo Barbáchano <[email protected]>
Changes
Reason
Not hardcoding the key in the rootfs decreases the changes the key is reused in some other context.
License Acceptance
By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md
.PR Checklist
PR.
CHANGELOG.md
.TODO
s link to an issue.contribution quality standards.
rust-vmm
.