CI: revamp rootfs artifacts #4858
Open
+384
−684
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pb8o
added
Priority: Low
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #4858 +/- ##
=======================================
Coverage 84.07% 84.07%
=======================================
Files 251 251
Lines 28052 28052
=======================================
Hits 23586 23586
Misses 4466 4466
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
pb8o
force-pushed
the
ci-no-ssh-keys
branch
8 times, most recently
from
b392ac5
to
39f7388
Compare
pb8o
changed the title
Manciukic
reviewed
Oct 22, 2024
| "sudo chown -Rc $USER: snapshot_artifacts", | ||
| "mv -v snapshot_artifacts/* snapshots/{instance}_{kv}", | ||
| "./tools/devtool -y test --no-build -- -m nonci -n4 integration_tests/functional/test_snapshot_phase1.py", | ||
| # punch holes in mem snapshot tiles and tar them so they are preserved in S3 |
There was a problem hiding this comment.
curiosity: doesn't firecracker support writing sparse memory snapshots? I see everybody has implemented their own way of punching holes >.<
There was a problem hiding this comment.
I think we don't do it, but it could be a nice improvement and we probably wouldn't have to gate it under an option.
Comment on lines
+89
to
+100
|
|
||
| kernel_dbg_dir = args.kernel.parent / "debug" | ||
| kernel_dbg = kernel_dbg_dir / args.kernel.name | ||
| print(f"uvm2 with kernel {kernel_dbg} ...") | ||
| uvm2 = vmfcty.build(kernel_dbg, args.rootfs) | ||
| uvm2.spawn() | ||
| uvm2.add_net_iface() | ||
| uvm2.basic_config(vcpu_count=args.vcpus, mem_size_mib=args.guest_mem_size // 2**20) | ||
| uvm2.start() | ||
| # trace-cmd needs this (DNS resolution?) | ||
| uvm2.help.enable_ip_forwarding() | ||
| files = uvm2.help.trace_cmd_guest(["-l", "read_msr"], cmd="sleep 5") |
There was a problem hiding this comment.
I didn't really follow this change. what is this tool used for? do we need 2 vms?
There was a problem hiding this comment.
sandbox is a tool to quickly get a microvm and an IPython shell. It is great for quick checks and to bootstrap new tests.
Just type ./tools/devtool sandbox in a working copy.
There was a problem hiding this comment.
And the 2nd VM just uses a debug kernel so we can try it out.
kalyazin
reviewed
Oct 22, 2024
| @@ -0,0 +1,6 @@ | |||
| CONFIG_IKCONFIG=y | |||
There was a problem hiding this comment.
if for some reason we need to disable a config option, would it work with this concat approach?
There was a problem hiding this comment.
Yep, both these styles work, and are normalized to the is not set style:
# CONFIG_SQUASHFS_ZSTD is not set
CONFIG_SQUASHFS_ZSTD=n
docs/getting-started.md
Outdated
|
|
||
| # Set user read permission on the ssh key | ||
| chmod 400 ./ubuntu-24.04.id_rsa | ||
| wget -O ubuntu-24.04.squashfs.orig "https://s3.amazonaws.com/spec.ccfc.min/firecracker-ci/v1.10/${ARCH}/ubuntu-24.04.squashfs" |
There was a problem hiding this comment.
Nit: could we choose another name than an *.orig? Maybe it's just me, but I tend to use this suffix when creating custom files and backing up the originals.
pb8o
force-pushed
the
ci-no-ssh-keys
branch
3 times, most recently
from
39f7388
to
b4c7ed5
Compare
This avoids the need to store and download the image from S3. Signed-off-by: Pablo Barbáchano <[email protected]>
Enabling ftrace in our kernels changed the performance of several tests, so it was reverted. Make a new set of kernels that will not be used for performance tests. While doing this, simplify our guest kernel config customization that relied on patches and use file concatenation instead. Turns out `make olddefconfig` produces the same result and we avoid the complexity of dealing with patches. Signed-off-by: Pablo Barbáchano <[email protected]>
Compress squashfs with zstd since that now we have CONFIG_SQUASHFS_ZSTD=y in all our guest kernels. In my tests it is 78MB vs 85MB (an 8.2% reduction) Signed-off-by: Pablo Barbáchano <[email protected]>
socat v1.8.0 in Ubuntu 24.04 has a bug when using `UDP-LISTEN` without
specifying the address family. It looks like:
E xioopen_ipdgram_listen(): unknown address family 0
We can work-around it by specifying IPv4.
See http://www.dest-unreach.org/socat/CHANGES v1.8.0.1
Signed-off-by: Pablo Barbáchano <[email protected]>
Update guest rootfs to Ubuntu 24.04 Signed-off-by: Pablo Barbáchano <[email protected]>
Generate SSH key after downloading artifacts, and add it to the rootfs. This avoids having an SSH key hardcoded in the rootfs. Downside is that we have to rebuild the rootfs, but that is fast. Signed-off-by: Pablo Barbáchano <[email protected]>
This returns a Popen object instead of waiting for the command to finish. It may be useful when we need to incrementally read the output of a long running process in the guest, without having to use screen. Signed-off-by: Pablo Barbáchano <[email protected]>
So that we don't have to install it in the future. Signed-off-by: Pablo Barbáchano <[email protected]>
For now it's a very simple one, but we can use it as a base to provide more complicated ones in the future. Signed-off-by: Pablo Barbáchano <[email protected]>
Signed-off-by: Pablo Barbáchano <[email protected]>
It's more hassle to keep this as a separate tool than including it in the tests, and we avoid having to treat it specially. Also this way we can run it in parallel easily. Signed-off-by: Pablo Barbáchano <[email protected]>
This is so we use less IO overall sending snapshot to/from S3. - Punch holes in the memory snapshots - Decrease guest memory from 1GB to 512MB as it's not important to the test. This decreases around 10x: Before: 27GB * 22 runs ~ 594GB After: 2.7GB * 22 runs ~ 59.4GB Signed-off-by: Pablo Barbáchano <[email protected]>
Also fix for when the patch kernel version is <100. Signed-off-by: Pablo Barbáchano <[email protected]>
It is unlikely that we will ever support more than one rootfs. Signed-off-by: Pablo Barbáchano <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Reason
Not hardcoding the key in the rootfs decreases the changes the key is reused in some other context.
License Acceptance
By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.PR Checklist
PR.
CHANGELOG.md.TODOs link to an issue.contribution quality standards.
rust-vmm.