feat: provisioning script #335

Callisto13 · 2021-12-21T16:03:18Z

provision.sh tool

Drawn a lot from firecracker's one of these, but not as much as i expected as their bits made my shellcheck yell at me a lot

There are docs included in this PR, but here is the help:

root@callisto:~# ./provision.sh -h
usage: ./provision.sh <COMMAND> <OPTIONS>

Script to provision hosts for running flintlock microvms

COMMANDS:

  all                    Complete setup for production ready host. Component versions
                         can be configured by setting the FLINTLOCK, CONTAINERD and FIRECRACKER
                         environment variables.
    OPTIONS:
      -y                 Autoapprove all prompts (danger)
      --skip-apt, -s     Skip installation of apt packages
      --thinpool, -t     Name of thinpool to create (default: flintlock or flintlock-dev)
      --disk, -d         Name blank unpartioned disk to use for direct lvm thinpool (ignored if --dev set)
      --dev              Set up development environment. Loop thinpools will be created.

  apt                    Install all apt packages required by flintlock

  firecracker            Install firecracker from feature branch
    OPTIONS:
      --version, -v      Version to install (default: latest)

  containerd             Install, configure and start containerd service
    OPTIONS:
      --version, -v      Version to install (default: latest)
      --thinpool, -t     Name of thinpool to include in config toml (default: flintlock-thinpool)
      --dev              Set up development environment. Containerd will keep state under 'dev' tagged paths.

  flintlock              Install and start flintlockd service
    OPTIONS:
      --version, -v      Version to install (default: latest)
      --dev              Assumes containerd has been provisioned in a dev environment

  direct_lvm             Set up direct_lvm thinpool
    OPTIONS:
      -y                 Autoapprove all prompts (danger)
      --thinpool, -t     Name of thinpool to create (default: flintlock)
      --disk, -d         Name blank unpartioned disk to use for direct lvm thinpool
      --skip-apt, -s     Skip installation of apt packages

  devpool                Set up loop device thinpool (development environments)
    OPTIONS:
      --thinpool, -t     Name of thinpool to create (default: flintlock-dev)

And an action shot:

codecov-commenter · 2021-12-21T16:29:55Z

Codecov Report

Merging #335 (f45c3ff) into main (aeec4d9) will decrease coverage by 0.15%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #335      +/-   ##
==========================================
- Coverage   58.29%   58.13%   -0.16%     
==========================================
  Files          51       51              
  Lines        2503     2503              
==========================================
- Hits         1459     1455       -4     
- Misses        929      932       +3     
- Partials      115      116       +1

Impacted Files	Coverage Δ
pkg/queue/queue.go	`93.93% <0.00%> (-6.07%)`	⬇️
infrastructure/containerd/event_service.go	`56.14% <0.00%> (-3.51%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aeec4d9...f45c3ff. Read the comment docs.

yitsushi

I'm about half-way (at ## CONTAINERD) and have a few comments already, so i'll just submit them now and continue with the rest.

hack/scripts/provision.sh

yitsushi · 2021-12-22T10:41:59Z

hack/scripts/provision.sh

+	local repo_name="$1"
+	local tag="$2"
+	local bin="$3"
+	echo "https://github.com/$repo_name/releases/download/$tag/$bin"


I really love all this variables names to document what are those arguments and
not just https://github.com/${1}/.....

yitsushi · 2021-12-22T10:45:32Z

hack/scripts/provision.sh

+	local url=$(curl -sL "$latest_url" | awk -F'"' '/tag_name/ {printf $4}')
+	echo "$url"


Why not:

Suggested change

local url=$(curl -sL "$latest_url" | awk -F'"' '/tag_name/ {printf $4}')

echo "$url"

curl -sL "$latest_url" | awk -F'"' '/tag_name/ {printf $4}'

i wanted to make the return more explicit, but yes it is unnecessary

yitsushi · 2021-12-22T10:46:03Z

hack/scripts/provision.sh

+	tag=$(git ls-remote --tags --sort="v:refname" "https://github.com/$repo_name" |
+		tail -n 1 | sed 's/.*\///; s/\^{}//')
+	echo "$tag"


same as latest_release_tag.

hack/scripts/provision.sh

yitsushi · 2021-12-22T11:23:08Z

hack/scripts/provision.sh

+		lvcreate -q --wipesignatures y -n thinpool "$volume_group" -l 95%VG || die "Failed to create logical volume for thinpool data"
+		say "Created logical volume for $volume_group thinpool data"
+
+		lvcreate -q --wipesignatures y -n thinpoolmeta "$thinpool" -l 1%VG || die "Failed to create logical volume for thinpool metadata"


Hmm.. That 95% and 1% combo seems strange.

where is the remaining 4%

the METADATA_SPARSE_SIZE="10G" DATA_SPARSE_SIZE="100G" pair has a 1:10 ratio.

It's even more fishy, we have fix size for spare files, but we use dynamic volume size with lvm.

If the disk full size is less than $((DATA_SPARSE_SIZE + METADATA_SPARSE_SIZE)) we should fail before we even try to erase it.

nvm, i see DATA_SPARSE_SIZE and METADATA_SPARSE_SIZE are used for dev only without lvm.

(leave this thread here to record my confusion 😆 )

i copied all the devpool stuff from our existing script, did not actually read any of it 😁

hack/scripts/provision.sh

Callisto13 · 2021-12-22T13:12:35Z

😬 sorry @yitsushi I had a mistake in the docs

Callisto13 added the kind/feature New feature or request label Dec 21, 2021

Callisto13 force-pushed the provision branch from 925bb8e to 9d3cb0a Compare December 21, 2021 16:29

Callisto13 marked this pull request as ready for review December 21, 2021 16:36

Callisto13 force-pushed the provision branch 2 times, most recently from 3eede98 to e4e203c Compare December 22, 2021 10:50

yitsushi reviewed Dec 22, 2021

View reviewed changes

hack/scripts/provision.sh Outdated Show resolved Hide resolved

yitsushi reviewed Dec 22, 2021

View reviewed changes

yitsushi previously approved these changes Dec 22, 2021

View reviewed changes

feat: provisioning script

141814d

Callisto13 dismissed yitsushi’s stale review via 141814d December 22, 2021 13:12

Callisto13 force-pushed the provision branch from f45c3ff to 141814d Compare December 22, 2021 13:12

yitsushi approved these changes Dec 22, 2021

View reviewed changes

Callisto13 merged commit 898d1d8 into liquidmetal-dev:main Dec 22, 2021

Callisto13 deleted the provision branch December 22, 2021 13:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: provisioning script #335

feat: provisioning script #335

Callisto13 commented Dec 21, 2021 •

edited

Loading

codecov-commenter commented Dec 21, 2021 •

edited

Loading

yitsushi left a comment

yitsushi Dec 22, 2021

yitsushi Dec 22, 2021

Callisto13 Dec 22, 2021

yitsushi Dec 22, 2021

yitsushi Dec 22, 2021

yitsushi Dec 22, 2021

yitsushi Dec 22, 2021

Callisto13 Dec 22, 2021

Callisto13 commented Dec 22, 2021

		local url=$(curl -sL "$latest_url" \| awk -F'"' '/tag_name/ {printf $4}')
		echo "$url"

	local url=$(curl -sL "$latest_url" \| awk -F'"' '/tag_name/ {printf $4}')
	echo "$url"
	curl -sL "$latest_url" \| awk -F'"' '/tag_name/ {printf $4}'

feat: provisioning script #335

feat: provisioning script #335

Conversation

Callisto13 commented Dec 21, 2021 • edited Loading

codecov-commenter commented Dec 21, 2021 • edited Loading

Codecov Report

yitsushi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Callisto13 commented Dec 22, 2021

Callisto13 commented Dec 21, 2021 •

edited

Loading

codecov-commenter commented Dec 21, 2021 •

edited

Loading