Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: v0.9.0 blog post && docs changes #69

Merged
merged 16 commits into from
Dec 6, 2024
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/components/Sidebar.astro
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ const sidebarPostList: StarlightRouteData["sidebar"] = blogPosts
badge: undefined,
href: url,
isCurrent: Astro.url.pathname.includes(url || ""),
label: ` ${p.frontmatter.title} ${new Date(p.frontmatter.date).toISOString().substring(0, 10)}`,
label: ` ${p.frontmatter.title} (${new Date(p.frontmatter.date).toISOString().substring(0, 10)})`,
type: "link",
} as SidebarEntry;
});
Expand Down
136 changes: 68 additions & 68 deletions src/content/docs/blog/caching-update.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,14 @@ authors:
img: https://avatars.githubusercontent.com/u/4626052
github: gmpinder
---

:::tip[TLDR]
A recent BlueBuild CLI update makes builds and updates much faster. [What do I need to do to take advantage?](#migration)
:::

Hello! I'm Jerry and I'm the creator and maintainer of the Rust-based `bluebuild` CLI tool. I started creating this tool as a way to extend the work that [XYNY](https://github.com/xynydev) did with the recipe standard, but make it so that I didn't have to manage the `Containerfile` directly. The tool is an integral part of the project that handles Containerfile generation from the recipe, the building of container images, and pushing them to the registry.

For the past month or so, we've been working on trying to make the building experience better for our users. One of the biggest concerns brought up was the size of updates when using `rpm-ostree upgrade` to pull your new changes. The size of the updates would end up being bigger than the original single `RUN build.sh` instruction that was used. While this was really great for the time, more advanced uses of the legacy `startingpoint` repo required users to know how to manage a `Containerfile`. This can be really intimidating to less-technical users. One of the `bluebuild` CLI's goals is to perform most of the optimizations for the user so they don't have to worry about setting that up.
For the past month or so, we've been working on trying to make the building experience better for our users. One of the biggest concerns brought up was the size of updates when using `rpm-ostree upgrade` to pull your new changes. The size of the updates would end up being bigger than the original single `RUN build.sh` instruction that was used. While this was really great for the time, more advanced uses of the legacy `startingpoint` repo required users to know how to manage a `Containerfile`. This can be really intimidating to less-technical users. One of the `bluebuild` CLI's goals is to perform most of the optimizations for the user so they don't have to worry about setting that up.

## Migration

Expand All @@ -23,11 +24,24 @@ I'm happy to announce that we have now released `v0.8.4` of `bluebuild` CLI (and
- This does not include all the other files/directories in your `./config/` like `./config/scripts/` or `./config/files/`
- Any `.yml`/`.yaml` file that contains information for the modules to build your image are your recipe files.
- Be sure to update any use of `from-file:` to account for any changes in your directory structure
- For example if the recipes were in `./config/recipes/` and extended recipe files in `./config/recipes/image_configs/`.
- One of the recipes calls `from-file: recipes/image_configs/common.yml`.
```
config
└─recipes
- For example if the recipes were in `./config/recipes/` and extended recipe files in `./config/recipes/image_configs/`.
- One of the recipes calls `from-file: recipes/image_configs/common.yml`.
```
config
└─recipes
├── image_configs
│ ├── common.yml
│ ├── gnome.yml
│ └── kde.yml
├── recipe-gnome-nvidia.yml
├── recipe-gnome.yml
├── recipe-kde-nvidia.yml
└── recipe-kde.yml
```
- You could move `recipes` up a level to be:
```
config
recipes
├── image_configs
│ ├── common.yml
│ ├── gnome.yml
Expand All @@ -36,24 +50,11 @@ I'm happy to announce that we have now released `v0.8.4` of `bluebuild` CLI (and
├── recipe-gnome.yml
├── recipe-kde-nvidia.yml
└── recipe-kde.yml
```
- You could move `recipes` up a level to be:
```
config
recipes
├── image_configs
│ ├── common.yml
│ ├── gnome.yml
│ └── kde.yml
├── recipe-gnome-nvidia.yml
├── recipe-gnome.yml
├── recipe-kde-nvidia.yml
└── recipe-kde.yml
```
- Then `from-file:` can have the path set to `image_configs/common.yml`
```
- Then `from-file:` can have the path set to `image_configs/common.yml`
- Be sure to update your workflow file too
- If your `recipe.yml` was located in `./config/` and you moved it to `./recipes/`, the GHA will already take that into account
- The thing to look out for is the recipe's relative location to the "root" of recipe locations
- If your `recipe.yml` was located in `./config/` and you moved it to `./recipes/`, the GHA will already take that into account
- The thing to look out for is the recipe's relative location to the "root" of recipe locations
2. It's also suggested to move the `./config/containerfiles/` directory into the root of your repo too.
- You can see the docs for the `containerfile` module [here](https://blue-build.org/reference/modules/containerfile/).
3. Update your GHA to be at least `v1.4`
Expand Down Expand Up @@ -82,29 +83,29 @@ Here's a snippet of my recipe before the change.
name: jp-laptop
description: The image of Wunker OS for JP's Laptop.
base_image: ghcr.io/ublue-os/bazzite
image_version: '39'
image_version: "39"
modules:
# ...
- type: script
scripts:
- setup-selinux-dockersock.sh
- type: script
scripts:
- setup-kubectl.sh
- type: rpm-ostree
repos:
- https://pkg.earthly.dev/earthly.repo
- https://cli.github.com/packages/rpm/gh-cli.repo
install:
- earthly
- neovim
- helix
- alacritty
- gh
- type: script
scripts:
- install-mkcert.sh
- install-codelldb.sh
# ...
- type: script
scripts:
- setup-selinux-dockersock.sh
- type: script
scripts:
- setup-kubectl.sh
- type: rpm-ostree
repos:
- https://pkg.earthly.dev/earthly.repo
- https://cli.github.com/packages/rpm/gh-cli.repo
install:
- earthly
- neovim
- helix
- alacritty
- gh
- type: script
scripts:
- install-mkcert.sh
- install-codelldb.sh
```

And here's the change. I'm removing `neovim` cause I'm a filthy `helix` user.
Expand All @@ -113,28 +114,28 @@ And here's the change. I'm removing `neovim` cause I'm a filthy `helix` user.
name: jp-laptop
description: The image of Wunker OS for JP's Laptop.
base_image: ghcr.io/ublue-os/bazzite
image_version: '39'
image_version: "39"
modules:
# ...
- type: script
scripts:
- setup-selinux-dockersock.sh
- type: script
scripts:
- setup-kubectl.sh
- type: rpm-ostree
repos:
- https://pkg.earthly.dev/earthly.repo
- https://cli.github.com/packages/rpm/gh-cli.repo
install:
- earthly
- helix
- alacritty
- gh
- type: script
scripts:
- install-mkcert.sh
- install-codelldb.sh
# ...
- type: script
scripts:
- setup-selinux-dockersock.sh
- type: script
scripts:
- setup-kubectl.sh
- type: rpm-ostree
repos:
- https://pkg.earthly.dev/earthly.repo
- https://cli.github.com/packages/rpm/gh-cli.repo
install:
- earthly
- helix
- alacritty
- gh
- type: script
scripts:
- install-mkcert.sh
- install-codelldb.sh
```

So we re-run the build.
Expand All @@ -146,7 +147,6 @@ So we re-run the build.
=> CACHED [stage-keys 1/1] COPY cosign.pub /keys/jp-desktop-gaming.pub 0.0s
=> CACHED [stage-4 2/16] RUN --mount=type=bind,from=stage-keys,src=/keys,dst=/tmp/keys mkdir -p /usr/etc/pki/containers/ && cp /tmp/keys/* /usr/et 0.0s
=> CACHED [stage-bins 1/3] COPY --from=gcr.io/projectsigstore/cosign /ko-app/cosign /bins/cosign 0.0s
=> CACHED [stage-bins 2/3] COPY --from=docker.io/mikefarah/yq /usr/bin/yq /bins/yq 0.0s
=> CACHED [stage-bins 3/3] COPY --from=ghcr.io/blue-build/cli:main-installer /out/bluebuild /bins/bluebuild 0.0s
=> CACHED [stage-4 3/16] RUN --mount=type=bind,from=stage-bins,src=/bins,dst=/tmp/bins mkdir -p /usr/bin/ && cp /tmp/bins/* /usr/bin/ && ostree 0.0s
=> CACHED [stage-4 4/16] RUN --mount=type=tmpfs,target=/var --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw --mount=type=bind 0.0s
Expand Down Expand Up @@ -185,6 +185,6 @@ No you don't. We will continue to support the legacy method for the time being.

## Final words

Hopefully I was able to explain how these changes will help you in your building ventures. We'll be working more on creating useful features as well as optimizing your building experience.
Hopefully I was able to explain how these changes will help you in your building ventures. We'll be working more on creating useful features as well as optimizing your building experience.

Happy building!
135 changes: 135 additions & 0 deletions src/content/docs/blog/v090-features-changes.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
---
title: New CLI Features & Breaking Changes — v0.9.0
description: Stay up-to-date on new CLI features and breaking changes related to custom modules.
date: 2024-12-02
authors:
- name: Gerald (Jerry) Pinder
img: https://avatars.githubusercontent.com/u/4626052
github: gmpinder
- name: xyny
img: https://avatars.githubusercontent.com/u/60004820
github: xynydev
---

:::caution[TLDR]
The BlueBuild CLI `v0.9.0` update introduces **breaking changes**. [Show me what you broke!](#breaking-changes)
:::

The BlueBuild CLI is the tool that builds all BlueBuild custom images. It is also installed on every image built with BlueBuild and can be used to build and tinker with images locally. Lately we _(gmpinder)_ have been hard at work on a new CLI version to automate many administrative tasks you have to do as a custom image creator. But enough of the fluff, let's get right to it!

## New Features
xynydev marked this conversation as resolved.
Show resolved Hide resolved

### Generate ISOs for a fresh install

You can now generate offline ISO installation images as shown in the [how-to guide](/how-to/generate-iso/). Under the hood this is a wrapper of [JasonN3's build-container-installer](https://github.com/JasonN3/build-container-installer). Feel free to report issues to the CLI repository first, and to the upstream project only if the issue is deemed to be caused by them.

### Quickly start a new BlueBuild project from the command line

You can now use the `bluebuild new` and `bluebuild init` commands to start a new BlueBuild project from the command line. This will set up a new local git repository for you based on the [template](https://github.com/blue-build/template/) and some short configuration questions. (`new` creates a project in a new directory, `init` in the current empty one)

Here's a sneak peek:

```bash
❯ bluebuild new ./weird-os
✔ What would you like to name your image? · weird-os
✔ What is the registry for the image? (e.g. ghcr.io or registry.gitlab.com) · ghcr.io
✔ What is the name of your org/username? · octocat
✔ Write a short description of your image: · this is my weird custom OS :3
✔ Are you building on Github or Gitlab? · Github
Private key written to cosign.key
Public key written to cosign.pub
[main (root-commit) 994d18c] chore: Initial Commit
11 files changed, 355 insertions(+)
create mode 100644 .github/dependabot.yml
create mode 100644 .github/workflows/build.yml
create mode 100644 .gitignore
create mode 100644 LICENSE
create mode 100644 README.md
create mode 100644 cosign.pub
create mode 100644 files/scripts/example.sh
create mode 100644 files/system/etc/.gitkeep
create mode 100644 files/system/usr/.gitkeep
create mode 100644 modules/.gitkeep
create mode 100644 recipes/recipe.yml
INFO => Created new BlueBuild project in ./weird-os
```

### Build stages for compiling programs from source while keeping your image clean

You can define stages with a new top-level `stages:` key in the recipe before the `modules:`. Stages use the same module system as the rest of the build, but you'll probably find the [`script`](/reference/modules/script/) module most useful.

After building something in a stage, you can use the newly stable [`copy`](/reference/modules/copy/) module to copy files from the stage. This maps directly to a `COPY` statement in the `Containerfile`.

Read more on the [dedicated Stages-page](https://blue-build.org/reference/stages/).

### Miscellaneous

- A new `login` command for logging into registries for all tools like `docker`, `skopeo`, `cosign`, etc.
- A new `validate` command and check before building that will tell you where you have errors in your recipes
- A new `prune` command to easily clean build space for the build drivers
- The `sigstore` driver can now handle signing if you don't have `cosign` installed
- A new `switch` command consolidates `upgrade`/`rebase` commands
- The ability to build multiple recipes at the same time

## Breaking Changes

### Related to custom modules

Internal changes and refactors always might affect your custom modules, but we try to be transparent when making such changes and announce them, so that you aren't left with broken builds without any apparent reason.

#### Removal of [`yq`](https://github.com/mikefarah/yq)

We no longer include `yq` by default in images built with BlueBuild, and have refactored all modules to use `jq` instead. This comes in tandem with our experimentation [with using Nushell](https://github.com/blue-build/modules/issues/212) to build some new modules, and a concern of `yq` being included unconditionally regardless of if it's actually needed in an image, increasing attack surface. Read more about this from the [related GitHub issue](https://github.com/blue-build/cli/issues/261).

What this means for you:

- Reading module configuration with `yq` and `get_yaml_array` will break with the new update
- You now have to use `jq` and `get_json_array` instead

Here's how you should refactor your module code:

```diff lang=bash
#!/usr/bin/env bash

# read a single variable from the configuration
- VAR=$(echo "$1" | yq -I=0 '.var')
+ VAR=$(echo "$1" | jq -r 'try .["var"]')
echo "$VAR"

# read an array from the configuration
- get_yaml_array ARRAY '.array[]' "$1"
+ get_json_array ARRAY 'try .["array"][]' "$1"
```

Read more on the [updated custom module how-to guide](/how-to/making-modules/).

#### New way to use a local modules

Local modules must now have the `source` property set to `local`. Local modules are those that live inside the same repository as your custom image, inside the `./modules/` folder. This change allows us to validate recipes properly while not causing errors when custom modules are used.

```yaml ins={4}
modules:
# use the module `./modules/custom-module/`
- type: custom-module
source: local
input:
- value 1
- value 2
```

### Phasing out official support for [blue-build/legacy-template](blue-build/legacy-template)

With the switch to `jq` and `get_json_array` as talked about above, the 'legacy template', aka the old 'startingpoint', is starting to become ever more broken. As such, we will no longer be updating the template to accommodate for new changes. The repository is now archived, and a message is left in the README.

If you're _one of the few people who still_ run their image builds from a repository based on it, and you are still not considering [migration to the new system](/blog/introducing-bluebuild/#how-to-migrate), _you shall be crowned the new absolute ruler of your repository, in charge of keeping the builds running without someone telling you what to do._

Some things that might help you on your quest are [our guide to running BlueBuild modules from a Containerfile directly](/how-to/minimal-setup/) and plain-old reading the source code of [BlueBuild's CLI](https://github.com/blue-build/cli), especially the file containing the [bash exports for modules](https://github.com/blue-build/cli/blob/main/scripts/exports.sh) and the [Containerfile template used by BlueBuild images](https://github.com/blue-build/cli/blob/main/template/templates/Containerfile.j2).

### Miscellaneous

- Minimum supported version of `buildah` is now `1.29`, up from `1.24`.

## Final words

Happy holidays, everyone! Have fun with the new features, and enjoy your free time whenever you have the chance.
_Stay tuned for the eventual `v1.0.0`!_
30 changes: 30 additions & 0 deletions src/content/docs/how-to/generate-iso.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
title: How to build an ISO based on your custom image of Fedora Atomic
description: Using the `bluebuild generate-iso` command
---

First you need to choose whether to generate the ISO from remote image or a recipe.

- Generate from remote image if...
- You're trying to install a custom image made by someone else
- You are actively building and publishing images (this is the default)
- Generate from a recipe if...
- You don't want to rely on a cloud to build your images for you and are willing to manually run builds locally (or set up some system to automate that for you)

1. Generate the ISO:

```bash
# Generate ISO from a built and published remote image
sudo bluebuild generate-iso --iso-name weird-os.iso image ghcr.io/octocat/weird-os

# Build image and generate ISO from a local recipe
sudo bluebuild generate-iso --iso-name weird-os.iso recipe recipes/recipe.yml
```

2. Flash the ISO onto a USB drive ([Fedora Media Writer](https://www.fedoraproject.org/en/workstation/download) is recommended) and boot it.

- The ISO file should be inside your working directory (wherever you ran the command).

- Once booted, you should be presented with a screen similar to this:

![](./ublue-iso.png)
8 changes: 5 additions & 3 deletions src/content/docs/how-to/making-modules.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -23,18 +23,20 @@ If you want to extend your image with custom functionality that requires configu

This guide only includes the bare minimum information to get you started on your coding adventure. Check out the [module reference](/reference/module/) for more technical information about modules.

When being launched, your module receives its configuration as a JSON string as the first argument. It can be read from in bash using `jq` or `yq` like this:
When being launched, your module receives its configuration as a JSON string as the first argument. It can be read from in bash using `jq` like this:

```bash title="modules/<name-of-your-module>/<name-of-your-module>.sh"
#!/usr/bin/env bash
set -euo pipefail

# read a single variable from the configuration
VAR=$(echo "$1" | yq -I=0 ".var") # `-I=0` makes sure the output isn't indented
# `try` makes the command output 'null' if the key is not found, otherwise it will error out and the build will fail
# the `.["var"]` syntax is optional and could be replaced with the less safe and more error-prone `.var` syntax
VAR=$(echo "$1" | jq -r 'try .["var"]')
echo "$VAR"

# read an array from the configuration
get_yaml_array ARRAY '.array[]' "$1"
get_json_array ARRAY 'try .["array"][]' "$1"
# loop over the array
for THING in "${ARRAY[@]}"; do
echo "$THING"
Expand Down
Loading