Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: replace apm-pipeline-library kibana-docker-image #153

Merged
merged 4 commits into from
Nov 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions .github/workflows/test-kibana-docker-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: test-kibana-docker-image

on:
workflow_dispatch: ~
push:
paths:
- '.github/workflows/test-kibana-docker-image.yml'
- 'kibana-docker-image/**'

permissions:
contents: read

jobs:
test:
name: kibana-${{ matrix.serverless && 'serverless' || 'cloud' }}-docker-image
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- serverless: false
# Serverless is not something we built anymore as part of the obs automation
# but use the existing docker images generated by Kibana CI.
# See https://github.com/elastic/kibana/pull/181851
# - serverless: true
steps:
- uses: actions/checkout@v4
- name: Free Disk Space (Ubuntu)
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be
with:
tool-cache: false
android: true
dotnet: true
haskell: true
large-packages: true
docker-images: true
# This needs to be set to false.
# See https://github.com/jlumbroso/free-disk-space/issues/12
# TL;DR
# Otherwise, the workflow is cancelled with an 143 exit code.
swap-storage: false

# IMPORTANT: the secrets below are managed through IASC (contact robots if you need further assistance)
- name: Log in to the Elastic Container registry
uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v3.3.0
with:
registry: ${{ secrets.ELASTIC_DOCKER_REGISTRY }}
username: ${{ secrets.ELASTIC_DOCKER_USERNAME }}
password: ${{ secrets.ELASTIC_DOCKER_PASSWORD }}

- uses: ./kibana-docker-image
id: kibana-docker-image
with:
serverless: ${{ matrix.serverless }}

- name: validate
run: |
echo "${STACK_VERSION:?}"
echo "${GIT_COMMIT_SHA:?}"
echo "${DOCKER_IMAGE_REF:?}"
docker pull "${DOCKER_IMAGE_REF}"
env:
DOCKER_IMAGE_REF: ${{ steps.kibana-docker-image.outputs.kibana-docker-image }}
GIT_COMMIT_SHA: ${{ steps.kibana-docker-image.outputs.kibana-commit-sha }}
STACK_VERSION: ${{ steps.kibana-docker-image.outputs.kibana-stack-version }}
68 changes: 68 additions & 0 deletions kibana-docker-image/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# <!--name-->kibana-docker-image<!--/name-->

[![usages](https://img.shields.io/badge/usages-white?logo=githubactions&logoColor=blue)](https://github.com/search?q=elastic%2Foblt-actions%2Fkibana-docker-image+%28path%3A.github%2Fworkflows+OR+path%3A**%2Faction.yml+OR+path%3A**%2Faction.yaml%29&type=code)
[![test-kibana-docker-image](https://github.com/elastic/oblt-actions/actions/workflows/test-kibana-docker-image.yml/badge.svg?branch=main)](https://github.com/elastic/oblt-actions/actions/workflows/test-kibana-docker-image.yml)

<!--description-->
An Action to build and push Kibana docker images given a git ref.
<!--/description-->

## Inputs

<!--inputs-->
| Name | Description | Required | Default |
|---------------------|----------------------------------------------------------------------------------|----------|------------------------|
| `github-repository` | The git repository to build the image from. | `false` | `elastic/kibana` |
| `git-ref` | The git ref of elastic/kibana to build the image from. (Default: default branch) | `false` | ` ` |
| `serverless` | Whether to build the serverless image or not. | `false` | `false` |
| `docker-registry` | Docker registry to publish the image to. | `false` | `docker.elastic.co` |
| `docker-namespace` | Docker namespace to publish the image to. | `false` | `observability-ci` |
| `checkout-path` | The path to checkout the git repository to. | `false` | `kibana-repo-checkout` |
<!--/inputs-->

## Outputs

<!--outputs-->
| Name | Description |
|------------------------|---------------------------------------------------------------------------|
| `kibana-docker-image` | The reference of the Docker image that was built in the format image:tag. |
| `kibana-commit-sha` | The git commit SHA of the image that was built. |
| `kibana-stack-version` | The elastic stack version of Kibana that was built. |
<!--/outputs-->

## Usage

<!--usage action="elastic/oblt-actions/**" version="env:VERSION"-->
```yaml
---
name: example

on: workflow_dispatch

jobs:
kibana-docker-image-cloud:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Log in to the Elastic Container registry
uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v3.3.0
with:
registry: ${{ secrets.ELASTIC_DOCKER_REGISTRY }}
username: ${{ secrets.ELASTIC_DOCKER_USERNAME }}
password: ${{ secrets.ELASTIC_DOCKER_PASSWORD }}

- uses: 'elastic/oblt-actions/kibana-docker-image@v1'
id: kibana-docker-image
with:
git-ref: main # git ref of elastic/kibana
serverless: true # Default: false

- name: docker pull
run: |
echo "${DOCKER_IMAGE:?}"
docker pull "${DOCKER_IMAGE}"
env:
DOCKER_IMAGE: ${{ steps.kibana-docker-image.outputs.kibana-docker-image }}
```
<!--/usage-->
80 changes: 80 additions & 0 deletions kibana-docker-image/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
---
name: 'kibana-docker-image'
description: |
An Action to build and push Kibana docker images given a git ref.
inputs:
github-repository:
description: The git repository to build the image from.
default: "elastic/kibana"
required: false
git-ref:
description: "The git ref of elastic/kibana to build the image from. (Default: default branch)"
required: false
serverless:
description: Whether to build the serverless image or not.
required: false
default: "false"
docker-registry:
description: Docker registry to publish the image to.
required: false
default: "docker.elastic.co"
docker-namespace:
description: Docker namespace to publish the image to.
required: false
default: "observability-ci"
checkout-path:
description: The path to checkout the git repository to.
required: false
default: "kibana-repo-checkout"

outputs:
kibana-docker-image:
description: The reference of the Docker image that was built in the format image:tag.
value: ${{ steps.vars.outputs.docker-reference }}
kibana-commit-sha:
description: The git commit SHA of the image that was built.
value: ${{ steps.vars.outputs.kibana-commit-sha }}
kibana-stack-version:
description: The elastic stack version of Kibana that was built.
value: ${{ steps.vars.outputs.kibana-stack-version }}

runs:
using: composite
steps:
- uses: actions/checkout@v4
with:
repository: ${{ inputs.github-repository }}
ref: ${{ inputs.git-ref }}
path: ${{ inputs.checkout-path }}

- uses: actions/setup-node@v4
with:
node-version-file: ${{ inputs.checkout-path }}/.node-version
cache: yarn
cache-dependency-path: ${{ inputs.checkout-path }}/yarn.lock

- name: Set up variables
id: vars
working-directory: ${{ inputs.checkout-path }}
run: ${{ github.action_path }}/setup-vars.sh
env:
DOCKER_REGISTRY: ${{ inputs.docker-registry }}
DOCKER_NAMESPACE: ${{ inputs.docker-namespace }}
SERVERLESS: ${{ inputs.serverless }}
shell: bash

- name: Build and Push
working-directory: ${{ inputs.checkout-path }}
run: ${{ github.action_path }}/build-and-push.sh
shell: bash
env:
SERVERLESS: ${{ inputs.serverless }}
DOCKER_NAMESPACE: ${{ steps.vars.outputs.docker-namespace }}
DOCKER_TAG: ${{ steps.vars.outputs.docker-tag }}
DOCKER_REFERENCE: ${{ steps.vars.outputs.docker-reference }}

- name: Verify image exists
run: docker manifest inspect "${DOCKER_REFERENCE}" > /dev/null 2>&1;
env:
DOCKER_REFERENCE: ${{ steps.vars.outputs.docker-reference }}
shell: bash
66 changes: 66 additions & 0 deletions kibana-docker-image/build-and-push.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
#!/usr/bin/env bash
#
# Builds a docker image for Kibana inspired by
# https://github.com/elastic/kibana/blob/main/.buildkite/scripts/build_kibana.sh
#
set -euo pipefail

if docker manifest inspect "${DOCKER_REFERENCE}" > /dev/null 2>&1; then
echo "Docker image ${DOCKER_REFERENCE} already exists. Skipping build."
exit 0
fi

export BABEL_DISABLE_CACHE=true
export FORCE_COLOR=1
export NODE_OPTIONS=" --max-old-space-size=4096"
export BUILD_TS_REFS_DISABLE="true"

echo "::group::Bootstrap"
time yarn kbn clean
time yarn kbn bootstrap
echo "::endgroup::"

# https://github.com/elastic/kibana/blob/main/.buildkite/scripts/build_kibana.sh#L11-L19
echo "::group::Build Linux package"
export KBN_NP_PLUGINS_BUILT=true
time node scripts/build \
--skip-os-packages \
--skip-canvas-shareable-runtime \
--skip-cdn-assets \
--skip-docker-contexts
echo "::endgroup::"

# https://github.com/elastic/kibana/blob/main/.buildkite/scripts/build_kibana.sh#L21-L34
echo "::group::Build docker images"
if [ "${SERVERLESS}" == "false" ] ; then
time node scripts/build \
--docker-images \
--docker-namespace="${DOCKER_NAMESPACE}" \
--docker-tag="${DOCKER_TAG}" \
--docker-push \
--skip-archives \
--skip-initialize \
--skip-cdn-assets \
--skip-docker-contexts \
--skip-docker-ubi \
--skip-docker-ubuntu \
--skip-generic-folders \
--skip-platform-folders \
--skip-docker-serverless
else
# enable Docker multiarch support
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
time node scripts/build \
--release \
--docker-cross-compile \
--docker-images \
--docker-namespace="${DOCKER_NAMESPACE}" \
--docker-tag="${DOCKER_TAG}" \
--docker-push \
--skip-cdn-assets \
--skip-docker-contexts \
--skip-docker-ubi \
--skip-docker-ubuntu \
--skip-docker-cloud
fi
echo "::endgroup::"
22 changes: 22 additions & 0 deletions kibana-docker-image/setup-vars.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/usr/bin/env bash

set -euo pipefail

docker_image="kibana-cloud"
if [[ "${SERVERLESS}" == "true" ]]; then
docker_image="kibana-serverless"
fi

kibana_commit_sha=$(git rev-parse HEAD)
kibana_stack_version="$(jq -r .version package.json)-SNAPSHOT"
docker_tag="${kibana_stack_version}-${kibana_commit_sha}"
docker_reference="${DOCKER_REGISTRY}/${DOCKER_NAMESPACE}/${docker_image}:${docker_tag}"
{
echo "kibana-stack-version=${kibana_stack_version}"
echo "kibana-commit-sha=${kibana_commit_sha}"
echo "docker-registry=${DOCKER_REGISTRY}"
echo "docker-namespace=${DOCKER_NAMESPACE}"
echo "docker-image=${docker_image}"
echo "docker-tag=${docker_tag}"
echo "docker-reference=${docker_reference}"
} >> "${GITHUB_OUTPUT}"
Loading