Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ap/switch to rules oci #1

Merged
merged 32 commits into from
Oct 13, 2023
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
9347ca6
rules_oci dependencies
apesternikov Aug 9, 2023
7c1da88
move provider. deprecate fields
apesternikov Aug 9, 2023
814e4c4
push_oci
apesternikov Aug 9, 2023
aa90e5a
upgrade bazel
apesternikov Aug 9, 2023
187238b
wip
apesternikov Aug 9, 2023
98a6758
tests are passing with oci
apesternikov Aug 9, 2023
2bbc400
removed rules_docker
apesternikov Aug 10, 2023
2c1d27c
update go and gazelle
apesternikov Aug 10, 2023
3417ed5
run gazelle
apesternikov Aug 10, 2023
d89d598
cleanup + better vendoring
apesternikov Aug 10, 2023
6900451
wip
apesternikov Aug 10, 2023
15e2560
cleanup & gues_runfiles update
apesternikov Aug 11, 2023
d60f696
remove blanket copyright statement
apesternikov Aug 11, 2023
60d2488
rename the ruleset
apesternikov Aug 11, 2023
32d2dfe
rename example to e2e
apesternikov Aug 11, 2023
93fc7d0
rename KustomizeInfo to GitopsArtifactsInfo
apesternikov Aug 11, 2023
2ca19ae
wip
apesternikov Aug 14, 2023
eae6f50
min supported bazel version for rules_go is 5.4.0
apesternikov Aug 16, 2023
69b480f
run on latest bazel release
apesternikov Aug 16, 2023
64a61e8
fix stamping
apesternikov Aug 17, 2023
a280982
move secrets examplt back to /examples and add test for generated files
apesternikov Aug 17, 2023
d992f25
wip
apesternikov Aug 17, 2023
7e6efd1
use test for buildifier
apesternikov Aug 17, 2023
47c5334
min required bazel version is 6.1.0
apesternikov Aug 17, 2023
aa6320b
rename K8sPushInfo to GitopsPushInfo
apesternikov Aug 18, 2023
4111ec3
e2e with it test
apesternikov Aug 23, 2023
102b031
tests
apesternikov Aug 23, 2023
4d3c694
exclude it tests
apesternikov Aug 23, 2023
2bff6e4
support for custom openapi schema with cloud run as test
apesternikov Sep 6, 2023
e4100c6
Update examples/secrets/BUILD.bazel
apesternikov Sep 6, 2023
eb2f076
wip
apesternikov Sep 6, 2023
2a7cabf
Merge branch 'ap/switch_to_rules_oci' of github.com:fasterci/rules_gi…
apesternikov Sep 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .bazelignore
Original file line number Diff line number Diff line change
@@ -1 +1 @@
examples
e2e
2 changes: 1 addition & 1 deletion .bazelversion
Original file line number Diff line number Diff line change
@@ -1 +1 @@
6.0.0
6.3.1
19 changes: 8 additions & 11 deletions .fasterci/config.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,9 @@
# configure vscode yaml support https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml
# yaml-language-server: $schema=https://fasterci.com/config.schema.json
workflows:

- &build_workflow
name: Faster CI / build (6.0.0)
name: Faster CI / build (6.1.0)
env:
USE_BAZEL_VERSION: "6.0.0"
image: us.gcr.io/fasterci/bazelbuilder:5e59f651dbb5
USE_BAZEL_VERSION: "6.1.0"
on:
push:
branches:
Expand All @@ -21,17 +18,17 @@ workflows:
- //...
test_targets:
- //...
- name: Build & test examples
working-directory: examples
- name: Build & test e2e
working-directory: e2e
bazel:
build_targets:
- //...
test_targets:
- //...
- name: Check
run: bazel run //:buildifier-check
test_flags:
- --test_size_filters=-large,-enormous

- <<: *build_workflow
name: Faster CI / build (5.3.1)
name: Faster CI / build (6.3.2)
env:
USE_BAZEL_VERSION: "5.3.1"
USE_BAZEL_VERSION: "6.3.2"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to keep this consistent with .bazelversion

4 changes: 2 additions & 2 deletions .github/workflows/workspace_snippet.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ WORKSPACE snippet:
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
name = "com_adobe_rules_gitops",
name = "rules_gitops",
sha256 = "${SHA}",
strip_prefix = "${PREFIX}",
urls = ["https://github.com/adobe/rules_gitops/archive/refs/tags/${TAG}.tar.gz"],
urls = ["https://github.com/fasterci/rules_gitops/archive/refs/tags/${TAG}.tar.gz"],
)
EOF

Expand Down
9 changes: 4 additions & 5 deletions BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@
# governing permissions and limitations under the License.

# gazelle:build_tags darwin,linux
# gazelle:exclude examples
# gazelle:exclude examples e2e
# gazelle:proto disable_global

load("@bazel_gazelle//:def.bzl", "gazelle")
load("@com_github_bazelbuild_buildtools//buildifier:def.bzl", "buildifier")
load("@com_github_bazelbuild_buildtools//buildifier:def.bzl", "buildifier", "buildifier_test")

licenses(["notice"]) # Apache 2.0

Expand Down Expand Up @@ -50,8 +50,8 @@ buildifier(
lint_mode = "fix",
)

buildifier(
name = "buildifier-check",
buildifier_test(
name = "buildifier_check",
lint_mode = "warn",
lint_warnings = [
"-module-docstring",
Expand All @@ -60,5 +60,4 @@ buildifier(
"-function-docstring-args",
"-function-docstring-return",
],
mode = "check",
)
5 changes: 0 additions & 5 deletions COPYRIGHT

This file was deleted.

26 changes: 13 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,9 @@ copy the WORKSPACE snippet into your `WORKSPACE` file.
The `k8s_deploy` creates rules that produce the `.apply` and `.gitops` targets `k8s_deploy` is defined in [k8s.bzl](./skylib/k8s.bzl). `k8s_deploy` takes the files listed in the `manifests`, `patches`, and `configmaps_srcs` attributes and combines (**renders**) them into one YAML file. This happens when you `bazel build` or `bazel run` a target created by the `k8s_deploy`. The file is created at `bazel-bin/path/to/package/name.yaml`. When you run a `.apply` target, it runs `kubectl apply` on this file. When you run a `.gitops` target, it copies this file to
the appropriate location in the same os separate repository.

For example, let's look at the [example's k8s_deploy](./examples/helloworld/BUILD). We can peek at the file containing the rendered K8s manifests:
For example, let's look at the [example's k8s_deploy](./e2e/helloworld/BUILD). We can peek at the file containing the rendered K8s manifests:
```bash
cd examples
cd e2e
bazel run //helloworld:mynamespace.show
```
When you run `bazel run ///helloworld:mynamespace.apply`, it applies this file into your personal (`{BUILD_USER}`) namespace. Viewing the rendered files with `.show` can be useful for debugging issues with invalid or misconfigured manifests.
Expand All @@ -57,7 +57,7 @@ When you run `bazel run ///helloworld:mynamespace.apply`, it applies this file i
| ------------------------- | -------------- | -----------
| ***cluster*** | `None` | The name of the cluster in which these manifests will be applied.
| ***namespace*** | `None` | The target namespace to assign to all manifests. Any namespace value in the source manifests will be replaced or added if not specified.
| ***user*** | `{BUILD_USER}` | The user passed to kubectl in .apply rule. Must exist in users ~/.kube/config
| ***user*** | value from ~/.kube/config | The user passed to kubectl in .apply rule. Must exist in users ~/.kube/config
| ***configmaps_srcs*** | `None` | A list of files (of any type) that will be combined into configmaps. See [Generating Configmaps](#generating-configmaps).
| ***configmaps_renaming*** | `None` | Configmaps/Secrets renaming policy. Could be None or 'hash'. 'hash' renaming policy is used to add a unique suffix to the generated configmap or secret name. All references to the configmap or secret in other manifests will be replaced with the generated name.
| ***secrets_srcs*** | `None` | A list of files (of any type) that will be combined into a secret similar to configmaps.
Expand All @@ -82,7 +82,7 @@ When you run `bazel run ///helloworld:mynamespace.apply`, it applies this file i
| ***image_registry*** | `docker.io` | The registry to push images to.
| ***image_repository*** | `None` | The repository to push images to. By default, this is generated from the current package path.
| ***image_repository_prefix*** | `None` | Add a prefix to the image_repository. Can be used to upload the images in
| ***image_pushes*** | `[]` | A list of labels implementing K8sPushInfo referring image uploaded into registry. See [Injecting Docker Images](#injecting-docker-images).
| ***image_pushes*** | `[]` | A list of labels implementing GitopsPushInfo referring image uploaded into registry. See [Injecting Docker Images](#injecting-docker-images).
| ***release_branch_prefix*** | `master` | A git branch name/prefix. Automatically run GitOps while building this branch. See [GitOps and Deployment](#gitops_and_deployment).
| ***deployment_branch*** | `None` | Automatic GitOps output will appear in a branch and PR with this name. See [GitOps and Deployment](#gitops_and_deployment).
| ***gitops_path*** | `cloud` | Path within the git repo where gitops files get generated into
Expand Down Expand Up @@ -229,9 +229,9 @@ spec:
<a name="injecting-docker-images"></a>
### Injecting Docker Images

Third-party Docker images can be referenced directly in K8s manifests, but for most apps, we need to run our own images. The images are built in the Bazel build pipeline using [rules_docker](https://github.com/bazelbuild/rules_docker). For example, the `java_image` rule creates an image of a Java application from Java source code, dependencies, and configuration.
Third-party Docker images can be referenced directly in K8s manifests, but for most apps, we need to run our own images. The images are built in the Bazel build pipeline using [rules_oci](https://github.com/bazel-contrib/rules_oci).

Here's a (very contrived) example of how this ties in with `k8s_deploy`. Here's the `BUILD` file located in the package `//examples`:
Here's a (very contrived) example of how this ties in with `k8s_deploy`. Here's the `BUILD` file located in the package `//e2e`:
```starlark
java_image(
name = "helloworld_image",
Expand All @@ -254,9 +254,9 @@ metadata:
name: helloworld
spec:
containers:
- image: //examples:helloworld_image # (2)
- image: //e2e:helloworld_image # (2)
```
There `images` attribute dictionary `(1)` defines the images available for the substitution. The manifest file references the fully qualified image target path `//examples:helloworld_image` `(2)`.
There `images` attribute dictionary `(1)` defines the images available for the substitution. The manifest file references the fully qualified image target path `//e2e:helloworld_image` `(2)`.

The `image` key value in the dictionary is used as an image push identifier. The best practice (as provided in the example) is to use image key that matches the [label name](https://docs.bazel.build/versions/master/skylark/lib/Label.html#name) of the image target.

Expand Down Expand Up @@ -315,7 +315,7 @@ Docker image and the files in the image. So for example, here's what will happen
1. A new `helloworld` manifest will be rendered using the new image
1. The new `helloworld` pod will be deployed

It is possible to use alternative ways to resolve images as long as respective rule implements K8sPushInfo provider. For example, this setup will mirror the referred image into a local registry and provide a reference to it. `k8s_deploy` will need to use `image_pushes` parameter:
It is possible to use alternative ways to resolve images as long as respective rule implements GitopsPushInfo provider. For example, this setup will mirror the referred image into a local registry and provide a reference to it. `k8s_deploy` will need to use `image_pushes` parameter:

```starlark
load("@com_fasterci_rules_mirror//mirror:defs.bzl", "mirror_image")
Expand Down Expand Up @@ -368,7 +368,7 @@ The *Create GitOps PRs* step usually is the last step of a CI pipeline. `rules_g

For the full list of `create_gitops_prs` command line options, run:
```bash
bazel run @com_adobe_rules_gitops//gitops/prer:create_gitops_prs
bazel run @rules_gitops//gitops/prer:create_gitops_prs
```

<a name="gitops-and-deployment-supported-git-servers"></a>
Expand Down Expand Up @@ -405,7 +405,7 @@ GIT_ROOT_DIR=$(git rev-parse --show-toplevel)
GIT_COMMIT_ID=$(git rev-parse HEAD)
GIT_BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD)
if [ "${GIT_BRANCH_NAME}" == "master"]; then
bazel run @com_adobe_rules_gitops//gitops/prer:create_gitops_prs -- \
bazel run @rules_gitops//gitops/prer:create_gitops_prs -- \
--workspace $GIT_ROOT_DIR \
--git_repo https://github.com/example/repo.git \
--git_mirror $GIT_ROOT_DIR/.git \
Expand Down Expand Up @@ -482,7 +482,7 @@ GIT_BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD) # => release/team-20
RELEASE_BRANCH_SUFFIX=${GIT_BRANCH_NAME#"release/team"} # => -20200101
RELEASE_BRANCH=${GIT_BRANCH_NAME%${RELEASE_BRANCH_SUFFIX}} # => release/team
if [ "${RELEASE_BRANCH}" == "release/team"]; then
bazel run @com_adobe_rules_gitops//gitops/prer:create_gitops_prs -- \
bazel run @rules_gitops//gitops/prer:create_gitops_prs -- \
--workspace $GIT_ROOT_DIR \
--git_repo https://github.com/example/repo.git \
--git_mirror $GIT_ROOT_DIR/.git \
Expand Down Expand Up @@ -575,7 +575,7 @@ The test code launches the script to perform the test setup. The test code shoul
The `@k8s_test//:kubeconfig` target referenced from `k8s_test_setup` rule serves the purpose of making Kubernetes configuration available in the test sandbox. The `kubeconfig` repository rule in the `WORKSPACE` file will need, at minimum, provide the cluster name.

```starlark
load("@com_adobe_rules_gitops//gitops:defs.bzl", "kubeconfig")
load("@rules_gitops//gitops:defs.bzl", "kubeconfig")

kubeconfig(
name = "k8s_test",
Expand Down
42 changes: 17 additions & 25 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -8,74 +8,66 @@
# OF ANY KIND, either express or implied. See the License for the specific language
# governing permissions and limitations under the License.

workspace(name = "com_adobe_rules_gitops")
workspace(name = "rules_gitops")

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
name = "io_bazel_rules_go",
sha256 = "dd926a88a564a9246713a9c00b35315f54cbd46b31a26d5d8fb264c07045f05d",
sha256 = "278b7ff5a826f3dc10f04feaf0b70d48b68748ccd512d7f98bf442077f043fe3",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.38.1/rules_go-v0.38.1.zip",
"https://github.com/bazelbuild/rules_go/releases/download/v0.38.1/rules_go-v0.38.1.zip",
"https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.41.0/rules_go-v0.41.0.zip",
"https://github.com/bazelbuild/rules_go/releases/download/v0.41.0/rules_go-v0.41.0.zip",
],
)

http_archive(
name = "com_github_bazelbuild_buildtools",
sha256 = "ca524d4df8c91838b9e80543832cf54d945e8045f6a2b9db1a1d02eec20e8b8c",
strip_prefix = "buildtools-6.0.1",
sha256 = "977a0bd4593c8d4c8f45e056d181c35e48aa01ad4f8090bdb84f78dca42f47dc",
strip_prefix = "buildtools-6.1.2",
urls = [
"https://github.com/bazelbuild/buildtools/archive/refs/tags/6.0.1.tar.gz",
"https://github.com/bazelbuild/buildtools/archive/refs/tags/v6.1.2.tar.gz",
],
)

load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")

go_rules_dependencies()

go_register_toolchains(version = "1.19.2")
go_register_toolchains(version = "1.20.6")

#
# Self dependencies
#
load("@com_adobe_rules_gitops//gitops:deps.bzl", "rules_gitops_dependencies")
load("@rules_gitops//gitops:deps.bzl", "rules_gitops_dependencies")

rules_gitops_dependencies()

load("@com_adobe_rules_gitops//gitops:repositories.bzl", "rules_gitops_repositories")
load("@rules_gitops//gitops:repositories.bzl", "rules_gitops_repositories")

rules_gitops_repositories()

#
# Development dependencies
#

load("@bazel_skylib//lib:unittest.bzl", "register_unittest_toolchains")

register_unittest_toolchains()

http_archive(
name = "com_google_protobuf",
sha256 = "d0f5f605d0d656007ce6c8b5a82df3037e1d8fe8b121ed42e536f569dec16113",
strip_prefix = "protobuf-3.14.0",
sha256 = "3bd7828aa5af4b13b99c191e8b1e884ebfa9ad371b0ce264605d347f135d2568",
strip_prefix = "protobuf-3.19.4",
urls = [
"https://mirror.bazel.build/github.com/protocolbuffers/protobuf/archive/v3.14.0.tar.gz",
"https://github.com/protocolbuffers/protobuf/archive/v3.14.0.tar.gz",
"https://github.com/protocolbuffers/protobuf/archive/v3.19.4.tar.gz",
],
)

load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")

protobuf_deps()

load(
"@io_bazel_rules_docker//go:image.bzl",
go_image_repositories = "repositories",
)

go_image_repositories()

load("@bazel_skylib//lib:unittest.bzl", "register_unittest_toolchains")

register_unittest_toolchains()

load("@com_github_bazelbuild_buildtools//buildifier:deps.bzl", "buildifier_dependencies")

buildifier_dependencies()
Expand Down
1 change: 1 addition & 0 deletions e2e/.bazelrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
build --action_env HOME --test_env HOME
1 change: 1 addition & 0 deletions e2e/.bazelversion
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
6.3.1
File renamed without changes.
File renamed without changes.
File renamed without changes.
31 changes: 20 additions & 11 deletions examples/WORKSPACE → e2e/WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
workspace(name = "examples")

local_repository(
name = "com_adobe_rules_gitops",
name = "rules_gitops",
path = "..",
)

Expand All @@ -22,24 +22,24 @@ load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
name = "io_bazel_rules_go",
sha256 = "099a9fb96a376ccbbb7d291ed4ecbdfd42f6bc822ab77ae6f1b5cb9e914e94fa",
sha256 = "278b7ff5a826f3dc10f04feaf0b70d48b68748ccd512d7f98bf442077f043fe3",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.35.0/rules_go-v0.35.0.zip",
"https://github.com/bazelbuild/rules_go/releases/download/v0.35.0/rules_go-v0.35.0.zip",
"https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.41.0/rules_go-v0.41.0.zip",
"https://github.com/bazelbuild/rules_go/releases/download/v0.41.0/rules_go-v0.41.0.zip",
],
)

load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")

go_rules_dependencies()

go_register_toolchains(version = "1.19.2")
go_register_toolchains(version = "1.20.6")

load("@com_adobe_rules_gitops//gitops:deps.bzl", "rules_gitops_dependencies")
load("@rules_gitops//gitops:deps.bzl", "rules_gitops_dependencies")

rules_gitops_dependencies()

load("@com_adobe_rules_gitops//gitops:repositories.bzl", "rules_gitops_repositories")
load("@rules_gitops//gitops:repositories.bzl", "rules_gitops_repositories")

rules_gitops_repositories()

Expand All @@ -49,9 +49,18 @@ rules_gitops_repositories()
# examples dependencies
#

load(
"@io_bazel_rules_docker//go:image.bzl",
go_image_repositories = "repositories",
load("@rules_oci//oci:pull.bzl", "oci_pull")

oci_pull(
name = "go_image_static",
digest = "sha256:ebd8cc37d22551dce0957ba8e58f03b22a8448bbf844c8c9ded4feef883b36bc",
image = "gcr.io/distroless/static",
)

go_image_repositories()
load("@rules_gitops//skylib:k8s.bzl", "kubeconfig")

kubeconfig(
name = "k8s_test",
cluster = "gke_rules-gitops-demo_us-central1_cluster-demo",
use_host_config = True,
)
File renamed without changes.
5 changes: 5 additions & 0 deletions e2e/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
module github.com/fasterci/rules_gitops/e2e

go 1.20

require github.com/adobe/rules_gitops v0.18.0
2 changes: 2 additions & 0 deletions e2e/go.sum
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
github.com/adobe/rules_gitops v0.18.0 h1:VXG5WNHCA26pA72aqcyeUjXYkBXDyAQxS33QFs4czHw=
github.com/adobe/rules_gitops v0.18.0/go.mod h1:vdARypk35U3M/1EAmFRptiSbzXg9FMQPe8D33Am7rHY=
Loading