Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

k8s_deploy doesn't template manifests when used with cc_image. #119

Open
lbhdc opened this issue Jan 27, 2023 · 2 comments
Open

k8s_deploy doesn't template manifests when used with cc_image. #119

lbhdc opened this issue Jan 27, 2023 · 2 comments

Comments

@lbhdc
Copy link

lbhdc commented Jan 27, 2023

Description of the problem / feature request:

I am trying to deploy a c++ project with rules_gitops, but the bazel labels aren't rendering in the manifests.

Feature requests: what underlying problem are you trying to solve with this feature?

Support cc_images in manifests

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

I was able to modify the hello world example to reproduce the problem.

Test cc app

// main.cc contents
#include <iostream>

int main() {
  std::cout << "it works\n";
}

Add cc binary and image to the BUILD file in hello world

diff --git a/examples/helloworld/BUILD b/examples/helloworld/BUILD
index 924f8a4..4b0bf28 100644
--- a/examples/helloworld/BUILD
+++ b/examples/helloworld/BUILD
@@ -8,12 +8,23 @@
 # OF ANY KIND, either express or implied. See the License for the specific language
 # governing permissions and limitations under the License.
 
+load("@io_bazel_rules_docker//cc:image.bzl", "cc_image")
 load("@io_bazel_rules_docker//go:image.bzl", "go_image")
 load("@io_bazel_rules_go//go:def.bzl", "go_binary", "go_library", "go_test")
 load("@com_adobe_rules_gitops//gitops:defs.bzl", "k8s_deploy")
 
 licenses(["notice"])  # Apache 2.0
 
+cc_binary(
+       name = "tester",
+       srcs = ["main.cc"],
+)
+
+cc_image(
+       name = "image2",
+       binary = ":tester",
+)
+
 go_library(
     name = "go_default_library",
     srcs = ["helloworld.go"],

Add the cc base image that cc_image depends on to the example workspace

diff --git a/examples/WORKSPACE b/examples/WORKSPACE
index 86ee6eb..f8b2f1b 100644
--- a/examples/WORKSPACE
+++ b/examples/WORKSPACE
@@ -55,3 +55,9 @@ load(
 )
 
 go_image_repositories()
+
+load(
+    "@io_bazel_rules_docker//cc:image.bzl",
+    cc_image_repos = "repositories",
+)
+cc_image_repos()

Point manifests to use the cc image instead of the go image.

diff --git a/examples/helloworld/deployment.yaml b/examples/helloworld/deployment.yaml
index 391baed..8f7eacf 100644
--- a/examples/helloworld/deployment.yaml
+++ b/examples/helloworld/deployment.yaml
@@ -11,11 +11,11 @@ spec:
     metadata:
       labels:
         app: helloworld
-        app_label_image_short_digest: "{{//helloworld:image.short-digest}}"
+        app_label_image_short_digest: "{{//helloworld:image2.short-digest}}"
     spec:
       containers:
         - name: helloworld
-          image: //helloworld:image
+          image: //helloworld:image2
           args:
             - --port=8080
           ports:

The output of bazel run //helloworld:mynamespace.show returns this yaml which not fill out the image values. Running show with the original go image correctly templates the manifests.

apiVersion: v1
kind: Service
metadata:
  labels:
    app: helloworld
  name: helloworld
  namespace: monolith
spec:
  ports:
  - name: web
    port: 80
    targetPort: 8080
  selector:
    app: helloworld
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: helloworld
  namespace: monolith
spec:
  replicas: 1
  selector:
    matchLabels:
      app: helloworld
  template:
    metadata:
      labels:
        app: helloworld
        app_label_image_short_digest: '{{//helloworld:image2.short-digest}}'
    spec:
      containers:
      - args:
        - --port=8080
        env:
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        image: //helloworld:image2
        name: helloworld
        ports:
        - containerPort: 8080
          name: web
        readinessProbe:
          httpGet:
            path: /healthz
            port: 8080
        resources:
          requests:

What operating system are you running Bazel on?

> uname -sr
Linux 6.1.7-100.fc36.x86_64

What's the output of bazel info release?

I am using basilisk, so it is picking up the version set in the .bazelversion from rules_gitopts. My project is on 6.0.0, and experiences the same problem as the above.

release 5.3.1

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

Replace this line with your answer.

Snippet of the WORKSPACE file that includes rules_gitops rules.

This is the workspace file from the examples with the patch from above applied to add in the cc_base_image workspace required by cc_image.

# Copyright 2020 Adobe. All rights reserved.
# This file is licensed to you under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. You may obtain a copy
# of the License at http://www.apache.org/licenses/LICENSE-2.0

# Unless required by applicable law or agreed to in writing, software distributed under
# the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS
# OF ANY KIND, either express or implied. See the License for the specific language
# governing permissions and limitations under the License.

workspace(name = "examples")

local_repository(
    name = "com_adobe_rules_gitops",
    path = "..",
)

# generated by workspace_snippet.sh
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

#---SNIP--- Below here is re-used in the workspace snippet published on releases

http_archive(
    name = "io_bazel_rules_go",
    sha256 = "099a9fb96a376ccbbb7d291ed4ecbdfd42f6bc822ab77ae6f1b5cb9e914e94fa",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.35.0/rules_go-v0.35.0.zip",
        "https://github.com/bazelbuild/rules_go/releases/download/v0.35.0/rules_go-v0.35.0.zip",
    ],
)

load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")

go_rules_dependencies()

go_register_toolchains(version = "1.19.2")

load("@com_adobe_rules_gitops//gitops:deps.bzl", "rules_gitops_dependencies")

rules_gitops_dependencies()

load("@com_adobe_rules_gitops//gitops:repositories.bzl", "rules_gitops_repositories")

rules_gitops_repositories()

#---END_SNIP--- marks the end of the the workspace snippet published on releases

#
# examples dependencies
#

load(
    "@io_bazel_rules_docker//go:image.bzl",
    go_image_repositories = "repositories",
)

go_image_repositories()

load(
    "@io_bazel_rules_docker//cc:image.bzl",
    cc_image_repos = "repositories",
)
cc_image_repos()

Have you found anything relevant by searching the web?

No, I have searched quite abit, but its kind of vague problem (failed templating) that is difficult to search for.

I browsed and searched the issue tracker for relavent issues but didn't see anything related.

I have tried reading through relevant portions of the rules, but did not see anything that jumped out at me.

I searched github for com_adobe_rules_gitops cc_image filename:BUILD to try to locate places where people have used cc_images with gitops, but github yields no results. (same for BUILD.bazel).

Any other information, logs, or outputs that you want to share?

Thanks for taking a look 😁

@eko
Copy link
Contributor

eko commented Feb 21, 2023

Hi,

I have the same issue using Bazel 6.0.0. You have to stay on Bazel 5.4.0 at the moment if you want to use this.

@lbhdc
Copy link
Author

lbhdc commented Feb 21, 2023

@eko I was able to get this working in 6.0.0, but its fragile.

The label of the image must be image and images will resolve if the start with @.

So something like this

image: "@//foo/bar:image"

In places where I was building multiple images in a single BUILD file, I had to break them up to accommodate this restriction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants