Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Free Camel K from runtime dependency #4025

Merged
merged 41 commits into from
Mar 13, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
a6afc74
feat(trait): builder enable pod strategy
squakez Jan 12, 2023
b719856
feat(trait): Quarkus Native to spin off a Builder Pod
squakez Jan 12, 2023
58bb96b
feat(build): use mvnw
squakez Jan 13, 2023
f2252d2
feat(build): enable maven wrapper for build
squakez Jan 17, 2023
154d35b
feat (build): remove maven dependency
squakez Jan 17, 2023
13beb31
feat(build): create a builder image on the fly
squakez Jan 19, 2023
ad352a1
feat(controller): CamelCatalog reconciliation
squakez Jan 20, 2023
0093a95
feat(controller): CamelCatalog reconciliation
squakez Jan 25, 2023
51aad23
chore(catalog): ping the registry to verify
squakez Jan 27, 2023
7bc102b
fix: wait for catalog
squakez Jan 30, 2023
e366941
feat(trait): enable any runtime version
squakez Jan 31, 2023
d43bb6e
chore: go mod tidy
squakez Feb 2, 2023
72ce537
chore: rename test func
squakez Feb 2, 2023
30cfdf6
feat(install): use a PVC to share artifacts among concurrent builds
squakez Feb 13, 2023
9555c8a
chore: make lint happy
squakez Feb 14, 2023
43e3384
feat(operator): enable storage configuration
squakez Feb 14, 2023
b73e751
chore(ci): prepare Kind for test
squakez Feb 14, 2023
040966c
chore(e2e): adjust values for testing
squakez Feb 17, 2023
2529cbe
chore: remove need for root
squakez Feb 20, 2023
3b14e58
fix: set builder security context
squakez Feb 28, 2023
34d4749
chore: builder routine strategy test
squakez Feb 28, 2023
5b5ddd9
chore: reuse kit test
squakez Feb 28, 2023
90e3b72
fix(e2e): metrics cannot access build log
squakez Feb 28, 2023
9d22bd4
chore(e2e): catalog reconciliation test
squakez Feb 28, 2023
4cd697c
feat(build): builder Pods run in operator namespace
squakez Mar 1, 2023
d7dbbe9
feat(cmd): add --no-storage installation option
squakez Mar 2, 2023
59500d5
fix(builder): global catalog reconciliation
squakez Mar 2, 2023
767b4b6
fix(e2e): olm test reorganization
squakez Mar 2, 2023
2e36511
fix(builder): Kaniko requires root privileges
squakez Mar 2, 2023
2664ede
chore: minor naming adjustments
squakez Mar 3, 2023
49a0b87
chore: exit on olm test fail
squakez Mar 3, 2023
10626fe
fix(install): add storage for kustomize
squakez Mar 6, 2023
acf9e51
fix(olm): support storage
squakez Mar 6, 2023
f64c206
chore(olm): install PVC before operator
squakez Mar 7, 2023
b38120f
chore: olm default ephemeral storage
squakez Mar 7, 2023
00ff68f
fix(install): kustomize patches pvc
squakez Mar 8, 2023
654efc3
fix(e2e): account for reconciliation requeues
squakez Mar 8, 2023
6d25ace
fix: operator pod to own global builds
squakez Mar 8, 2023
376f5af
fix(e2e): tweaking tests
squakez Mar 9, 2023
75fc8b1
fix(e2e): catalog builder tweaking
squakez Mar 9, 2023
4036bb8
chore: include info in test dump
squakez Mar 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
# limitations under the License.
# ---------------------------------------------------------------------------

name: e2e-upgrade
description: 'End-to-End tests for upgrading to new version'
name: e2e-install-olm
description: 'End-to-End tests for OLM installation and upgrade procedure'

inputs:
cluster-config-data:
Expand Down Expand Up @@ -105,7 +105,7 @@ runs:
shell: bash
run: |
# Note different parameters due to alternative installation
./.github/actions/e2e-upgrade/exec-tests.sh \
./.github/actions/e2e-install-olm/exec-tests.sh \
-b "${{ steps.released-kamel-cli.outputs.released-kamel-binary }}" \
-d "${{ steps.build-kamel.outputs.build-bundle-image-bundle-index }}" \
-l "${{ steps.config-cluster.outputs.cluster-image-registry-pull-host }}" \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -128,4 +128,4 @@ export KAMEL_K_TEST_OPERATOR_CURRENT_IMAGE=${CUSTOM_IMAGE}:${CUSTOM_VERSION}
export CAMEL_K_TEST_SAVE_FAILED_TEST_NAMESPACE=${SAVE_FAILED_TEST_NS}

# Then run integration tests
DO_TEST_PREBUILD=false GOTESTFMT="-json 2>&1 | gotestfmt" make test-install-upgrade
DO_TEST_PREBUILD=false GOTESTFMT="-json 2>&1 | gotestfmt" make test-install-olm
10 changes: 5 additions & 5 deletions .github/workflows/install.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ concurrency:
cancel-in-progress: true

jobs:
install-it:
non-olm:
runs-on: ubuntu-20.04
steps:
- name: Checkout code
Expand All @@ -87,13 +87,13 @@ jobs:
-q "${{ github.event.inputs.log-level }}" \
-t "${{ github.event.inputs.test-filters }}"

- name: Install Tests
- name: Install and upgrade test (OLM)
uses: ./.github/actions/e2e-install
with:
cluster-config-data: ${{ secrets.E2E_CLUSTER_CONFIG }}
cluster-kube-config-data: ${{ secrets.E2E_KUBE_CONFIG }}

upgrade-it:
olm:
runs-on: ubuntu-20.04

steps:
Expand All @@ -111,8 +111,8 @@ jobs:
-q "${{ github.event.inputs.log-level }}" \
-t "${{ github.event.inputs.test-filters }}"

- name: Upgrade Tests
uses: ./.github/actions/e2e-upgrade
- name: Install and upgrade test (OLM)
uses: ./.github/actions/e2e-install-olm
with:
cluster-config-data: ${{ secrets.E2E_CLUSTER_CONFIG }}
cluster-kube-config-data: ${{ secrets.E2E_KUBE_CONFIG }}
4 changes: 2 additions & 2 deletions .github/workflows/native.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ concurrency:
cancel-in-progress: true

jobs:
install-native-high-memory:
higher-memory:
if: contains(github.event.pull_request.labels.*.name, 'trigger native test')
runs-on: macos-12

Expand All @@ -87,7 +87,7 @@ jobs:
cluster-kube-config-data: ${{ secrets.E2E_KUBE_CONFIG }}
high-memory: 'true'

install-native:
lower-memory:
if: contains(github.event.pull_request.labels.*.name, 'trigger native test')
runs-on: macos-12

Expand Down
14 changes: 7 additions & 7 deletions addons/strimzi/duck/client/internalclientset/fake/register.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 7 additions & 7 deletions addons/strimzi/duck/client/internalclientset/scheme/register.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

45 changes: 27 additions & 18 deletions build/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,34 +13,43 @@
# See the License for the specific language governing permissions and
# limitations under the License.

FROM quay.io/quarkus/ubi-quarkus-mandrel:22.2.0.0-Final-java11
FROM eclipse-temurin:17-jdk

ARG MAVEN_VERSION="3.8.6"
ARG MAVEN_DEFAULT_VERSION="3.8.6"
ARG MAVEN_HOME="/usr/share/maven"
ARG SHA="f790857f3b1f90ae8d16281f902c689e4f136ebe584aba45e4b1fa66c80cba826d3e0e52fdd04ed44b4c66f6d3fe3584a057c26dfcac544a60b301e6d0f91c26"
ARG BASE_URL="https://archive.apache.org/dist/maven/maven-3/${MAVEN_VERSION}/binaries"
ARG MAVEN_DIST_URL="https://archive.apache.org/dist/maven/maven-3/${MAVEN_DEFAULT_VERSION}/binaries/apache-maven-${MAVEN_DEFAULT_VERSION}-bin.zip"
ARG MVNW_DIR="/usr/share/maven/mvnw/"

USER 0

RUN mkdir -p ${MAVEN_HOME} \
&& curl -Lso /tmp/maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz \
&& echo "${SHA} /tmp/maven.tar.gz" | sha512sum -c - \
&& tar -xzC ${MAVEN_HOME} --strip-components=1 -f /tmp/maven.tar.gz \
&& rm -v /tmp/maven.tar.gz \
&& ln -s ${MAVEN_HOME}/bin/mvn /usr/bin/mvn \
&& rm ${MAVEN_HOME}/lib/maven-slf4j-provider*

ADD build/_maven_output /tmp/artifacts/m2
ADD build/_kamelets /kamelets
# Maven configuration
RUN mkdir -p ${MAVEN_HOME}
COPY build/_maven_overlay/ /usr/share/maven/lib/
ADD build/_maven_overlay/logback.xml /usr/share/maven/conf/

ENV MAVEN_OPTS="${MAVEN_OPTS} -Dlogback.configurationFile=/usr/share/maven/conf/logback.xml"
# Prepare the maven wrapper required in the build Pod strategy
COPY build/mvnw/mvnw.tar ${MVNW_DIR}
RUN tar -xC ${MVNW_DIR} -f ${MVNW_DIR}mvnw.tar \
&& rm -v ${MVNW_DIR}mvnw.tar \
&& sed -i "s;distributionUrl=.*;distributionUrl=${MAVEN_DIST_URL};" ${MVNW_DIR}.mvn/wrapper/maven-wrapper.properties
# Used by mvnw to download maven dist into it
ENV MAVEN_USER_HOME="${MAVEN_HOME}"
# Install a default mvnw distribution at build time and prepare the config for formatting log
RUN ${MVNW_DIR}/mvnw --version | grep "Maven home:" | sed 's/Maven home: //' >> ${MVNW_DIR}default \
&& cp -r /usr/share/maven/lib/. $(cat ${MVNW_DIR}default)/lib \
&& rm $(cat ${MVNW_DIR}default)/lib/maven-slf4j-provider*
ENV MAVEN_OPTS="${MAVEN_OPTS} -Dlogback.configurationFile=${MAVEN_HOME}/conf/logback.xml"

ADD build/_maven_output /tmp/local/m2
ADD build/_kamelets /kamelets

RUN chgrp -R 0 /tmp/artifacts/m2 \
&& chmod -R g=u /tmp/artifacts/m2 \
RUN mkdir -p /etc/maven/m2 \
&& chgrp -R 0 /etc/maven/m2 \
&& chmod -R g=u /etc/maven/m2 \
&& chgrp -R 0 /kamelets \
&& chmod -R g=u /kamelets
&& chmod -R g=u /kamelets \
&& chgrp -R 0 ${MAVEN_HOME} \
&& chmod -R g=u ${MAVEN_HOME}

USER 1000

Expand Down
Binary file added build/mvnw/mvnw.tar
Binary file not shown.
3 changes: 1 addition & 2 deletions cmd/util/json-schema-gen/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"os"
"reflect"
"strings"
Expand Down Expand Up @@ -99,7 +98,7 @@ func generate(crdFilename, dslFilename, path string, isArray bool, destination s
if err != nil {
return err
}
return ioutil.WriteFile(destination, result, 0o600)
return os.WriteFile(destination, result, 0o600)
}

func remapRef(ref string) string {
Expand Down
3 changes: 1 addition & 2 deletions cmd/util/vfs-gen/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ package main
import (
"flag"
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
Expand Down Expand Up @@ -144,7 +143,7 @@ limitations under the License.
var finalContent []byte
finalContent = append(finalContent, []byte(header)...)
finalContent = append(finalContent, content...)
if err := ioutil.WriteFile(resourceFile, finalContent, 0o600); err != nil {
if err := os.WriteFile(resourceFile, finalContent, 0o600); err != nil {
log.Fatalln(err)
}
}
Expand Down
7 changes: 7 additions & 0 deletions config/crd/bases/camel.apache.org_builds.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,10 @@ spec:
spec:
description: BuildSpec defines the Build operation to be executed
properties:
operatorNamespace:
description: The namespace of the operator in charge of this Build
reconciliation.
type: string
strategy:
description: The strategy that should be used to perform the Build.
enum:
Expand Down Expand Up @@ -654,6 +658,9 @@ spec:
canceled, and its phase set to BuildPhaseFailed.
format: duration
type: string
toolImage:
description: The container image to be used to run the build.
type: string
type: object
status:
description: BuildStatus defines the observed state of Build
Expand Down
50 changes: 50 additions & 0 deletions config/crd/bases/camel.apache.org_camelcatalogs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,10 @@ spec:
jsonPath: .spec.runtime.provider
name: Runtime Provider
type: string
- description: The catalog phase
jsonPath: .status.phase
name: Phase
type: string
name: v1
schema:
openAPIV3Schema:
Expand Down Expand Up @@ -398,6 +402,52 @@ spec:
type: object
status:
description: the actual state of the catalog
properties:
conditions:
description: a list of events happened for the CamelCatalog
items:
description: CamelCatalogCondition describes the state of a resource
at a certain point.
properties:
lastTransitionTime:
description: Last time the condition transitioned from one status
to another.
format: date-time
type: string
lastUpdateTime:
description: The last time this condition was updated.
format: date-time
type: string
message:
description: A human-readable message indicating details about
the transition.
type: string
reason:
description: The reason for the condition's last transition.
type: string
status:
description: Status of the condition, one of True, False, Unknown.
type: string
type:
description: Type of CamelCatalog condition.
type: string
required:
- status
- type
type: object
type: array
image:
description: the container image available for building an application
with this catalog
type: string
observedGeneration:
description: ObservedGeneration is the most recent generation observed
for this Catalog.
format: int64
type: integer
phase:
description: the actual phase
type: string
type: object
type: object
served: true
Expand Down
4 changes: 4 additions & 0 deletions config/crd/bases/camel.apache.org_integrationkits.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,10 @@ spec:
items:
type: string
type: array
strategy:
description: The strategy to use, either `pod` or `routine`
(default routine)
type: string
verbose:
description: Enable verbose logging on build components that
support it (e.g. Kaniko build pod).
Expand Down
8 changes: 8 additions & 0 deletions config/crd/bases/camel.apache.org_integrationplatforms.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -407,6 +407,10 @@ spec:
items:
type: string
type: array
strategy:
description: The strategy to use, either `pod` or `routine`
(default routine)
type: string
verbose:
description: Enable verbose logging on build components that
support it (e.g. Kaniko build pod).
Expand Down Expand Up @@ -1990,6 +1994,10 @@ spec:
items:
type: string
type: array
strategy:
description: The strategy to use, either `pod` or `routine`
(default routine)
type: string
verbose:
description: Enable verbose logging on build components that
support it (e.g. Kaniko build pod).
Expand Down
4 changes: 4 additions & 0 deletions config/crd/bases/camel.apache.org_integrations.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6184,6 +6184,10 @@ spec:
items:
type: string
type: array
strategy:
description: The strategy to use, either `pod` or `routine`
(default routine)
type: string
verbose:
description: Enable verbose logging on build components that
support it (e.g. Kaniko build pod).
Expand Down
4 changes: 4 additions & 0 deletions config/crd/bases/camel.apache.org_kameletbindings.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6464,6 +6464,10 @@ spec:
items:
type: string
type: array
strategy:
description: The strategy to use, either `pod` or `routine`
(default routine)
type: string
verbose:
description: Enable verbose logging on build components
that support it (e.g. Kaniko build pod).
Expand Down
8 changes: 8 additions & 0 deletions config/manager/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,18 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- operator-pvc.yaml
- operator-deployment.yaml
- operator-service-account.yaml

patchesStrategicMerge:
- patch-toleration.yaml
- patch-node-selector.yaml
- patch-resource-requirements.yaml
patches:
- path: patch-log-level.yaml
target:
kind: Deployment
- path: patch-install-default-pvc.yaml
target:
kind: Deployment
1 change: 1 addition & 0 deletions config/manager/operator-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -78,3 +78,4 @@ spec:
port: 8081
initialDelaySeconds: 20
periodSeconds: 10

Loading