Skip to content

Commit

Permalink
feat: promote with traits
Browse files Browse the repository at this point in the history
  • Loading branch information
squakez committed Sep 21, 2024
1 parent 2ecb2b2 commit 3a52c35
Show file tree
Hide file tree
Showing 6 changed files with 233 additions and 15 deletions.
13 changes: 13 additions & 0 deletions docs/modules/ROOT/pages/pipes/promoting.adoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
[[promoting-pipes]]
= Promoting Pipes across environments

As soon as you have an Pipes running in your cluster, you will be challenged to move that Pipe to an higher environment. Ie, you can test your Pipe in a **development** environment, and, as soon as you're happy with the result, you will need to move it into a **production** environment.

[[cli-promote]]
== CLI `promote` command

You may already be familiar with this command as seen when xref:running/promoting.adoc[promoting Integrations across environments]. The command is smart enough to detect when you want to promote a Pipe or an Integration and it works exactly in the same manner.
Expand Down Expand Up @@ -49,3 +51,14 @@ status: {}
```

As you may already have seen with the Integration example, also here the Pipe is reusing the very same container image. From a release perspective we are guaranteeing the **immutability** of the Pipe as the container used is exactly the same of the one we have tested in development (what we change are just the configurations, if any).

[[traits]]
== Moving traits

NOTE: this feature is available starting from version 2.5

When you use the `promote` subcommand, you're also keeping the status of any configured trait along with the new promoted Pipe. The tool is in fact in charge to recover the trait configuration of the source Pipe and port it over to the new Pipe promoted.

This is particularly nice when you have certain traits which are requiring the scan the source code (for instance, Service trait). In this way, when you promote the new Pipe, the traits will be automatically configured to copy any parameter, replicating the very exact behavior between the source and destination environment.

With this approach, you won't need to worry any longer about any trait which was requiring the source to be attached in order to automatically scan for features.
12 changes: 12 additions & 0 deletions docs/modules/ROOT/pages/running/promoting.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

As soon as you have an Integration running in your cluster, you will be challenged to move that Integration to an higher environment. Ie, you can test your Integration in a **development** environment, and, as soon as you're happy with the result, you will need to move it into a **production** environment.

[[cli-promote]]
== CLI `promote` command

Camel K has an opinionated way to achieve the promotion goal through the usage of `kamel promote` command. With this command you will be able to easily move an Integration from one namespace to another without worrying about any low level detail such as resources needed by the Integration. You only need to make sure that both the source operator and the destination operator are using the same container registry and that the destination namespace provides the required Configmaps, Secrets or Kamelets required by the Integration.
Expand Down Expand Up @@ -52,3 +53,14 @@ hello, I am production!
Something nice is that since the Integration is reusing the very same container image, the execution of the new application will be immediate. Also from a release perspective we are guaranteeing the **immutability** of the Integration as the container used is exactly the same of the one we have tested in development (what we change are just the configurations).

Please notice that the Integration running in test is not altered in any way and will be running until any user will stop it.

[[traits]]
== Moving traits

NOTE: this feature is available starting from version 2.5

When you use the `promote` subcommand, you're also keeping the status of any configured trait along with the new promoted Integration. The tool is in fact in charge to recover the trait configuration of the source Integration and port it over to the new Integration promoted.

This is particularly nice when you have certain traits which are requiring the scan the source code (for instance, Service trait). In this way, when you promote the new Integration, the traits will be automatically configured to copy any parameter, replicating the very exact behavior between the source and destination environment.

With this approach, you won't need to worry any longer about any trait which was requiring the source to be attached in order to automatically scan for features.
36 changes: 36 additions & 0 deletions pkg/apis/camel/v1/pipe_types_support.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import (
"encoding/json"
"fmt"

scase "github.com/stoewer/go-strcase"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
Expand Down Expand Up @@ -70,6 +71,41 @@ func (in *Pipe) SetOperatorID(operatorID string) {
SetAnnotation(&in.ObjectMeta, OperatorIDAnnotation, operatorID)
}

// SetTrait converts a trait into the related annotation.
func (in *Pipe) SetTraits(traits *Traits) error {
var mappedTraits map[string]map[string]interface{}
data, err := json.Marshal(traits)
if err != nil {
return err
}
err = json.Unmarshal(data, &mappedTraits)
if err != nil {
return err
}

addons := mappedTraits["addons"]
delete(mappedTraits, "addons")
if in.Annotations == nil && (len(mappedTraits) > 0 || len(addons) > 0) {
in.Annotations = make(map[string]string)
}
for id, trait := range mappedTraits {
for k, v := range trait {
in.Annotations[fmt.Sprintf("%s%s.%s", TraitAnnotationPrefix, id, scase.KebabCase(k))] = fmt.Sprintf("%v", v)
}
}
for id, trait := range addons {
castedMap, ok := trait.(map[string]interface{})
if !ok {
return fmt.Errorf("could not cast trait addon %v", trait)
}
for k, v := range castedMap {
in.Annotations[fmt.Sprintf("%s%s.%s", TraitAnnotationPrefix, id, scase.KebabCase(k))] = fmt.Sprintf("%v", v)
}
}

return nil
}

// GetCondition returns the condition with the provided type.
func (in *PipeStatus) GetCondition(condType PipeConditionType) *PipeCondition {
for i := range in.Conditions {
Expand Down
45 changes: 45 additions & 0 deletions pkg/apis/camel/v1/pipe_types_support_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,10 @@ import (
"encoding/json"
"testing"

"github.com/apache/camel-k/v2/pkg/apis/camel/v1/trait"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"k8s.io/utils/ptr"
)

func TestNumberConversion(t *testing.T) {
Expand All @@ -46,3 +48,46 @@ func TestNumberConversion(t *testing.T) {
assert.Equal(t, "123.123", res["float32"])
assert.Equal(t, "1111123.123", res["float64"])
}

func TestSetTraits(t *testing.T) {
traits := Traits{
Affinity: &trait.AffinityTrait{
Trait: trait.Trait{
Enabled: ptr.To(true),
},
PodAffinity: ptr.To(true),
},
Addons: map[string]AddonTrait{
"master": toAddonTrait(t, map[string]interface{}{
"enabled": true,
"resourceName": "test-lock",
"labelKey": "test-label",
"labelValue": "test-value",
}),
},
Knative: &trait.KnativeTrait{
Trait: trait.Trait{
Enabled: ptr.To(true),
},
ChannelSources: []string{
"channel-a", "channel-b",
},
},
}

expectedAnnotations := map[string]string(map[string]string{
"trait.camel.apache.org/affinity.enabled": "true",
"trait.camel.apache.org/affinity.pod-affinity": "true",
"trait.camel.apache.org/knative.channel-sources": "[channel-a channel-b]",
"trait.camel.apache.org/knative.enabled": "true",
"trait.camel.apache.org/master.enabled": "true",
"trait.camel.apache.org/master.label-key": "test-label",
"trait.camel.apache.org/master.label-value": "test-value",
"trait.camel.apache.org/master.resource-name": "test-lock",
})

pipe := NewPipe("my-pipe", "my-ns")
err := pipe.SetTraits(&traits)
assert.NoError(t, err)
assert.Equal(t, expectedAnnotations, pipe.Annotations)
}
51 changes: 36 additions & 15 deletions pkg/cmd/promote.go
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,10 @@ func (o *promoteCmdOptions) run(cmd *cobra.Command, args []string) error {

// Pipe promotion
if promotePipe {
destPipe := o.editPipe(sourcePipe, sourceIntegration, sourceKit)
destPipe, err := o.editPipe(sourcePipe, sourceIntegration, sourceKit)
if err != nil {
return err
}
if o.OutputFormat != "" {
return showPipeOutput(cmd, destPipe, o.OutputFormat, c.GetScheme())
}
Expand Down Expand Up @@ -242,6 +245,9 @@ func (o *promoteCmdOptions) editIntegration(it *v1.Integration, kit *v1.Integrat
dstIt.Annotations = cloneAnnotations(it.Annotations, o.ToOperator)
dstIt.Labels = cloneLabels(it.Labels)
dstIt.Spec.IntegrationKit = nil
if it.Status.Traits != nil {
dstIt.Spec.Traits = *it.Status.Traits
}
if dstIt.Spec.Traits.Container == nil {
dstIt.Spec.Traits.Container = &traitv1.ContainerTrait{}
}
Expand Down Expand Up @@ -322,14 +328,40 @@ func cloneLabels(lbs map[string]string) map[string]string {
return newMap
}

func (o *promoteCmdOptions) editPipe(kb *v1.Pipe, it *v1.Integration, kit *v1.IntegrationKit) *v1.Pipe {
func (o *promoteCmdOptions) editPipe(kb *v1.Pipe, it *v1.Integration, kit *v1.IntegrationKit) (*v1.Pipe, error) {
contImage := it.Status.Image
// Pipe
dst := v1.NewPipe(o.To, kb.Name)
dst.Spec = *kb.Spec.DeepCopy()
dst.Annotations = cloneAnnotations(kb.Annotations, o.ToOperator)
dst.Labels = cloneLabels(kb.Labels)
dst.Annotations[fmt.Sprintf("%scontainer.image", v1.TraitAnnotationPrefix)] = contImage
traits := it.Status.Traits
if traits == nil {
traits = &v1.Traits{}
}
if traits.Container == nil {
traits.Container = &traitv1.ContainerTrait{}
}
traits.Container.Image = contImage
if kit != nil {
// We must provide the classpath expected for the IntegrationKit. This is calculated dynamically and
// would get lost when creating the non managed build Integration. For this reason
// we must report it in the promoted Integration.
mergedClasspath := getClasspath(kit, dst.Annotations[fmt.Sprintf("%sjvm.classpath", v1.TraitAnnotationPrefix)])
if traits.JVM == nil {
traits.JVM = &traitv1.JVMTrait{}
}
traits.JVM.Classpath = mergedClasspath
// We must also set the runtime version so we pin it to the given catalog on which
// the container image was built
if traits.Camel == nil {
traits.Camel = &traitv1.CamelTrait{}
}
traits.Camel.RuntimeVersion = kit.Status.RuntimeVersion
}
if err := dst.SetTraits(traits); err != nil {
return nil, err
}
if dst.Spec.Source.Ref != nil {
dst.Spec.Source.Ref.Namespace = o.To
}
Expand All @@ -344,18 +376,7 @@ func (o *promoteCmdOptions) editPipe(kb *v1.Pipe, it *v1.Integration, kit *v1.In
}
}

if kit != nil {
// We must provide the classpath expected for the IntegrationKit. This is calculated dynamically and
// would get lost when creating the non managed build Integration. For this reason
// we must report it in the promoted Integration.
mergedClasspath := getClasspath(kit, dst.Annotations[fmt.Sprintf("%sjvm.classpath", v1.TraitAnnotationPrefix)])
dst.Annotations[fmt.Sprintf("%sjvm.classpath", v1.TraitAnnotationPrefix)] = mergedClasspath
// We must also set the runtime version so we pin it to the given catalog on which
// the container image was built
dst.Annotations[fmt.Sprintf("%scamel.runtime-version", v1.TraitAnnotationPrefix)] = kit.Status.RuntimeVersion
}

return &dst
return &dst, nil
}

func (o *promoteCmdOptions) replaceResource(res k8sclient.Object) (bool, error) {
Expand Down
91 changes: 91 additions & 0 deletions pkg/cmd/promote_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import (
"testing"

v1 "github.com/apache/camel-k/v2/pkg/apis/camel/v1"
"github.com/apache/camel-k/v2/pkg/apis/camel/v1/trait"
"github.com/apache/camel-k/v2/pkg/platform"
"github.com/apache/camel-k/v2/pkg/util/defaults"
"github.com/apache/camel-k/v2/pkg/util/test"
Expand All @@ -30,6 +31,7 @@ import (
"github.com/stretchr/testify/require"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/utils/ptr"
)

const cmdPromote = "promote"
Expand Down Expand Up @@ -371,3 +373,92 @@ spec:
status: {}
`, output)
}

func TestIntegrationWithSavedTraitsDryRun(t *testing.T) {
srcPlatform := v1.NewIntegrationPlatform("default", platform.DefaultPlatformName)
srcPlatform.Status.Version = defaults.Version
srcPlatform.Status.Build.RuntimeVersion = defaults.DefaultRuntimeVersion
srcPlatform.Status.Phase = v1.IntegrationPlatformPhaseReady
dstPlatform := v1.NewIntegrationPlatform("prod-namespace", platform.DefaultPlatformName)
dstPlatform.Status.Version = defaults.Version
dstPlatform.Status.Build.RuntimeVersion = defaults.DefaultRuntimeVersion
dstPlatform.Status.Phase = v1.IntegrationPlatformPhaseReady
defaultIntegration, defaultKit := nominalIntegration("my-it-test")
defaultIntegration.Status.Traits = &v1.Traits{
Service: &trait.ServiceTrait{
Trait: trait.Trait{
Enabled: ptr.To(true),
},
},
}
srcCatalog := createTestCamelCatalog(srcPlatform)
dstCatalog := createTestCamelCatalog(dstPlatform)

promoteCmdOptions, promoteCmd, _ := initializePromoteCmdOptions(t, &srcPlatform, &dstPlatform, &defaultIntegration, &defaultKit, &srcCatalog, &dstCatalog)
output, err := test.ExecuteCommand(promoteCmd, cmdPromote, "my-it-test", "--to", "prod-namespace", "-o", "yaml", "-n", "default")
assert.Equal(t, "yaml", promoteCmdOptions.OutputFormat)
require.NoError(t, err)
assert.Equal(t, `apiVersion: camel.apache.org/v1
kind: Integration
metadata:
creationTimestamp: null
name: my-it-test
namespace: prod-namespace
spec:
traits:
camel:
runtimeVersion: 1.2.3
container:
image: my-special-image
jvm:
classpath: /path/to/artifact-1/*:/path/to/artifact-2/*
service:
enabled: true
status: {}
`, output)
}

func TestPipeWithSavedTraitsDryRun(t *testing.T) {
srcPlatform := v1.NewIntegrationPlatform("default", platform.DefaultPlatformName)
srcPlatform.Status.Version = defaults.Version
srcPlatform.Status.Build.RuntimeVersion = defaults.DefaultRuntimeVersion
srcPlatform.Status.Phase = v1.IntegrationPlatformPhaseReady
dstPlatform := v1.NewIntegrationPlatform("prod-namespace", platform.DefaultPlatformName)
dstPlatform.Status.Version = defaults.Version
dstPlatform.Status.Build.RuntimeVersion = defaults.DefaultRuntimeVersion
dstPlatform.Status.Phase = v1.IntegrationPlatformPhaseReady
defaultKB := nominalPipe("my-kb-test")
defaultKB.Annotations = map[string]string{
"camel.apache.org/operator.id": "camel-k",
"my-annotation": "my-value",
}
defaultKB.Labels = map[string]string{
"my-label": "my-value",
}
defaultIntegration, defaultKit := nominalIntegration("my-kb-test")
srcCatalog := createTestCamelCatalog(srcPlatform)
dstCatalog := createTestCamelCatalog(dstPlatform)

promoteCmdOptions, promoteCmd, _ := initializePromoteCmdOptions(t, &srcPlatform, &dstPlatform, &defaultKB, &defaultIntegration, &defaultKit, &srcCatalog, &dstCatalog)
output, err := test.ExecuteCommand(promoteCmd, cmdPromote, "my-kb-test", "--to", "prod-namespace", "-o", "yaml", "-n", "default")
assert.Equal(t, "yaml", promoteCmdOptions.OutputFormat)
require.NoError(t, err)
assert.Equal(t, `apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
annotations:
my-annotation: my-value
trait.camel.apache.org/camel.runtime-version: 1.2.3
trait.camel.apache.org/container.image: my-special-image
trait.camel.apache.org/jvm.classpath: /path/to/artifact-1/*:/path/to/artifact-2/*
creationTimestamp: null
labels:
my-label: my-value
name: my-kb-test
namespace: prod-namespace
spec:
sink: {}
source: {}
status: {}
`, output)
}

0 comments on commit 3a52c35

Please sign in to comment.