Redesign scope of xk6-kubernetes API

[pablochacin]

The xk6-kubernetes extension was initially developed with the intention of has been used for chaos experiments, providing basic functionalities such as deleting resources (e.g. namespaces, secrets) and running work loads (for example, jobs). It also provided some other basic functions for creating resources but adding little or no additional abstractions over the native k8s api.

The functions provided by xk6-kubernetes can also be used when setting up tests. However, some common tasks such as exposing a deployment as a service require considerable boiler plate code and can became tedious to program.

At the same time, as the requirements for running chaos experiments has become more complex, some specialized functions has been added, such as attaching ephemeral containers to running pods, or executing commands in a container.

It seems clear that the current design of xk6-kubernetes mixes functions that are too generic for been useful when setting up tests or too specialized for been in a general purpose k8s extension. Therefore the following changes are proposed:

1. Focus on providing helpers that abstract common tasks such as the proposed in Create Helpers for exposing pod/deployment in a cluster #62
2. Cover other requirements by providing generic apply functions for creating resources using yaml definitions.
3. Move specialized functionality introduced to support chaos experiments (ephemeral containers, command execution) to xk6-chaos

Regarding the generic apply method, as handling inline yaml can be cumbersome (in particular if we want to substitute certain fields using variables) other alternatives can be explored:

• rendering inline yaml using inline text templates (similar to using helm charts)
• generating yaml using tools such as kustomize, embedding this tool(s) into the library to prevent dependencies with external binaries in the test environment.

Edit:
The generic interface could also be provided by using using a generic golang client that handles resources as generic structs which should be easily mapped to javascript objects.

One potential benefit of using a generic apply method is the possibility of removing the dependencies to many of the k8s api packages, reducing the size of the extension.

#66 (comment)

[javaducky]

I'm not sure if the xk6-kubernetes extension was initially created for chaos experiments, but was definitely well-suited for it. I personally see xk6-kubernetes as a way to deploy an isolated service for any reliability testing, whether that be simply load testing or chaos testing...at least until we possibly create xk6-terraform :)

1. Focus on providing helpers that abstract common tasks such as the proposed in Create Helpers for exposing pod/deployment in a cluster #62

Yes...the deployment is rather ugly given the necessity of creating multiple resources in separate calls. A helper method to ease this would be welcome for sure.

2. Cover other requirements by providing generic apply functions for creating resources using yaml definitions.

Just wanted to note the original contributor mentioned that he added the apply functionality because attempts at using the JavaScript syntax for creating some of the resources were unsuccessful.

💯 to using Generics somehow to have a single apply method without regard to the resource type(s) included within the YAML.

From your later mention about incorporating tools to allow variable substitutions in YAML, I could see using something like Kustomize or Ytt, or even Kapp for deploying. (Caveat: I have not researched these tools to know enough about how well we could incorporate them into xk6-kubernetes)

3. Move specialized functionality introduced to support chaos experiments (ephemeral containers, command execution) to xk6-chaos

My only concern is the duplication of Kubernetes base functionality. If xk6-chaos could somehow inherit and build upon the capabilities of xk6-kubernetes would be ideal IMO.

[pablochacin]

I'm not sure if the xk6-kubernetes extension was initially created for chaos experiments

That is what I understood, but this is not really relevant so I'm modifying the text.

I personally see xk6-kubernetes as a way to deploy an isolated service for any reliability testing

I agree. This is mentioned in my comment. The intention is to make this the primary use case instead of just a generic wrapper over basic k8s api.

100 to using Generics somehow to have a single apply method without regard to the resource type(s) included within the YAML.

My concern here is that providing yaml text from javascript is not really convenient. I would like to explore other alternatives more idiomatic to javascript (even if providing yaml is also supported).

I could see using something like Kustomize or Ytt, or even Kapp for deploying.

My intention is not to use external tools, but to integrate them into the library to avoid having dependencies of other binaries in the node running the tests. For instance, kustomize can be used as a library. I modified the text to reflect this better.

My only concern is the duplication of Kubernetes base functionality. If xk6-chaos could somehow inherit and build upon the capabilities of xk6-kubernetes would be ideal IMO.

That is the idea. Otherwise, I think they should remain in xk6-kubernetes

[dgzlopes]

I'm not sure if the xk6-kubernetes extension was initially created for chaos experiments

It was :D

I'm totally in line with this proposal! Although, as I discussed with Pablo, a design doc showcasing the end-user experience of this extension (as we are doing with Chaos) would be pretty nice.

Also, I think this process of making the extension leaner and smaller, plus creating more useful abstractions (instead of wrapping all the raw Kubernetes stuff) will get us closer to having this into Core k6 at some point. That way, anyone can spawn a new deployment, clone an existing one or apply some Kustomizations before running their tests.

[pablochacin]

a design doc showcasing the end-user experience of this extension (as we are doing with Chaos) would be pretty nice.

Good point, I will do it!

[javaducky]

@pablochacin

My concern here is that providing yaml text from javascript is not really convenient.

I didn't mean to imply that I was in favor of defining YAML within the JavaScript, however, being able to possibly load from a YAML file would be nice.

I'll look forward to your design thoughts!

[pablochacin]

The generic API including the method Apply for creating resources from a YAML are covered in #70

[pablochacin]

The helpers are introduced in #72

[pablochacin]

Specialized functions such as attaching ephemeral containers have been moved to the xk6-disruptor extension

[pablochacin]

@javaducky can we close this issue? I think we have covered all the scoped of the changes proposed here.

[javaducky]

@pablochacin sounds good to me!