Support Antrea installation using antctl CLI

[antoninbas]

Describe what you are trying to solve
While we were updating the ROADMAP, @tnqn suggested that we could add support for a new installation method for Antrea.
We currently support 2 methods:

1. using the K8s manifest directly (e.g, https://github.com/antrea-io/antrea/releases/download/v1.14.1/antrea.yml)
2. using the antrea/antrea Helm chart (https://github.com/antrea-io/antrea/blob/main/docs/helm.md)

The 1st method is very straightforward and the only prerequisite is kubectl. However, any customization has to be done manually (by editing the YAML) and is error-prone. It is fairly common for users to make a typo when editing the antrea config in the YAML for example (invalid indentation, etc.). Some features also require editing both antrea-agent.conf and antrea-controller.conf, and it is easy for users to miss that.

The 2nd method requires Helm to be installed, but allows for easy customization. Of course, we are constrained by what Helm can do:

• cannot upgrade components in a specific order (Controller first, then Agents)
• CRD management is not always straightforward: [Helm chart] Should CRDs be placed in the special crds/ directory? #3665
• cannot run custom validations / sanity-checks pre-installation
• cannot run custom validations or logic post-installation

Describe the solution you have in mind
Several other projects in the cloud-native ecosystem support CLI-based installation (Cilium and Calico come to mind).
Users can easily customize the installation, and developers have the opportunity to execute custom logic before and after the installation, with direct access to the K8s API.
This proposal is to add the antctl install command, which would take care of installing an "arbitrary" Antrea version (we should support some version skew between antctl and the Antrea version being installed). Similarly, we would add the "antctl upgrade" command for upgrades.

Describe how your solution impacts user flows
Users have access to a new installation method, which should be as convenient as installing the Helm chart, while at the same time providing additional value / features.

Describe the main design/architecture of your solution
This is a possible workflow / implementation for antctl install [--version x.y.z]:

1. Download the Helm chart for Antrea (latest release or version provided by user)
2. Run some sanity checks (is the K8s version supported, is there another CNI already installed, ...)
3. Render the Helm template based on customization values provided by user
4. Extract individual resources from the generated manifest, and apply them in the correct order using the K8s API (like kubectl would)
5. Run post-installation logic (e.g., restart existing Pods) and custom validations (basic connectivity checks)

The CLI command could take care of CNI migration under the hood (https://github.com/antrea-io/antrea/blob/main/docs/migrate-to-antrea.md), if another CNI was detected in step 2.

Note that we use Helm to render the chart template into a K8s manifest, but we create resources through the K8s API. This is the only way to meet our goal of installing / updating components in a specific order.

antctl install could also take care of (optionally) installing other Antrea components (FlowAggregator, antrea-ui).

Implementation for antctl upgrade would be similar.

Ideally, antctl would also support Windows installation, but that may come later.

Alternative solutions that you considered
An alternative solution could be to make the Antrea operator a "first-class citizen". The operator would take care of the actual lifecycle management of Antrea components & resources. In that case antctl install would simply take care of installing the operator by installing its Helm chart. This is a higher-impact change however, compared to the solution described above.

[antoninbas]

We have submitted this issue as a project idea for the LFX mentorship program: cncf/mentoring#1129. So no-one should ideally work on this issue until we know if the proposal is accepted and if we can match a mentee to work on it.
See https://docs.linuxfoundation.org/lfx/mentorship for more information on the program.

[peaceLT]

Hello, I am very interested in this project and would like to participate in this season of LFX. Could you please guide me on how to get started?

[antoninbas]

@peaceLT all the information is at the link I provided above: https://docs.linuxfoundation.org/lfx/mentorship

While we appreciate the enthusiasm, we'd prefer for such questions to be asked on Slack, so we can keep this issue focused on technical discussions about the proposal.

[Mayaleeeee]

Hello @antoninbas,

I'm Maya, an Open Source Product Designer, UX Researcher, Design Maintainer and Technical Steering Committee (TSC) at the AsyncAPI Initiatives.

I've explored the LFX mentorship guidelines and am genuinely intrigued by this issue. Recently, I enhanced the UI/UX of the AsyncAPI CLI, addressing both design and error message improvements.

I'd appreciate joining the Slack channel, but I couldn't locate it. Could you kindly share the link with me?

Best,
Maya

[antoninbas]

@Mayaleeeee Slack info is in the readme

[Mayaleeeee]

@Mayaleeeee Slack info is in the readme

Seen and joined the channel.
Thank you @antoninbas

[kanha-gupta]

@antoninbas This is a great opportunity to learn. do we require a proposal to submit ?

[antoninbas]

For all the folks who have applied or are considering applying to one of the Antrea projects for the LFX mentorship program, we have published instructions to complete test tasks: #5976. We will review your submissions for these tasks alongside other material (resume, cover letter) when selecting mentees. The deadline for submitting is February 20th 5PM PST.

[ambuj-1211]

Hey where to share the proposal or ideas related to this issue?
Or the task that is to be done will be the only selection criteria.

[antoninbas]

We had a discussion with @kanha-gupta and we believe that at a high level, the task should be decomposed as follows:

1. support for post-installation validations through a new antctl command (syntax TBD)

• this would be useful regardless of which installation method is used to deploy Antrea (manifest, Helm, or in the future antctl install)
• similar in spirit to cilium connectivity test, which is available for Cilium

2. support pre-installation validations through a new antctl command

• kubeadm pre-flight checks would be a good reference for this one
• some validation checks would depend on the configuration in which Antrea will be installed, so may be harder to really make it a "standalone" command

3. work on antctl install itself

• it is still unclear whether we want to go the operator route or not

[roopeshsn]

We had a discussion with @kanha-gupta and we believe that at a high level, the task should be decomposed as follows:

1. support for post-installation validations through a new antctl command (syntax TBD)

• this would be useful regardless of which installation method is used to deploy Antrea (manifest, Helm, or in the future antctl install)
• similar in spirit to cilium connectivity test, which is available for Cilium

2. support pre-installation validations through a new antctl command

• kubeadm pre-flight checks would be a good reference for this one
• some validation checks would depend on the configuration in which Antrea will be installed, so may be harder to really make it a "standalone" command

3. work on antctl install itself

• it is still unclear whether we want to go the operator route or not

I would vote for choosing the operator route.

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment, or this will be closed in 90 days

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment, or this will be closed in 90 days