Feedback: Feature Request - Pull based deployment method (GitOps style)

[pelithne]

Describe the solution you'd like.
The ability to use a pull based approach to deploying workloads into a container app environment. An example of pull based deployment method is GitOps.

In other words, to have a "client" running in the environment, which periodically monitors a git repository and automatically pulls and applies any changes made to a specific branch in that repo (for instance when a PR is merged into the main branch).

The Git repository will be the single source of truth, and will contain the descriptions of the desired state of the running application(s), and there is an automated process to make the environment match the described state in the repository.

In the case of Kubernetes, the client would apply any changes to a yaml-manifest (or helm chart). For a container app environment, a bicep file or ARM Template (or a collection of those) should be applied instead.

Some requirements:

• The time interval between synchronizations towards the repo should be configurable.
• It should also be possible to trigger a synchronization with an event.
• The client that runs in the environment needs to have permission to apply changes, which means that it needs to have an identity in Azure AD.
• Should support both open and closed git repositories.
• Git credentials should be stored in a secure manner (e.g. keyvault).

Example Scenario: Easily deploy to private environment.
When an environment is fully private, it becomes a challenge to deploy using tools like Github actions and Azure DevOps, since connectivity to the control plane is needed. This can be solved by opening up for (huge) ip ranges of the hosted agents used by Azure DevOps or Github, but this is rarely an acceptable solution. Another alternative is to use self hosted agents, in a VNET with access to the control plane, but this introduces operational cost and cost of ownership, as well as adding complexity.

A cleaner solution is to use a pull based deployment, from inside the container app environment. This removes the need to whitelist ip addresses, and/or the need to host virtual machines for self hosted agents.

Flux and ArgoCD are examples of GitOps tools, which are used for GitOps in Kubernetes.

Describe alternatives you've considered.
The other alternative already exists, with CD pipelines. Compared to a typical CD pipeline, this method has the advantage that it operates in an "inside out" manner.

Additional context.
Gitops is quickly becoming a popular solution for deploying applications and infrastructure to Kubernetes. There is reason to believe that this method of deployment will be requested for container apps as well.

[denniszielke]

+1 This would really improve the productivity for using multiple container apps.

[marcindulak]

Some more requirements:

6. possibility to temporarily disable the sync (for example when doing manual emergency deployments), see the analogous request in Provide a way to pause syncing an application argoproj/argo-cd#4808
7. support for a specific repository/branch/commit/folder to be deployed https://argo-cd.readthedocs.io/en/stable/operator-manual/declarative-setup/#applications
8. UI and cli diff support, like in helm diff or kapp diff https://carvel.dev/kapp/docs/develop/diff/. See also Kustomize and Helm diff support fluxcd/flux2#820
9. manual gating Gate after commit and before deployment fluxcd/flux2#870 (comment)
10. automated promotion between environments Best practices for promotion between clusters argoproj/argo-cd#5667

[simongottschlag]

@pelithne we are looking into building something like this until there's official support. My thought is that we will have a reconciler running as a background task in a managed environment and then have a separate RG where all other apps are created. We will have to create our own definition of an app etc that can be stored in a readable way in git.

I'll ping you when (if) we have something to try out.

[joelnotified]

+1

We've got an awesome setup using Flux with AKS today, and we'd like to see something similar in ACA if we were to consider moving (ACA looks super promising). And aside from the benefits listed above, we're really liking the speed at which we can experiment in non production environments: Just push changes to a git repo and seconds later your changes are live. Revert a commit and you're back.

[joshuadmatthews]

@pelithne ACR Tasks can handle some of these requirements. They can listen for repo changes and automatically build images and push them into a network restricted registry. In my setup I am triggering the ACR tasks manually from azure pipelines, and then once the ACR task has executed I use CLI to update the container app to the new image.

This setup allows not to open up huge ip ranges and also not needing vms.

[simongottschlag]

Hi!

I've released an initial MVP of a GitOps engine for Azure Container Apps: https://github.com/XenitAB/azcagit

Feel free to test it out. 👍

[nwmcsween]

This could also be modeled using tf-controller in fluxcd and terraform, although I don't really know the pros and cons to using tf-controller vs azcagit @simongottschlag?

[calleo]

This could also be modeled using tf-controller in fluxcd and terraform, although I don't really know the pros and cons to using tf-controller vs azcagit @simongottschlag?

Wouldn't I have to install the fluxcd operator in the ACA environment for that to work? Is that even possible?

[nwmcsween]

This could also be modeled using tf-controller in fluxcd and terraform, although I don't really know the pros and cons to using tf-controller vs azcagit @simongottschlag?

Wouldn't I have to install the fluxcd operator in the ACA environment for that to work? Is that even possible?

No. You would deploy fluxcd, install tf-controller and workload identity, create terraform ACA deployment in git, create tf-controller with workload ident creds with ref to aca git repo, a bit more complex but allows a lot more.

[simongottschlag]

This could also be modeled using tf-controller in fluxcd and terraform, although I don't really know the pros and cons to using tf-controller vs azcagit @simongottschlag?

Wouldn't I have to install the fluxcd operator in the ACA environment for that to work? Is that even possible?

No. You would deploy fluxcd, install tf-controller and workload identity, create terraform ACA deployment in git, create tf-controller with workload ident creds with ref to aca git repo, a bit more complex but allows a lot more.

Or just use azcagit 😊🤷‍♂️

[ezYakaEagle442]

Hi,
@denniszielke @dariagrigoriu could you please add this RFE in the ACA Roadmap ?
Would it be possible to leverage the AKS underlying Flux Add-on ?

Other use case related to the Flux ecosystem:

• a CI/CD Runner deployed as a self-hosted agent in AKS could be combined with Flagger to automatically upgrade the Runner container image.
• What about ACA ? Is there a way today to to automatically upgrade the Runner container image deployed in ACA ?

Thank you,
Steve ex GBB ;)

[magohl]

Anything new related to pull based GitOps for ACA?

Agree on the comments that something related to Flux would be ideal.

azcagit look great (thanks @simongottschlag) and for a small hobby project i would not hesitate to use it but for a long term commitment it does not feel ideal.

[ameya-karbonhq]

Any idea when this will be out or at least if MS can share Roadmap for future releases?

[cveld]

This could also be modeled using tf-controller in fluxcd and terraform, although I don't really know the pros and cons to using tf-controller vs azcagit @simongottschlag?

Wouldn't I have to install the fluxcd operator in the ACA environment for that to work? Is that even possible?

No. You would deploy fluxcd, install tf-controller and workload identity, create terraform ACA deployment in git, create tf-controller with workload ident creds with ref to aca git repo, a bit more complex but allows a lot more.

I don't fully understand. Would fluxcd with tf-controller already run in ACA as we know it today or would this require any evolution?

[simonjj]

@simongottschlag this is quite awesome! We're a little late in noticing, thank you very much for sharing. How do people feel about getting something like this implemented into ACA? Can you share (comment or thumbs up) if you:

• deployed @simongottschlag project into your environment
• implemented something similar on your own
• would love to have something like this

[simonjj]

yes, it took me 5 times to get this "just right"

[magohl]

@simonjj something like that implemented into ACA would absolutely have me look at ACA as an alternative to AKS in more scenarios.

[Tim-Hodge]

Have implemented a simple yaml -> container app GitOps tool using Pulumi, but would welcome something built in or a simple way to use a popular tool like Flux.

[fritzfs]

Any idea when this will be out or at least if MS can share Roadmap for future releases?

This seems to be the roadmap - https://github.com/orgs/microsoft/projects/540

[fritzfs]

This could also be modeled using tf-controller in fluxcd and terraform, although I don't really know the pros and cons to using tf-controller vs azcagit @simongottschlag?

Wouldn't I have to install the fluxcd operator in the ACA environment for that to work? Is that even possible?

No. You would deploy fluxcd, install tf-controller and workload identity, create terraform ACA deployment in git, create tf-controller with workload ident creds with ref to aca git repo, a bit more complex but allows a lot more.

I don't fully understand. Would fluxcd with tf-controller already run in ACA as we know it today or would this require any evolution?

I don't see how this can be achieved without the Microsoft effort. They already integrated flux for Azure Acr and Azure Kubernetes Service (https://learn.microsoft.com/en-gb/azure/azure-arc/kubernetes/conceptual-gitops-flux2) so let's hope they can embed it here also ...

[ameyaagashe]

@fritzfs I do not see GiOps for ACA still on MS Azure Roadmap! Help me rectify if I am wrong.

[ameyaagashe]

How do I contribute to ACA? I want to start developing a GitOps Operator for ACA. We live in a world of modern software engineering, and it isn't very pleasant not to have GitOps in ACA.