Skip to content

Commit

Permalink
Merge pull request #12 from cloudogu/feature/argocd_helm
Browse files Browse the repository at this point in the history
Feature/argocd helm
  • Loading branch information
pmarkiewka authored Apr 22, 2021
2 parents 018330e + 01e7a6a commit fc8c663
Show file tree
Hide file tree
Showing 36 changed files with 661 additions and 475 deletions.
40 changes: 19 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,8 @@ working example bundled with the complete infrastructure for a gitops deep dive.
- [Real life examples](#real-life-examples)
- [Usage](#usage)
- [Default Folder Structure](#default-folder-structure)
- [FluxV1](#fluxv1)
- [Plain-k8s](#plain-k8s)
- [Helm](#helm)
- [FluxV2](#fluxv2)
- [ArgoCD](#argocd)
- [Plain-k8s](#plain-k8s)
- [Helm](#helm)
- [GitOps-Config](#gitops-config)
- [SCM-Provider](#scm-provider)
- [Stages](#stages)
Expand Down Expand Up @@ -89,6 +86,7 @@ def gitopsConfig = [
repositoryUrl: 'fluxv1/gitops'
],
application: 'spring-petclinic',
gitopsTool: 'FLUX',
stages: [
staging: [
deployDirectly: true
Expand Down Expand Up @@ -120,6 +118,7 @@ def gitopsConfig = [
cesBuildLibVersion: <cesBuildLibVersion> /* Default: a recent cesBuildLibVersion see deployViaGitops.groovy */ ,
cesBuildLibCredentialsId: <cesBuildLibCredentialsId> /* Default: '', empty due to default public github repo */,
application: 'spring-petclinic',
gitopsTool: 'FLUX'
mainBranch: 'master' /* Default: 'main' */,
deployments: [
sourcePath: 'k8s' /* Default: 'k8s' */,
Expand Down Expand Up @@ -157,9 +156,14 @@ deployViaGitops(gitopsConfig)
### Real life examples

**FluxV1:**
* [using petclinic with helm](https://github.com/cloudogu/k8s-gitops-playground/blob/main/applications/petclinic/fluxv1/helm/Jenkinsfile)
* [using petclinic with plain-k8s](https://github.com/cloudogu/k8s-gitops-playground/blob/main/applications/petclinic/fluxv1/plain-k8s/Jenkinsfile)
* [using helm with nginx and extra resources](https://github.com/cloudogu/k8s-gitops-playground/blob/main/applications/nginx/fluxv1/Jenkinsfile)
* [using petclinic with helm and extra k8s-resources and extra files](https://github.com/cloudogu/k8s-gitops-playground/blob/main/applications/petclinic/fluxv1/helm/Jenkinsfile)
* [using petclinic with plain-k8s and extra files](https://github.com/cloudogu/k8s-gitops-playground/blob/main/applications/petclinic/fluxv1/plain-k8s/Jenkinsfile)
* [using nginx with helm and extra files](https://github.com/cloudogu/k8s-gitops-playground/blob/main/applications/nginx/fluxv1/Jenkinsfile)

**ArgoCD:**
* [using petclinic with helm and extra k8s-resources and extra files](https://github.com/cloudogu/k8s-gitops-playground/blob/main/applications/petclinic/argocd/helm/Jenkinsfile)
* [using petclinic with plain-k8s](https://github.com/cloudogu/k8s-gitops-playground/blob/main/applications/petclinic/argocd/plain-k8s/Jenkinsfile)


---

Expand Down Expand Up @@ -201,9 +205,7 @@ A default project structure in your application repo could look like the example
and/or helm resources bundled in a folder. This specific resources folder (here `k8s`) will later be specified by the
`sourcePath` within the deployments section of your `gitopsConfig`.

### FluxV1

#### Plain-k8s
### Plain-k8s
```
├── application
├── config.yamllint.yaml // not necessarily needed
Expand All @@ -217,7 +219,7 @@ and/or helm resources bundled in a folder. This specific resources folder (here
└── service.yaml
```

#### Helm
### Helm
```
├── application
├── config.yamllint.yaml // not necessarily needed
Expand All @@ -238,14 +240,6 @@ and/or helm resources bundled in a folder. This specific resources folder (here
└── values-staging.yaml
```

### FluxV2

Upcoming

### ArgoCD

Upcoming

---

## GitOps-Config
Expand All @@ -254,12 +248,16 @@ You can find a complete yet simple example [here](#examples).

**Properties**

First of all there are some mandatory properties e.g. the information about your gitops repository and the application repository.
First of all there are some mandatory properties e.g. the information about your gitops repository, the application repository and the gitops tool to be used.

```
application: 'spring-petclinic' // Name of the application. Used as a folder in GitOps repo
```

```
gitopsTool: 'ARGO' // Name of the gitops tool. Currently supporting 'FLUX' (for now only fluxV1) and 'ARGO' (for now supporting only helm charts from git repos)
```

and some optional parameters (below are the defaults) for the configuration of the dependency to the ces-build-lib or the default name for the git branch:
```
cesBuildLibRepo: 'https://github.com/cloudogu/ces-build-lib',
Expand Down
14 changes: 7 additions & 7 deletions src/com/cloudogu/gitopsbuildlib/deployment/Deployment.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,13 @@ abstract class Deployment {
def create(String stage) {
createFoldersAndCopyK8sResources(stage)
createFileConfigmaps(stage)
createPreValidation(stage)
preValidation(stage)
validate(stage)
createPostValidation(stage)
postValidation(stage)
}

abstract createPreValidation(String stage)
abstract createPostValidation(String stage)
abstract preValidation(String stage)
abstract postValidation(String stage)

def validate(String stage) {
gitopsConfig.validators.each { validatorConfig ->
Expand All @@ -36,11 +36,11 @@ abstract class Deployment {
def sourcePath = gitopsConfig.deployments.sourcePath
def application = gitopsConfig.application

script.sh "mkdir -p ${stage}/${application}/${sourcePath}/"
script.sh "mkdir -p ${stage}/${application}/extraResources/"
script.sh "mkdir -p ${configDir}/"
// copy extra resources like sealed secrets
script.echo "Copying k8s payload from application repo to gitOps Repo: '${sourcePath}/${stage}/*' to '${stage}/${application}/${sourcePath}'"
script.sh "cp -r ${script.env.WORKSPACE}/${sourcePath}/${stage}/* ${stage}/${application}/${sourcePath}/ || true"
script.echo "Copying k8s payload from application repo to gitOps Repo: '${sourcePath}/${stage}/*' to '${stage}/${application}/extraResources/'"
script.sh "cp -r ${script.env.WORKSPACE}/${sourcePath}/${stage}/* ${stage}/${application}/extraResources/ || true"
script.sh "cp ${script.env.WORKSPACE}/*.yamllint.yaml ${configDir}/ || true"
}

Expand Down
Original file line number Diff line number Diff line change
@@ -1,41 +1,59 @@
package com.cloudogu.gitopsbuildlib.deployment
package com.cloudogu.gitopsbuildlib.deployment.helm

import com.cloudogu.gitopsbuildlib.deployment.repotype.GitRepo
import com.cloudogu.gitopsbuildlib.deployment.repotype.HelmRepo
import com.cloudogu.gitopsbuildlib.deployment.repotype.RepoType
import com.cloudogu.gitopsbuildlib.deployment.Deployment
import com.cloudogu.gitopsbuildlib.deployment.helm.helmrelease.ArgoCDRelease
import com.cloudogu.gitopsbuildlib.deployment.helm.helmrelease.FluxV1Release
import com.cloudogu.gitopsbuildlib.deployment.helm.helmrelease.HelmRelease
import com.cloudogu.gitopsbuildlib.deployment.helm.repotype.GitRepo
import com.cloudogu.gitopsbuildlib.deployment.helm.repotype.HelmRepo
import com.cloudogu.gitopsbuildlib.deployment.helm.repotype.RepoType

class Helm extends Deployment {

protected RepoType helm
protected RepoType chartRepo
protected HelmRelease helmRelease

Helm(def script, def gitopsConfig) {
super(script, gitopsConfig)
if (gitopsConfig.deployments.helm.repoType == 'GIT') {
helm = new GitRepo(script)
chartRepo = new GitRepo(script)
} else if (gitopsConfig.deployments.helm.repoType == 'HELM') {
helm = new HelmRepo(script)
chartRepo = new HelmRepo(script)
}
if(gitopsConfig.gitopsTool == 'FLUX') {
helmRelease = new FluxV1Release(script)
} else if(gitopsConfig.gitopsTool == 'ARGO') {
helmRelease = new ArgoCDRelease(script)
}
}

@Override
def createPreValidation(String stage) {
def preValidation(String stage) {
def helmConfig = gitopsConfig.deployments.helm
def application = gitopsConfig.application
def sourcePath = gitopsConfig.deployments.sourcePath

// writing the merged-values.yaml via writeYaml into a file has the advantage, that it gets formatted as valid yaml
// This makes it easier to read in and indent for the inline use in the helmRelease.
// It enables us to reuse the `fileToInlineYaml` function, without writing a complex formatting logic.
script.writeFile file: "${stage}/${application}/mergedValues.yaml", text: helm.mergeValues(helmConfig, ["${script.env.WORKSPACE}/${sourcePath}/values-${stage}.yaml", "${script.env.WORKSPACE}/${sourcePath}/values-shared.yaml"] as String[])
script.writeFile file: "${stage}/${application}/mergedValues.yaml", text: chartRepo.mergeValues(helmConfig, ["${script.env.WORKSPACE}/${sourcePath}/values-${stage}.yaml", "${script.env.WORKSPACE}/${sourcePath}/values-shared.yaml"] as String[])

updateYamlValue("${stage}/${application}/mergedValues.yaml", helmConfig)
script.writeFile file: "${stage}/${application}/helmRelease.yaml", text: helm.createHelmRelease(helmConfig, application, getNamespace(stage), "${stage}/${application}/mergedValues.yaml")

script.writeFile file: "${stage}/${application}/applicationRelease.yaml", text: helmRelease.create(helmConfig, application, getNamespace(stage), "${stage}/${application}/mergedValues.yaml")

// since the values are already inline (helmRelease.yaml) we do not need to commit them into the gitops repo
script.sh "rm ${stage}/${application}/mergedValues.yaml"
}

@Override
def createPostValidation(String stage) {
def postValidation(String stage) {
def helmConfig = gitopsConfig.deployments.helm

// clean the gitrepo helm chart folder since the helmRelease.yaml ist now created
if (helmConfig.repoType == 'GIT') {
script.sh "rm -rf chart || true"
}
}

private void updateYamlValue(String yamlFilePath, Map helmConfig) {
Expand All @@ -53,28 +71,4 @@ class Helm extends Deployment {
}
script.writeYaml file: yamlFilePath, data: data, overwrite: true
}


//TODO helmValuesFromFile not yet implemented
// private String createFromFileValues(String stage, Map gitopsConfig) {
// String values = ""
//
// gitopsConfig.helmValuesFromFile.each {
// if (stage in it['stage']) {
// values = fileToInlineYaml(it['key'], "${script.env.WORKSPACE}/k8s/${it['file']}")
// }
// }
// return values
// }
//
// private String fileToInlineYaml(String key, String filePath) {
// String values = ""
// String indent = " "
//
// def fileContent = readFile filePath
// values += "\n ${key}: |\n${indent}"
// values += fileContent.split("\\n").join("\n" + indent)
//
// return values
// }
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
package com.cloudogu.gitopsbuildlib.deployment.helm.helmrelease

import com.cloudogu.gitopsbuildlib.docker.DockerWrapper

class ArgoCDRelease extends HelmRelease{

protected DockerWrapper dockerWrapper

ArgoCDRelease(def script) {
super(script)
dockerWrapper = new DockerWrapper(script)
}

@Override
String create(Map helmConfig, String application, String namespace, String mergedValuesFile) {

String helmRelease = ""
if (helmConfig.repoType == 'GIT') {
helmRelease = createResourcesFromGitRepo(helmConfig, application, mergedValuesFile)
} else if (helmConfig.repoType == 'HELM') {
// TODO not yet implemented
}
return helmRelease
}

private String createResourcesFromGitRepo(Map helmConfig, String application, String mergedValuesFile) {
String helmRelease = ""

def chartPath = ''
if (helmConfig.containsKey('chartPath')) {
chartPath = helmConfig.chartPath
}

dockerWrapper.withHelm {
String templateScript = "helm template ${application} chart/${chartPath} -f ${mergedValuesFile}"
helmRelease = script.sh returnStdout: true, script: templateScript
}
// this line removes all empty lines since helm template creates some and the helm kubeval validator will throw an error if there are emtpy lines present
helmRelease = helmRelease.replaceAll("(?m)^[ \t]*\r?\n", "")
return helmRelease
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
package com.cloudogu.gitopsbuildlib.deployment.helm.helmrelease

class FluxV1Release extends HelmRelease{

FluxV1Release(def script) {
super(script)
}

@Override
String create(Map helmConfig, String application, String namespace, String mergedValuesFile) {
def values = fileToInlineYaml(mergedValuesFile)
def chart = getChart(helmConfig)
return """apiVersion: helm.fluxcd.io/v1
kind: HelmRelease
metadata:
name: ${application}
namespace: ${namespace}
annotations:
fluxcd.io/automated: "false"
spec:
releaseName: ${application}
chart:${chart}
values:
${values}
"""
}

private String gitRepoChart(Map helmConfig) {

def chartPath = "."
if (helmConfig.containsKey('chartPath') && helmConfig.chartPath) {
chartPath = helmConfig.chartPath
}

return """
git: ${helmConfig.repoUrl}
ref: ${helmConfig.version}
path: ${chartPath}"""
}

private String helmRepoChart(Map helmConfig) {
return """
repository: ${helmConfig.repoUrl}
name: ${helmConfig.chartName}
version: ${helmConfig.version}"""
}

private String getChart(Map helmConfig) {
if (helmConfig.repoType == 'GIT') {
return gitRepoChart(helmConfig)
} else if (helmConfig.repoType == 'HELM') {
return helmRepoChart(helmConfig)
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
package com.cloudogu.gitopsbuildlib.deployment.helm.helmrelease

abstract class HelmRelease {

protected script

HelmRelease(def script) {
this.script = script
}

abstract String create(Map helmConfig, String application, String namespace, String mergedValuesFile)

String fileToInlineYaml(String fileContents) {
String values = ""
String indent = " "
String fileContent = script.readFile fileContents
fileContent.split("\n").each { line ->
if(line.size() > 0) {
values += indent + line + "\n"
} else {
values += line + "\n"
}
}
// remove unnecessary last blank line
return values.substring(0, values.lastIndexOf('\n'))
}
}
Loading

0 comments on commit fc8c663

Please sign in to comment.