Manifests validation script

[jgato]

Hi all,
based on this documentation from @mvazquezc
RHsyseng/telco-operations#25
I am thinking on a kind of script that would validate the Manifests (Siteconfigs/PGT) before pushing to ArgoCD.
The motivation: many times you add a new manifest, or do some changes, push the change to your branch, from there a PR to main (or the branch synced with ArgoCD), accept the PR, then sync and... you realize you did a little mistake forgetting to add a file to kustomization, or some incorrect naming, and similar stuff. Repeating all the process again.
The script would be included as part of a CI/CD pipeline, avoiding to merge to main if there are errors, or similar thigs.
I am working in something like this:

    #! /bin/bash
    BASEDIR=$1
    ZTP_SITE_GENERATOR_IMG="quay.io/redhat_emp1/ztp-site-generator:4.10.0-1"

    if [[ $1 == "-h" || $1 == "--help" ]]; then
        echo "Usage:"
        echo "  $(basename $0) PATH_WITH_MANIFESTS"
        exit 1
    fi

    if [[ ! -d $BASEDIR ]]; then
        echo "FATAL: $BASEDIR is not a directory" >&2
        exit 1
    fi

    echo "Cheking yaml syntax"
    yamllint ${BASEDIR} -d relaxed

    if [[ $? != 0  ]]; then
        echo "Error on yamls systax"
        exit 1
    fi

    export KUSTOMIZE_PLUGIN_HOME=/tmp/ztp-kustomize-plugin/

    mkdir -p /tmp/ztp-kustomize-plugin/
    podman cp $(podman create --name policgentool --rm ${ZTP_SITE_GENERATOR_IMG=}):/kustomize/plugin/ran.openshift.io /tmp/ztp-kustomize-plugin/
    podman rm -f policgentool

    kustomize build ${BASEDIR} --enable-alpha-plugins | oc apply --dry-run=client -f -

    if [[ $? != 0  ]]; then
        echo "Error processing manifests"
        exit 1
    fi
    exit 0

It seems that kustomize build does not capture as many errors as ArgoCD does. For example: including a non-existing file in kustomization.yaml.
In ArgoCD you will have the error:

rpc error: code = Unknown desc = Manifest generation error (cached): `kustomize build .site-policies --enable-alpha-plugins` failed exit status 1: Error: loading generator plugins: accumulation err='accumulating resources from 'file-no-exists.yaml': evalsymlink failure on '.site-policies/file-no-exists.yaml' : lstat .site-policies/file-no-exists.yaml: no such file or directory': evalsymlink failure on '.site-policies/file-no-exists.yaml' : lstat .site-policies/file-no-exists.yaml: no such file or directory

but the script above executes with no errors.

any ideas? do you think it makes sense to have something like this?

[lack]

I think a dry-run script like this would be very valuable. We recently completed some related work in #1112 that implements a related mechanism, to generate individual sites' CRs for evaluation and testing purposes, and this feels like a nice follow-on to that.

Approach-wise, rather than having the script copy the contents out of our container, maybe it's worth looking in to the kustomize Containerized KRM Functions mechanism? No cp needed that way...

[jgato]

Hi @lack ,
I am not an expert on Kustomize, but I will check how to use these KRM functions.
Why do you think the extracted kustomize plugin for siteconfigs/PGT detects less error than when it is executed inside Argocd?

Many thanks,

[lack]

My suspicion would be either a difference in the version of kustomize, or because the error you see is talking about symlinks, perhaps an error in how we patched the ArgoCD container...

[jgato]

I am using the same version of ztp-sitegenerate. So the one in the patch is the one I am using in the script.
The symlink error seems oka. I mean, I have forced it, creating a kustomize file with references to files that does not exists. This is correctly captured in ArgoCD, but the script says everything is oka.

[jgato]

I did an interesting discovering:
https://kubernetes.io/blog/2019/01/14/apiserver-dry-run-and-kubectl-diff/
So, dry-run is too simple and it will not discover many issues:
"this runs a local dry-run that doesn't talk to the server: it doesn't have server validation and doesn't go through validating admission controllers. As an example, Custom resource names are only validated on the server so a local dry-run won't help."
But there exists the possibility of using "APIServer dry-run ". I will give it a look.

[jgato]

Ok, I understood that part of the errors I wanted to capture came from syncs with the OCP cluster. So, returned by the API-Server. With that dry-run=server now I can capture the desired errors.
I have done a simple script for that:
https://github.com/jgato/ztp-prevalidate
pretty simple, but it seems useful in combination, for example, with git hooks.