Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

interface validation checks in ci #211

Closed
UnicornChance opened this issue Feb 29, 2024 · 2 comments
Closed

interface validation checks in ci #211

UnicornChance opened this issue Feb 29, 2024 · 2 comments
Labels
ci Issues pertaining to CI / Pipelines / Testing enhancement New feature or request

Comments

@UnicornChance
Copy link
Contributor

Expected result

Grafana, Neuvector and Keycloak all have interface validation checks in their respective src/tasks.yaml. At this time they are commented out because CI does not create istio for the single package tests.

Actual Result

Interface validations are not passing in ci, either they need to pass or not run with some sort of conditional logic. Their ideal for local testing, so maintaining them is worth the hassle.

Related Issue

defenseunicorns/maru-runner#66

@UnicornChance UnicornChance added the possible-bug Something may not be working label Feb 29, 2024
@mjnagel mjnagel added the ci Issues pertaining to CI / Pipelines / Testing label Mar 28, 2024
@mjnagel
Copy link
Contributor

mjnagel commented May 1, 2024

Updating this ticket with some new changes - Istio now runs in all single package tests but we still have similar considerations for testing interactions with other pieces of the stack like monitoring/keycloak.

One avenue to help here is splitting validation into two separate test runs

Individual packages could then have tasks like:

tasks:
  - name: single-pkg-validate
    actions:
      - cmd: <hit grafana VS>
      - cmd: <basic cypress test?>
  - name: all-validate
     actions:
       - task: single-pkg-validate
       - cmd: <sso login check>
       - cmd: <validate grafana dashboards / datasources>

This would allow us to reuse tests where it makes sense but also allow augmenting our full test suite without causing issues on the single package side.

@mjnagel mjnagel added enhancement New feature or request and removed possible-bug Something may not be working labels Jul 2, 2024
@mjnagel
Copy link
Contributor

mjnagel commented Oct 21, 2024

Going to close this for now @UnicornChance - I think the intent with splitting up tests will be met as we build out the e2e tests. Also worth noting that all dependencies deploy in our new layer CI, so things like SSO logins can always be checked.

@mjnagel mjnagel closed this as completed Oct 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci Issues pertaining to CI / Pipelines / Testing enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants