Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: initial minio operator and tenant #4

Merged
merged 47 commits into from
Aug 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
ed9a71d
updates to templates and additional vars
zachariahmiller Aug 2, 2024
652095d
add non-ui test and copy secret process
zachariahmiller Aug 5, 2024
48b6ce3
fix runners
zachariahmiller Aug 5, 2024
98c5b1f
commnet out upgrade tasks
zachariahmiller Aug 5, 2024
4efeb1c
comment out upgrade tasks and notify
zachariahmiller Aug 5, 2024
0f57ba3
bump uds-common task version
zachariahmiller Aug 5, 2024
3688f31
bump uds-common action versions
zachariahmiller Aug 5, 2024
9edbdf4
update test task and test tasks called in ci
zachariahmiller Aug 5, 2024
94c1324
remove nonexistant field from job spec, add tenant.pools helm overrid…
zachariahmiller Aug 5, 2024
4f1dd59
lint fixes, registry1 values updates
zachariahmiller Aug 5, 2024
0d3305c
more lint fixes, fix test condition
zachariahmiller Aug 5, 2024
5094350
more lint fixes
zachariahmiller Aug 5, 2024
ffbbb15
update registry1 image ref
zachariahmiller Aug 5, 2024
f4a7972
rein in netpols
zachariahmiller Aug 5, 2024
4a4c859
lint
zachariahmiller Aug 5, 2024
ea77f08
add configuration.md, cleanup
zachariahmiller Aug 6, 2024
1544a65
lint fix
zachariahmiller Aug 6, 2024
7d6e1d5
update readme
zachariahmiller Aug 6, 2024
a588e18
doc note on SSO
zachariahmiller Aug 6, 2024
374ae49
uncomment scorecard.yaml
zachariahmiller Aug 6, 2024
306d9a7
Update docs/configuration.md
zachariahmiller Aug 6, 2024
0685948
update releaseplease action, refactor tests, add missing values file
zachariahmiller Aug 6, 2024
45e6c70
diable upgrade in ci matrix until there is upgrade to test
zachariahmiller Aug 6, 2024
dfdb470
lint fix
zachariahmiller Aug 6, 2024
7c19e89
fix monitoring
zachariahmiller Aug 6, 2024
1cec83f
fix missed copy paste description issue
zachariahmiller Aug 6, 2024
cca4475
refactor values
zachariahmiller Aug 6, 2024
3bf55c4
lint fix
zachariahmiller Aug 6, 2024
3dae954
remove registry1 and update to 6.0.1
zachariahmiller Aug 6, 2024
4e6a20c
update to 6.0.1
zachariahmiller Aug 7, 2024
105ff97
update minio, add restiry1 images back in, add job automation to prov…
zachariahmiller Aug 19, 2024
af8da90
fix .gitignore and add job that was ignored in last commit
zachariahmiller Aug 19, 2024
08de937
update default value of tenant level copyPassword to false
zachariahmiller Aug 19, 2024
b7fa924
lint fixes
zachariahmiller Aug 19, 2024
1f5c5b5
Apply suggestions from code review
zachariahmiller Aug 20, 2024
ef52437
Apply suggestions from code review
zachariahmiller Aug 20, 2024
cccb211
remove root cred copy, remove ability to override password generation…
zachariahmiller Aug 20, 2024
40033a8
remove problematic zarf var, update documentation.
zachariahmiller Aug 20, 2024
dc762e6
update docs
zachariahmiller Aug 20, 2024
8a18aa3
more doc fixes
zachariahmiller Aug 20, 2024
dc40f7d
more doc fixes
zachariahmiller Aug 20, 2024
2e47bd4
config chart domain to zarf var
zachariahmiller Aug 21, 2024
3537980
Update chart/templates/app-secret.yaml
zachariahmiller Aug 21, 2024
0f5db61
refactor secret and job templates based on PR discussion
zachariahmiller Aug 21, 2024
982350f
update doc to reflect last set of changes
zachariahmiller Aug 21, 2024
0f2dd28
update user still being in wrong place in test bundle override after …
zachariahmiller Aug 22, 2024
f7263ff
update runs on to public runners
zachariahmiller Aug 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/commitlint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,4 @@ on:
jobs:
validate:
name: Validate
uses: defenseunicorns/uds-common/.github/workflows/commitlint.yaml@264ec430c4079129870820e70c4439f3f3d57cbc # v0.3.9
uses: defenseunicorns/uds-common/.github/workflows/commitlint.yaml@afd3aea72462ac21a715cb5dbc2654b42073ac40 # v0.11.0
2 changes: 1 addition & 1 deletion .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
fetch-depth: 0

- name: Environment setup
uses: defenseunicorns/uds-common/.github/actions/setup@264ec430c4079129870820e70c4439f3f3d57cbc # v0.3.9
uses: defenseunicorns/uds-common/.github/actions/setup@afd3aea72462ac21a715cb5dbc2654b42073ac40 # v0.11.0
with:
registry1Username: ${{ secrets.IRON_BANK_ROBOT_USERNAME }}
registry1Password: ${{ secrets.IRON_BANK_ROBOT_PASSWORD }}
Racer159 marked this conversation as resolved.
Show resolved Hide resolved
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/tag-and-release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
steps:
- name: Create release tag
id: tag
uses: google-github-actions/release-please-action@a37ac6e4f6449ce8b3f7607e4d97d0146028dc0b # v4.1.0
uses: googleapis/release-please-action@7987652d64b4581673a76e33ad5e98e3dd56832f # v4.1.3
- id: release-flag
run: echo "release_created=${{ steps.tag.outputs.release_created || false }}" >> $GITHUB_OUTPUT

Expand All @@ -36,7 +36,7 @@ jobs:
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1

- name: Environment setup
uses: defenseunicorns/uds-common/.github/actions/setup@264ec430c4079129870820e70c4439f3f3d57cbc # v0.3.9
uses: defenseunicorns/uds-common/.github/actions/setup@afd3aea72462ac21a715cb5dbc2654b42073ac40 # v0.11.0
with:
registry1Username: ${{ secrets.IRON_BANK_ROBOT_USERNAME }}
registry1Password: ${{ secrets.IRON_BANK_ROBOT_PASSWORD }}
Expand All @@ -51,6 +51,6 @@ jobs:

- name: Save logs
if: always()
uses: defenseunicorns/uds-common/.github/actions/save-logs@264ec430c4079129870820e70c4439f3f3d57cbc # v0.3.9
uses: defenseunicorns/uds-common/.github/actions/save-logs@afd3aea72462ac21a715cb5dbc2654b42073ac40 # v0.11.0
with:
suffix: '${{ matrix.flavor }}-${{ github.run_id }}-${{ github.run_attempt }}'
10 changes: 5 additions & 5 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,32 +34,32 @@ permissions:
jobs:
run-test:
name: ${{ matrix.type }} ${{ matrix.flavor }}
runs-on: uds-ubuntu-big-boy-8-core
runs-on: ubuntu-latest
timeout-minutes: 25
strategy:
matrix:
flavor: [upstream, registry1]
type: [install, upgrade]
type: [install] #todo add upgrade after first release

steps:
- name: Checkout repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1

- name: Environment setup
uses: defenseunicorns/uds-common/.github/actions/setup@264ec430c4079129870820e70c4439f3f3d57cbc # v0.3.9
uses: defenseunicorns/uds-common/.github/actions/setup@afd3aea72462ac21a715cb5dbc2654b42073ac40 # v0.11.0
with:
registry1Username: ${{ secrets.IRON_BANK_ROBOT_USERNAME }}
registry1Password: ${{ secrets.IRON_BANK_ROBOT_PASSWORD }}
ghToken: ${{ secrets.GITHUB_TOKEN }}

- name: Test
uses: defenseunicorns/uds-common/.github/actions/test@264ec430c4079129870820e70c4439f3f3d57cbc # v0.3.9
uses: defenseunicorns/uds-common/.github/actions/test@afd3aea72462ac21a715cb5dbc2654b42073ac40 # v0.11.0
with:
flavor: ${{ matrix.flavor }}
type: ${{ matrix.type }}

- name: Save logs
if: always()
uses: defenseunicorns/uds-common/.github/actions/save-logs@264ec430c4079129870820e70c4439f3f3d57cbc # v0.3.9
uses: defenseunicorns/uds-common/.github/actions/save-logs@afd3aea72462ac21a715cb5dbc2654b42073ac40 # v0.11.0
with:
suffix: ${{ matrix.type }}-${{ matrix.flavor }}-${{ github.run_id }}-${{ github.run_attempt }}
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Welcome to the Minio-operator UDS Package
# Welcome to the Minio Operator UDS Package

Thank you for your interest in this Defense Unicorns UDS Package!

Expand Down
15 changes: 12 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,21 @@
# 🏭 UDS Minio-operator Package
# 🏭 UDS Minio Operator Package

[![Latest Release](https://img.shields.io/github/v/release/defenseunicorns/uds-package-minio-operator)](https://github.com/defenseunicorns/uds-package-minio-operator/releases)
[![Build Status](https://img.shields.io/github/actions/workflow/status/defenseunicorns/uds-package-minio-operator/tag-and-release.yaml)](https://github.com/defenseunicorns/uds-package-minio-operator/actions/workflows/tag-and-release.yaml)
[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/defenseunicorns/uds-package-minio-operator/badge)](https://api.securityscorecards.dev/projects/github.com/defenseunicorns/uds-package-minio-operator)

This package is designed for use as part of a bundle deployed on [UDS Core](https://github.com/defenseunicorns/uds-core).

> [!WARNING]
> `uds-package-minio-operator` is in development and is absolutely not ready for general consumption. If you would like to improve this package, please open a PR.

This package is designed for use as part of a [UDS Software Factory](https://github.com/defenseunicorns/uds-software-factory) bundle deployed on [UDS Core](https://github.com/defenseunicorns/uds-core).

## Flavors

| Flavor | Description | Example Creation |
| ------ | ----------- | ---------------- |
| upstream | Uses upstream images within the package. | `zarf package create . -f upstream` |
| registry1 | Uses registry1 images within the package. | `zarf package create . -f registry1` |

## Releases

Expand All @@ -24,4 +29,8 @@ The released packages can be found in [ghcr](https://github.com/defenseunicorns/

## Contributing

Please see the [CONTRIBUTING.md](./CONTRIBUTING.md)
Please see the [CONTRIBUTING.md](./CONTRIBUTING.md)

## Development

When developing this package it is ideal to utilize the json schemas for UDS Bundles, Zarf Packages and Maru Tasks. This involves configuring your IDE to provide schema validation for the respective files used by each application. For guidance on how to set up this schema validation, please refer to the [guide](https://github.com/defenseunicorns/uds-common/blob/main/docs/development-ide-configuration.md) in uds-common.
31 changes: 25 additions & 6 deletions bundle/uds-bundle.yaml
Original file line number Diff line number Diff line change
@@ -1,14 +1,33 @@
kind: UDSBundle
metadata:
name: uds-package-minio
metadata:
name: minio-test
description: A UDS Bundle for minio operator (+tenant)
# x-release-please-start-version
version: 0.0.1
# x-release-please-end
version: 0.1.0
# x-release-please-end

packages:
packages:
- name: minio-operator
path: ../
# x-release-please-start-version
ref: 5.0.15-uds.0
ref: 6.0.2-uds.0
# x-release-please-end
overrides:
minio-operator:
uds-minio-config:
values:
# Test helm overrides to provision app specific buckets, policies and creds
- path: apps
value:
- name: mc-cli
namespace: mc-cli
remoteSelector:
job-name: minio-job
zachariahmiller marked this conversation as resolved.
Show resolved Hide resolved
bucketNames:
- mc-cli-test-bucket
policy: ""
copyPassword:
zachariahmiller marked this conversation as resolved.
Show resolved Hide resolved
enabled: true
secretName: ""
secretIDKey: ""
secretPasswordKey: ""
2 changes: 1 addition & 1 deletion bundle/uds-config.yaml
Original file line number Diff line number Diff line change
@@ -1 +1 @@
variables:
variables:
31 changes: 31 additions & 0 deletions chart/templates/app-secret.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{{- range .Values.apps }}
{{- $password := (randAlphaNum 32) | b64enc }}
{{- $user := (.user | default .name) | b64enc }}
{{- $secretName := .copyPassword.secretName | default (printf "minio-%s" .name) }}
{{- $secretIDKey := .copyPassword.secretIDKey | default "accessKey" }}
{{- $secretPasswordKey := .copyPassword.secretPasswordKey | default "secretKey" }}
apiVersion: v1
kind: Secret
metadata:
name: {{ (printf "minio-%s" .name) }}
namespace: minio
type: Opaque
data:
{{- $data := dict "accessKey" $user }}
{{- $data = merge $data (dict "secretKey" $password) }}
{{- toYaml $data | nindent 2 }}
---
{{- if .copyPassword.enabled }}
apiVersion: v1
kind: Secret
metadata:
name: {{ $secretName }}
namespace: {{ .namespace | default $.Release.Namespace }}
type: Opaque
data:
{{- $data := dict $secretIDKey $user }}
{{- $data = merge $data (dict $secretPasswordKey $password) }}
{{- toYaml $data | nindent 2 }}
{{- end }}
---
{{- end }}
43 changes: 29 additions & 14 deletions chart/templates/config-secret.yaml
Original file line number Diff line number Diff line change
@@ -1,21 +1,36 @@
{{- $password := .Values.rootPassword | default (randAlphaNum 32) }}
{{- $user := .Values.rootUser | default "minio" }}

apiVersion: v1
kind: Secret
metadata:
name: uds-minio-dev
name: {{ .Values.name }}
namespace: minio
type: Opaque
stringData:
config.env: |-
# TODO @zachariahmiller make the root creds set to random generated values unless overridden
export MINIO_ROOT_USER={{ .Values.minio.rootUser }}
export MINIO_ROOT_PASSWORD={{ .Values.minio.rootPassword }}
export MINIO_IDENTITY_OPENID_DISPLAY_NAME={{ .Values.minio.displayName }}
export MINIO_IDENTITY_OPENID_CLAIM_NAME={{ .Values.minio.identityOpenidClaimName }}
export MINIO_IDENTITY_OPENID_CLIENT_ID={{ .Values.minio.identityOpenidClientId }}
export MINIO_IDENTITY_OPENID_CLIENT_SECRET={{ .Values.minio.identityOpenidClientSecret }}
export MINIO_IDENTITY_OPENID_CONFIG_URL={{ .Values.minio.identityOpenidConfigUrl }}
export MINIO_IDENTITY_OPENID_REDIRECT_URI_DYNAMIC={{ .Values.minio.identityOpenidRedirectUriDynamic }}
export MINIO_IDENTITY_OPENID_SCOPES={{ .Values.minio.identityOpenidScopes }}
export MINIO_IDENTITY_OPENID_ENABLE=on
export MINIO_BROWSER_REDIRECT_URL=https://minio-console.uds.dev
#export MINIO_IDENTITY_OPENID_ROLE_POLICY="consoleAdmin"
export MINIO_ROOT_USER={{ $user | quote }}
export MINIO_ROOT_PASSWORD={{ $password | quote }}
export MINIO_PROMETHEUS_AUTH_TYPE="public"
{{- if .Values.sso.enabled }}
export MINIO_IDENTITY_OPENID_DISPLAY_NAME={{ .Values.name }}
export MINIO_IDENTITY_OPENID_CLAIM_NAME=policy
export MINIO_IDENTITY_OPENID_CLIENT_ID="{{ printf "%s-sso" .Values.name }}"
export MINIO_IDENTITY_OPENID_CLIENT_SECRET={{ .Values.identityOpenidClientSecret }}
export MINIO_IDENTITY_OPENID_REDIRECT_URI_DYNAMIC=on
export MINIO_IDENTITY_OPENID_SCOPES=openid
export MINIO_IDENTITY_OPENID_ENABLE=on
export MINIO_BROWSER_REDIRECT_URL="https://minio-console.{{ .Values.domain }}"
export MINIO_IDENTITY_OPENID_CONFIG_URL="{{ default (printf "https://sso.%s/realms/uds/.well-known/openid-configuration" .Values.domain) .Values.identityOpenidConfigUrl }}"
{{- end }}

---
apiVersion: v1
kind: Secret
metadata:
name: {{ .Values.name }}-creds
namespace: minio
stringData:
{{- $data := dict "rootUser" $user }}
{{- $data = merge $data (dict "rootPassword" $password) }}
{{- toYaml $data | nindent 2 }}
63 changes: 63 additions & 0 deletions chart/templates/job.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
{{- range .Values.apps }}
{{- $secretName := (printf "minio-%s" .name) }}
apiVersion: batch/v1
kind: Job
metadata:
name: {{ .name }}-minio-setup
namespace: minio
spec:
template:
spec:
containers:
- name: minio-setup
image: {{ $.Values.mcImage }}
command: ["/bin/sh", "-c"]
args:
- |
mc alias set myminio $MINIO_SERVER $MINIO_ACCESS_KEY $MINIO_SECRET_KEY;
{{- range $bucket := .bucketNames }}
mc mb myminio/{{ $bucket }};
{{- end }}
mc admin policy create myminio {{ .name }}-policy /config/policy.json;
mc admin user add myminio $USER $USER_SECRET;
mc admin policy attach myminio {{ .name }}-policy --user $USER;
env:
- name: MINIO_SERVER
value: "http://uds-minio-hl.minio.svc.cluster.local:9000"
- name: MC_CONFIG_DIR
value: "/tmp/mc/"
- name: USER
valueFrom:
secretKeyRef:
name: {{ $secretName }}
key: "accessKey"
- name: USER_SECRET
valueFrom:
secretKeyRef:
name: {{ $secretName }}
key: "secretKey"
- name: MINIO_ACCESS_KEY
valueFrom:
secretKeyRef:
name: {{ $.Values.name }}-creds
key: rootUser
- name: MINIO_SECRET_KEY
valueFrom:
secretKeyRef:
name: {{ $.Values.name }}-creds
key: rootPassword
volumeMounts:
- name: policy-config
mountPath: /config
- name: config-volume
mountPath: /tmp
restartPolicy: OnFailure
volumes:
- name: policy-config
configMap:
name: {{ .name }}-minio-policy
- name: config-volume
emptyDir: {}
backoffLimit: 4
---
{{- end }}
54 changes: 54 additions & 0 deletions chart/templates/policy-cm.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
{{- range .Values.apps }}
{{- if hasKey . "bucketNames" }}
{{- $bucketNames := .bucketNames | default list }}

apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .name }}-minio-policy
namespace: minio
data:
policy.json: |
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": [
{{- range $index, $bucket := $bucketNames }}
"arn:aws:s3:::{{ $bucket }}"{{ if lt (add $index 1) (len $bucketNames) }},{{ end }}
{{- end }}
]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": [
{{- range $index, $bucket := $bucketNames }}
"arn:aws:s3:::{{ $bucket }}/*"{{ if lt (add $index 1) (len $bucketNames) }},{{ end }}
{{- end }}
]
}
]
}
---
{{- else }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .name }}-minio-policy
namespace: minio
data:
policy.json: |
{{ .policy | indent 4 }}
---
{{- end }}
{{- end }}
Loading