Skip to content

operator‐1951

Jump to bottom
Allan Roger Reid edited this page Dec 4, 2024 · 3 revisions

Setup

See end of document

Test cases

FAIL EXPECTED

Test 0

Summary

  1. Create a tenant
  2. Add a pool, ensuring multiple pools are present
  3. Attempt to edit and save any pool
  4. Observe the edit fail image

PASS EXPECTED

Implement fix

cd ~/operator && git checkout enable-edit-pool

Compile and Push newly compile operator to docker. See above instructions for building in setup at end of document

Deploy minio

~/operator/kubectl-minio/kubectl-minio init --image 10.62.75.169:5000/minio/operator:db8785e4 --console-image 10.62.75.169:5000/minio/operator:db8785e4
SA_TOKEN=$(kubectl -n minio-operator get secret console-sa-secret -o jsonpath="{.data.token}" | base64 --decode)
echo $SA_TOKEN
kubectl patch service -n minio-operator console -p '{"spec":{"ports":[{"name": "http","port": 9090,"protocol": "TCP","nodePort":31090}],"type": "NodePort"}}'

Login

http://k8s-master.minio.training:31090/login

Test 1

Summary:

  1. Create tenant using console
  2. Attempt to edit pool-0 i.e. change Volume Size and Storage Class
  3. Expand using Console
  4. Attempt to edit pool-1 i.e. change Volume Size and Storage Class
image
  1. Verify change in tenant yaml
  2. Verify pods running and minio stable

Test 2

Summary:

  1. Using previous tenant from Test 1
  2. Expand using Console
  3. Define runc 
cat <<EOF | kubectl apply -f -
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: runc
handler: runc
EOF
  1. Attempt to edit pool-1 i.e. change Security Context and Custom Runtime Configurations
securityContext:
      fsGroup: 1000
      fsGroupChangePolicy: Always
      runAsGroup: 1000
      runAsNonRoot: true
      runAsUser: 1000
image
  1. Verify change in tenant yaml
  2. Verify pods running and minio stable
image

Test 3

Summary:

  1. Using previous tenant from Test 1
  2. Expand using Console
  3. Attempt to edit pool-0 i.e. change Pod Placement
image
  1. Verify change in tenant yaml
  2. Verify pods running and minio stable
image

SETUP

Create a kubernetes cluster such as in https://github.com/allanrogerr/public/wiki/vm-broker-vanilla-k8s-cluster-on-multiple-instances,-same-node

SSH into control plane

ssh -p 20029 [email protected] -o "ServerAliveInterval=5" -o "ServerAliveCountMax=100000" -o "StrictHostKeyChecking=off"

Persist sessions after logout

loginctl enable-linger ubuntu

Clone operator

cd ~
git clone https://github.com/allanrogerr/operator.git
cd operator && git checkout master

Install essential packages

sudo apt-get update && sudo apt-get upgrade -y
sudo apt-get install vim -y
sudo apt-get install zip -y
sudo apt-get install make -y
sudo apt install build-essential -y

Install go

cd ~
wget https://go.dev/dl/go1.21.3.linux-amd64.tar.gz
sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.21.3.linux-amd64.tar.gz
cat <<EOF >> $HOME/.profile 
export PATH=$PATH:/usr/local/go/bin:~/go/bin
EOF
cat $HOME/.profile 
source $HOME/.profile
go version

Install nvm

sudo apt-get install nodejs -y
sudo apt-get install npm -y
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
source ~/.bashrc
nvm install && nvm use && sudo npm install -g yarn

Install podman on each node. Push newly compile operator to docker

sudo apt-get install podman -y
sudo apt-get install podman-docker -y

podman run -d -p 5000:5000 --restart always --name registry registry:2

Note: make local registry insecure e.g. in /etc/containers/registries.conf on each node set

[[registry]]
insecure=true
location="10.62.75.169:5000"

Edit Makefile

VERSION ?= $(shell git describe --tags --always)
VERSIONV ?= $(shell git describe --tags --always | sed 's,v,,g')

binary:
        @CGO_ENABLED=0 GOOS=linux go build -trimpath -o minio-operator ./cmd/operator

docker: operator
        @docker buildx build --no-cache --platform linux/amd64 -t $(TAG) .

Install https://github.com/mvdan/gofumpt then,

CI=true
cd ~/operator && make build && make assets && make docker
podman tag localhost/minio/operator:dev 10.62.75.169:5000/minio/operator:0.6
podman images
podman push 10.62.75.169:5000/minio/operator:0.6 --tls-verify=false
podman search 10.62.75.169:5000/ --tls-verify=false
or
curl -X GET http://10.62.75.169:5000/v2/_catalog

Add on each node (also pull on each node e.g podman pull 10.62.75.169:5000/minio/operator:64cb15eb)

sudo vi /etc/containerd/config.toml
[plugins."io.containerd.grpc.v1.cri".registry.configs]
        [plugins."io.containerd.grpc.v1.cri".registry.configs."10.62.75.169:5000".tls] # edited line
          ca_file = "" # edited line
          cert_file = "" # edited line
          insecure_skip_verify = true # edited line
          key_file = "" # edited line

      [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."10.62.75.169:5000"] # edited line
          endpoint = ["http://10.62.75.169:5000"] # edited line

Install mc

mkdir -p ~/mc && cd ~/mc && rm -rf mc* && wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc && cd ~

Build & install kubectl-minio plugin

cd ~/operator/kubectl-minio
CGO_ENABLED=1 go build --ldflags "-s -w" -trimpath -o ../kubectl-minio

Deploy minio

~/operator/kubectl-minio/kubectl-minio init --image 10.62.75.169:5000/minio/operator:0.6 --console-image 10.62.75.169:5000/minio/operator:0.6
SA_TOKEN=$(kubectl -n minio-operator get secret console-sa-secret -o jsonpath="{.data.token}" | base64 --decode)
echo $SA_TOKEN
kubectl patch service -n minio-operator console -p '{"spec":{"ports":[{"name": "http","port": 9090,"protocol": "TCP","nodePort":31090}],"type": "NodePort"}}'

Set default storage class

kubectl patch storageclass local-path -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
kubectl get storageclass
Clone this wiki locally