Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MinIO server deployment in vHive (Kubernetes) #101

Merged
merged 3 commits into from
Feb 23, 2021
Merged

MinIO server deployment in vHive (Kubernetes) #101

merged 3 commits into from
Feb 23, 2021

Conversation

ustiugov
Copy link
Member

@ustiugov ustiugov commented Jan 21, 2021 •
edited
Loading

This PR introduces MinIO object storage deployment in a Kubernetes cluster.

Current setup:

  • The storage class is local-storage that requires node affinity, hence a node name and an existing path on it.
  • The MinIO service is exposed by a custom IP (i.e., ClusterIP publishing type) that is specified in service.yaml

Prerequisites

Modify the YAML config of the MinIO service

The only type of storage that is supported is local storage (at least for now).

Set up the MinIO service (after the cluster is up and running):

# create a folder in the local storage (on <MINIO_NODE_NAME> that is one of the Kubernetes nodes)
sudo mkdir -p <MINIO_PATH>

cd ./configs/storage/minio

# create a persistent volume (PV) and the corresponding PV claim
# specify the node name that would host the MinIO objects
# (use `hostname` command for the local node)
MINIO_NODE_NAME=<MINIO_NODE_NAME> MINIO_PATH=<MINIO_PATH> envsubst < pv.yaml | kubectl apply -f -
kubectl apply -f pv-claim.yaml
# create a storage app and the corresponding service
kubectl apply -f deployment.yaml
kubectl apply -f service.yaml

How to check if MinIO is ready

Check if a minio-* pod is in the Running state with kubectl get pods

Using the MinIO client

An example of how the service can be accessed, using the MinIO client ("minio" is the access key, "minio123" is the secret key):

mc alias set myminio http://10.96.0.46:9000 minio minio123
mc mb myminio/mybucket
mc cp README.md myminio/mybucket/tmp

Cleanup procedure

Note that files in the bucket are persistent in the local filesystem despite persistent volume removal.

kubectl delete deployment minio-deployment
kubectl delete pvc minio-pv-claim
kubectl delete svc minio-service
kubectl delete pv minio-pv

Signed-off-by: Dmitrii Ustiugov [email protected]

@ustiugov ustiugov added the enhancement New feature or request label Jan 21, 2021
@ustiugov ustiugov added this to the Support function composition milestone Jan 21, 2021
@ustiugov ustiugov self-assigned this Jan 21, 2021
@ustiugov ustiugov marked this pull request as draft January 21, 2021 21:24
@ustiugov ustiugov force-pushed the minio_in_k8s branch 3 times, most recently from a70bba6 to 49e6a83 Compare February 22, 2021 17:31
@ustiugov ustiugov marked this pull request as ready for review February 22, 2021 19:28
@ustiugov ustiugov requested a review from shyamjesal February 22, 2021 19:28
@ustiugov
Copy link
Member Author

This PR can be merged, I think.

shyamjesal
shyamjesal previously approved these changes Feb 23, 2021
View reviewed changes
@ustiugov ustiugov requested a review from shyamjesal February 23, 2021 13:25
amariucaitheodor
amariucaitheodor reviewed Feb 23, 2021
View reviewed changes
Copy link
Collaborator

@amariucaitheodor amariucaitheodor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, if you've tested it and it works then I'll go ahead and benchmark vHive storage transfer next.

@ustiugov ustiugov merged commit 7242618 into master Feb 23, 2021
@ustiugov ustiugov deleted the minio_in_k8s branch February 23, 2021 15:26
HermioneKT pushed a commit that referenced this pull request Jan 31, 2024
@KTCyber @HermioneKT
# This is a combination of 154 commits.
6674807 
# This is the 1st commit message:

refactor

# This is the commit message #2:

modify powerNodeSelector

# This is the commit message #3:

modify powerNodeSelector

# This is the commit message #4:

modify powerNodeSelector

# This is the commit message #5:

modify namespace

# This is the commit message #6:

modify namespace

# This is the commit message #7:

add namespace

# This is the commit message #8:

init power manager script

# This is the commit message #9:

add go code

# This is the commit message #10:

test

# This is the commit message #11:

test

# This is the commit message #12:

test

# This is the commit message #13:

test

# This is the commit message #14:

go.sum

# This is the commit message #15:

go sum

# This is the commit message #16:

go mod tidy

# This is the commit message #17:

test

# This is the commit message #18:

test

# This is the commit message #19:

test

# This is the commit message #20:

test

# This is the commit message #21:

test

# This is the commit message #22:

test

# This is the commit message #23:

test

# This is the commit message #24:

test

# This is the commit message #25:

test

# This is the commit message #26:

test

# This is the commit message #27:

test

# This is the commit message #28:

test

# This is the commit message #29:

print output

# This is the commit message #30:

test

# This is the commit message #31:

test

# This is the commit message #32:

test

# This is the commit message #33:

test

# This is the commit message #34:

test

# This is the commit message #35:

test

# This is the commit message #36:

test

# This is the commit message #37:

test

# This is the commit message #38:

test

# This is the commit message #39:

test

# This is the commit message #40:

refactor

# This is the commit message #41:

change val

# This is the commit message #42:

test

# This is the commit message #43:

test

# This is the commit message #44:

test

# This is the commit message #45:

exp1

# This is the commit message #46:

add url

# This is the commit message #47:

test

# This is the commit message #48:

test

# This is the commit message #49:

test auth

# This is the commit message #50:

test

# This is the commit message #51:

test

# This is the commit message #52:

test

# This is the commit message #53:

test

# This is the commit message #54:

test

# This is the commit message #55:

fix bug

# This is the commit message #56:

remove log

# This is the commit message #57:

test

# This is the commit message #58:

test

# This is the commit message #59:

test

# This is the commit message #60:

fix loop

# This is the commit message #61:

increase invocation

# This is the commit message #62:

test

# This is the commit message #63:

test

# This is the commit message #64:

test

# This is the commit message #65:

test

# This is the commit message #66:

test

# This is the commit message #67:

test

# This is the commit message #68:

test

# This is the commit message #69:

Test

# This is the commit message #70:

test

# This is the commit message #71:

test

# This is the commit message #72:

test

# This is the commit message #73:

test

# This is the commit message #74:

test

# This is the commit message #75:

test

# This is the commit message #76:

test

# This is the commit message #77:

test

# This is the commit message #78:

test

# This is the commit message #79:

test

# This is the commit message #80:

test

# This is the commit message #81:

test

# This is the commit message #82:

test

# This is the commit message #83:

test

# This is the commit message #84:

test

# This is the commit message #85:

test

# This is the commit message #86:

test

# This is the commit message #87:

test

# This is the commit message #88:

test

# This is the commit message #89:

test

# This is the commit message #90:

test1

# This is the commit message #91:

test

# This is the commit message #92:

test

# This is the commit message #93:

test

# This is the commit message #94:

test

# This is the commit message #95:

test

# This is the commit message #96:

test

# This is the commit message #97:

test

# This is the commit message #98:

test

# This is the commit message #99:

test

# This is the commit message #100:

test

# This is the commit message #101:

test

# This is the commit message #102:

test

# This is the commit message #103:

test

# This is the commit message #104:

test

# This is the commit message #105:

test

# This is the commit message #106:

test

# This is the commit message #107:

test

# This is the commit message #108:

test

# This is the commit message #109:

test1

# This is the commit message #110:

test

# This is the commit message #111:

test

# This is the commit message #112:

Test

# This is the commit message #113:

test

# This is the commit message #114:

test

# This is the commit message #115:

test

# This is the commit message #116:

test

# This is the commit message #117:

test

# This is the commit message #118:

test

# This is the commit message #119:

Test

# This is the commit message #120:

Test

# This is the commit message #121:

test

# This is the commit message #122:

test

# This is the commit message #123:

Test

# This is the commit message #124:

test

# This is the commit message #125:

test

# This is the commit message #126:

test

# This is the commit message #127:

test

# This is the commit message #128:

test

# This is the commit message #129:

test

# This is the commit message #130:

test

# This is the commit message #131:

test

# This is the commit message #132:

test

# This is the commit message #133:

test

# This is the commit message #134:

test

# This is the commit message #135:

test

# This is the commit message #136:

test

# This is the commit message #137:

test

# This is the commit message #138:

test

# This is the commit message #139:

test

# This is the commit message #140:

test

# This is the commit message #141:

test

# This is the commit message #142:

test

# This is the commit message #143:

Test

# This is the commit message #144:

Test

# This is the commit message #145:

Test

# This is the commit message #146:

Test

# This is the commit message #147:

test

# This is the commit message #148:

test

# This is the commit message #149:

test

# This is the commit message #150:

test

# This is the commit message #151:

test

# This is the commit message #152:

test

# This is the commit message #153:

test

# This is the commit message #154:

Test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@amariucaitheodor amariucaitheodor amariucaitheodor approved these changes

@shyamjesal shyamjesal Awaiting requested review from shyamjesal

Assignees

@ustiugov ustiugov

Labels
enhancement New feature or request
Projects
None yet
Milestone
Support function composition
Development

Successfully merging this pull request may close these issues.
3 participants
@ustiugov @shyamjesal @amariucaitheodor