COSI-17: Add Dev Container for COSI Driver Development #12

anurag4DSB · 2024-11-13T08:40:40Z

Generated via copilot

🎉 Dev Containers 🚀

This PR introduces a Dev Container for the Scality COSI Driver, streamlining development and testing environments. Key contributions include:

COSI-17: Add Dev Container Configuration
- 🛠️ Implemented a Dev Container setup that includes Minikube, S3, and IAM services using Docker Compose.
- ⚙️ Pre-configured with Go, Docker-in-Docker, AWS CLI, kubectl, and Helm.
- 🤖 Automated Kubernetes cluster setup and service deployment through a custom setup script.
Updated Cleanup Script and Docker References
- 🧹 Refined the cleanup script (cleanup_cosi_resources.sh) to properly delete COSI resources and adjust Docker container references.
- 📝 Issue: COSI-17
COSI-17: Add Dev Container Documentation
- 📚 Added comprehensive documentation on using the Dev Container, running tests, and setting up COSI resources.
- 📝 Included detailed steps and troubleshooting guidance for a seamless development experience.

Summary 📝

🧰 Dev Container configuration for consistent setup.
🔄 Automated resource setup and cleanup scripts.
📖 Comprehensive Dev Container documentation for developers.
🏷️ Issue: COSI-17

The aim is to make participation easier with the introduction of Dev Container.

codecov · 2024-11-13T10:06:19Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.52%. Comparing base (9e51a25) to head (c0f97c6).
Report is 4 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main      #12   +/-   ##
=======================================
  Coverage   75.52%   75.52%           
=======================================
  Files           4        4           
  Lines         237      237           
=======================================
  Hits          179      179           
  Misses         51       51           
  Partials        7        7

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

anurag4DSB · 2024-11-13T13:51:39Z

.devcontainer/devcontainer.json

+    "customizations": {
+        "codespaces": {
+            "repositories": {
+                "scality/cloudserver": {
+                    "permissions": {
+                        "packages": "read"
+                    }
+                },
+                "scality/vault": {
+                    "permissions": {
+                        "packages": "read"
+                    }
+                }
+            }
+        }
+    },
+    "postCreateCommand": "bash .devcontainer/setup-dev-container.sh"


We are looking into with Platform enigneering why these permissions are't working yet.

Till then we have docker login in the script.

dvasilas · 2024-11-13T16:21:00Z

.devcontainer/setup-dev-container.sh

+minikube start
+
+echo "Logging into GHCR (GitHub Container Registry)..."
+echo "$REPOSITORY_USER_TOKEN" | docker login ghcr.io -u "$REPOSITORY_USER" --password-stdin


Where are these env variables being set ? 🤔

Secrets are available as ENV variables in codespaces

docs/development/dev-container-setup.md

dvasilas · 2024-11-13T16:28:03Z

docs/development/dev-container-setup.md

+Change `COSI_S3_ENDPOINT` to the IP (`hostname -I | awk '{print $1}'`) of codespace instance.
+
+#### Example
+
+```bash
+$ hostname -I | awk '{print $1}'
+10.0.1.236
+```
+
+New secret file with updated `COSI_S3_ENDPOINT`
+
+```
+apiVersion: v1
+kind: Secret
+metadata:
+  name: s3-secret-for-cosi
+  namespace: default
+type: Opaque
+stringData:
+  COSI_S3_ACCESS_KEY_ID: PBUOB68AVF39EVVAFNFL  # Plain text access key, generated in the CI
+  COSI_S3_SECRET_ACCESS_KEY: P+PK+uMB9spUc21huaQoOexqdJoV00tSnl+pc7t7  # Plain text secret key
+  COSI_S3_ENDPOINT: http://10.0.1.236:8000  # Plain text endpoint
+  COSI_S3_REGION: us-west-1  # Plain text region
+```


format: I think you could add indentation to everything under 1 as you do for 2 and 3

I removed this part as its automated.

dvasilas · 2024-11-13T16:34:07Z

.devcontainer/devcontainer.json

+    // Features to add to the dev container. More info: https://containers.dev/features.
+    "features": {
+        "ghcr.io/devcontainers/features/go:1": {
+            "version": "latest"


Is using the latest go version intentional (instead of pinning the same version as in go.mod) ?

It was intentional to use the latest, as we want to update regularly.
I can use the same as go.mod if you insist.

dvasilas · 2024-11-13T16:37:43Z

.github/scripts/cleanup_cosi_resources.sh

@@ -41,6 +41,9 @@ log_and_run echo "Deleting Bucket Class and Bucket Claim..."
 log_and_run kubectl delete -f cosi-examples/bucketclass.yaml || { echo "Bucket Class not found." | tee -a "$LOG_FILE"; }
 log_and_run kubectl delete -f cosi-examples/bucketclaim.yaml || { echo "Bucket Claim not found." | tee -a "$LOG_FILE"; }

+log_and_run echo "Deleting s3-secret-for-cosi secret..."
+log_and_run kubectl delete secret s3-secret-for-cosi --namespace=default || { echo "Secret s3-secret-for-cosi not found." | tee -a "$LOG_FILE"; }


Not sure if it makes any difference, but in other kubectl commands the scality-object-storage namespace is being used.

the secret is intentionally being deployed in default namespace test the rbac and COSi driver across namespaces as cosi driver reads the secrets.

anurag4DSB · 2024-11-13T16:59:16Z

rebased and pushed

docs/development/dev-container-setup.md

fredmnl

LGTM

You have a typo in one of your commit messages (sodi instead of cosi)

After having used them, would you recommend dev containers development environment for other projects as well?

anurag4DSB · 2024-11-14T16:02:22Z

LGTM

You have a typo in one of your commit messages (sodi instead of cosi)

After having used them, would you recommend dev containers development environment for other projects as well?

Thanks, I will edit the commit message. :)

I recommend using dev containers for every repo. The main reason is that they ensure a consistent development environment for everyone. I prefer developing locally with tools like Minikube and building images on my machine. However, if someone needs to contribute in the future, setting up everything locally can take time, especially for people who don’t contribute regularly.

With dev containers, contributors can quickly start working without spending time setting up the environment. In the how to guide, we specify how to run the CI and end-to-end tests using dev containers, so once I’m familiar with the tools, one can switch back to local development if needed.

- the container sets up minikube, s3 and iam services.

Issue: COSI-17

anurag4DSB force-pushed the feature/COSI-17-add-dev-container-to-cosi-driver branch 4 times, most recently from 276ff69 to d39c1ba Compare November 13, 2024 13:38

anurag4DSB marked this pull request as ready for review November 13, 2024 13:39

anurag4DSB changed the title ~~Feature/cosi 17 add dev container to cosi driver~~ COSI-17: Add Dev Container for COSI Driver Development Nov 13, 2024

anurag4DSB commented Nov 13, 2024

View reviewed changes

dvasilas approved these changes Nov 13, 2024

View reviewed changes

anurag4DSB force-pushed the feature/COSI-17-add-dev-container-to-cosi-driver branch from d39c1ba to 1636d3f Compare November 13, 2024 16:58

fredmnl assigned fredmnl and unassigned fredmnl Nov 14, 2024

fredmnl self-requested a review November 14, 2024 08:57

anurag4DSB commented Nov 14, 2024

View reviewed changes

docs/development/dev-container-setup.md Outdated Show resolved Hide resolved

anurag4DSB commented Nov 14, 2024

View reviewed changes

docs/development/dev-container-setup.md Outdated Show resolved Hide resolved

anurag4DSB commented Nov 14, 2024

View reviewed changes

docs/development/dev-container-setup.md Outdated Show resolved Hide resolved

fredmnl approved these changes Nov 14, 2024

View reviewed changes

anurag4DSB force-pushed the feature/COSI-17-add-dev-container-to-cosi-driver branch from 1eb8281 to dae6da4 Compare November 14, 2024 16:26

anurag4DSB added 3 commits November 14, 2024 17:32

COSI-17: add-dev-container-for-cosi-driver

2513676

- the container sets up minikube, s3 and iam services.

Updated cleanup script and docker container references

dfdaf11

Issue: COSI-17

COSI-17: add-dev-container-docs

c0f97c6

anurag4DSB force-pushed the feature/COSI-17-add-dev-container-to-cosi-driver branch from dae6da4 to c0f97c6 Compare November 14, 2024 16:32

anurag4DSB merged commit 5ca4892 into main Nov 14, 2024
7 checks passed

anurag4DSB deleted the feature/COSI-17-add-dev-container-to-cosi-driver branch November 14, 2024 16:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

COSI-17: Add Dev Container for COSI Driver Development #12

COSI-17: Add Dev Container for COSI Driver Development #12

anurag4DSB commented Nov 13, 2024 •

edited

Loading

codecov bot commented Nov 13, 2024 •

edited

Loading

anurag4DSB Nov 13, 2024

anurag4DSB Nov 13, 2024

dvasilas Nov 13, 2024

anurag4DSB Nov 14, 2024

dvasilas Nov 13, 2024

anurag4DSB Nov 14, 2024

dvasilas Nov 13, 2024

anurag4DSB Nov 14, 2024 •

edited

Loading

dvasilas Nov 13, 2024

anurag4DSB Nov 14, 2024

anurag4DSB commented Nov 13, 2024

fredmnl left a comment

anurag4DSB commented Nov 14, 2024

COSI-17: Add Dev Container for COSI Driver Development #12

COSI-17: Add Dev Container for COSI Driver Development #12

Conversation

anurag4DSB commented Nov 13, 2024 • edited Loading

🎉 Dev Containers 🚀

Summary 📝

codecov bot commented Nov 13, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anurag4DSB Nov 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anurag4DSB commented Nov 13, 2024

fredmnl left a comment

Choose a reason for hiding this comment

anurag4DSB commented Nov 14, 2024

anurag4DSB commented Nov 13, 2024 •

edited

Loading

codecov bot commented Nov 13, 2024 •

edited

Loading

anurag4DSB Nov 14, 2024 •

edited

Loading