Create default passwords when dev mode is set. #441 #442

cmoulliard · 2024-11-08T13:25:20Z

Add new parameter to enable/disable: dev mode
Set the default password for gitea
[Feature] Have a --dev parameter able to configure gitea and argocd with default: username/password #441

Signed-off-by: cmoulliard <[email protected]>

…a new password. cnoe-io#441 Signed-off-by: cmoulliard <[email protected]>

cmoulliard · 2024-11-08T13:57:03Z

Question: Is it the best place to add the code able to generate a new password (= developer) when devMode is enabled -

idpbuilder/pkg/controllers/localbuild/argo.go

Line 53 in 71fefc7

    
           if result, err := argocd.Install(ctx, resource, r.Client, r.Scheme, r.Config); err != nil {

? @nabuskey

nabuskey · 2024-11-08T14:45:39Z

Yeah after install() comes back with no errors.

… - developer. Create a kubeClient part of the k8s util package. cnoe-io#441 Signed-off-by: cmoulliard <[email protected]>

Signed-off-by: cmoulliard <[email protected]>

cmoulliard · 2024-11-09T11:56:59Z

If what I did cannot work, we have 2 options:

Create an Argocd PR to propose to use password instead of generating one usng a new parameter --dev or --insecure-password or --dev-password`
OR
Add to our project the 2 Kube secrets (argocd-secret and argocd-initial-admin-secret) with content populated and where we set the property adminAccount.Enabled to false

nabuskey · 2024-11-09T15:40:12Z

I am still not sure if the flag name is what we want to go with.

Is proving static password the only thing needed for dev mode? Does it describe what it does? If we add more config changes for dev mode in the future, do we want to provide ways to control each change?

If the names I proposed in the issue are too long, can we get away with a short flag?

I do like the idea of having a static default passwords for sure.

cmoulliard · 2024-11-10T08:42:08Z

Is proving static password the only thing needed for dev mode? Does it describe what it does? If we add more config changes for dev mode in the future, do we want to provide ways to control each change?

If we continue to develop idpbuilder, having a devMode help a lot as we do for quarkus devMode able to launch testcontainers: backstage, DB, argocd, gitea, etc and where developers can access many information using dev UI - https://quarkus.io/guides/dev-mode-differences#dev-mode-features :-)

cmoulliard · 2024-11-10T08:42:46Z

If the names I proposed in the issue are too long, can we get away with a short flag?

Let's review that later when we have a PR which is working as --dev is short, enough to test the PR

cmoulliard · 2024-11-10T08:43:01Z

I do like the idea of having a static default passwords for sure.

I will make a try to see if that works

cmoulliard · 2024-11-12T09:05:00Z

Apparently, we should be (to be investigated of course) to set the accounts directly within the argocd secret as discussed here: argoproj/argo-cd#4096 (comment)

I was able to play with a user developer where I installed or customized some resources.

RBAC

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/name: argocd-rbac-cm
    app.kubernetes.io/part-of: argocd
  name: argocd-rbac-cm
  namespace: argocd
data:
  policy.csv: |
    p, role:developer, applications, *, *, allow
    g, developer, role:developer

ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/name: argocd-cm
    app.kubernetes.io/part-of: argocd
  name: argocd-cm
  namespace: argocd
data:
  accounts.developer: apiKey, login
  application.resourceTrackingMethod: annotation
  resource.exclusions: |
    - kinds:
        - ProviderConfigUsage
      apiGroups:
        - "*"
  timeout.reconciliation: 60s

As the following secret is created on the flight by argocd when admin.enabled and generate an admin password using bcrypt, we should find a way to patch it with the developer password

Argocd Secret

apiVersion: v1
kind: Secret
metadata:
  labels:
    app.kubernetes.io/name: argocd-secret
    app.kubernetes.io/part-of: argocd
  name: argocd-secret
  namespace: argocd
type: Opaque
data:
  accounts.developer.password: JDJhJDEwJEYwakhFSmJMYTEvREQzWUZsOWtxN2U0TUZXUXdDclhZWXZNTXhYVXEwdGhVWWUuLi9WWUZL
  accounts.developer.passwordMtime: MjAyNC0xMS0xMlQwODo1NDoyNVo=
...

…veloper's password. cnoe-io#441 Signed-off-by: cmoulliard <[email protected]>

cmoulliard · 2024-11-12T10:20:13Z

The PR supports now to log on to the UI of Argocd using (developer/developer) with RBAC Applications - Admin or gitea (developer/developer)

Can you please review it ?
@nabuskey

Signed-off-by: cmoulliard <[email protected]>

…oper Signed-off-by: cmoulliard <[email protected]>

cmoulliard · 2024-11-12T10:56:36Z

Apparently we cannot use for gitea developer/developer as we got such an error reported when we create a cluster

Nov 12 11:52:39 ERROR Reconciler error controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository name=argocd namespace=idpbuilder-dabou namespace=idpbuilder-dabou name=argocd reconcileID=1f80e800-276a-4b74-a868-b54ff4a26de5 err=creating repository: failed to create git repository: The repository with the same name already exists. 
Nov 12 11:52:39 ERROR Reconciler error controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository name=nginx namespace=idpbuilder-dabou namespace=idpbuilder-dabou name=nginx reconcileID=798effc7-600a-4035-8e39-eb8bcdcb79b3 err=creating repository: failed to create git repository: The repository with the same name already exists. 
Nov 12 11:52:39 ERROR Reconciler error controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository name=gitea namespace=idpbuilder-dabou namespace=idpbuilder-dabou name=gitea reconcileID=bbb8fbdd-2803-4904-b0c5-c680e2a9c0b9 err=creating repository: failed to create git repository: The repository with the same name already exists.

I will then revert to the user giteaAdmin in the meantime

… fails Signed-off-by: cmoulliard <[email protected]>

…for argocd Signed-off-by: cmoulliard <[email protected]>

nabuskey

If you are having problems setting passwords on install, we can change them through APIs after install.

nabuskey · 2024-11-19T22:17:37Z

hack/argo-cd/argocd-rbac-dev.yaml

+  namespace: argocd
+data:
+  policy.csv: |
+    p, role:developer, applications, *, *, allow


What's the purpose of having a separate account that has a very similar permissions as admin?

This is an account for the developers and it only allows to handle applications

Can you tell me if this is inline with what you are thinking?

Use a random password for the developer and admin account if the dev password flag is unset.

Use a static, known password for the developer and admin account if the dev password flag is set.

Option 2 => Use a known password for the developer and admin account if the dev password flag is set

Ok that sounds good to me. Looks like Gitea static password isn't working for some reason?

Ok that sounds good to me. Looks like Gitea static password isn't working for some reason?

For gitea when using dev-mode, we should still use as user: giteaAdmin and password = developer

Since you are adding a user called developer to argocd, we may as well add the developer user in Gitea.

nabuskey · 2024-11-19T22:25:54Z

pkg/controllers/localbuild/argo.go

+			return ctrl.Result{}, fmt.Errorf("Error marshalling patch data: %w", err)
+		}
+
+		kubeClient, err := k8s.GetKubeClient()


The reconciler has client already. You can just use that instead. e.g. r.Client.. You can also change passwords through ArgoCD api.

You can also change passwords through ArgoCD api.

That will require to be done in a 2nd step while here the idea is to have the account developer/developer available when packages are installed too

r.Client fixed. See: 7d81b27

You can also change passwords through ArgoCD api.

That will require to be done in a 2nd step while here the idea is to have the account developer/developer available when packages are installed too

Yeah I am starting to think we should set them through API for both Gitea and ArgoCD in every invocation of idpbuilder. Instead of relying on manifests.

We should then (maybe) create some job(s) which are applied post installation of the core package like gitea or argocd and able to execute some additional steps based on parameters passed.

Until now, the first action could be to create a new user: developer able to log on using as user/pwd => developer/developer or idpbuilder/developer if the parameter passed is --dev-mode.

A second job could also patch existing resources if by example we pass a different DNS domain = --host

etc

flag renamed from "dev" to "dev-mode". See: a8c3016

nabuskey · 2024-11-19T22:28:18Z

pkg/controllers/localbuild/argo.go

+		err = kubeClient.Get(ctx, client.ObjectKey{Name: argocdAdminSecretName, Namespace: argocdNamespace}, &s)
+		if err != nil {
+			return ctrl.Result{}, fmt.Errorf("getting argocd secret: %w", err)
+		}
+
+		// Patching the argocd-secret with the user's hashed password
+		err = kubeClient.Patch(ctx, &s, client.RawPatch(types.StrategicMergePatchType, patchBytes))
+		if err != nil {
+			return ctrl.Result{}, fmt.Errorf("Error patching the Secret: %w", err)
+		}


You can use server side apply instead here to simplify this process.

Ok. I will review

What do you mean by "server side" ? My code is similar to what we do here with the Gitea -

idpbuilder/pkg/controllers/localbuild/gitea.go

Line 189 in f31a08e

return r.Client.Patch(ctx, &u, client.Apply, client.ForceOwnership, client.FieldOwner(v1alpha1.FieldManager))

...

The way it's done in Gitea is server side apply. If you look further up, you'll see it's working with unstructured object to avoid having ownership of fields we do not care about. Sometimes we see errors like "object has been updated and version doesn't match". Server side apply avoids that.

This talks about it: https://eng.d2iq.com/blog/conflict-resolution-kubernetes-server-side-apply/

Here's where unstructured object is used:

idpbuilder/pkg/controllers/localbuild/gitea.go

Lines 163 to 166 in f31a08e

u := unstructured.Unstructured{}

u.SetName(giteaAdminSecret)

u.SetNamespace(giteaNamespace)

u.SetGroupVersionKind(corev1.SchemeGroupVersion.WithKind("Secret"))

Here's another example:

idpbuilder/pkg/util/util.go

Lines 78 to 86 in d14289a

func ApplyAnnotation(ctx context.Context, kubeClient client.Client, obj client.Object, annotations map[string]string, opts ...client.PatchOption) error {

// MUST be unstructured to avoid managing fields we do not care about.

u := unstructured.Unstructured{}

u.SetAnnotations(annotations)

u.SetName(obj.GetName())

u.SetNamespace(obj.GetNamespace())

u.SetGroupVersionKind(obj.GetObjectKind().GroupVersionKind())

return kubeClient.Patch(ctx, &u, client.Apply, opts...)

}

Fixed. See: 80a785b

nabuskey · 2024-11-19T22:29:23Z

pkg/controllers/localbuild/argo.go

+	argocdDevModePassword         = "developer"
+	argocdAdminSecretName         = "argocd-secret"
+	argocdInitialAdminSecretName  = "argocd-initial-admin-secret"
+	argocdInitialAdminPasswordKey = "argocd-initial-admin-secret"
+	argocdNamespace               = "argocd"


These are already defined in other packages I think. Either export them or move them to globals or APIs

Ok. I will review

I removed 2 non used constants and others are non declared in another go file or package. See: 78e1e40

nabuskey · 2024-11-19T22:35:23Z

pkg/cmd/create/root.go

@@ -20,6 +20,7 @@ import (
 const (
 	recreateClusterUsage   = "Delete cluster first if it already exists."
 	buildNameUsage         = "Name for build (Prefix for kind cluster name, pod names, etc)."
+	devModeUsage           = "When enabled, the platform will run the core packages with developer password."


I am just not sold on the name dev mode. idpbuilder is primary used for development and testing purposes already. I think we should use more explicit name with a short flag.

Even if we accept dev mode, we shouldn't limit the flag name to just passwords. dev mode could mean a lot of different things. This should be a meta flag.

idpbuilder is primary used for development and testing purposes already. I think we should use more explicit name with a short flag.

If this is the case, then we should install gitea and argocd using a non generated password ;-)

2. Even if we accept dev mode, we shouldn't limit the flag name to just passwords. dev mode could mean a lot of different things. This should be a meta flag.

What about using the the parameter: --devPwd or --devPassword as boolean @nabuskey

I am ok with dev-pwd or dev-password since we use snake case for our flags. We can also have --dev-mode flag that is essentially a convenient flag that enables dev passwords and any other QOL stuff for dev purposes.

Finally we aligned our paths => I will then use --dev-mode

To be clear, I meant to have two flags. (--dev-password OR --dev-pwd ) AND --dev-mode because we could enable other QOL features under --dev-mode other than static passwords.

nabuskey · 2024-11-19T22:41:52Z

pkg/k8s/util.go


 	"github.com/cnoe-io/idpbuilder/pkg/util"
 	"k8s.io/apimachinery/pkg/runtime"
 	"sigs.k8s.io/controller-runtime/pkg/client"
 )

+var (
+	setupLog = ctrl.Log.WithName("k8s")


I would rather not produce log messages in utility functions. Let's wrap the error instead. e.g. fmt.Errorf("Error creating kubernetes client: %w"

Fixed with: 8158551

…orf() Signed-off-by: cmoulliard <[email protected]>

Signed-off-by: cmoulliard <[email protected]>

…out. Signed-off-by: cmoulliard <[email protected]>

Signed-off-by: cmoulliard <[email protected]>

cmoulliard · 2024-11-22T11:06:06Z

I pushed a commit: 3c3511d to show the developer username/password @nabuskey

Remark: This code should be reviewed and improved as the secret containing the developer username/password is argocd-secret where the password has been bcrypted and by consequence we cannot get and decode it from the secret
This is why it has been included as such and will be displayed every time no matter if --dev-mode has been used or not

nabuskey · 2024-11-22T15:53:42Z

pkg/cmd/get/secrets.go

+	// TODO: The following code should be reviewed and improved as the secret containing the developer username/password is argocd-secret
+	// where the password has been bcrypted and by consequence we cannot get and decode it from the secret
+	// This is why we are going to add it here BUT it will be displayed every time no matter if --dev-mode has been used or not
+	if strings.Contains(s.Name, "gitea") {
+		data.Data["username-developer"] = "giteAdmin"
+		data.Data["password-developer"] = "developer"
+	} else if strings.Contains(s.Name, "argocd") {
+		data.Data["username-developer"] = "developer"
+		data.Data["password-developer"] = "developer"
+	}
+


We can get the localbuild object from K8s since it's essentially our way of storing configurations. You added the DevMode field so you can just check the value. We should change that field name btw. devPassword makes sense. There's a typo: giteAdmin -> giteaAdmin

We can get the localbuild object from K8s since it's essentially our way of storing configurations.

Yes but the flag --dev-mode or --dev-password is not passed at all to the command: idp get secrets and users will not like to perform such a command idp get secrets --dev-mode.

nabuskey · 2024-11-22T16:02:00Z

The more I think about it, I think it's cleaner to change passwords and add users post install. This enables flows like:

Create a cluster without specifying the static password flag. e.g. idpbuilder create
Run the same command except we specify the static password flag. idpbuilder create --dev-password

cmoulliard · 2024-11-22T17:07:58Z

The more I think about it, I think it's cleaner to change passwords and add users post install

I'm also in favor to go down that road as I mentioned also in a different comments where we could use job or something else able to perform additional config steps or let's say post install steps.

2 remarks:

Such approach will nevertheless don't allow the user to watch directly using developr password the information avaialble from the web application using UI except if they use the generated password
We should avoid to request to the user to re-apply the command idpbuilder create with additional flags as that will create confusion.
A better approach would be to have a different command to install the packages and their config, execute some "actions" on a core package or package using a job, etc.
Idea about such commands could be:
idp package add <package_name> or <package1_name,package2_name2,etc>
idp package execute <job1> // for the post-installation tasks:

cmoulliard · 2024-11-27T07:42:35Z

As the result about what I did and also what I discovered imply that finally we hack the existing code to support to generate a developer password, should we close it and come with a better solution as proposed here: #442 (comment)

WDYT ? @nabuskey

nabuskey · 2024-11-27T18:37:41Z

2 remarks:
Such approach will nevertheless don't allow the user to watch directly using developr password the information avaialble from the web application using UI except if they use the generated password
We should avoid to request to the user to re-apply the command idpbuilder create with additional flags as that will create confusion.
A better approach would be to have a different command to install the packages and their config, execute some "actions" on a core package or package using a job, etc.
Idea about such commands could be:
idp package add <package_name> or <package1_name,package2_name2,etc>
idp package execute // for the post-installation tasks:

What I meant was to do this in post-install (e.g. the core packages are ready) during the reconciliation loop. Essentially around here:

idpbuilder/pkg/controllers/localbuild/controller.go

Line 81 in 03559e6

go r.installCorePackages(instCtx, req, &localBuild, errChan)

Instead of using manifests to set initial passwords, change them to a known value if the dev password flag is set.

We should avoid to request to the user to re-apply the command idpbuilder create with additional flags as that will create confusion.

I agree with this. We should have a separate command to configure / reconfigure things.

package execute
This implies we need to create a configuration file for each package. I have been resisting the idea of creating yet another configuration file. Maybe we end up needing to create one because current approach of using argocd resource hooks is not too user friendly. BUT for this particular issue of setting static passwords, it's not necessary imo.

cmoulliard added 3 commits November 8, 2024 14:24

WIP. Create default passwords when dev mode is set. cnoe-io#441

3a974cb

Signed-off-by: cmoulliard <[email protected]>

Set the devMode to BuildCustomization

a9c683f

Signed-off-by: cmoulliard <[email protected]>

Pass the proper object containing devMode to the function populating …

6d85851

…a new password. cnoe-io#441 Signed-off-by: cmoulliard <[email protected]>

cmoulliard added 3 commits November 8, 2024 16:10

Adding the code to patch the argocd-secret to use the hashed password…

8dedc1e

… - developer. Create a kubeClient part of the k8s util package. cnoe-io#441 Signed-off-by: cmoulliard <[email protected]>

Use cost 0 as argocd code cnoe-io#441

704bc78

Signed-off-by: cmoulliard <[email protected]>

Re-creating too the inital admin secret for argocd. cnoe-io#441

ee9dc06

Signed-off-by: cmoulliard <[email protected]>

This comment was marked as resolved.

Sign in to view

Move the current time function. Generate the missing field. cnoe-io#441

d392287

Signed-off-by: cmoulliard <[email protected]>

cmoulliard marked this pull request as ready for review November 8, 2024 17:21

cmoulliard requested a review from nabuskey November 8, 2024 17:32

This comment was marked as resolved.

Sign in to view

Include needed argocd k8s resources and change the code to set the de…

0c62564

…veloper's password. cnoe-io#441 Signed-off-by: cmoulliard <[email protected]>

cmoulliard added 2 commits November 12, 2024 11:20

Fix wrong key as account should be accounts

1e1b980

Signed-off-by: cmoulliard <[email protected]>

Change the gitea user from giteaAdmin to developer for a user's devel…

3d884da

…oper Signed-off-by: cmoulliard <[email protected]>

cmoulliard added 2 commits November 12, 2024 11:58

Reverting to giteaAdmin till we know why a different user - developer…

f31a08e

… fails Signed-off-by: cmoulliard <[email protected]>

Change the number from 58 to 59 as we install a new ConfigMap - RBAC …

c355ba1

…for argocd Signed-off-by: cmoulliard <[email protected]>

nabuskey reviewed Nov 19, 2024

View reviewed changes

cmoulliard added 3 commits November 21, 2024 15:22

Remove from the util go file the setupLog and replace it with fmt.Err…

8158551

…orf() Signed-off-by: cmoulliard <[email protected]>

Removed non used argocd constants

78e1e40

Signed-off-by: cmoulliard <[email protected]>

Use r.client instead of k8s.GetKubeClient()

7d81b27

Signed-off-by: cmoulliard <[email protected]>

cmoulliard added 3 commits November 21, 2024 17:48

Rename the flag from dev to dev-mode

a8c3016

Signed-off-by: cmoulliard <[email protected]>

Use an unstructured object to avoid managing fields we do not care ab…

80a785b

…out. Signed-off-by: cmoulliard <[email protected]>

Include the developer username and password to the command get secret

3c3511d

Signed-off-by: cmoulliard <[email protected]>

nabuskey reviewed Nov 22, 2024

View reviewed changes

	u := unstructured.Unstructured{}
	u.SetName(giteaAdminSecret)
	u.SetNamespace(giteaNamespace)
	u.SetGroupVersionKind(corev1.SchemeGroupVersion.WithKind("Secret"))

	func ApplyAnnotation(ctx context.Context, kubeClient client.Client, obj client.Object, annotations map[string]string, opts ...client.PatchOption) error {
	// MUST be unstructured to avoid managing fields we do not care about.
	u := unstructured.Unstructured{}
	u.SetAnnotations(annotations)
	u.SetName(obj.GetName())
	u.SetNamespace(obj.GetNamespace())
	u.SetGroupVersionKind(obj.GetObjectKind().GroupVersionKind())
	return kubeClient.Patch(ctx, &u, client.Apply, opts...)
	}

Create default passwords when dev mode is set. #441 #442

Are you sure you want to change the base?

Create default passwords when dev mode is set. #441 #442

Conversation

cmoulliard commented Nov 8, 2024

cmoulliard commented Nov 8, 2024

nabuskey commented Nov 8, 2024

This comment was marked as resolved.

cmoulliard commented Nov 9, 2024

nabuskey commented Nov 9, 2024

cmoulliard commented Nov 10, 2024

cmoulliard commented Nov 10, 2024

cmoulliard commented Nov 10, 2024

This comment was marked as resolved.

cmoulliard commented Nov 12, 2024 • edited Loading

cmoulliard commented Nov 12, 2024 • edited Loading

cmoulliard commented Nov 12, 2024

nabuskey left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nabuskey Nov 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nabuskey Nov 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmoulliard commented Nov 22, 2024

nabuskey Nov 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nabuskey commented Nov 22, 2024

cmoulliard commented Nov 22, 2024

cmoulliard commented Nov 27, 2024

nabuskey commented Nov 27, 2024

cmoulliard commented Nov 12, 2024 •

edited

Loading

cmoulliard commented Nov 12, 2024 •

edited

Loading

nabuskey Nov 21, 2024 •

edited

Loading

nabuskey Nov 21, 2024 •

edited

Loading

nabuskey Nov 22, 2024 •

edited

Loading