How to run a command from reconcile inside of a pod/container?

[camilamacedo86]

Type of question

Usage

Question

What did you do?
Getting a pod and trying to kubectl exec by using the client libs available.

What did you expect to see?
Do something as :

	cmd := []string{
		"/bin/sh",
		"-c",
		"pg_dump mobile_security_service | gzip > mobile_security_service.gz",
	}

       command(cmd).Run()

API example:

/api/v1/namespaces/project-1/pods/pod-1-lmlzj/exec?command=/bin/bash&command=-c&command=/bin/bash&stdin=true&stderr=true&stdout=true&tty=true

[lilic]

@camilamacedo86 Can you explain a bit more, not sure I understand the reconcile part.

[cmoulliard]

Why don't you use the k8s client to execute this command - see https://github.com/snowdrop/kreate/blob/78f0b5ee8e27117b78fe9d6d5192bc5b04c0e5db/pkg/k8s/client.go#L243 ? @camilamacedo86

[camilamacedo86]

Hi @lilic, I'd like to execute a command inside of the container in the reconcile func like kubectl exec

[camilamacedo86]

I tried the following impl and it did not work as well. The restClient is nil. I do not think will be possible to use it inside of the container.

// Get the config
	cfg, err := config.GetConfig()
	if err != nil {
		return reconcile.Result{}, err
	}

	// Get The Rest client for this config
	restClient, _ := rest.RESTClientFor(cfg);

	// Do the request
	req := restClient.Post().
		Namespace(pod.Namespace).
		Resource("pods").
		Name(pod.Name).
		SubResource("exec").
		VersionedParams(&corev1.PodExecOptions{
			Command: cmd,
		}, scheme.ParameterCodec)

	// Connect to url (constructed from req) using SPDY (HTTP/2) protocol which allows bidirectional streams.
	exec, err := remotecommand.NewSPDYExecutor(cfg, "POST", req.URL())
	if err != nil {
		reqLogger.Error(err, "cmd", cmd, "pod", pod.Name)
		return reconcile.Result{}, err
	}

	// initialize the transport of the standard shell streams
	err = exec.Stream(remotecommand.StreamOptions{})
	if err != nil {
		reqLogger.Error(err, "cmd", cmd, "pod", pod.Name)
		return reconcile.Result{}, err

	}

Could you help me here? How can I execute a command inside of the containers of the pods managed by the operator? PS.: I cannot add the command via args when the deployment is made for example. In my case, I would like to do a bkp of the database and move the file for another place.

[camilamacedo86]

@shawn-hurley @lilic @cmoulliard any ideas?

[shawn-hurley]

I think you need to change some of the options of your impl to get it to work.

https://github.com/snowdrop/kreate/blob/78f0b5ee8e27117b78fe9d6d5192bc5b04c0e5db/pkg/k8s/client.go#L251

https://github.com/snowdrop/kreate/blob/78f0b5ee8e27117b78fe9d6d5192bc5b04c0e5db/pkg/k8s/client.go#L270

[camilamacedo86]

Hi @shawn-hurley,

Really tks. However, I am still with an issue before that. The following code will return null.

	// Get The Rest client for this config
	restClient, _ := rest.RESTClientFor(cfg);

PS.: The cfg is that one injects in the manager. Have you any idea to solve it?

[cmoulliard]

some of the options of your impl to get it to work.

Such examples will work as the restclient is created outside of the k8s cluster, We need for the Camila use case, something like this but able to working within the cluster

https://gist.github.com/cmoulliard/ec4dac1275042643bab7f4ba2867604f#file-gistfile1-txt-L279

[lilic]

@camilamacedo86 if you want to do the request directly we do it like this:

operator-sdk/test/e2e/memcached_test.go

Lines 792 to 801 in c8ead55

	func proxyViaPod(kubeClient kubernetes.Interface, namespace, podName, podPortName, path string) *rest.Request {
	return kubeClient.
	CoreV1().
	RESTClient().
	Get().
	Namespace(namespace).
	Resource("pods").
	SubResource("proxy").
	Name(fmt.Sprintf("%s:%s", podName, podPortName)).
	Suffix(path)

and to get the client we do:

operator-sdk/pkg/test/framework.go

Lines 72 to 79 in c8ead55

	kubeconfig, kcNamespace, err = k8sInternal.GetKubeconfigAndNamespace(*kubeconfigPath)
	if *singleNamespace && namespace == "" {
	namespace = kcNamespace
	}
	if err != nil {
	return fmt.Errorf("failed to build the kubeconfig: %v", err)
	}
	kubeclient, err := kubernetes.NewForConfig(kubeconfig)

You should be able to do direct requests then, haven't tried it myself.

[camilamacedo86]

Hi @lilic I am not sure if it would attend this need as well. Note that is required the kubeconfigPath. From inside of the cluster it is one path (when the operator been executed) and from outside of the cluster (when we run operator-sdk up local) is another path. I believe that the test file added here is used in the e2e which is running outside of the cluster.

[camilamacedo86]

Marco shared: https://github.com/zalando/postgres-operator/blob/master/pkg/cluster/exec.go#L44

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[anikin-aa]

Guys, any ideas here ? Or we should just use kubeClient ?

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci-robot]

@openshift-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[sarroutbi]

Marco shared: https://github.com/zalando/postgres-operator/blob/master/pkg/cluster/exec.go#L44

Hello @camilamacedo86. Is this alternative currently working? If so ... have you noticed any issue with it (memory leaks or related)?

I have used a similar approach, but my controller crashes continuously. It seems to be related to this:
kubernetes/kubernetes#105830