Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement publishing API UNIX socket on Windows platforms #23409

Merged
merged 1 commit into from
Oct 11, 2024

Conversation

arixmkii
Copy link
Contributor

Fixes #23408

gvproxy and win-sshproxy have capabilities to serve this type of enpoint. This change only adds one additional API enpoint publishing by appending proxy command lines.

Originally developed within #13006 but could be generalized to any VM using gvproxy or win-sshproxy. This has been verified to work for Hyper-V and WSL machines.

Example how it looks for WSL machine.
commands

  • podman machine list (to show what kind of machine it is)
  • podman machine inspect (to demo published endpoint in info)
  • where curl (to verify it will use Windows one)
  • curl to call the enpoint
C:\qcw-utils\shells>podman machine list
NAME                     VM TYPE     CREATED         LAST UP            CPUS        MEMORY      DISK SIZE
podman-machine-default*  wsl         43 seconds ago  Currently running  2           2GiB        100GiB

C:\qcw-utils\shells>podman machine inspect
[
     {
          "ConfigDir": {
               "Path": "C:\\Users\\User\\.config\\containers\\podman\\machine\\wsl"
          },
          "ConnectionInfo": {
               "PodmanSocket": {
                    "Path": "C:\\Users\\User\\AppData\\Local\\Temp\\podman\\podman-machine-default-api.sock"
               },
               "PodmanPipe": {
                    "Path": "\\\\.\\pipe\\podman-machine-default"
               }
          },
          "Created": "2024-07-26T12:57:29.14137+03:00",
          "LastUp": "0001-01-01T00:00:00Z",
          "Name": "podman-machine-default",
          "Resources": {
               "CPUs": 2,
               "DiskSize": 100,
               "Memory": 2048,
               "USBs": []
          },
          "SSHConfig": {
               "IdentityPath": "C:\\Users\\User\\.local\\share\\containers\\podman\\machine\\machine",
               "Port": 50239,
               "RemoteUsername": "user"
          },
          "State": "running",
          "UserModeNetworking": false,
          "Rootful": false,
          "Rosetta": false
     }
]

C:\qcw-utils\shells>where curl
C:\Windows\System32\curl.exe

C:\qcw-utils\shells>curl --unix-socket C:\\Users\\User\\AppData\\Local\\Temp\\podman\\podman-machine-default-api.sock h
ttp://d/v5.0.0/libpod/info
{"host":{"arch":"amd64","buildahVersion":"1.36.0","cgroupManager":"cgroupfs","cgroupVersion":"v1","cgroupControllers":[],"conmon":{"package":"conmon-2.1.10-1.fc40.x86_64","path":"/usr/bin/conmon","version":"conmon version 2.1.10, commit: "},"cpus":4,"cpuUtilization":{"userPercent":0.86,"systemPercent":1.54,"idlePercent":97.6},"databaseBackend":"sqlite","distribution":{"distribution":"fedora","variant":"container","version":"40"},"eventLogger":"journald","freeLocks":2048,"hostname":"glider-mk2","idMappings":{"gidmap":[{"container_id":0,"host_id":1000,"size":1},{"container_id":1,"host_id":524288,"size":65536}],"uidmap":[{"container_id":0,"host_id":1000,"size":1},{"container_id":1,"host_id":524288,"size":65536}]},"kernel":"5.15.153.1-microsoft-standard-WSL2","logDriver":"journald","memFree":16180596736,"memTotal":16773939200,"networkBackend":"netavark","networkBackendInfo":{"backend":"netavark","version":"netavark 1.11.0","package":"netavark-1.11.0-1.fc40.x86_64","path":"/usr/libexec/podman/netavark","dns":{"version":"aardvark-dns 1.11.0","package":"aardvark-dns-1.11.0-1.fc40.x86_64","path":"/usr/libexec/podman/aardvark-dns"}},"ociRuntime":{"name":"crun","package":"crun-1.15-1.fc40.x86_64","path":"/usr/bin/crun","version":"crun version 1.15\ncommit: e6eacaf4034e84185fd8780ac9262bbf57082278\nrundir: /run/user/1000/crun\nspec: 1.0.0\n+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL"},"os":"linux","remoteSocket":{"path":"/run/user/1000/podman/podman.sock","exists":true},"rootlessNetworkCmd":"pasta","serviceIsRemote":false,"security":{"apparmorEnabled":false,"capabilities":"CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT","rootless":true,"seccompEnabled":true,"seccompProfilePath":"/usr/share/containers/seccomp.json","selinuxEnabled":false},"slirp4netns":{"executable":"","package":"","version":""},"pasta":{"executable":"/usr/bin/pasta","package":"passt-0^20240624.g1ee2eca-1.fc40.x86_64","version":"pasta 0^20240624.g1ee2eca-1.fc40.x86_64\nCopyright Red Hat\nGNU General Public License, version 2 or later\n  \u003chttps://www.gnu.org/licenses/old-licenses/gpl-2.0.html\u003e\nThis is free software: you are free to change and redistribute it.\nThere is NO WARRANTY, to the extent permitted by law.\n"},"swapFree":4294967296,"swapTotal":4294967296,"uptime":"0h 1m 29.00s","variant":"","linkmode":"dynamic"},"store":{"configFile":"/home/user/.config/containers/storage.conf","containerStore":{"number":0,"paused":0,"running":0,"stopped":0},"graphDriverName":"overlay","graphOptions":{},"graphRoot":"/home/user/.local/share/containers/storage","graphRootAllocated":1081101176832,"graphRootUsed":790700032,"graphStatus":{"Backing Filesystem":"extfs","Native Overlay Diff":"true","Supports d_type":"true","Supports shifting":"false","Supports volatile":"true","Using metacopy":"false"},"imageCopyTmpDir":"/var/tmp","imageStore":{"number":0},"runRoot":"/run/user/1000/containers","volumePath":"/home/user/.local/share/containers/storage/volumes","transientStore":false},"registries":{"search":["docker.io"]},"plugins":{"volume":["local"],"network":["bridge","macvlan","ipvlan"],"log":["k8s-file","none","passthrough","journald"],"authorization":null},"version":{"APIVersion":"5.1.2","Version":"5.1.2","GoVersion":"go1.22.5","GitCommit":"","BuiltTime":"Wed Jul 10 03:00:00 2024","Built":1720569600,"OsArch":"linux/amd64","Os":"linux"}}

Does this PR introduce a user-facing change?

Additionally provide API access via UNIX sockets to Podman running in Podman Machine on Windows

@arixmkii
Copy link
Contributor Author

I can't find any tests for sockets or pipes in e2e tests of the machine, so, I can't extend these tests for additional verification. There is limited value trying to test gvproxy or win-sshproxy command builders as it would be close to testing getters/setters.

@baude
Copy link
Member

baude commented Aug 7, 2024

@l0rd any opinions on this?

@l0rd
Copy link
Member

l0rd commented Aug 7, 2024

@l0rd any opinions on this?

I am fine with the original issue. I think that's useful. Now a few considerations:

  1. AF_UNIX requires build 17061 or higher. Podman requires build 19041 or higher. We are fine on that side.
  2. We need automated tests to make sure that we don't break this on both hyper-v / wsl
  3. That's an optional feature that can break the machine start. Therefore we should have a config option to disable it

@arixmkii
Copy link
Contributor Author

arixmkii commented Aug 8, 2024

Therefore we should have a config option to disable it

I will work on this. If you someone can share a good example of a feature toggle I can use as inspiration this would speed up things definitely.

We need automated tests to make sure that we don't break this on both hyper-v / wsl

I will try to sketch some tests and then they could be improved during review process.

Having this accepted as an idea for a feature is good enough for me now to continue improving this changeset.

@arixmkii
Copy link
Contributor Author

Added tests for the API. Client for Unix sockets and Named Pipes. Curl for Unix sockets. To not skip tests of Client on Windows it is needed to have a fix in go-connections utilities first docker/go-connections#116

Will work on the config changes for the feature toggle.

gvproxy and win-sshproxy have capabilities to serve this type of enpoint.
This change only adds one additional API enpoint publishing by appending
proxy command lines.

Signed-off-by: Arthur Sengileyev <[email protected]>
@rhatdan
Copy link
Member

rhatdan commented Sep 3, 2024

@l0rd @baude PTAL

@l0rd
Copy link
Member

l0rd commented Sep 4, 2024

@arixmkii thank you for updating the PR. I didn't had the chance to review it yet but I plan to do it tomorrow.

Copy link
Member

@l0rd l0rd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to manually test the Unix socket on Windows and the tests looks good (I don't think docker/go-connections#116 is a blocker). Looking forward for the config toggle.

@baude
Copy link
Member

baude commented Sep 17, 2024

code changes LGTM ... @l0rd are you good with merging this?

@l0rd
Copy link
Member

l0rd commented Sep 17, 2024

There is no way to disable the exposure of the Unix API socket yet but in the current state the PR already adds some value so +1 for me to merge if @arixmkii is ok.

@arixmkii
Copy link
Contributor Author

@baude @l0rd would be nice to merge this if possible. I don't see reasonable code improvements for this feature, but feature toggle will be an additional feature itself.

@l0rd
Copy link
Member

l0rd commented Sep 20, 2024

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 20, 2024
Copy link
Contributor

openshift-ci bot commented Oct 11, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: arixmkii, l0rd

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 11, 2024
@openshift-merge-bot openshift-merge-bot bot merged commit f311979 into containers:main Oct 11, 2024
91 checks passed
@arixmkii arixmkii deleted the unix-api-socket branch November 4, 2024 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. machine release-note
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Publish API UNIX socket for Windows Podman machines
4 participants