Add support for Unix sockets on Windows

[matthewvalimaki]

On Windows 10 I have gcloud setup and working. I downloaded the binary and without any arguments (per Using automatic instance discovery with gcloud credentials) I saw:

2017/04/18 11:51:31 Using gcloud's active project: myproject-12345
2017/04/18 11:51:33 mkdir myproject-12345:us-central1:test: The filename, directory name, or volume label syntax is incorrect.
2017/04/18 11:51:33 errors parsing config:
        mkdir myproject-12345:us-central1:test: The filename, directory name, or volume label syntax is incorrect.

I guess this is to be expected as Windows does not support : in directory names, but perhaps by default the character should be - or changed for Windows specifically.

After changing the separator in code to - it worked but then I hit:

2017/04/18 11:58:57 Using gcloud's active project: myproject-12345
2017/04/18 11:58:59 errors parsing config:
        invalid "myproject-12345:us-central1:test": unsupported network: unix

And in code it specifically checks if OS is Windows and removes unix from supported list. I wonder if this could be default tcp on Windows?

In the end I got things to work with this: .\cloud_sql_proxy.exe -instances=myproject-12345:us-central1:test=tcp:5433.

[Carrotman42]

This is specifically for postgres+Windows. Note that non-tcp isn't supported for Windows+MySQL instances, but I think another (more specific) error comes out.

[chatterjee1977]

@matthewvalimaki You are a life saver!!!! Wish I had checked the Issues section here first. Could you enumerate for my understanding why the tcp:5433 ?

[matthewvalimaki]

@chatterjee1977 the tcp is required to tell the tool not to use unix (which is not supported on Windows anyhow) and the port is a requirement by the tool itself. Documentation says For connectivity over TCP, you must specify a tcp port as part of the instance string. For example, the following example opens a loopback TCP socket on port 3306, which will be proxied to connect to the instance.

The tool does not make any guesses when it comes to protocol or port when on Windows. My question in the original was if tcp could be made default for Windows just like unix is default for the other systems.

[chatterjee1977]

On a separate page located here
https://cloud.google.com/sql/docs/mysql/connect-admin-proxy

This has the steps to invoke this from a Windows machine with the following command line after creating a service account

./cloud_sql_proxy -instances=<INSTANCE_CONNECTION_NAME>=tcp:3306
-credential_file=<PATH_TO_KEY_FILE> &

[Carrotman42]

Other than defaulting to TCP on windows, another option is to add support named pipes in windows, which works for just MySQL (not postgres).

I've considered doing tcp by default on windows, but I am mostly afraid of people being confused about which port the Proxy starts listening on. There's definitely some things that can be done here, but I don't personally have a lot of time to look into it. Hopefully the code is simple enough for people to send pull requests (but admittedly, the logic to set up new connections is a bit complex).

[dmitry-weirdo]

Hello, guys.

I'm trying to connect to my postgreSQL instance and get the same error when starting cloud_sql_proxy_x64.exe on Windows:

mkdir dpopov-gcp-spring-boot:us-central1:dpopov-postgres: The filename, directory name, or volume label syntax is incorrect.

--instances parameter with =tcp:3306 or tcp:5433 does not seem to work, producing the same output.

Could you please write how exactly do you worked around Linux directories creation on Windows?

Update: @matthewvalimaki as I understand from your message, you've rebuilt the exe-file with fixed separators. Could you please send/attach this file somehow? Or at least show where to fix it in the sources?

[Carrotman42]

Any sort of attempt to work with unix sockets will not work on windows; you must use the tcp option to get things working.

Can you share the exact command line which you used which included the extra TCP part? The fact that you say it failed with the same output makes me think you have a typo. It should not print the same output

[dmitry-weirdo]

Hello, @Carrotman42, thanks for answering.

Hm, I double-checked the spelling and tried this:
cloud_sql_proxy_x64.exe -instances=dpopov-gcp-spring-boot:us-central1:dpopov-postgres=tcp:5433

and it seems to be working!

c:\java\gcp\cloud_sql_proxy>cloud_sql_proxy_x64.exe -instances=dpopov-gcp-spring-boot:us-central1:dpopov-postgres=tcp:5433
2018/08/24 17:27:13 Listening on 127.0.0.1:5433 for dpopov-gcp-spring-boot:us-central1:dpopov-postgres
2018/08/24 17:27:13 Ready for new connections

Thanks a lot for pointing!

Ha-ha, and I already found a place to replace the directory in proxy.go, method parseInstanceConfig, branch if strings.HasPrefix(strings.ToLower(in.DatabaseVersion), "postgres") {, replace : with - in instance variable. It works but I get the same

        invalid "dpopov-gcp-spring-boot:us-central1:dpopov-postgres": unsupported network: unix 

as in the first comment.

But it looks like the public version of exe is actually working, and I've just mistyped the tcp parameter.
Thanks a lot for your help!

[Carrotman42]

Note: the new version of Go supports unix sockets on windows: https://tip.golang.org/doc/go1.12#syscall

Not sure if MySQL's CLI supports it, though.

[royappa]

This feature would be helpful, because I develop on Windows with local firebase emulation and proxy. Currently I have to instrument the firebase function with host+ip connectivity to be able to use the local emulator, whereas in the cloud it uses socketPath. It would be nice to not have to switch between the two. Thank you.

[devfemibadmus]

and still none of the above is able to solve this

[enocom]

It's not ideal to have a mismatch between a dev environment (Windows with TCP sockets) and a prod environment (Linux with a Unix socket), but right now that's the way to workaround this limitation.

One option would be to connect to a database using the host as an environment variable that would be the full Unix socket path in the deployed function, and localhost with a port on your dev machine. That's basically what @royappa is doing, I think.

[enocom]

There is support for Unix sockets on Windows in Go now. At a minimum, I'd like to adopt an approach that works for the various CLIs that support it (mysql, psql). I don't think mssql-cli would support it though.

[enocom]

FYI As part of the V2 work, we'll be able to support Unix sockets on Windows. #1182

[enocom]

The v2 release includes support for Unix sockets on Windows.

Get the latest version here: https://github.com/GoogleCloudPlatform/cloud-sql-proxy/releases/tag/v2.0.0.preview.0.

See the documentation here: https://github.com/GoogleCloudPlatform/cloud-sql-proxy#configuring-unix-domain-sockets.

[royappa]

Thank you!