hostname args ineffective in background mode on openconnect

[entrity]

When I use vpn-slice + openconnect in the background, I observe some bad behaviour: the presence of any hostname arguments for vpn-slice appears to eliminate the effect of vpn-slice.

sudo openconnect --background -s "vpn-slice 10.0.0.0/8 example.com"
# => Does not update /etc/hosts

sudo openconnect --background -s "vpn-slice 10.0.0.0/8"
# => Updates /etc/hosts, but not for the hostnames that I need to specify manually

sudo openconnect -s "vpn-slice 10.0.0.0/8 example.com"
# => Updates /etc/hosts, including the hostnames that I need to specify manually
# ...but this does not move to the background

I don't want to leave a terminal open for this; I want to run this in the background. I've even tried using sudo setsid openconnect (with --passwd-on-stdin) instead of using --background, but the effect is the same.

[dlenski]

When I use vpn-slice + openconnect in the background, I observe some bad behaviour: the presence of any hostname arguments for vpn-slice appears to eliminate the effect of vpn-slice.

OS? Version of vpn-slice? Version of openconnect?

sudo openconnect --background -s "vpn-slice 10.0.0.0/8 example.com"
# => Does not update /etc/hosts

Run with vpn-slice --verbose --dump so we can see what it is (and isn't) doing on each hook where it's called by OpenConnect.

sudo openconnect --background -s "vpn-slice 10.0.0.0/8"
# => Updates /etc/hosts, but not for the hostnames that I need to specify manually

Same, please collect additional logging information.

[entrity]

Thanks for the follow up!

OS: Ubuntu 22.04.3 LTS

vpn-slice 0.16.1

OpenConnect version v8.20-1
Using GnuTLS 3.7.3. Features present: TPMv2, PKCS#11, RSA software token, HOTP software token, TOTP software token, Yubikey OATH, System keys, DTLS, ESP
Supported protocols: anyconnect (default), nc, gp, pulse, f5, fortinet, array
Default vpnc-script (override with --script): /usr/share/vpnc-scripts/vpnc-script

The output for my command which uses both vpn-slice and --background is given in the following txt file: openconnect-vpn-slice-background-output.txt. After this command, my hosts file looks just as it does on startup (no insertions from vpn-slice). My (redacted) command is:

sudo openconnect --protocol=anyconnect --user=Myself --background --script "vpn-slice --verbose --dump XXX.XXX.XXX.XXX/8 hostname1 hostname2 hostname3 hostname4 hostname5 hostname6 hostname7 hostname8 hostname9 hostname10 hostname11 hostname12 hostname13 hostname14 hostname15 hostname16 hostname17 hostname18 hostname19" example.vpn.com

The output for my command which uses only vpn-slice (no --background) is given by the following txt file: openconnect-vpn-slice-foreground-output.txt. And after this command, my hosts file includes lines like XXX.XXX.XXX.XXX hostname1 # vpn-slice-tun0 AUTOCREATED.

[dlenski]

OS: Ubuntu 22.04.3 LTS

Just to double-check: you're not running this on WSL, are you? See #145 if so 🤨

OpenConnect version v8.20-1

Quite an old version at this point, but I can't think of any plausibly relevant recent changes that would affect this.

The output for my command which uses both vpn-slice and --background is given in the following txt file: openconnect-vpn-slice-background-output.txt.

This log file shows that vpn-slice is not being invoked at all, which makes no sense at all. If OpenConnect fails to invoke it, or if vpn-slice returns an error, OpenConnect should noisily warn about that, and your log doesn't show that.

What's going on with this message about SSH CM sockets at the end of the log? How is SSH involved here at all? 🧐

Continuing in background; pid 157125
removing SSH ControlMaster sockets...

My suspicion is that you have some additional wrapper layer or script here involved SSH, which is interfering somehow.

[entrity]

1. Confirmed, I am not using WSL.
2. I too noticed that vpn-slice appears to be not invoked at all when --background is used. Very strange, but I've performed this any number of times, and the behaviour is consistent.
3. Sorry about that removing SSH ControlMaster sockets... message. It's a red herring. That's something I put in manually at the end of my script to clean up some sockets every time I re-establish my vpn connection. (In my ssh config file, I have ControlMaster configured so that I can re-use sockets.)

sudo openconnect \
	--protocol=anyconnect \
	--user=Myself \
	--script "vpn-slice --verbose --dump 10.0.0.0/8 hostname1 hostname2 hostname3 hostname4 hostname5 hostname6 hostname7 hostname8 hostname9 hostname10 hostname11 hostname12 hostname13 hostname14 hostname15 hostname16 hostname17 hostname18 hostname19" \
	vpn.example.com

>&2 echo "removing SSH ControlMaster sockets..."
if compgen -G ~/.ssh/dev-cm-*; then
	rm ~/.ssh/dev-cm-*
fi

[dlenski]

Does it make any difference if you change --script "vpn-slice …" to --script "/full/absolute/path/to/vpn-slice …"?

Can you confirm that sudo /full/absolute/path/to/vpn-slice --self-test works?

[entrity]

Sure thing. The self-test works:

myself@machine:~$ sudo /usr/local/bin/vpn-slice --self-test
[sudo] password for myself: 
***************************************************************************
*** Self-test passed. Try using vpn-slice with openconnect or vpnc now. ***
***************************************************************************

...And I have the same outcome when specifying the absolute path to vpn-slice:

sudo openconnect --protocol=anyconnect --user=Myself --background --script /usr/local/bin/vpn-slice --verbose --dump 10.0.0.0/8 hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname

output.txt

My vpn-slice was installed using PyPI: sudo pip3 install "vpn-slice[dnspython,setproctitle]".

[dlenski]

@entrity, I have absolutely no idea why OpenConnect appears not to be invoking the vpnc-script at all.

Run it through strace. Do you see the expected fork and exec* system calls happening, where it should be invoking the script and putting itself into the background?

[entrity]

I may have hit on something that gives useful information: when I run my openconnect command with stderr redirected (to a file or /dev/null), the intended behaviour appears: openconnect moves to the background and my hosts file gets the updates from vpn-slice. Yes, having stderr redirected is the only difference between my successful case and my failure case.

strace

<<<$PASSWORD sudo strace -o trace.log -f openconnect --background --passwd-on-stdin --protocol=anyconnect --user=Myself --script "/usr/local/bin/vpn-slice --verbose --dump 10.0.0.0/8 hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname" hostname

• Running strace without -f results in no appearances of fork or execl. Running it with -f appears to prevent openconnect from moving to the background; the traced process remains in the foreground, and hitting Ctrl+c after a long wait results in the end of the vpn connection. Thus, I seem to be unable to run this trace with --background truly in effect, so the significance of my traces is in question.
• I log no fork sys calls, but vfork does appear in two varieties: vfork( <unfinished ...> and <... vfork resumed>).
• The execl I see from the linked code doesn't appear, but my trace shows a few identical execve calls for vpn-slice:

execve("/usr/local/bin/vpn-slice", ["/usr/local/bin/vpn-slice", "--verbose", "--dump", "10.0.0.0/8", "hostname", "hostname", "hostname", "hostname", "hostname", "hostname", "hostname", "hostname", "hostname", "hostname", "hostname", "hostname", "hostname", "hostname", "hostname", "hostname", "hostname", "hostname", "hostname"], 0x55b29d573868 /* 394 vars */ <unfinished ...>

[dlenski]

I may have hit on something that gives useful information: when I run my openconnect command with stderr redirected (to a file or /dev/null), the intended behaviour appears: openconnect moves to the background and my hosts file gets the updates from vpn-slice. Yes, having stderr redirected is the only difference between my successful case and my failure case.

Good find. So this is definitely an OpenConnect issue rather than a vpn-slice issue.

I can't find anything that we might've changed since v8.20 that could be relevant to this, but please check if it still exists in v9.21. If so, report it (at https://gitlab.com/openconnect/openconnect/-/issues).

[dlenski]

@amimof is experiencing the same issue ("redirecting stderr makes --background work") on v9.01. See #145 (comment)

[johnlane]

Just noticed this myself whe migrating to a new laptop. It seemed to me that the slicing did work (I could verify this with ip route get to specific addresses) but the hosts file was not updated so any explicit host name mappings did not work (because they were not in the hosts file).

Reading this issue led me to realise the difference was that, on the old laptop, OpenConnect version 9.01 seems to completely ignore --background and therefore did not impact vpn-slice writing to the hosts file. On my new laptop, OpenConnect version 9.12 does react to --background and that prevented the hosts file from being updated. I wondered why the output from OpenConnect was different - it was the background arg now having an effect

I didn't even realise that the backgrounding wasn't working before (it was in the script but I didn't really pay any attention to it - I just left it running in a terminal window because it "just worked"). On the new one the same script did indeed background but I could not resolve any explicit hosts that vpn-slice would normally write to the hosts file. Removing the --background arg allows it to work exactly as before (albeit with the need to leave a terminal window open).

Just another perspective in case it helps. If it's of interest I am using ArchLinux and connecting to a GlobalProtect VPN.

[ghost]

Openconnect issue https://gitlab.com/openconnect/openconnect/-/issues/692