Terminus doesn't detect changes to dev/multidev environment when env is in SFTP mode

[danielbachhuber]

After I've created a new multidev environment and installed a plugin using WP-CLI, running terminus site code diffstat doesn't indicate any changed files. If you open the dashboard for the environment, wait for the code to refresh, and then execute the previous command again, it will indicate changed files.

I'd expect terminus site code diffstat to indicate changed files without requiring the dashboard to be refreshed.

Reproduction steps

Replace TERMINUS_SITE and TERMINUS_ENV with a test site you have access to

$ export TERMINUS_SITE=wp-saml-auth
$ export TERMINUS_ENV=test-2
$ export PANTHEON_SITE_URL="$TERMINUS_ENV-$TERMINUS_SITE.pantheonsite.io"
$ terminus site create-env --to-env=$TERMINUS_ENV --from-env=dev
............................
[2016-06-29 11:15:17] [info] Created Multidev environment "test-2"
$ terminus site set-connection-mode --mode=sftp
..........
[2016-06-29 11:19:41] [info] Enabled on-server development via SFTP for "test-2"
$ terminus wp "plugin install wp-native-php-sessions --activate"
[2016-06-29 11:21:05] [info] Running wp plugin install wp-native-php-sessions --activate on wp-saml-auth-test-2
    cmd: 'plugin install wp-native-php-sessions --activate'
    site: 'wp-saml-auth'
    env: 'test-2'
Installing WordPress Native PHP Sessions (0.5)
Downloading install package from https://downloads.wordpress.org/plugin/wp-native-php-sessions.0.5.zip...
Unpacking the package...
Installing the plugin...
Plugin installed successfully.
Activating 'wp-native-php-sessions'...
Success: Plugin 'wp-native-php-sessions' activated.
$ terminus site code diffstat
[2016-06-29 11:21:35] [info] No changes on server.
$ terminus site code commit --message="Set up WP Native Sessions for testing"
There are no changed files. Commit anyway? [y/n]
..
[2016-06-29 11:23:12] [info] Committed code changes in "test-2"

[stevector]

@TeslaDethray, do you know of any workarounds that could trigger a refresh of the code status from the command line? This bug changes the way @danielbachhuber are writing tests for some of the Pantheon WordPress plugins. We also got a question about it on the power users group: https://groups.google.com/a/pantheon.io/forum/#!topic/power-users/eKVU0A5NFIU

[uberhacker]

Not sure if this is directly related but I noticed Terminus lets you switch from sftp to git mode regardless if there are pending changes or not. This is unlike the dashboard which will not let you switch unless you create a commit of the pending changes. Should the same check be added to Terminus as a possible resolution to this issue?

[danielbachhuber]

Not sure if this is directly related but I noticed Terminus lets you switch from sftp to git mode regardless if there are pending changes or not. This is unlike the dashboard which will not let you switch unless you create a commit of the pending changes. Should the same check be added to Terminus as a possible resolution to this issue?

This sounds unrelated to the original problem in this issue. I'd encourage you to open a new issue.

[ataylorme]

I am running into a similar issue when updating WordPress plugins via Terminus and wp-cli then doing a code commit with Terminus.

Steps to reproduce:

1. Choose a WordPress site on Pantheon that has plugin updates available.
2. Create a multidev environment to update plugins on
3. Set the multidev to SFTP mode with terminus site set-connection-mode --site=${SITE_UUID} --env=${MULTIDEV} --mode=sftp
4. Run plugin updates with wp-cli terminus wp "plugin update --all" --site=${SITE_UUID} --env=${MULTIDEV}
5. Commit the code via Terminus without visiting the Pantheon dashboard terminus site code commit --site=${SITE_UUID} --env=${MULTIDEV} --message="update WordPress plugins"

Terminus responds with the message There are no changed files. Commit anyway? [y/n] even though there are changed files. Visiting the dashboard shows files changed and ready to be committed.

[ataylorme]

@danielbachhuber For what it's worth I ended up using --yes in Terminus when making a code commit to force the commit even though Terminus doesn't think there is a change.

[danielbachhuber]

For what it's worth I ended up using --yes in Terminus when making a code commit to force the commit even though Terminus doesn't think there is a change.

This doesn't solve the problem for me :( The current problem is that a plugin can't be activated, because the filesystem isn't refreshed.

[jrisebor]

I'd just like to chime in and say I'm also having this issue with Terminus 1.0.0.

Adding --yes does not solve the problem for me.

The dashboard seems to be able to pick up the sftp changes pretty quickly (or refreshing the page seems to guarantee it), and after that everything works as expected, but If I don't load the environment's dashboard page first, Terminus usually doesn't notice that there are changes.

[ataylorme]

If I don't load the environment's dashboard page first, Terminus usually doesn't notice that there are changes.

I can confirm this in Terminus 1.x as well.

@TeslaDethray this is causing issues this script I'm using to auto deploy WordPress updates with visual regression.

Basically I spin up a new multidev environment, switch it to SFTP mode and then check for theme/plugin updates with WP-CLI. After running WordPress updates in SFTP mode terminus env:commit is used to commit the changes. Terminus reports There is no code to commit, even though there is, resulting in no commit being made.

See this CirclCI log for details.

This causes the rest of the script, running a visual regression test between live and the multidev and merging/deploying the updates to live if it passes, to fail since there is no commit to merge.

edit: adding screenshots

[ataylorme]

This has now been causing timeouts on the script in CircleCI. See build #338.

Plugin updates exist on the server from running WP-CLI in SFTP mode but the code commit fails as Terminus/Pantheon doesn't detect the file changes. That leads to the multidev merge failing.

[ari-gold]

Workaround (not a very nice one):
Open site dashboard
terminus dashboard <site.env>

[ataylorme]

@ari Loading the dashboard works but doesn't solve the issue in the context of scripting as the next commands in the script (like code commit) fail due to the change not being detected.

If a user has to hit the dashboard or do things manually it takes away from the power of Terminus, especially in the context of automation.

[ari]

Who are you? Why are you emailing me?

[greg-1-anderson]

@ari: @ataylorme meant @ari-gold

[joshkoenig]

I suspect that this is because the watcher process (which is invoked by dashboard running) isn't spun up. There aught to be a way for Terminus to spin up a watcher, then request the diffstat, then spin it down?

[TeslaDethray]

The issue I've had in fixing this issue isn't that I can't fix it, but that I can't duplicate it anymore. For me, Terminus never fails to know when I have new code to commit or not. I fixed the branch name being checked by Terminus back in February, but the change was not released for a while after.
https://github.com/pantheon-systems/terminus/blob/master/src/Models/UpstreamStatus.php#L44

[TeslaDethray]

It looks like this was fixed on Feb 2, but the change was not released until Mar 9. @ataylorme Can you still dupe this with version 1.2.1?

[TeslaDethray]

With multidev:

With dev:

[ataylorme]

@TeslaDethray here are the logs for the update script we reviewed via screen share.

Seems this only happens to site where the site dashboard hasn't been visited in some time and the file watcher becomes inactive.

If you need more info from me just let me know - happy to help debug further. Once updates run on my script I'll let you know if env:wake works.

[populist]

I am seeing the error as part of work I am doing on an automatic update script for Drupal. The error happens even after the wake command has been explicitly run, but when the Multidev environment on which the SFTP code is placed has not been accessed via the dashboard.

[ronan]

A workaround for this (now merged into master) is to add a --force flag to env:commit if you know there are committable files but they're not showing up in env:diffstat.

This flag does exactly what you think it does. If there are no files to commit it'll commit anyway leaving you with an empty commit.

We will hopefully get a more permanent solution into the pipeline soon.

[dalin-]

FWIW I had luck with running env:clear-cache before the commit.

[G-Rath]

I've just hit this when attempting to setup automatic deploys for one of our sites - is my understanding correct that the issue is with the service that detects if files have been changed, rather than the files themselves having actually changed?

e.g. if I've just rsynced new stuff to the site, calling terminus env:commit --force will commit those files even if the dashboard/service/whatever doesn't realise there are changes yet.

I'm going to add --force to our CI to see if that does help, but wanted to check-in here too :)