Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for symfony/lock #153

Merged
merged 4 commits into from
Oct 20, 2018
Merged

Conversation

digilist
Copy link
Contributor

@digilist digilist commented Oct 14, 2018

This PR adds the ability to use a symfony/lock storage to prevent command overlapping, as discussed in #125.

I did not remove the file based locking, as this might be BC break (because of the lockFile() method etc., which is not private). However, this could be removed with the next major release.

This should also solve the request of @IkeLutra, because he can now use the FlockStorage and define the path.

Additionally, I added a unit test that verifies that the task lockig is working properly.

@PabloKowalczyk
Copy link
Collaborator

Hello @digilist, thank you very much! I appreciate your work, but, sorry, you should target branch 1.x, as this is active development branch. Could you please backport your PR to PHP 5.6? If you want you can close this PR and open new.
Thanks again, good work.

@digilist digilist force-pushed the symfony_lock branch 2 times, most recently from e094713 to 6981116 Compare October 16, 2018 12:01
@digilist digilist changed the base branch from master to 1.x October 16, 2018 12:02
@digilist
Copy link
Contributor Author

No problem, I was able to rebase the commits and change the target branch of this PR.

@digilist digilist force-pushed the symfony_lock branch 3 times, most recently from 65e0e48 to a88c9b4 Compare October 16, 2018 12:13
@PabloKowalczyk
Copy link
Collaborator

Ah yes, very nice. There are some problems with AppVeyor CI, i will monitor it, but fail(s) can be unrelated to your changes.

composer.json Outdated Show resolved Hide resolved
tests/Functional/PreventOverlappingTest.php Outdated Show resolved Hide resolved
@PabloKowalczyk
Copy link
Collaborator

Could you also add example, of locking to different path (as mentiond in #125), to README?

@digilist
Copy link
Contributor Author

I added some examples to the README.

Regarding AppVeyor I do not see how the failing tests could be related to my changes and do not think it is 😕

@PabloKowalczyk
Copy link
Collaborator

Looks like the issue with AppVeyor is related to this PR (last build https://ci.appveyor.com/project/lavary/crunz/builds/19558199/job/5egmyi1aw64uhw08), CI killed the build after 1h, something blocked the end of PHPUnit. On the other hand my PR without symfony/lock was successful (#155).

I have some idea. On Lock component's docs (https://symfony.com/doc/3.4/components/lock.html#flockstore) we can read:

The FlockStore uses the file system on the local computer to create the locks. It does not support expiration, but the lock is automatically released when the PHP process is terminated

Also some warning:

Beware that some file systems (such as some types of NFS) do not support locking. In those cases, it's better to use a directory on a local disk drive or a remote store based on Redis or Memcached.

This case reminds me also bug PHPCsFixer PHP-CS-Fixer/PHP-CS-Fixer#3460 which was related to file locks on Vagrant's NFS.

So, to the conclusion, i think that weird behavior of AppVeoyr is related to FlockStore. For some reason PHP can't release Lock (maybe AppVeoyor uses NFS, IDK) and process hangs for 1h.
IMO this issue can be easy fixed by swapping lock store to SemaphoreStore.
WDYT?

@digilist digilist force-pushed the symfony_lock branch 2 times, most recently from c13ce46 to 3760210 Compare October 19, 2018 11:33
@digilist
Copy link
Contributor Author

Interesting idea, thanks. It looks like the file locking in general is the problem. Even the "old" code does not work, it's not only symfony lock. And as there was no test case for the locking previously, it is also no longer surprising for me, that it is failing with my changes now 😜

I only noticed just now, that AppVeyor is based on Windows, so the problem is (probably) related to Windows.

I changed the condition of the loop from while($event->isLocked()) to while($event->getProcess()->isRunning()) and the build is no longer timing out. Now, the last assertion fails, because the event is still locked, but I do not yet know why.

I will continue my testing to find a solution. I will clean up my test commits later - and I am sorry for that many pushes (if you get an email from github for every push ;)). I currently see no other way to debug it, I have no Windows system at hand.

@PabloKowalczyk
Copy link
Collaborator

If you want some WIndows for tests i think that you can use one from https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/ and run it in VirtualBox, hope it helps :)

@digilist
Copy link
Contributor Author

digilist commented Oct 20, 2018

Thanks! I found the issue. It was related to my overriden manageStartedEvents() function, where I forgot to call the afterCallbacks of the event (after the process finished). So the lock file on Windows was not deleted and the process was locked forever.

I noticed some issue on Windows here: If the lock file is not deleted, because the EventRunner dies (for whatever reason), the event will never start again. This is because the event is considered locked if the lock file exists (https://github.com/lavary/crunz/blob/1.x/src/Event.php#L1028).

I found this solution to check if a process is actually still running: https://stackoverflow.com/questions/13648304/detecting-if-a-windows-process-and-application-is-running. Probably, we can fix it this way, but I would do it in another PR.

@PabloKowalczyk
Copy link
Collaborator

Very nice, good job! All checks green, Is it ready to merge now?

@digilist
Copy link
Contributor Author

I would say so, yes.

@PabloKowalczyk PabloKowalczyk merged commit 27639ee into lavary:1.x Oct 20, 2018
@PabloKowalczyk
Copy link
Collaborator

Thanks @digilist!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants