Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove watchdog to avoid non-deterministic parsing of AMBER output. #27

Merged
merged 5 commits into from
Mar 21, 2023

Conversation

lohedges
Copy link
Contributor

This PR fixes #26 by removing the use of the watchdog package that was used to automatically parse the AMBER energy (mdinfo) file. While this works fairly well for simulations that log data infrequently, it produces non-deterministic results when updates are rapid. For example, the mtime resolution on macOS is larger than on Linux, so fewer data points are recorded for a simulation reporting at the same time interval. (If the interval is small enough.) We now parse the AMBER log file manually using a similar approach to the one used for other engines that we support, i.e. using pygtail to get the updated log lines, then checking those for new records of interest. I've also added a simple test to verify that the correct number of records are recorded.

  • I confirm that I have merged the latest version of devel into this branch before issuing this pull request (e.g. by running git pull origin devel): [y]
  • I confirm that I have permission to release this code under the GPL3 license: [y]

Suggested reviewers:

@chryswoods

@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 15:25 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 15:25 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 15:25 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 15:25 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 15:25 — with GitHub Actions Inactive
@lohedges lohedges added the bug Something isn't working label Mar 20, 2023
chryswoods
chryswoods previously approved these changes Mar 20, 2023
Copy link
Contributor

@chryswoods chryswoods left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - shame that watchdog didn't work, but better to have a unified solution on top of pygtail.

@lohedges
Copy link
Contributor Author

Yes, it's one of those things that has worked better or worse over time depending on implementation changes behind the scenes in the package. It's quite tricky, since they try to do things in a cross-platform way, so not all file system events are available on all platforms. It also seems that AMBER keeps the file open for the duration of the simulation, so it's being written as I had anticipated anyway. There are other similar solutions, but these rely on polling the file, so would also suffer from data loss depending on the polling frequency.

@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 16:21 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 16:21 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 16:21 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 16:21 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 16:21 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 16:21 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 20:27 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 22:11 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 22:11 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 22:11 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 22:11 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 20, 2023 22:11 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 21, 2023 07:58 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 21, 2023 07:58 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 21, 2023 07:58 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 21, 2023 07:58 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 21, 2023 07:58 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 21, 2023 08:08 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 21, 2023 08:08 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 21, 2023 08:08 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 21, 2023 08:08 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 21, 2023 08:08 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 21, 2023 09:15 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 21, 2023 09:15 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 21, 2023 09:15 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 21, 2023 09:15 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 21, 2023 09:15 — with GitHub Actions Inactive
@lohedges lohedges temporarily deployed to biosimspace-build March 21, 2023 09:15 — with GitHub Actions Inactive
@lohedges lohedges merged commit 5dc845f into devel Mar 21, 2023
@lohedges lohedges deleted the fix_remove_watchog branch March 21, 2023 13:54
@lohedges lohedges mentioned this pull request Apr 26, 2023
lohedges pushed a commit that referenced this pull request Oct 12, 2023
Co-authored-by: William (Zhiyi) Wu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] AMBER time series records are non-deterministic
2 participants