Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

watchdog spams file_opened events #949

Closed
rbachleitner opened this issue Feb 27, 2023 · 5 comments
Closed

watchdog spams file_opened events #949

rbachleitner opened this issue Feb 27, 2023 · 5 comments

Comments

@rbachleitner
Copy link

Hi there,

I don't know what happened since 2.2.1,
but in 2.3.0 we noticed that watchdog was spamming file_opened events on many if not
all files in the --patterns flag and restarting our service.

We are using a watchmedo auto-restart like follows

watchmedo", \
  "auto-restart", \
  "--patterns", \
  "*.py", \
  "--recursive", \
  "--", \
  "python3",
  ... flask commands follow

(formatting due to using it in a Dockerfile);

Our *.py files didn't show any actual change (checked via git & ls -a),
but watchdog was logging a lot of file_opened events and sending the KeyboardInterrrupt.
For now we reverted to 2.2.1.

@BoboTiG
Copy link
Collaborator

BoboTiG commented Feb 27, 2023

Thanks for the report.

You are right, the new "file opened" event is included by default in all events: https://github.com/gorakhargosh/watchdog/blob/v2.3.0/src/watchdog/events.py#L289

I should have catch that in #941, and will revert the change (users wanting to catch those events will have to use .on_opened()). Until we can actually filter by event types.

@BoboTiG
Copy link
Collaborator

BoboTiG commented Feb 27, 2023

Could you try the version from the master branch (as of commit 346ee46)? If it fixes your issue, the release will be done right after.

@jthurner
Copy link

Thanks, latest master works in not triggering autorestart on FileOpenedEvent.

@BoboTiG
Copy link
Collaborator

BoboTiG commented Feb 28, 2023

Thanks @jthurner, release is on the way :)

@BoboTiG
Copy link
Collaborator

BoboTiG commented Feb 28, 2023

v2.3.1 is out!

wxx9248 pushed a commit to wxx9248/Pickle-Rush that referenced this issue Mar 1, 2023
Bumps [watchdog](https://github.com/gorakhargosh/watchdog) from 2.3.0 to
2.3.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/gorakhargosh/watchdog/releases">watchdog's
releases</a>.</em></p>
<blockquote>
<h2>2.3.1</h2>
<ul>
<li>Run <code>black</code> on the entire source code</li>
<li>Bundle the <code>requirements-tests.txt</code> file in the source
distribution (<a
href="https://github-redirect.dependabot.com/gorakhargosh/watchdog/issues/939">#939</a>)</li>
<li>[watchmedo] Exclude <code>FileOpenedEvent</code> events from
<code>AutoRestartTrick</code>, and <code>ShellCommandTrick</code>, to
restore watchdog &lt; 2.3.0 behavior. A better solution should be found
in the future. (<a
href="https://github-redirect.dependabot.com/gorakhargosh/watchdog/issues/949">#949</a>)</li>
<li>[watchmedo] Log <code>FileOpenedEvent</code>, and
<code>FileClosedEvent</code>, events in <code>LoggerTrick</code></li>
</ul>
<p>:heart_decoration: Thanks to our beloved contributors: <a
href="https://github.com/BoboTiG"><code>@​BoboTiG</code></a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/gorakhargosh/watchdog/blob/master/changelog.rst">watchdog's
changelog</a>.</em></p>
<blockquote>
<p>2.3.1</p>
<pre><code>
2023-02-28 • `full history
&lt;https://github.com/gorakhargosh/watchdog/compare/v2.3.0...v2.3.1&gt;`__
<ul>
<li>Run <code>black</code> on the entire source code</li>
<li>Bundle the <code>requirements-tests.txt</code> file in the source
distribution
(<code>[#939](gorakhargosh/watchdog#939)
&amp;lt;https://github.com/gorakhargosh/watchdog/pull/939&amp;gt;</code>__)</li>
<li>[watchmedo] Exclude <code>FileOpenedEvent</code> events from
<code>AutoRestartTrick</code>, and <code>ShellCommandTrick</code>, to
restore watchdog &lt; 2.3.0 behavior. A better solution should be found
in the future.
(<code>[#949](gorakhargosh/watchdog#949)
&amp;lt;https://github.com/gorakhargosh/watchdog/pull/949&amp;gt;</code>__)</li>
<li>[watchmedo] Log <code>FileOpenedEvent</code>, and
<code>FileClosedEvent</code>, events in <code>LoggerTrick</code></li>
<li>Thanks to our beloved contributors: <a
href="https://github.com/BoboTiG"><code>@​BoboTiG</code></a>
</code></pre></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/gorakhargosh/watchdog/commit/60a97bf12f5670787b351c68e1c61f5df35d0210"><code>60a97bf</code></a>
Version 2.3.1</li>
<li><a
href="https://github.com/gorakhargosh/watchdog/commit/346ee46b52caddf4a031c5f307488b9e77d3dff4"><code>346ee46</code></a>
Bundle the <code>requirements-tests.txt</code> file in the source
distribution</li>
<li><a
href="https://github.com/gorakhargosh/watchdog/commit/d38ab5a4977b5d5774a99c81ad42470be0df9995"><code>d38ab5a</code></a>
Second run for <code>black</code></li>
<li><a
href="https://github.com/gorakhargosh/watchdog/commit/b4a8a1467cd7f12119a8ba448af4f8e58f7d3edd"><code>b4a8a14</code></a>
[watchmedo] Log <code>FileOpenedEvent</code>, and
<code>FileClosedEvent</code>, events in `LoggerTr...</li>
<li><a
href="https://github.com/gorakhargosh/watchdog/commit/25a2d1fec55fc32f97055ba827c8ce7b959acb34"><code>25a2d1f</code></a>
[watchmedo] Exclude <code>FileOpenedEvent</code> events from
<code>AutoRestartTrick</code>, and `Sh...</li>
<li><a
href="https://github.com/gorakhargosh/watchdog/commit/54f2cf005a996fde77f3815fdd1111efac2e216d"><code>54f2cf0</code></a>
Run <code>black</code> on the entire source code</li>
<li><a
href="https://github.com/gorakhargosh/watchdog/commit/16015f514af0a2cb86907012abee3f1df877f580"><code>16015f5</code></a>
Bump the version</li>
<li>See full diff in <a
href="https://github.com/gorakhargosh/watchdog/compare/v2.3.0...v2.3.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=watchdog&package-manager=pip&previous-version=2.3.0&new-version=2.3.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
unkcpz added a commit to aiidalab/aiidalab that referenced this issue Mar 6, 2023
fixes #358

The watchdog file system monitor introduces a new event handler INOPNED for when file is opened. The new "file opened" event is included by default in all events: https://github.com/gorakhargosh/watchdog/blob/v2.3.0/src/watchdog/events.py#L289

The details are discussed at gorakhargosh/watchdog#949, we encounter the same issue. But we use on_any_event explicitly, we adopt the same change as gorakhargosh/watchdog@25a2d1f to avoid spamming by the opened event update.
netbsd-srcmastr referenced this issue in NetBSD/pkgsrc May 5, 2023
3.0.0
~~~~~

- Drop support for Python 3.6.
- ``watchdog`` is now PEP 561 compatible, and tested with ``mypy``
- Fix missing ``>`` in ``FileSystemEvent.__repr__()``  (`#980 <https://github.com/gorakhargosh/watchdog/pull/980>`__)
- [ci] Lots of improvements
- [inotify] Return from ``InotifyEmitter.queue_events()`` if not launched when thread is inactive (`#963 <https://github.com/gorakhargosh/watchdog/pull/963>`__)
- [tests] Stability improvements
- [utils] Remove handling of ``threading.Event.isSet`` spelling (`#962 <https://github.com/gorakhargosh/watchdog/pull/962>`__)
- [watchmedo] Fixed tricks YAML generation (`#965 <https://github.com/gorakhargosh/watchdog/pull/965>`__)
- Thanks to our beloved contributors: @kurtmckee, @altendky, @agroszer, @BoboTiG

2.3.1
~~~~~

- Run ``black`` on the entire source code
- Bundle the ``requirements-tests.txt`` file in the source distribution (`#939 <https://github.com/gorakhargosh/watchdog/pull/939>`__)
- [watchmedo] Exclude ``FileOpenedEvent`` events from ``AutoRestartTrick``, and ``ShellCommandTrick``, to restore watchdog < 2.3.0 behavior. A better solution should be found in the future. (`#949 <https://github.com/gorakhargosh/watchdog/pull/949>`__)
- [watchmedo] Log ``FileOpenedEvent``, and ``FileClosedEvent``, events in ``LoggerTrick``
- Thanks to our beloved contributors: @BoboTiG

2.3.0
~~~~~

- [inotify] Add support for ``IN_OPEN`` events: a ``FileOpenedEvent`` event will be fired (`#941 <https://github.com/gorakhargosh/watchdog/pull/941>`__)
- [watchmedo] Add optional event debouncing for ``auto-restart``, only restarting once if many events happen in quick succession (``--debounce-interval``) (`#940 <https://github.com/gorakhargosh/watchdog/pull/940>`__)
- [watchmedo] Exit gracefully on ``KeyboardInterrupt`` exception (Ctrl+C) (`#945 <https://github.com/gorakhargosh/watchdog/pull/945>`__)
- [watchmedo] Add option to not auto-restart the command after it exits (``--no-restart-on-command-exit``) (`#946 <https://github.com/gorakhargosh/watchdog/pull/946>`__)
- Thanks to our beloved contributors: @BoboTiG, @dstaple, @taleinat, @cernekj

2.2.1
~~~~~

- Enable ``mypy`` to discover type hints as specified in PEP 561 (`#933 <https://github.com/gorakhargosh/watchdog/pull/933>`__)
- [ci] Set the expected Python version when building release files
- [ci] Update actions versions in use
- [watchmedo] [regression] Fix usage of missing ``signal.SIGHUP`` attribute on non-Unix OSes (`#935 <https://github.com/gorakhargosh/watchdog/pull/935>`__)
- Thanks to our beloved contributors: @BoboTiG, @simon04, @piotrpdev

2.2.0
~~~~~

- [build] Wheels are now available for Python 3.11 (`#932 <https://github.com/gorakhargosh/watchdog/pull/932>`__)
- [documentation] HTML documentation builds are now tested for errors (`#902 <https://github.com/gorakhargosh/watchdog/pull/902>`__)
- [documentation] Fix typos here, and there (`#910 <https://github.com/gorakhargosh/watchdog/pull/910>`__)
- [fsevents2] The ``fsevents2`` observer is now deprecated (`#909 <https://github.com/gorakhargosh/watchdog/pull/909>`__)
- [tests] The error message returned by musl libc for error code ``-1`` is now allowed (`#923 <https://github.com/gorakhargosh/watchdog/pull/923>`__)
- [utils] Remove unnecessary code in ``dirsnapshot.py`` (`#930 <https://github.com/gorakhargosh/watchdog/pull/930>`__)
- [watchmedo] Handle shutdown events from ``SIGHUP`` (`#912 <https://github.com/gorakhargosh/watchdog/pull/912>`__)
- Thanks to our beloved contributors: @kurtmckee, @babymastodon, @QuantumEnergyE, @timgates42, @BoboTiG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants