Added known limitation section to the docs #796

07pepa · 2022-07-16T07:48:37Z

added known limitations to documentation

Thanks for submitting a PR, your contribution is really appreciated!

Here's a quick checklist that should be present in PRs:

Make sure to include reasonable tests for your change if necessary
We use towncrier for changelog management, so please add a news file into the changelog folder following these guidelines:
- Name it $issue_id.$type for example 588.bugfix;
- If you don't have an issue_id change it to the PR id after creating it
- Ensure type is one of removal, feature, bugfix, vendor, doc or trivial
- Make sure to use full sentences with correct case and punctuation, for example:
```
Fix issue with non-ascii contents in doctest text files.
```

arthuRHD

Nice and explicit documentation

RonnyPfannschmidt · 2022-07-20T07:03:19Z

Im a bit torn between limitations ynd design constraints, but nicely done

07pepa · 2022-07-20T07:55:38Z

i missed changes in testing..

- added known limitations to documentation

07pepa · 2022-07-20T07:59:53Z

it would be nice to document what to run when testing (linux and windows ) so contributors are shure they are doing it correctly

nicoddemus

Overall looks good, left a few comments/requests. Thanks!

docs/index.rst

* moved doc file to rst file * moved know limitations section to its own file * added quick workarounds section that documents quick fixes

07pepa · 2022-07-21T22:11:48Z

@nicoddemus i tried to implement all your sugestion and om some of them i expanded. if i missed anything let me know

changelog/796.doc.rst

nicoddemus

Thanks @07pepa for the contribution, appreciate it!

07pepa · 2022-07-23T10:22:27Z

Thanks for merging

4767: Update pytest-xdist requirement from ~=2.5.0 to ~=3.0.2 r=jenshnielsen a=dependabot[bot] Updates the requirements on [pytest-xdist](https://github.com/pytest-dev/pytest-xdist) to permit the latest version. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pytest-dev/pytest-xdist/blob/master/CHANGELOG.rst">pytest-xdist's changelog</a>.</em></p> <blockquote> <h1>pytest-xdist 3.0.2 (2022-10-25)</h1> <h2>Bug Fixes</h2> <ul> <li><code>[#813](pytest-dev/pytest-xdist#813) <https://github.com/pytest-dev/pytest-xdist/issues/813></code>_: Cancel shutdown when a crashed worker is restarted.</li> </ul> <h2>Deprecations</h2> <ul> <li> <p><code>[#825](pytest-dev/pytest-xdist#825) <https://github.com/pytest-dev/pytest-xdist/issues/825></code>_: The <code>--rsyncdir</code> command line argument and <code>rsyncdirs</code> config variable are deprecated.</p> <p>The rsync feature will be removed in pytest-xdist 4.0.</p> </li> <li> <p><code>[#826](pytest-dev/pytest-xdist#826) <https://github.com/pytest-dev/pytest-xdist/issues/826></code>_: The <code>--looponfail</code> command line argument and <code>looponfailroots</code> config variable are deprecated.</p> <p>The loop-on-fail feature will be removed in pytest-xdist 4.0.</p> </li> </ul> <h2>Improved Documentation</h2> <ul> <li> <p><code>[#791](pytest-dev/pytest-xdist#791) <https://github.com/pytest-dev/pytest-xdist/issues/791></code>_: Document the <code>pytest_xdist_auto_num_workers</code> hook.</p> </li> <li> <p><code>[#796](pytest-dev/pytest-xdist#796) <https://github.com/pytest-dev/pytest-xdist/issues/796></code>_: Added known limitations section to documentation.</p> </li> <li> <p><code>[#829](pytest-dev/pytest-xdist#829) <https://github.com/pytest-dev/pytest-xdist/issues/829></code>_: Document the <code>-n logical</code> option.</p> </li> </ul> <h2>Features</h2> <ul> <li> <p><code>[#792](pytest-dev/pytest-xdist#792) <https://github.com/pytest-dev/pytest-xdist/issues/792></code>_: The environment variable <code>PYTEST_XDIST_AUTO_NUM_WORKERS</code> can now be used to specify the default for <code>-n auto</code> and <code>-n logical</code>.</p> </li> <li> <p><code>[#812](pytest-dev/pytest-xdist#812) <https://github.com/pytest-dev/pytest-xdist/issues/812></code>_: Partially restore old initial batch distribution algorithm in <code>LoadScheduling</code>.</p> <p>pytest orders tests for optimal sequential execution - i. e. avoiding unnecessary setup and teardown of fixtures. So executing tests in consecutive chunks is important for optimal performance.</p> <p>In v1.14, initial test distribution in <code>LoadScheduling</code> was changed to round-robin, optimized for the corner case, when the number of tests is less than <code>2 * number of nodes</code>. At the same time, it became worse for all other cases.</p> <p>For example: if some tests use some "heavy" fixture, and these tests fit into the initial batch, with round-robin distribution the fixture will be created</p> </li> </ul>  </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/eed37d4771f7379156303f63e0d09131f1ac8dee"><code>eed37d4</code></a> Update CHANGELOG</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/3e9284b6e0b474acaf6f25f8f8db68c78c9cd187"><code>3e9284b</code></a> Merge remote-tracking branch 'upstream/master' into release-3.0.0</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/ccdab727a29cef7075b853e550eb03aa8de076c1"><code>ccdab72</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/831">#831</a> from pytest-dev/fix-setup</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/0c981d35475ab6eaee2e6a5ead44442bf07d6e61"><code>0c981d3</code></a> fix and update packaging</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/38dcf52ae69188b0249eb6307fe6745e2e4174dc"><code>38dcf52</code></a> Release 3.0.1</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/f2633f1b372dbc58a221cd612c9cb4e79f59eea3"><code>f2633f1</code></a> Fetch all tags, as required by setuptools-scm</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/192193197bee27228e1e4b44855b3777397e31e5"><code>1921931</code></a> Update CHANGELOG for 3.0</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/0f58a14def50b9b9112ec25cef97b9593ce86b8a"><code>0f58a14</code></a> Fix towncrier command for latest version</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/bd23c2460f00129fa0ace7151c8ae18c0a70d5ad"><code>bd23c24</code></a> Fix changelog fragments</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/794f28bcf4a3711acd074e9637ce8674fb6215be"><code>794f28b</code></a> Use modern syntax for towncrier</li> <li>Additional commits viewable in <a href="https://github.com/pytest-dev/pytest-xdist/compare/v2.5.0...v3.0.2">compare view</a></li> </ul> </details> <br /> You can trigger a rebase of this PR by commenting ``@dependabot` rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - ``@dependabot` rebase` will rebase this PR - ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it - ``@dependabot` merge` will merge this PR after your CI passes on it - ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it - ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging - ``@dependabot` reopen` will reopen this PR if it is closed - ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [pytest-xdist](https://togithub.com/pytest-dev/pytest-xdist) ([changelog](https://pytest-xdist.readthedocs.io/en/latest/changelog.html)) | `==2.5.0` -> `==3.3.1` | [![age](https://badges.renovateapi.com/packages/pypi/pytest-xdist/3.3.1/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/pytest-xdist/3.3.1/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/pytest-xdist/3.3.1/compatibility-slim/2.5.0)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/pytest-xdist/3.3.1/confidence-slim/2.5.0)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>pytest-dev/pytest-xdist</summary> ### [`v3.3.1`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-331-2023-05-19) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v3.3.0...v3.3.1) \=============================== ## Bug Fixes - `#907 <https://github.com/pytest-dev/pytest-xdist/issues/907>`\_: Avoid remote calls during startup as `execnet` by default does not ensure remote affinity with the main thread and might accidentally schedule the pytest worker into a non-main thread, which breaks numerous frameworks, for example `asyncio`, `anyio`, `PyQt/PySide`, etc. A more safe correction will require thread affinity in `execnet` (`pytest-dev/execnet#96 <https://github.com/pytest-dev/execnet/issues/96>`\__). ### [`v3.3.0`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-330-2023-05-12) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v3.2.1...v3.3.0) \=============================== ## Features - `#555 <https://github.com/pytest-dev/pytest-xdist/issues/555>`\_: Improved progress output when collecting nodes to be less verbose. ### [`v3.2.1`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-321-2023-03-12) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v3.2.0...v3.2.1) \=============================== ## Bug Fixes - `#884 <https://github.com/pytest-dev/pytest-xdist/issues/884>`\_: Fixed hang in `worksteal` scheduler. ### [`v3.2.0`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-320-2023-02-07) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v3.1.0...v3.2.0) \=============================== ## Improved Documentation - `#863 <https://github.com/pytest-dev/pytest-xdist/issues/863>`\_: Document limitations for debugging due to standard I/O of workers not being forwarded. Also, mention remote debugging as a possible workaround. ## Features - `#855 <https://github.com/pytest-dev/pytest-xdist/issues/855>`\_: Users can now configure `load` scheduling precision using `--maxschedchunk` command line option. - `#858 <https://github.com/pytest-dev/pytest-xdist/issues/858>`*: New `worksteal` scheduler, based on the idea of `work stealing <https://en.wikipedia.org/wiki/Work_stealing>`*. It's similar to `load` scheduler, but it should handle tests with significantly differing duration better, and, at the same time, it should provide similar or better reuse of fixtures. ## Trivial Changes - `#870 <https://github.com/pytest-dev/pytest-xdist/issues/870>`\_: Make the tests pass even when `$PYTEST_XDIST_AUTO_NUM_WORKERS` is set. ### [`v3.1.0`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-310-2022-12-01) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v3.0.2...v3.1.0) \=============================== ## Features - `#789 <https://github.com/pytest-dev/pytest-xdist/issues/789>`\_: Users can now set a default distribution mode in their configuration file: .. code-block:: ini [pytest] addopts = --dist loadscope - `#842 <https://github.com/pytest-dev/pytest-xdist/issues/842>`\_: Python 3.11 is now officially supported. ## Removals - `#842 <https://github.com/pytest-dev/pytest-xdist/issues/842>`\_: Python 3.6 is no longer supported. ### [`v3.0.2`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-302-2022-10-25) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v2.5.0...v3.0.2) \=============================== ## Bug Fixes - `#813 <https://github.com/pytest-dev/pytest-xdist/issues/813>`\_: Cancel shutdown when a crashed worker is restarted. ## Deprecations - `#825 <https://github.com/pytest-dev/pytest-xdist/issues/825>`\_: The `--rsyncdir` command line argument and `rsyncdirs` config variable are deprecated. The rsync feature will be removed in pytest-xdist 4.0. - `#826 <https://github.com/pytest-dev/pytest-xdist/issues/826>`\_: The `--looponfail` command line argument and `looponfailroots` config variable are deprecated. The loop-on-fail feature will be removed in pytest-xdist 4.0. ## Improved Documentation - `#791 <https://github.com/pytest-dev/pytest-xdist/issues/791>`\_: Document the `pytest_xdist_auto_num_workers` hook. - `#796 <https://github.com/pytest-dev/pytest-xdist/issues/796>`\_: Added known limitations section to documentation. - `#829 <https://github.com/pytest-dev/pytest-xdist/issues/829>`\_: Document the `-n logical` option. ## Features - `#792 <https://github.com/pytest-dev/pytest-xdist/issues/792>`\_: The environment variable `PYTEST_XDIST_AUTO_NUM_WORKERS` can now be used to specify the default for `-n auto` and `-n logical`. - `#812 <https://github.com/pytest-dev/pytest-xdist/issues/812>`\_: Partially restore old initial batch distribution algorithm in `LoadScheduling`. pytest orders tests for optimal sequential execution - i. e. avoiding unnecessary setup and teardown of fixtures. So executing tests in consecutive chunks is important for optimal performance. In v1.14, initial test distribution in `LoadScheduling` was changed to round-robin, optimized for the corner case, when the number of tests is less than `2 * number of nodes`. At the same time, it became worse for all other cases. For example: if some tests use some "heavy" fixture, and these tests fit into the initial batch, with round-robin distribution the fixture will be created `min(n_tests, n_workers)` times, no matter how many other tests there are. With the old algorithm (before v1.14), if there are enough tests not using the fixture, the fixture was created only once. So restore the old behavior for typical cases where the number of tests is much greater than the number of workers (or, strictly speaking, when there are at least 2 tests for every node). ## Removals - `#468 <https://github.com/pytest-dev/pytest-xdist/issues/468>`\_: The `--boxed` command-line option has been removed. If you still need this functionality, install `pytest-forked <https://pypi.org/project/pytest-forked>`\__ separately. ## Trivial Changes - `#468 <https://github.com/pytest-dev/pytest-xdist/issues/468>`\_: The `py` dependency has been dropped. - `#822 <https://github.com/pytest-dev/pytest-xdist/issues/822>`\_: Replace internal usage of `py.log` with a custom solution (but with the same interface). - `#823 <https://github.com/pytest-dev/pytest-xdist/issues/823>`\_: Remove usage of `py._pydir` as an rsync candidate. - `#824 <https://github.com/pytest-dev/pytest-xdist/issues/824>`\_: Replace internal usages of `py.path.local` by `pathlib.Path`. </details> --- ### Configuration 📅 **Schedule**: Branch creation - "every weekend" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/fluencelabs/spell).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

arthuRHD approved these changes Jul 20, 2022

View reviewed changes

RonnyPfannschmidt approved these changes Jul 20, 2022

View reviewed changes

Added known limitation section

c49cfdc

- added known limitations to documentation

nicoddemus requested changes Jul 21, 2022

View reviewed changes

Improve known limitations documentation

b52e51b

* moved doc file to rst file * moved know limitations section to its own file * added quick workarounds section that documents quick fixes

nicoddemus added 2 commits July 22, 2022 10:27

Update 796.rst

f30656f

Rename 796.rst to 796.doc.rst

43f04d7

nicoddemus reviewed Jul 22, 2022

View reviewed changes

changelog/796.doc.rst Show resolved Hide resolved

Update known-limitations.rst

0c836be

nicoddemus approved these changes Jul 22, 2022

View reviewed changes

nicoddemus changed the title ~~Added known limitation section~~ Added known limitation section to the docs Jul 22, 2022

nicoddemus merged commit 6113277 into pytest-dev:master Jul 22, 2022

07pepa deleted the add_known_limitation_section branch July 23, 2022 10:22

mikelane mentioned this pull request Oct 26, 2022

Bump pytest-xdist from 2.5.0 to 3.0.2 mikelane/reddit-get#83

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added known limitation section to the docs #796

Added known limitation section to the docs #796

07pepa commented Jul 16, 2022 •

edited

Loading

arthuRHD left a comment

RonnyPfannschmidt commented Jul 20, 2022

07pepa commented Jul 20, 2022

07pepa commented Jul 20, 2022

nicoddemus left a comment

07pepa commented Jul 21, 2022 •

edited

Loading

nicoddemus left a comment

07pepa commented Jul 23, 2022

Added known limitation section to the docs #796

Added known limitation section to the docs #796

Conversation

07pepa commented Jul 16, 2022 • edited Loading

arthuRHD left a comment

Choose a reason for hiding this comment

RonnyPfannschmidt commented Jul 20, 2022

07pepa commented Jul 20, 2022

07pepa commented Jul 20, 2022

nicoddemus left a comment

Choose a reason for hiding this comment

07pepa commented Jul 21, 2022 • edited Loading

nicoddemus left a comment

Choose a reason for hiding this comment

07pepa commented Jul 23, 2022

07pepa commented Jul 16, 2022 •

edited

Loading

07pepa commented Jul 21, 2022 •

edited

Loading