Option to set the default distribution algorithm #789

DavideCanton · 2022-07-06T09:50:54Z

Hi, I have the following scenario:

I have some tests that need to be run in the same worker due to the fact they use a shared object (I know it's not very idiomatic for unit tests, but it is really the simpler way to check what I need to check)
I saw the --dist loadgroup option, but unfortunately it is not the default, so I should remember to specify every time the command line option
I can't add it to the addopts setting since it errors if I don't specify either -n or -tx, and I want to be able to choose to run the tests the standard way too.

What could work for me is a setting to set the default algorithm to use when --dist is not specified, so that:

if I run pytest standalone, it doesn't run in parallel
if I run pytest -n auto, it uses the algorithm specified in the setting
if I run pytest -n auto --dist loadfile, it overrides the setting with the loadfile algorithm.

The text was updated successfully, but these errors were encountered:

antazoey · 2022-11-22T20:46:24Z

I need this as well.

Something like this would be nice:

[pytest.xdist]
dist = "loadscope"

such that whenever -n is used, it will automatically use loadscope distribution. Like the issue author states, it helps so I don't have to remember to use --dist loadscope because I have some tests scattered about that share a resource (a special, complicated process running).

DavideCanton · 2022-11-22T23:22:26Z

I could try setting up a PR if someone is interested in merging it

nicoddemus · 2022-11-23T13:50:41Z

I don't think we need a separate option, I would prefer if we can make it possible to set --dist X using addopts, without erroring out if -n is not passed too.

antazoey · 2022-11-23T14:02:39Z

I don't think we need a separate option, I would prefer if we can make it possible to set --dist X using addopts, without erroring out if -n is not passed too.

This works for me

DavideCanton · 2022-11-28T00:25:40Z

I don't think we need a separate option, I would prefer if we can make it possible to set --dist X using addopts, without erroring out if -n is not passed too.

How I could run pytest without erroring if -n is not passed?

nicoddemus · 2022-11-28T17:36:52Z

I think it should be possible to implement that (can't look at the code right now).

DavideCanton · 2022-11-28T22:01:42Z

It can be implemented since we had to do it by using pytest hooks, but it's quite messy since you have to dig deep inside how the workers read the options

Fix pytest-dev#789

nicoddemus · 2022-11-30T16:58:32Z

Opened #851. 👍

Bumps [pytest-xdist](https://github.com/pytest-dev/pytest-xdist) from 3.0.2 to 3.1.0. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pytest-dev/pytest-xdist/blob/master/CHANGELOG.rst">pytest-xdist's changelog</a>.</em></p> <blockquote> <h1>pytest-xdist 3.1.0 (2022-12-01)</h1> <h2>Features</h2> <ul> <li> <p><code>[#789](pytest-dev/pytest-xdist#789) <https://github.com/pytest-dev/pytest-xdist/issues/789></code>_: Users can now set a default distribution mode in their configuration file:</p> <p>.. code-block:: ini</p> <pre><code>[pytest] addopts = --dist loadscope </code></pre> </li> <li> <p><code>[#842](pytest-dev/pytest-xdist#842) <https://github.com/pytest-dev/pytest-xdist/issues/842></code>_: Python 3.11 is now officially supported.</p> </li> </ul> <h2>Removals</h2> <ul> <li><code>[#842](pytest-dev/pytest-xdist#842) <https://github.com/pytest-dev/pytest-xdist/issues/842></code>_: Python 3.6 is no longer supported.</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/92a76bb0139c0c95f89470bfbbd98289321b5543"><code>92a76bb</code></a> Release 3.1.0</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/6226965dac4df7f7a3b034bc02deb8150d09e2b3"><code>6226965</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/851">#851</a> from nicoddemus/789-default-dist-mode</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/7a0bc4c1c9a2d8da2f04ad02fc83b2417493863c"><code>7a0bc4c</code></a> Let users configure dist mode in the configuration file</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/c6bcd204eabeacc75de6ff75339bb731d55ae0cc"><code>c6bcd20</code></a> [pre-commit.ci] pre-commit autoupdate (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/849">#849</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/99c80c37a44b83f045a577b8ba55c912969a1916"><code>99c80c3</code></a> Fix typo <code>psutils</code> -> <code>psutil</code> (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/848">#848</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/e14895a96ed7a5c353c6d246c62f6efd47009188"><code>e14895a</code></a> [pre-commit.ci] pre-commit autoupdate (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/846">#846</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/bb27210cf981c80cd1af3e97b160c79cd8d2d56d"><code>bb27210</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/844">#844</a> from pytest-dev/pre-commit-ci-update-config</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/4a33933e308dcf4cd4b8ba8906fc1720e419191a"><code>4a33933</code></a> Use ternary operator to remove mypy error</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/41620d2f469fae063acedde46cb0ed815021f995"><code>41620d2</code></a> [pre-commit.ci] pre-commit autoupdate</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/6b6f13385096c0fc862b420560b520b6960f718f"><code>6b6f133</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/842">#842</a> from nicoddemus/drop-py36-add-py311</li> <li>Additional commits viewable in <a href="https://github.com/pytest-dev/pytest-xdist/compare/v3.0.2...v3.1.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pytest-xdist&package-manager=pip&previous-version=3.0.2&new-version=3.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

4981: Update types-setuptools requirement from ~=67.1.0 to ~=67.2.0 r=jenshnielsen a=dependabot[bot] Updates the requirements on [types-setuptools](https://github.com/python/typeshed) to permit the latest version. <details> <summary>Commits</summary> <ul> <li>See full diff in <a href="https://github.com/python/typeshed/commits">compare view</a></li> </ul> </details> <br /> You can trigger a rebase of this PR by commenting ``@dependabot` rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - ``@dependabot` rebase` will rebase this PR - ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it - ``@dependabot` merge` will merge this PR after your CI passes on it - ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it - ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging - ``@dependabot` reopen` will reopen this PR if it is closed - ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> 4982: Update pytest-xdist requirement from ~=3.1.0 to ~=3.2.0 r=jenshnielsen a=dependabot[bot] Updates the requirements on [pytest-xdist](https://github.com/pytest-dev/pytest-xdist) to permit the latest version. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pytest-dev/pytest-xdist/blob/master/CHANGELOG.rst">pytest-xdist's changelog</a>.</em></p> <blockquote> <h1>pytest-xdist 3.2.0 (2023-02-07)</h1> <h2>Improved Documentation</h2> <ul> <li><code>[#863](pytest-dev/pytest-xdist#863) <https://github.com/pytest-dev/pytest-xdist/issues/863></code>_: Document limitations for debugging due to standard I/O of workers not being forwarded. Also, mention remote debugging as a possible workaround.</li> </ul> <h2>Features</h2> <ul> <li> <p><code>[#855](pytest-dev/pytest-xdist#855) <https://github.com/pytest-dev/pytest-xdist/issues/855></code>_: Users can now configure <code>load</code> scheduling precision using <code>--maxschedchunk</code> command line option.</p> </li> <li> <p><code>[#858](pytest-dev/pytest-xdist#858) <https://github.com/pytest-dev/pytest-xdist/issues/858></code><em>: New <code>worksteal</code> scheduler, based on the idea of <code>work stealing <https://en.wikipedia.org/wiki/Work_stealing></code></em>. It's similar to <code>load</code> scheduler, but it should handle tests with significantly differing duration better, and, at the same time, it should provide similar or better reuse of fixtures.</p> </li> </ul> <h2>Trivial Changes</h2> <ul> <li><code>[#870](pytest-dev/pytest-xdist#870) <https://github.com/pytest-dev/pytest-xdist/issues/870></code>_: Make the tests pass even when <code>$PYTEST_XDIST_AUTO_NUM_WORKERS</code> is set.</li> </ul> <h1>pytest-xdist 3.1.0 (2022-12-01)</h1> <h2>Features</h2> <ul> <li> <p><code>[#789](pytest-dev/pytest-xdist#789) <https://github.com/pytest-dev/pytest-xdist/issues/789></code>_: Users can now set a default distribution mode in their configuration file:</p> <p>.. code-block:: ini</p> <pre><code>[pytest] addopts = --dist loadscope </code></pre> </li> <li> <p><code>[#842](pytest-dev/pytest-xdist#842) <https://github.com/pytest-dev/pytest-xdist/issues/842></code>_: Python 3.11 is now officially supported.</p> </li> </ul> <h2>Removals</h2> <ul> <li><code>[#842](pytest-dev/pytest-xdist#842) <https://github.com/pytest-dev/pytest-xdist/issues/842></code>_: Python 3.6 is no longer supported.</li> </ul> <h1>pytest-xdist 3.0.2 (2022-10-25)</h1> <p>Bug Fixes</p>  </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/5c065198e98df484d8193a5fba5480db541fac37"><code>5c06519</code></a> Release 3.2.0</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/c695763e92951e20bd10c783c47884032826ea6d"><code>c695763</code></a> [pre-commit.ci] pre-commit autoupdate (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/869">#869</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/9373ddb3f841a009bc9669b514923b6ecba724e7"><code>9373ddb</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/870">#870</a> from hroncok/delenv</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/8fd1bfd182e87f46e2c2ff6dbb43c0cd43d0cc00"><code>8fd1bfd</code></a> Tests: Unset PYTEST_XDIST_AUTO_NUM_WORKERS when the behavior without the envv...</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/017cc72b7090dc4bb7e7ad3d0caab024feb977a8"><code>017cc72</code></a> Document limitations for debugging as well as a workaround (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/867">#867</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/cf19f76d86aa149d21242da520879b8e0ee80013"><code>cf19f76</code></a> Fix minor typos in the documentation (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/866">#866</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/691a0751bf7081224287d13c0863fc981a838380"><code>691a075</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/865">#865</a> from amezin/fix-loadsched-tests</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/e9860923bf899b283627526bf9a23cb688204d42"><code>e986092</code></a> Fix some LoadScheduling tests</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/d1dfad3e927e225c967f16928b39eee43a3d6457"><code>d1dfad3</code></a> Implement work-stealing scheduler (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/862">#862</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/9b0b5b1495b024940e3f5c71583711be46df255b"><code>9b0b5b1</code></a> Add --maxschedchunk CLI option (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/857">#857</a>)</li> <li>Additional commits viewable in <a href="https://github.com/pytest-dev/pytest-xdist/compare/v3.1.0...v3.2.0">compare view</a></li> </ul> </details> <br /> You can trigger a rebase of this PR by commenting ``@dependabot` rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - ``@dependabot` rebase` will rebase this PR - ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it - ``@dependabot` merge` will merge this PR after your CI passes on it - ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it - ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging - ``@dependabot` reopen` will reopen this PR if it is closed - ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [pytest-xdist](https://togithub.com/pytest-dev/pytest-xdist) ([changelog](https://pytest-xdist.readthedocs.io/en/latest/changelog.html)) | `==2.5.0` -> `==3.3.1` | [![age](https://badges.renovateapi.com/packages/pypi/pytest-xdist/3.3.1/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/pytest-xdist/3.3.1/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/pytest-xdist/3.3.1/compatibility-slim/2.5.0)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/pytest-xdist/3.3.1/confidence-slim/2.5.0)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>pytest-dev/pytest-xdist</summary> ### [`v3.3.1`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-331-2023-05-19) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v3.3.0...v3.3.1) \=============================== ## Bug Fixes - `#907 <https://github.com/pytest-dev/pytest-xdist/issues/907>`\_: Avoid remote calls during startup as `execnet` by default does not ensure remote affinity with the main thread and might accidentally schedule the pytest worker into a non-main thread, which breaks numerous frameworks, for example `asyncio`, `anyio`, `PyQt/PySide`, etc. A more safe correction will require thread affinity in `execnet` (`pytest-dev/execnet#96 <https://github.com/pytest-dev/execnet/issues/96>`\__). ### [`v3.3.0`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-330-2023-05-12) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v3.2.1...v3.3.0) \=============================== ## Features - `#555 <https://github.com/pytest-dev/pytest-xdist/issues/555>`\_: Improved progress output when collecting nodes to be less verbose. ### [`v3.2.1`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-321-2023-03-12) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v3.2.0...v3.2.1) \=============================== ## Bug Fixes - `#884 <https://github.com/pytest-dev/pytest-xdist/issues/884>`\_: Fixed hang in `worksteal` scheduler. ### [`v3.2.0`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-320-2023-02-07) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v3.1.0...v3.2.0) \=============================== ## Improved Documentation - `#863 <https://github.com/pytest-dev/pytest-xdist/issues/863>`\_: Document limitations for debugging due to standard I/O of workers not being forwarded. Also, mention remote debugging as a possible workaround. ## Features - `#855 <https://github.com/pytest-dev/pytest-xdist/issues/855>`\_: Users can now configure `load` scheduling precision using `--maxschedchunk` command line option. - `#858 <https://github.com/pytest-dev/pytest-xdist/issues/858>`*: New `worksteal` scheduler, based on the idea of `work stealing <https://en.wikipedia.org/wiki/Work_stealing>`*. It's similar to `load` scheduler, but it should handle tests with significantly differing duration better, and, at the same time, it should provide similar or better reuse of fixtures. ## Trivial Changes - `#870 <https://github.com/pytest-dev/pytest-xdist/issues/870>`\_: Make the tests pass even when `$PYTEST_XDIST_AUTO_NUM_WORKERS` is set. ### [`v3.1.0`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-310-2022-12-01) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v3.0.2...v3.1.0) \=============================== ## Features - `#789 <https://github.com/pytest-dev/pytest-xdist/issues/789>`\_: Users can now set a default distribution mode in their configuration file: .. code-block:: ini [pytest] addopts = --dist loadscope - `#842 <https://github.com/pytest-dev/pytest-xdist/issues/842>`\_: Python 3.11 is now officially supported. ## Removals - `#842 <https://github.com/pytest-dev/pytest-xdist/issues/842>`\_: Python 3.6 is no longer supported. ### [`v3.0.2`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-302-2022-10-25) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v2.5.0...v3.0.2) \=============================== ## Bug Fixes - `#813 <https://github.com/pytest-dev/pytest-xdist/issues/813>`\_: Cancel shutdown when a crashed worker is restarted. ## Deprecations - `#825 <https://github.com/pytest-dev/pytest-xdist/issues/825>`\_: The `--rsyncdir` command line argument and `rsyncdirs` config variable are deprecated. The rsync feature will be removed in pytest-xdist 4.0. - `#826 <https://github.com/pytest-dev/pytest-xdist/issues/826>`\_: The `--looponfail` command line argument and `looponfailroots` config variable are deprecated. The loop-on-fail feature will be removed in pytest-xdist 4.0. ## Improved Documentation - `#791 <https://github.com/pytest-dev/pytest-xdist/issues/791>`\_: Document the `pytest_xdist_auto_num_workers` hook. - `#796 <https://github.com/pytest-dev/pytest-xdist/issues/796>`\_: Added known limitations section to documentation. - `#829 <https://github.com/pytest-dev/pytest-xdist/issues/829>`\_: Document the `-n logical` option. ## Features - `#792 <https://github.com/pytest-dev/pytest-xdist/issues/792>`\_: The environment variable `PYTEST_XDIST_AUTO_NUM_WORKERS` can now be used to specify the default for `-n auto` and `-n logical`. - `#812 <https://github.com/pytest-dev/pytest-xdist/issues/812>`\_: Partially restore old initial batch distribution algorithm in `LoadScheduling`. pytest orders tests for optimal sequential execution - i. e. avoiding unnecessary setup and teardown of fixtures. So executing tests in consecutive chunks is important for optimal performance. In v1.14, initial test distribution in `LoadScheduling` was changed to round-robin, optimized for the corner case, when the number of tests is less than `2 * number of nodes`. At the same time, it became worse for all other cases. For example: if some tests use some "heavy" fixture, and these tests fit into the initial batch, with round-robin distribution the fixture will be created `min(n_tests, n_workers)` times, no matter how many other tests there are. With the old algorithm (before v1.14), if there are enough tests not using the fixture, the fixture was created only once. So restore the old behavior for typical cases where the number of tests is much greater than the number of workers (or, strictly speaking, when there are at least 2 tests for every node). ## Removals - `#468 <https://github.com/pytest-dev/pytest-xdist/issues/468>`\_: The `--boxed` command-line option has been removed. If you still need this functionality, install `pytest-forked <https://pypi.org/project/pytest-forked>`\__ separately. ## Trivial Changes - `#468 <https://github.com/pytest-dev/pytest-xdist/issues/468>`\_: The `py` dependency has been dropped. - `#822 <https://github.com/pytest-dev/pytest-xdist/issues/822>`\_: Replace internal usage of `py.log` with a custom solution (but with the same interface). - `#823 <https://github.com/pytest-dev/pytest-xdist/issues/823>`\_: Remove usage of `py._pydir` as an rsync candidate. - `#824 <https://github.com/pytest-dev/pytest-xdist/issues/824>`\_: Replace internal usages of `py.path.local` by `pathlib.Path`. </details> --- ### Configuration 📅 **Schedule**: Branch creation - "every weekend" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/fluencelabs/spell).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

nicoddemus added a commit to nicoddemus/pytest-xdist that referenced this issue Nov 30, 2022

Let users configure dist mode in the configuration file

7a0bc4c

Fix pytest-dev#789

nicoddemus mentioned this issue Nov 30, 2022

Let users configure dist mode in the configuration file #851

Merged

nicoddemus mentioned this issue Dec 1, 2022

Thoughts on command-line vs configuration options pytest-dev/pytest#10551

Open

nicoddemus closed this as completed in #851 Dec 1, 2022

mikelane mentioned this issue Dec 5, 2022

Bump pytest-xdist from 3.0.2 to 3.1.0 mikelane/reddit-get#97

Merged

skshetry mentioned this issue Dec 6, 2022

build(deps-dev): Bump pytest-xdist from 3.0.2 to 3.1.0 iterative/dvc#8656

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Option to set the default distribution algorithm #789

Option to set the default distribution algorithm #789

DavideCanton commented Jul 6, 2022

antazoey commented Nov 22, 2022 •

edited

Loading

DavideCanton commented Nov 22, 2022

nicoddemus commented Nov 23, 2022

antazoey commented Nov 23, 2022

DavideCanton commented Nov 28, 2022

nicoddemus commented Nov 28, 2022

DavideCanton commented Nov 28, 2022

nicoddemus commented Nov 30, 2022

Option to set the default distribution algorithm #789

Option to set the default distribution algorithm #789

Comments

DavideCanton commented Jul 6, 2022

antazoey commented Nov 22, 2022 • edited Loading

DavideCanton commented Nov 22, 2022

nicoddemus commented Nov 23, 2022

antazoey commented Nov 23, 2022

DavideCanton commented Nov 28, 2022

nicoddemus commented Nov 28, 2022

DavideCanton commented Nov 28, 2022

nicoddemus commented Nov 30, 2022

antazoey commented Nov 22, 2022 •

edited

Loading