-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix race condition in test case envs dir deletion #8438
Closed
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ries_on_server test case
wouterdb
approved these changes
Dec 4, 2024
Hugo-Inmanta
approved these changes
Dec 4, 2024
Processing this pull request |
inmantaci
pushed a commit
that referenced
this pull request
Dec 4, 2024
…ories_on_server` test case. (PR #8438) # Description Fix race condition in `test_environment_delete_removes_env_directories_on_server` test case. The registration of the compile request happens asynchronously with respect to the `notify_change()` API call. So the `is_compiling()` API call can happen before the compile request is created or before the compilation has started. ``` 02:45:43 __________ test_environment_delete_removes_env_directories_on_server ___________ 02:45:43 02:45:43 server = <inmanta.server.protocol.Server object at 0x7f67cc481c50> 02:45:43 client = <inmanta.protocol.endpoints.Client object at 0x7f67cc786610> 02:45:43 02:45:43 async def test_environment_delete_removes_env_directories_on_server( 02:45:43 server, 02:45:43 client, 02:45:43 ) -> None: 02:45:43 """ 02:45:43 Make sure the environment_delete endpoint deletes the environment directory on the server. 02:45:43 """ 02:45:43 state_dir: Optional[str] = config.Config.get("config", "state-dir") 02:45:43 assert state_dir is not None 02:45:43 env_dir = py.path.local(state_dir).join("server", "environments") 02:45:43 02:45:43 result = await client.create_project("env-test") 02:45:43 assert result.code == 200 02:45:43 project_id = result.result["project"]["id"] 02:45:43 02:45:43 result = await client.create_environment(project_id=project_id, name="env1") 02:45:43 assert result.code == 200 02:45:43 env_id = result.result["environment"]["id"] 02:45:43 02:45:43 result: Result = await client.notify_change(env_id) 02:45:43 assert result.code == 200 02:45:43 02:45:43 async def wait_for_compile() -> bool: 02:45:43 result = await client.is_compiling(env_id) 02:45:43 return result.code == 204 02:45:43 02:45:43 await utils.retry_limited(wait_for_compile, 15) 02:45:43 02:45:43 > assert os.path.exists(os.path.join(env_dir, env_id)) 02:45:43 E AssertionError: assert False 02:45:43 E + where False = <function exists at 0x7f680b8b7420>('/tmp/tmpmm37oya2/server/environments/a75c6da0-c84b-4ab7-b1d9-b661c109bc96') 02:45:43 E + where <function exists at 0x7f680b8b7420> = <module 'posixpath' (frozen)>.exists 02:45:43 E + where <module 'posixpath' (frozen)> = os.path 02:45:43 E + and '/tmp/tmpmm37oya2/server/environments/a75c6da0-c84b-4ab7-b1d9-b661c109bc96' = <function join at 0x7f6809a305e0>(local('/tmp/tmpmm37oya2/server/environments'), 'a75c6da0-c84b-4ab7-b1d9-b661c109bc96') 02:45:43 E + where <function join at 0x7f6809a305e0> = <module 'posixpath' (frozen)>.join 02:45:43 E + where <module 'posixpath' (frozen)> = os.path 02:45:43 02:45:43 tests/server/test_compilerservice.py:1979: AssertionError ``` # Self Check - [ ] ~~Attached issue to pull request~~ - [x] Changelog entry - [x] Type annotations are present - [x] Code is clear and sufficiently documented - [x] No (preventable) type errors (check using make mypy or make mypy-diff) - [x] Sufficient test cases (reproduces the bug/tests the requested feature) - [x] Correct, in line with design - [ ] ~~End user documentation is included or an issue is created for end-user documentation~~ - [ ] ~~If this PR fixes a race condition in the test suite, also push the fix to the relevant stable branche(s) (see [test-fixes](https://internal.inmanta.com/development/core/tasks/build-master.html#test-fixes) for more info)~~
Merged into branches master in 9b4077a |
inmantaci
pushed a commit
that referenced
this pull request
Dec 4, 2024
…ories_on_server` test case. (PR #8438) # Description Fix race condition in `test_environment_delete_removes_env_directories_on_server` test case. The registration of the compile request happens asynchronously with respect to the `notify_change()` API call. So the `is_compiling()` API call can happen before the compile request is created or before the compilation has started. ``` 02:45:43 __________ test_environment_delete_removes_env_directories_on_server ___________ 02:45:43 02:45:43 server = <inmanta.server.protocol.Server object at 0x7f67cc481c50> 02:45:43 client = <inmanta.protocol.endpoints.Client object at 0x7f67cc786610> 02:45:43 02:45:43 async def test_environment_delete_removes_env_directories_on_server( 02:45:43 server, 02:45:43 client, 02:45:43 ) -> None: 02:45:43 """ 02:45:43 Make sure the environment_delete endpoint deletes the environment directory on the server. 02:45:43 """ 02:45:43 state_dir: Optional[str] = config.Config.get("config", "state-dir") 02:45:43 assert state_dir is not None 02:45:43 env_dir = py.path.local(state_dir).join("server", "environments") 02:45:43 02:45:43 result = await client.create_project("env-test") 02:45:43 assert result.code == 200 02:45:43 project_id = result.result["project"]["id"] 02:45:43 02:45:43 result = await client.create_environment(project_id=project_id, name="env1") 02:45:43 assert result.code == 200 02:45:43 env_id = result.result["environment"]["id"] 02:45:43 02:45:43 result: Result = await client.notify_change(env_id) 02:45:43 assert result.code == 200 02:45:43 02:45:43 async def wait_for_compile() -> bool: 02:45:43 result = await client.is_compiling(env_id) 02:45:43 return result.code == 204 02:45:43 02:45:43 await utils.retry_limited(wait_for_compile, 15) 02:45:43 02:45:43 > assert os.path.exists(os.path.join(env_dir, env_id)) 02:45:43 E AssertionError: assert False 02:45:43 E + where False = <function exists at 0x7f680b8b7420>('/tmp/tmpmm37oya2/server/environments/a75c6da0-c84b-4ab7-b1d9-b661c109bc96') 02:45:43 E + where <function exists at 0x7f680b8b7420> = <module 'posixpath' (frozen)>.exists 02:45:43 E + where <module 'posixpath' (frozen)> = os.path 02:45:43 E + and '/tmp/tmpmm37oya2/server/environments/a75c6da0-c84b-4ab7-b1d9-b661c109bc96' = <function join at 0x7f6809a305e0>(local('/tmp/tmpmm37oya2/server/environments'), 'a75c6da0-c84b-4ab7-b1d9-b661c109bc96') 02:45:43 E + where <function join at 0x7f6809a305e0> = <module 'posixpath' (frozen)>.join 02:45:43 E + where <module 'posixpath' (frozen)> = os.path 02:45:43 02:45:43 tests/server/test_compilerservice.py:1979: AssertionError ``` # Self Check - [ ] ~~Attached issue to pull request~~ - [x] Changelog entry - [x] Type annotations are present - [x] Code is clear and sufficiently documented - [x] No (preventable) type errors (check using make mypy or make mypy-diff) - [x] Sufficient test cases (reproduces the bug/tests the requested feature) - [x] Correct, in line with design - [ ] ~~End user documentation is included or an issue is created for end-user documentation~~ - [ ] ~~If this PR fixes a race condition in the test suite, also push the fix to the relevant stable branche(s) (see [test-fixes](https://internal.inmanta.com/development/core/tasks/build-master.html#test-fixes) for more info)~~
Processing #8440. |
sanderr
pushed a commit
that referenced
this pull request
Dec 9, 2024
Bumps [pip](https://github.com/pypa/pip) from 24.2 to 24.3.1. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pypa/pip/blob/main/NEWS.rst">pip's changelog</a>.</em></p> <blockquote> <h1>24.3.1 (2024-10-27)</h1> <h2>Bug Fixes</h2> <ul> <li>Allow multiple nested inclusions of the same requirements file again. (<code>[#13046](pypa/pip#13046) <https://github.com/pypa/pip/issues/13046></code>_)</li> </ul> <h1>24.3 (2024-10-27)</h1> <h2>Deprecations and Removals</h2> <ul> <li>Deprecate wheel filenames that are not compliant with :pep:<code>440</code>. (<code>[#12918](pypa/pip#12918) <https://github.com/pypa/pip/issues/12918></code>_)</li> </ul> <h2>Features</h2> <ul> <li>Detect recursively referencing requirements files and help users identify the source. (<code>[#12653](pypa/pip#12653) <https://github.com/pypa/pip/issues/12653></code>_)</li> <li>Support for :pep:<code>730</code> iOS wheels. (<code>[#12961](pypa/pip#12961) <https://github.com/pypa/pip/issues/12961></code>_)</li> </ul> <h2>Bug Fixes</h2> <ul> <li>Display a better error message when an already installed package has an invalid requirement. (<code>[#12953](pypa/pip#12953) <https://github.com/pypa/pip/issues/12953></code>_)</li> <li>Ignore <code>PIP_TARGET</code> and <code>pip.conf</code> <code>global.target</code> when preparing a build environment. (<code>[#8438](pypa/pip#8438) <https://github.com/pypa/pip/issues/8438></code>_)</li> <li>Restore support for macOS 10.12 and older (via truststore). (<code>[#12901](pypa/pip#12901) <https://github.com/pypa/pip/issues/12901></code>_)</li> <li>Allow installing pip in editable mode in a virtual environment on Windows. (<code>[#12666](pypa/pip#12666) <https://github.com/pypa/pip/issues/12666></code>_)</li> </ul> <h2>Vendored Libraries</h2> <ul> <li>Upgrade certifi to 2024.8.30</li> <li>Upgrade distlib to 0.3.9</li> <li>Upgrade truststore to 0.10.0</li> <li>Upgrade urllib3 to 1.26.20</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pypa/pip/commit/05293b6b55eca86490b7c2944bcc558a56064f0d"><code>05293b6</code></a> Bump for release</li> <li><a href="https://github.com/pypa/pip/commit/6a5db8b107bb0063c69dc5ccd39dbfef14ca7a32"><code>6a5db8b</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/13047">#13047</a> from sbidoul/fix-13046</li> <li><a href="https://github.com/pypa/pip/commit/7be54ced1cca2c850e79e8fbe9ec2b76947b2b6f"><code>7be54ce</code></a> Don't fail when the same req file is included more than once</li> <li><a href="https://github.com/pypa/pip/commit/4f6aeb17ed540e181b9ad1dea8d7b5389effd21b"><code>4f6aeb1</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/13044">#13044</a> from sbidoul/release/24.3</li> <li><a href="https://github.com/pypa/pip/commit/e1b1d51fe8d0f4b84b77206173ceb656caa2edeb"><code>e1b1d51</code></a> Bump for development</li> <li><a href="https://github.com/pypa/pip/commit/cdba22f49b425fe4a57a8daf992fd6335c8010a1"><code>cdba22f</code></a> Bump for release</li> <li><a href="https://github.com/pypa/pip/commit/27f8374e8dd49141bd2397c0e8e8093cf3676ff7"><code>27f8374</code></a> Update AUTHORS.txt</li> <li><a href="https://github.com/pypa/pip/commit/c79d01953357913f421f192f51ffa9bab0a75ba0"><code>c79d019</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/13033">#13033</a> from sbidoul/vendoring-24.3-sbi</li> <li><a href="https://github.com/pypa/pip/commit/3ca89215a96f9b05619fc52bb778c19f26b84a9f"><code>3ca8921</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/13041">#13041</a> from sethmlarson/truststore-0.10.0</li> <li><a href="https://github.com/pypa/pip/commit/0cc7375ff0a42ddfa19f23f42cb96d6d7c06d29b"><code>0cc7375</code></a> Upgrade vendored truststore to 0.10.0</li> <li>Additional commits viewable in <a href="https://github.com/pypa/pip/compare/24.2...24.3.1">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pip&package-manager=pip&previous-version=24.2&new-version=24.3.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>
inmantaci
pushed a commit
that referenced
this pull request
Jan 16, 2025
Bumps [pip](https://github.com/pypa/pip) from 24.2 to 24.3.1. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pypa/pip/blob/main/NEWS.rst">pip's changelog</a>.</em></p> <blockquote> <h1>24.3.1 (2024-10-27)</h1> <h2>Bug Fixes</h2> <ul> <li>Allow multiple nested inclusions of the same requirements file again. (<code>[#13046](pypa/pip#13046) <https://github.com/pypa/pip/issues/13046></code>_)</li> </ul> <h1>24.3 (2024-10-27)</h1> <h2>Deprecations and Removals</h2> <ul> <li>Deprecate wheel filenames that are not compliant with :pep:<code>440</code>. (<code>[#12918](pypa/pip#12918) <https://github.com/pypa/pip/issues/12918></code>_)</li> </ul> <h2>Features</h2> <ul> <li>Detect recursively referencing requirements files and help users identify the source. (<code>[#12653](pypa/pip#12653) <https://github.com/pypa/pip/issues/12653></code>_)</li> <li>Support for :pep:<code>730</code> iOS wheels. (<code>[#12961](pypa/pip#12961) <https://github.com/pypa/pip/issues/12961></code>_)</li> </ul> <h2>Bug Fixes</h2> <ul> <li>Display a better error message when an already installed package has an invalid requirement. (<code>[#12953](pypa/pip#12953) <https://github.com/pypa/pip/issues/12953></code>_)</li> <li>Ignore <code>PIP_TARGET</code> and <code>pip.conf</code> <code>global.target</code> when preparing a build environment. (<code>[#8438](pypa/pip#8438) <https://github.com/pypa/pip/issues/8438></code>_)</li> <li>Restore support for macOS 10.12 and older (via truststore). (<code>[#12901](pypa/pip#12901) <https://github.com/pypa/pip/issues/12901></code>_)</li> <li>Allow installing pip in editable mode in a virtual environment on Windows. (<code>[#12666](pypa/pip#12666) <https://github.com/pypa/pip/issues/12666></code>_)</li> </ul> <h2>Vendored Libraries</h2> <ul> <li>Upgrade certifi to 2024.8.30</li> <li>Upgrade distlib to 0.3.9</li> <li>Upgrade truststore to 0.10.0</li> <li>Upgrade urllib3 to 1.26.20</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pypa/pip/commit/05293b6b55eca86490b7c2944bcc558a56064f0d"><code>05293b6</code></a> Bump for release</li> <li><a href="https://github.com/pypa/pip/commit/6a5db8b107bb0063c69dc5ccd39dbfef14ca7a32"><code>6a5db8b</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/13047">#13047</a> from sbidoul/fix-13046</li> <li><a href="https://github.com/pypa/pip/commit/7be54ced1cca2c850e79e8fbe9ec2b76947b2b6f"><code>7be54ce</code></a> Don't fail when the same req file is included more than once</li> <li><a href="https://github.com/pypa/pip/commit/4f6aeb17ed540e181b9ad1dea8d7b5389effd21b"><code>4f6aeb1</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/13044">#13044</a> from sbidoul/release/24.3</li> <li><a href="https://github.com/pypa/pip/commit/e1b1d51fe8d0f4b84b77206173ceb656caa2edeb"><code>e1b1d51</code></a> Bump for development</li> <li><a href="https://github.com/pypa/pip/commit/cdba22f49b425fe4a57a8daf992fd6335c8010a1"><code>cdba22f</code></a> Bump for release</li> <li><a href="https://github.com/pypa/pip/commit/27f8374e8dd49141bd2397c0e8e8093cf3676ff7"><code>27f8374</code></a> Update AUTHORS.txt</li> <li><a href="https://github.com/pypa/pip/commit/c79d01953357913f421f192f51ffa9bab0a75ba0"><code>c79d019</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/13033">#13033</a> from sbidoul/vendoring-24.3-sbi</li> <li><a href="https://github.com/pypa/pip/commit/3ca89215a96f9b05619fc52bb778c19f26b84a9f"><code>3ca8921</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/13041">#13041</a> from sethmlarson/truststore-0.10.0</li> <li><a href="https://github.com/pypa/pip/commit/0cc7375ff0a42ddfa19f23f42cb96d6d7c06d29b"><code>0cc7375</code></a> Upgrade vendored truststore to 0.10.0</li> <li>Additional commits viewable in <a href="https://github.com/pypa/pip/compare/24.2...24.3.1">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pip&package-manager=pip&previous-version=24.2&new-version=24.3.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fix race condition in
test_environment_delete_removes_env_directories_on_server
test case. The registration of the compile request happens asynchronously with respect to thenotify_change()
API call. So theis_compiling()
API call can happen before the compile request is created or before the compilation has started.Self Check
Attached issue to pull requestEnd user documentation is included or an issue is created for end-user documentationIf this PR fixes a race condition in the test suite, also push the fix to the relevant stable branche(s) (see test-fixes for more info)