Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate pytest-xdist Plugin for Parallel and Concurrent Testing #3870

Conversation

AndresCdo
Copy link
Contributor

@AndresCdo AndresCdo commented May 5, 2023

Background

Our current testing suite takes a significant amount of time to execute, which slows down the development process and reduces the team's productivity. To address this issue, we have decided to integrate the pytest-xdist plugin into our project. This plugin enables parallel and concurrent testing, which will help speed up our test execution and provide faster feedback to developers.

Changes

  • Added pytest-xdist to the project's requirements.txt file.
  • Modified CI/CD pipeline configuration to include parallel testing options for pytest. The following files were modified in .github folder:
    1. docker-ci.yml
    2. ci.yml

Test Plan

GitHub CI workflow.

PR Quality Checklist

  • My pull request is atomic and focuses on a single change.
  • I have thoroughly tested my changes with multiple different prompts.
  • I have considered potential risks and mitigations for my changes.
  • I have documented my changes clearly and comprehensively.
  • I have not snuck in any "extra" small tweaks changes

@vercel
Copy link

vercel bot commented May 5, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 10, 2023 6:43pm

@github-actions github-actions bot added the size/s label May 5, 2023
@AndresCdo
Copy link
Contributor Author

#3863

@codecov
Copy link

codecov bot commented May 5, 2023

Codecov Report

Patch and project coverage have no change.

Comparison is base (8f31196) 60.99% compared to head (0011c09) 60.99%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #3870   +/-   ##
=======================================
  Coverage   60.99%   60.99%           
=======================================
  Files          73       73           
  Lines        3310     3310           
  Branches      542      542           
=======================================
  Hits         2019     2019           
  Misses       1152     1152           
  Partials      139      139           

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@AndresCdo AndresCdo changed the title Integrate pytest-parallel Plugin for Parallel and Concurrent Testing Integrate pytest-xdist Plugin for Parallel and Concurrent Testing May 6, 2023
@vercel vercel bot temporarily deployed to Preview May 6, 2023 18:10 Inactive
@waynehamadi
Copy link
Contributor

@AndresCdo pipeline not passing

@vercel vercel bot temporarily deployed to Preview May 8, 2023 10:10 Inactive
@AndresCdo
Copy link
Contributor Author

@AndresCdo pipeline not passing

Please review it again

@AndresCdo AndresCdo marked this pull request as ready for review May 8, 2023 10:18
@AndresCdo AndresCdo requested a review from a team May 8, 2023 10:18
@vercel vercel bot temporarily deployed to Preview May 8, 2023 13:32 Inactive
@k-boikov
Copy link
Contributor

LGTM

@vercel vercel bot temporarily deployed to Preview May 10, 2023 18:43 Inactive
@waynehamadi waynehamadi merged commit 2513178 into Significant-Gravitas:master May 12, 2023
@AndresCdo AndresCdo deleted the feature/pytest-parallel-integration branch May 12, 2023 16:30
ppetermann pushed a commit to ppetermann/Auto-GPT that referenced this pull request May 13, 2023
…gnificant-Gravitas#3870)

* Adds pytest-parallel dependencies

* Implement pytest-parallel for faster tests

* Uses pytest-xdist

* Auto number of workers processes

* Update ci.yml

---------

Co-authored-by: Nicholas Tindle <[email protected]>
p-i- added a commit that referenced this pull request May 14, 2023
* Feature/tighten up ci pipeline (#3700)

* Fix docker volume mounts (#3710)

Co-authored-by: Reinier van der Leer <[email protected]>
Co-authored-by: Nicholas Tindle <[email protected]>

* Feature/enable intuitive logs for community challenge step 1 (#3695)

* Feature/enable intuitive logs summarization (#3697)

* Move task_complete command out of prompt (#3663)

* feat: move task_complete command out of prompt

* fix: formatting fixes

* Add the shutdown command to the test agents

* tests: update test vcrs

---------

Co-authored-by: James Collins <[email protected]>

* Allow users to Disable Commands via the .env (#3667)

* Document Disabling command categories (#3669)

* feat: move task_complete command out of prompt

* fix: formatting fixes

* feat: add command disabling

* docs: document how to disable command categories

* Enable denylist handling for plugins (#3688)

Co-authored-by: Luke Kyohere <[email protected]>
Co-authored-by: Nicholas Tindle <[email protected]>

* Fix call to `plugin.post_planning` (#3414)

Co-authored-by: Nicholas Tindle <[email protected]>

* create information retrieval challenge a (#3770)

Co-authored-by: Richard Beales <[email protected]>

* fix typos (#3798)

* Update run.bat (#3783)

Co-authored-by: Richard Beales <[email protected]>

* Update run.sh (#3752)

Co-authored-by: Richard Beales <[email protected]>

* ADD: Bash block in the contributing markdown (#3701)

Co-authored-by: Richard Beales <[email protected]>

* BUGFIX: Selenium Driver object reference was included in the browsing results for some reason (#3642)

* * there is really no need to return the  reference to the Selenium driver along with the text summary and list of links.

* * removing unused second return value from browse_website()

* * updated cassette

* * updated YAML cassette for test_browse_website

* * after requirements reinstall, another update YAML cassette for test_browse_website

* * another update YAML cassette for test_browse_website, only as a placholder commit to trigger re-testing due to some docker TCP timeout issue

* * another update YAML cassette for test_browse_website

---------

Co-authored-by: batyu <batyu@localhost>

* Update CONTRIBUTING.md

* Self feedback Improvement (#3680)

* Improved `Self-Feedback`

* minor tweak

* Test: Updated `test_get_self_feedback.py`

* community challenges in the wiki (#3764)

* Update README.md

* Update PULL_REQUEST_TEMPLATE.md

Added link to wiki Contributing page

* Add link to wiki Contributing page

* fix

* Add link to wiki page  on Contributing

* Implement Logging of User Input in logs/Debug Folder (#3867)

* Adds USER_INPUT_FILE_NAME

* Update agent.py

* Update agent.py

Log only if console_input is not the authorise_key

* Reformatting

* add information retrieval challenge to the wiki (#3876)

* add code owners policy (#3981)

* add code owners

* added @ to codeowners

* switched to team ownership

* Memory Challenge C (#3908)

* Memory Challenge C

* Working cassettes

* Doc fixes

* Linting and doc fix

* Updated cassette

* One more cassette try

---------

Co-authored-by: merwanehamadi <[email protected]>

* memory challenge c inconsistent (#3985)

* Improve & fix memory challenge docs. (#3989)

Co-authored-by: Kaan Osmanagaoglu <[email protected]>

* Feature/centralize prompt (#3990)

Co-authored-by: xiao.hu <[email protected]>

* Use correct reference to prompt_generator in autogpt/llm/chat.py (#4011)

* fix typos (#3998)

Co-authored-by: Minfeng Lu <[email protected]>
Co-authored-by: Richard Beales <[email protected]>

* fix typo in the getting started docs (#3997)

Co-authored-by: Richard Beales <[email protected]>

* Fix path to workspace directory in setup guide (#3927)

Co-authored-by: Nicholas Tindle <[email protected]>

* document that docker-compose 1.29.0 is minimally required (#3963)

Co-authored-by: Nicholas Tindle <[email protected]>

* Integrate pytest-xdist Plugin for Parallel and Concurrent Testing (#3870)

* Adds pytest-parallel dependencies

* Implement pytest-parallel for faster tests

* Uses pytest-xdist

* Auto number of workers processes

* Update ci.yml

---------

Co-authored-by: Nicholas Tindle <[email protected]>

* explain temperature setting in env file (#4140)

Co-authored-by: Richard Beales <[email protected]>

* Catch JSON error in summary_memory.py (#3996)

Co-authored-by: k-boikov <[email protected]>

* Update duckduckgo dependency - min should be 2.9.5 (#4142)

Co-authored-by: k-boikov <[email protected]>

* Update Dockerfile - add missing scripts and plugins directories. (#3706)

Co-authored-by: k-boikov <[email protected]>

* Updated memory setup links (#3829)

Co-authored-by: k-boikov <[email protected]>

* Parse package versions so upgrades can be forced (#4149)

* parse package versions so upgrades can be forced

* better version from @collijk

* fix typo in autopgt/agent/agent.py (#3747)

Co-authored-by: merwanehamadi <[email protected]>
Co-authored-by: Richard Beales <[email protected]>
Co-authored-by: k-boikov <[email protected]>

* Fix `milvus_memory_test.py` mock `Config` (#3424)

Co-authored-by: k-boikov <[email protected]>

* Implemented showing the number of preauthorised commands left. #1035 (#3322)

Co-authored-by: mayubi <[email protected]>
Co-authored-by: Nicholas Tindle <[email protected]>
Co-authored-by: k-boikov <[email protected]>

* Challenge: Kubernetes and documentation (#4121)

* challenge_kubes_and_readme

* docs

* testing

* black and isort

* revision

* lint

* comments

* blackisort

* docs

* docs

* deleting_cassette

* suggestions

* misspelling_errors

---------

Co-authored-by: merwanehamadi <[email protected]>

* Make sdwebui tests pass (when SD is running) (#3721)

Co-authored-by: Nicholas Tindle <[email protected]>

* Add Edge browser support using EdgeChromiumDriverManager (#3058)

Co-authored-by: Nicholas Tindle <[email protected]>
Co-authored-by: k-boikov <[email protected]>

* Added --install-plugin-deps to Docker (#4151)

Co-authored-by: Nicholas Tindle <[email protected]>

* Feature/basic proxy (#4164)

* basic proxy (#54)

* basic proxy (#55)

* basic proxy

* basic proxy

* basic proxy

* basic proxy

* add back double quotes

* add more specific files

* write file

* basic proxy

* Put back double quotes

* test new CI (#4168)

* test new CI

* test new CI

* remove double quotes

* Feature/test new ci pipeline 2 (#4169)

* test new CI

* remove double quotes

* make it a variable

* make it a variable

* Test New CI Pipeline (#4170)

* introduce dummy prompt change

* introduce dummy prompt change

* empty commit

* empty commit

* empty commit

* push to origin repo

* add s to quote

* Feature/fix rate limiting issue Step 1 (#4173)


* temporarilly remove 3.11

* add back 3.11 (#4185)

* Revert "Put back 3.11 until it's removed as a requirement" (#4191)

---------

Co-authored-by: Reinier van der Leer <[email protected]>
Co-authored-by: merwanehamadi <[email protected]>
Co-authored-by: Peter Petermann <[email protected]>
Co-authored-by: Nicholas Tindle <[email protected]>
Co-authored-by: James Collins <[email protected]>
Co-authored-by: Luke K <[email protected]>
Co-authored-by: Luke Kyohere <[email protected]>
Co-authored-by: Robin Richtsfeld <[email protected]>
Co-authored-by: RainRat <[email protected]>
Co-authored-by: itsmarble <[email protected]>
Co-authored-by: Ambuj Pawar <[email protected]>
Co-authored-by: bszollosinagy <[email protected]>
Co-authored-by: batyu <batyu@localhost>
Co-authored-by: Pi <[email protected]>
Co-authored-by: AbTrax <[email protected]>
Co-authored-by: Andres Caicedo <[email protected]>
Co-authored-by: Douglas Schonholtz <[email protected]>
Co-authored-by: Kaan <[email protected]>
Co-authored-by: Kaan Osmanagaoglu <[email protected]>
Co-authored-by: xiao.hu <[email protected]>
Co-authored-by: Tomasz Kasperczyk <[email protected]>
Co-authored-by: minfeng-ai <[email protected]>
Co-authored-by: Minfeng Lu <[email protected]>
Co-authored-by: Shlomi <[email protected]>
Co-authored-by: Itai Steinherz <[email protected]>
Co-authored-by: Boostrix <[email protected]>
Co-authored-by: Kristian Jackson <[email protected]>
Co-authored-by: k-boikov <[email protected]>
Co-authored-by: Eduardo Salinas <[email protected]>
Co-authored-by: prom3theu5 <[email protected]>
Co-authored-by: dominic-ks <[email protected]>
Co-authored-by: andrey13771 <[email protected]>
Co-authored-by: Marwand Ayubi <[email protected]>
Co-authored-by: mayubi <[email protected]>
Co-authored-by: Media <[email protected]>
Co-authored-by: Cenny <[email protected]>
Co-authored-by: Abdelkarim Habouch <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

5 participants