Add support for loading multiple modules from pyproject.toml #12551
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
|
@bendnorman is attempting to deploy a commit to the Elementl Team on Vercel. A member of the Team first needs to authorize it. |
erinkcochran87
requested review from
gibsondan
and removed request for
erinkcochran87
|
This seems totally reasonable to me. @schrockn do you feel OK about the syntax here? The alternative would be to have separate 'module_name' (when its singular) and 'module_names' (when it's plural) but a single key is probably better for consistency. |
|
My bad on the tardiness to the review here. One alternative to consider here is a different more "advanced" entry to allow for future extensions.
and also also the specification of attribute and location_name here. The most generic and future-proof being: My recommendation is a new |
## Summary & Motivation > Copied the same motivation that was stated back here, where it wasn't proceeded further #12551 It would be nice to load multiple modules using pyproject.toml so users don't have to type out all the modules they want to load using the CLI command. See issue #11439 and discussion #11167. I tried to add the recommendations mentioned here: #12551 (comment) ## How I Tested These Changes I added a new unit test in dagster_tests/cli_tests/test_toml_loading.py ## Changes: ### Add: Support for loading multiple modules from `pyproject.toml` in Dagster (#4bad886) ------------------------------------------------------------------------------------- * Updated documentation to include example of loading multiple modules (`DagsterDevTabs.mdx`) * Added `is_valid_modules_list` function to validate module lists in `pyproject.toml` (`load_target.py`) * Modified `get_origins_from_toml` to support loading multiple modules from `pyproject.toml` (`load_target.py`) * Added new test case `test_load_multiple_modules_from_toml` to verify loading multiple modules (`test_toml_loading.py`) * Created a new TOML test file `multiple_modules.toml` to simulate loading multiple modules (`toml_tests/multiple_modules.toml`)
Graphite Automations"Label and add CE on all Docs" took an action on this PR • (09/28/24)3 reviewers were added and 1 label was added to this PR based on Pedram Navid's automation. |
|
Closing stale PR. |
|
This was resolved by #22327 👍 |
Summary & Motivation
It would be nice to load multiple modules using
pyproject.tomlso users don't have to type out all the modules they want to load using the CLI command. See issue #11439 and discussion #11167.How I Tested These Changes
I added a new unit test in
dagster_tests/cli_tests/test_toml_loading.py