Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve documentation about namespaces #2825

Closed
noklam opened this issue Jul 20, 2023 · 8 comments
Closed

Improve documentation about namespaces #2825

noklam opened this issue Jul 20, 2023 · 8 comments
Labels
Component: Documentation 📄 Issue/PR for markdown and API documentation

Comments

@noklam
Copy link
Contributor

noklam commented Jul 20, 2023

Description

A long Slack thread and I found there are couples of things not explained clearly for modular pipeline.

Context

  1. image - it appears that parameters is namespaced too but I am not sure yet
  1. The docs use a cooking analogy which explain the modularity of pipeline, but there isn't any code example and it is quite hard to understand without playing with code. Diagrams like this could potentially helpful, at least this is how I learn.

The view of this documentation is significant.(for reference, API docs total view are around 2000)
image

@astrojuanlu
Copy link
Member

Related (but not quite the same) #2633

@noklam
Copy link
Contributor Author

noklam commented Jul 25, 2023

#2839 Partially fix 1., but I think we need to improve this documentation.

@noklam
Copy link
Contributor Author

noklam commented Jul 25, 2023

Update the description and adding more backlink for context.

@noklam
Copy link
Contributor Author

noklam commented Aug 15, 2023

More evidence. https://linen-slack.kedro.org/t/14140948/hi-i-m-looking-at-namespaces-and-modular-pipelines-must-say-#cfa0d7eb-477b-4740-ab46-0682a6c04ebd

Hi, I’m looking at namespaces and modular pipelines. Must say, it’s somewhat confusing, and also some things seems to

have changed in newer versions of kedro.
For one, how should I understand this note a bit further down in the docs https://docs.kedro.org/en/0.18.12/nodes_and_pipelines/modular_pipelines.html#using-a-modular-pipeline-multiple-times
parameters references will not be namespaced, but params: references will.
In my mental model, it should not matter if I specify the whole parameter dictionary as “parameters” or if I give an individual element of it “params:elem”. This seems like a detail of how I implemented my node function. But indeed, when I tried this out, “params:elem”-type parameters will be “hidden” under the given namespace when creating a namespaced pipeline from a template one, while the “parameters” dict will stay global.
So basically “params:elem” and the “elem” attribute of “parameters” will refer to different values, for instance when I have a node function that takes input=["parameters", "params:elem"].
What’s the reasoning here, conceptually?
Thanks,
Viveca (edited)

@astrojuanlu
Copy link
Member

Also related: #2723

@astrojuanlu
Copy link
Member

Notice we used to have a "Namespace pipelines" page https://docs.kedro.org/en/0.17.7/03_tutorial/06_namespace_pipelines.html

@astrojuanlu astrojuanlu changed the title Improve modular pipeline documentations Improve documentation about namespaces May 27, 2024
@merelcht
Copy link
Member

Did #3948 address this enough or do we want to spend more time on improving the docs for namespaces?

@astrojuanlu
Copy link
Member

For the record https://docs.kedro.org/en/stable/nodes_and_pipelines/namespaces.html

Indeed further improvements can be made separately

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Documentation 📄 Issue/PR for markdown and API documentation
Projects
Archived in project
Development

No branches or pull requests

4 participants