You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Should this mechanism internal of the Composable implementation?
Should this mechanism external of the Composable implementation?
Nevertheless, it seems composition rather than inheritance.
This ability will open up functionalities like:
Disable certain middleware in certain levels
Change parameters of some specific components
Serve as a kind of a dashboard to control the behavior of a pipeline.
Should this being constructed and passed around by the context?
Use contextvars?
Initialization
Basic initialization:
# way 1params= {
"args1": 1,
"args2": 2,
"node1": {
"__name__": "x",
"arg2": "some-arg2"
},
}
pipeline=Pipeline(**params)
# which is similar to way 2pipeline=Pipeline(args1=1, args2=2, node1={"__name__": "x", "arg2": "some-arg2"})
Pros:
Toward serializable params. Easier to perform plug-n-play.
Cons:
Too much nesting, hard to read and examine. The params might benefit from being flattened out (e.g. "node1.name", "node1.arg2"...), but this approach only works if we pass params as a dictionary. When we pass the params as __init__'s kwargs, we have to use the dictionary notation.
For complicated and nested pipeline, people don't usually initialize with hard-coded values in __init__ (way 1), since it is by nature will be very long, hard to observe, and does not allow plug-n-play. Instead, they would likely want something like way 2, because they can swap out values easily (changing params or swapping nodes is just a matter of manipulating a dictionary from some UI).
Update params or nodes dynamically
Update params and nodes of an existing pipeline.
pipeline=Pipeline(arg1="arg1", arg2="arg2")
pipeline.node1.node2.paramxyz="1000"# reload the module `node2` with new value for `paramxyz`pipeline.node1=cls# reload the module `node1` with new value for class
The text was updated successfully, but these errors were encountered:
Thinking points:
contextvars
?Initialization
Basic initialization:
Pros:
Cons:
params
as a dictionary. When we pass theparams
as__init__
's kwargs, we have to use the dictionary notation.For complicated and nested pipeline, people don't usually initialize with hard-coded values in
__init__
(way 1), since it is by nature will be very long, hard to observe, and does not allow plug-n-play. Instead, they would likely want something like way 2, because they can swap out values easily (changing params or swapping nodes is just a matter of manipulating a dictionary from some UI).Update params or nodes dynamically
Update params and nodes of an existing pipeline.
The text was updated successfully, but these errors were encountered: