Create subsection "Nested workflow" in the docs

[agoscinski]

Moves the graph builder, if task and while task examples into this subsection. Also removed the graph_builder.ipynb since it is now generated. Solves issue #195

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.56%. Comparing base (5937b88) to head (120aa9a).
Report is 88 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #277      +/-   ##
==========================================
+ Coverage   75.75%   80.56%   +4.81%     
==========================================
  Files          70       66       -4     
  Lines        4615     5156     +541     
==========================================
+ Hits         3496     4154     +658     
+ Misses       1119     1002     -117     

Flag	Coverage Δ
python-3.11	80.48% <ø> (+4.82%)	⬆️
python-3.12	80.48% <ø> (?)
python-3.9	80.52% <ø> (+4.78%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[agoscinski]

test_pause_task_after_submit seems a bit unstable it failed already two times in different jobs

[superstar54]

Looking at the if and while task again, because we now have new If task and while task, thus there are two ways to implement if

• if task, dose not need nested workgraph
• use graph_builder, thus nested workgraph.

thus it's better to keep if and while separately, and not move to the nested_workgraph.

I suggest at the end of the graph_builder.py, we make a link to show the application in the if and while.

[agoscinski]

I dont think the graph_builder example is really about the dynamic part as it just forwards inputs like any task does it. We could make another example that runs different tasks depending on the inputs. Something like

@task.calcfunction()
def add_one(x):
    return x.value+1

@task.calcfunction()
def modulo_five(x):
    return x % 5

@graph_builder(outputs = [...])
def my_modular(i: orm.Int):
    wg = WorkGraph()
    if i.value < 5:
        task = wg.add_task(add_one)
    else:
        task = wg.add_task(modulo_five)
    # need wg.selector to expose for linking?
    return wg

Then one could say that this does not preserve provenance and directly interlude to the If workgraph

[agoscinski]

Added an example for a dynamic use case of the graph_builder

[superstar54]

Hi @agoscinski , Thanks for the work.

I would suggest not creating a dynamic-workflows section, but moving all three notebooks into the howto, so that user can see the dynamic, if and while immediately.

[superstar54]

Then one could say that this does not preserve provenance and directly interlude to the If workgraph

Why ti does not preserve provenance?

[agoscinski]

I would suggest not creating a dynamic-workflows section, but moving all three notebooks into the howto, so that user can see the dynamic, if and while immediately.

Okay, but the howto's starting to get messy and need some structure at some point, but I think for now one still can put everything there

[agoscinski]

Why ti does not preserve provenance?

I mean more that it does not store provenance as transparently as using the If WorkGraph, but looking at the provenance graph, It is not really capturing the if-then-else logic, but just only some additional information about the condition. But at least in the GUI the if-then-else is capture more transparently.

[agoscinski]

I merged now the nested and dynamic example because it seemed strange to see them separate

[superstar54]

Hi @agoscinski , thanks for the update!

The dynamic_graph_builder.py is not used, I think you want to remove it.

[superstar54]

LGTM, thanks!