🐛 view_lineage() never completes if run output_artifacts contains a subset of input_artifacts

[Zethson]

Report

Discovered while playing with Nextflow use case.

af.view_lineage() never completes if af.run.output_artifacts contains Artifacts that are also in af.run.input_artifacts. I think this should error instead.

Version information

No response

[sunnyosun]

if af.run.output_artifacts contains Artifacts that are also in af.run.input_artifacts

How does this happen? Did you do that manually? In tracked notebooks, it never happens because we specifically account for it.

[Zethson]

I think it was something like the following in a script:

run = ln.context.run

# register input
input_af = ln.Artifact.from_dir(args.input)
ln.save(input_af)
run.input_artifacts.set(input_af)

output_af = ln.Artifact.from_dir(args.output)
ln.save(output_af)

[falexwolf]

Yes, we're not allowing cycles. It gotta be a DAG and @sunnyosun's viz code probably assumes it is one.

We could run some kind of check first but that's also really expensive and will likely slow things down 🤔

I agree it should error; probably it's stuck in a recursion

[sunnyosun]

input_af = ln.Artifact.from_dir(args.input)
ln.save(input_af)
run.input_artifacts.set(input_af)

This is problematic, you are registering the artifacts as output with ln.Artifact.from_dir(args.input), then as inputs via run.input_artifacts.set(input_af).

You need to pass ln.Artifact.from_dir(args.input, run=False)