2.0 Pipelines (parametrization)

[jorgeorpinel]

Based on https://github.com/iterative/dvc/wiki/Parametrization

• DVC Files guide - dvc.yaml section

  • General vars info
  • Parameters files
  • Global/Local vars
  • Multi-stages (foreach "loops")
  • Specific/multiple/partial params files in vars
  • Other notes e.g. that "parametrized variables are also displayed by params diff" and about wdir.

• params cmd ref.

• term "parametrize" -> "parametErize" + intro "templat(e|ing)"

• Follow this up with guide: make proper DVC File guide + separate reference? #2059 immediately after.

• Pending: terminology research (see 2.0 Pipelines (parametrization) #2052 (review)).

[jorgeorpinel]

Note: This example has no changes (which would be out of scope) but I moved the one about Python param files to the bottom so it looks like this is added.

It's the next example the one I added.

[shcheklein]

Okay, it's impossible to iterate on this PR further (GH breaks at this stage, it's impossible to follow all the comments, etc). Let's merge and work separately on the next version. The biggest concern is still structure:

All these titles do not look good to me. They are fine for Basic Concepts I guess, but not as a top level section in the User Guide. They should higher level.

After that we define the expected structure we can start iterating on the actual content.

• advanced-dvc.yaml - is too heavy and a mix of a reference (precise spec) and guide (titles are high level, no structure - story like)
• do we need two seprate dvc.yaml sections? why do we consider basic templating and advanced feature?
• .dvc file - is clearly a reference thing. Where do we put it?

[shcheklein]

@skshetry I agree that structure is not ideal, we'll iterate on it next thing. Let's merge it and do it in the next PR, this is one is not manageable.

Re the split/non-split - to the large extent if feels that it's a matter of preferences and we won't find a solution that fits all. I think the current approach is to keep pages reasonably short, to a single point (e.g. it's bad to have .dvc and the whole dvc.yaml - they are too different concepts. Can dvc.yaml fit a single page - good question-let's see in the next PR. I really hope so, I'm though worried that about some points I describe below, e.g. complexity of the right hand bar).

Usual reasons to split (besides personal preferences, which should not serve as an argument unless we do a focus group testing):

• have a clear link to separate page that describes one thing (good for SEO, since term becomes part of the URL, page has tailored meta section, etc)
• when you have too much on a single page- you won't have any navigation at all (due to multiple levels of nesting). Or we'll have to complicate right hand side bar.

Why is it neglected so much?

Nothing is neglected (e.g. we changed Get started when we had time from 10+ pages to 4). But I find traversing through those pages difficult is not an argument that we can use as a guideline to my mind. If you have some good structure or guideline in mind, please suggest it.

[jorgeorpinel]

Merged and moved discussion to #2059 (comment).

[jorgeorpinel]

It feels like we are splitting pages arbitrarily (just look at installation instructions).

@skshetry I don't see how Install pages are spit arbitrarily. But if you'd like to elaborate or have better examples feel free to open a separate issue. Thanks