[WIP] concepts: move from previous docs to new section, etc.

[jeremydesroches]

UPDATE: Latest status at #1944 (comment) (at the time of writing)

Per #550, creates a new Basic Concepts section in the user guide.

Initially an outline of extracted descriptions from requested topics/tooltips for scoping: single document vs. multiple and depth of description.

Summary of goals and plans from #550 (comment):

• There should be a page or section (maybe called Basic Concepts in UG, see example in this closed PR) — properly written on a user-level language (closer to UC) — explaining some concepts and DVC philosophy

  concepts like Cache, Remote, Wokspace, etc, and their relationships (with diagrams if needed)

• Possibly extract concept explanations from cmd refs (e.g. cache, dag, metrics/plots, remote)
• Figure out a connection to glossary tooltips (currently special md files with Frontmatter in content/docs/user-guide/basic-concepts/ — see glossary: create Glossary page (besides Basic Concepts?) #1395): Just link to the Concepts? Use a similar engine (with Frontmatter)? Merge completely?
• Find other issues that would be included in solving this (e.g. user-guide: cache/remote/workspace relationships #53, term: ambiguous use of "external" and "workspace" #1127).
• Consider also a related DVC Principles section (more philosophical, maybe in the Concepts section index)? See #550 (comment) above. Too much for this PR

Concepts to be included

• Workspace
• (Versioning through) Codification ?
• DVC Metafiles (may be part of ^)
• DVC Cache
• Data Pipelines
• Remote Storage
• Metrics and Plots
• Parameters

---

Concepts without main place in existing docs? (for a future PR)

• DVC Project (glossary entry exists, may be combined with workspace)

Other notes:

• rename DVC Dirs&Files guide and DVC metafiles concept page later per [WIP] concepts: move from previous docs to new section, etc. #1944 (comment)

  But don't extract detailed spec from guide (just the concept explanation)! It will be a separate task to figure that part out.

[shcheklein]

@jeremydesroches @jorgeorpinel qq- how will it replace/augment already existing basic concepts? I see the intention (one single page vs many pages), but we'll have to duplicate pretty much the information and support it later.

[jeremydesroches]

how will it replace/augment already existing basic concepts? I see the intention (one single page vs many pages), but we'll have to duplicate pretty much the information and support it later.

I think that the existing basic concepts, which are set up like a glossary, could be renamed as such. The Basic Concepts doc expands on select concepts.

The glossary approach is quite useful and it would be odd to exclude the "concepts". Instead, I suggest linking to the concept from the glossary items that have them.

For example, the dvc-cache.md glossary/tooltip links to DVC Files and Directories, so we might extract that to the Basic Concepts doc and apply a parallel structure to the other concepts.

With the above approach you could continue building out the glossary while also expanding on the more conceptual items. It avoids having to decide if a given term is best in the glossary or concepts — it can be both.

In practice all "concepts" should probably have a glossary entry, but not all glossary terms are concepts.

[jorgeorpinel]

qq- how will it replace/augment already existing basic concepts?

Meaning the glossary/tooltips @shcheklein ? Yes. That is one of the questions:

• Figure out a connection to glossary tooltips (currently special md files with Frontmatter in content/docs/user-guide/basic-concepts/

Not sure... But the tooltips are meant as very short summaries so we may need to maintain both? The longer explanations can grow considerably, e.g. DVC remotes as currently explained in https://dvc.org/doc/command-reference/remote.

one single page vs many pages

In fact we're not very sure of this format. May need one page per concept both because some concepts will be long to explain, and for SEO purposes (see #550 (comment)).

[shcheklein]

Okay, to share a bit of context they way I see it. Existing "basic concepts" framework was made in way to accommodate longer texts (that's the actual basic concepts thingy) + glossary (that comes from the short description that is provided via frontmatter) + it can drive SEO (again via frontmatter).

Make take from this discussion is that we should be doing something else only if we want to have everything on the single page.

In practice all "concepts" should probably have a glossary entry, but not all glossary terms are concepts.

this is true, but it doesn't break the existing framework in any way, right?

[jeremydesroches]

Existing "basic concepts" framework was made in way to accommodate longer texts (that's the actual basic concepts thingy) + glossary (that comes from the short description that is provided via frontmatter) + it can drive SEO (again via frontmatter).

Thanks @shcheklein, this context helps a lot.

Make take from this discussion is that we should be doing something else only if we want to have everything on the single page.

Given the above context that's how it seems to me, too. To me this is a win-win — I also like the frontmatter approach to create a tooltip, while still having underlying (expanded) content for clickthrough or to drive SEO.

That would mean we're taking the multiple document approach... sound OK @jorgeorpinel ?

In practice all "concepts" should probably have a glossary entry, but not all glossary terms are concepts.

this is true, but it doesn't break the existing framework in any way, right?

Right, we could add the concepts pages (or page) using the existing framework.

[jorgeorpinel]

Existing "basic concepts" framework was made to accommodate longer texts + glossary (short description via frontmatter).

Oh, I didn't realize md files in content/docs/user-guide/basic-concepts/ support a description in Frontmatter. Currently they only use the name and match fields. If that's the case then let's definitely just edit them there, but we may need a separate ticket to implement displaying them as pages of a special Basic Concepts section with the engine.

we're taking the multiple document approach... sound OK?

Yes @jeremydesroches. But the most important thing you can provide for starters is the content, whatever the final structure is. Please focus on that in parallel with this discussion. I feel like we still need to catch up on progress on that front.

Thanks

[jeremydesroches]

Oh, I didn't realize md files in content/docs/user-guide/basic-concepts/ support a description in Frontmatter...

I might be missing something but I don't follow how the description frontmatter would work and what the advantage is... I tried adding a description: block to test and saw no change to the tooltip behaviour. Is the idea that the frontmatter description would be the tooltip, and the body works like a normal .md page (which we'd add to navigation)? Edit: resolved, it works now.

In any case, I'll work on content for the concepts in parallel so we can place them in whatever structure is decided.

[shcheklein]

I tried adding a description: block to test and saw no change to the tooltip behaviour.

that can be fixed I think. @rogermparent can we make the engine take the text for tooltips from the frontmatter's description in this case?

@jeremydesroches the question is- should we use the same description in this case for SEO, or should we create two different fields? like glossary and description?

[jeremydesroches]

Good question. The constraints for the meta description (less than 160 characters, doesn't like markup) would make it difficult to address both with the same content. For example, the existing descriptions/definitions have links and are longer than 160 characters.

Your suggestion would work. Even if there was no unique meta description, good answers to the query in the body copy will get picked up automatically.

[rogermparent]

that can be fixed I think. @rogermparent can we make the engine take the text for tooltips from the frontmatter's description in this case?

Yes, we can specifically parse a field as Markdown at build time and use its HTML output to provide glossary text.
If we decide to co-locate pure glossary entries and full Basic Concepts pages in basic-concepts, we can have glossary entries fall back to the body if that field is not present.

[rogermparent]

I just made PR #1954 against this branch, it includes the ability to override glossary entries with a new frontmatter entry named tooltip that also supports Markdown like the current glossary entries. Check out the example "DVC Project" glossary entry on this page of the deploy.

[jeremydesroches]

Hey @rogermparent, the tooltip markdown works great but it seems that the glossary pages are being skipped for rendering. So if I add them to the navigation with sidebar.json or link to them directly the slug is there but the page is rendered blank.

To mock up what (I think) we are after here, I added a new "Concepts" folder and broke out the topics on separate pages. This isn't the latest content but I wanted to check on the direction.

My understanding of the desired behaviour is that glossary items could be added to the nav and rendered, appearing like these Concepts entries. The frontmatter on each would define the tooltip and the markdown would get rendered as a doc page. This would leave us with a single-source for each concept and/or glossary definition.

Does that sound right @jorgeorpinel and @shcheklein ?

[jorgeorpinel]

What's the best way (technically) to do that from here

First I'd merge the latest master...

You know what @jeremydesroches I can split this PR when ready. Let's focus on addressing feedback from your changes so far and especially on the SEO stuff. As each part seems ready I'll open parallel PRs with the actual intention to close them and ask Ivan to review those. Thanks

[jorgeorpinel]

p.s. You can list them as comments in each file change if you'd like @jeremydesroches, seems like the easiest approach to me.

p.p.s. This one (dvc-metafiles) doesn't have SEO metadata yet.

[jeremydesroches]

Based on your comment way up there ☝️ it seems like you are saying you don't want to move the Files content or create this item at all. Is that right @jorgeorpinel? If I'm mistaken on this, not a problem to research and create the metadata.

Otherwise, I think I've addressed the remaining feedback so we can start splitting up this PR to work on the pieces in order as you suggested.

[jorgeorpinel]

You're right but now we just need to revert moving content from https://dvc.org/doc/user-guide/dvc-files-and-directories to here. I guess I can just leave it out when I split this PR, so 👍

[jeremydesroches]

Looks like SEO metadata is missing from the dependency, dvc-cache, dvc-metafiles, output concepts (there are no changes to those files). All the frontmatter in fact (inc. draft tooltip)

Not following you here... dependency and output are glossary terms but aren't in the list of concepts. For dvc-cache, it is a full concept but it does have the frontmatter in the latest. 🤔 So I was only going to add frontmatter to dvc-metafiles. Is that different from what you were thinking @jorgeorpinel ?

[jorgeorpinel]

dvc-cache, it is a full concept but it does have the frontmatter in the latest. 🤔 So I was only going to add frontmatter to dvc-metafiles

Sounds good @jeremydesroches thanks!

[jorgeorpinel]

Ping @jeremydesroches if you no longer have the capacity to finish the SEO-related parts (see conversations above) just lmk please. Thanks.

[jeremydesroches]

Ping @jeremydesroches if you no longer have the capacity to finish the SEO-related parts (see conversations above) just lmk please. Thanks.

Sorry for the delay @jorgeorpinel. I'll get back to you on these items and most of the others today.

[jorgeorpinel]

No worries @jeremydesroches. I'm going to have low capacity for a few weeks anyway but I'll come back to splitting this ASAP. It looks like so far you've completed your contributions so thanks a bunch! unfortunately I have to close this PR but your commits will be preserved in following PRs which I'll link below for the record (later). Closing for now.