Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Usage -> User Guide (initial) #382

Closed
wants to merge 19 commits into from
Closed

Usage -> User Guide (initial) #382

wants to merge 19 commits into from

Conversation

jorgeorpinel
Copy link
Contributor

@jorgeorpinel jorgeorpinel commented Nov 18, 2022

And moves some ex-/usage info to the cmd /ref (index page).

This is to start focusing the purpose of the different top-level pages (currently there's bits of info of everything in different places).

In review app: https://cml-dev-config-ref-ahaolkflhty.herokuapp.com/doc/user-guide

and move some info to cmd ref.
@jorgeorpinel jorgeorpinel added the documentation Markdown files label Nov 18, 2022
@jorgeorpinel jorgeorpinel requested a review from a team November 18, 2022 08:35
@shcheklein shcheklein temporarily deployed to cml-dev-config-ref-mipoc3rfgb7 November 18, 2022 08:36 Inactive
@shcheklein shcheklein temporarily deployed to cml-dev-config-ref-mipoc3rfgb7 November 18, 2022 08:39 Inactive
@jorgeorpinel jorgeorpinel marked this pull request as ready for review November 18, 2022 08:40
@jorgeorpinel jorgeorpinel requested a review from a team as a code owner November 18, 2022 08:40
@jorgeorpinel jorgeorpinel changed the title Usage->Config Usage -> Config Nov 18, 2022
content/docs/config.md Outdated Show resolved Hide resolved
Comment on lines 94 to 95
[![](/img/GitLab_CML_report.png '=400')](https://gitlab.com/iterative.ai/cml-base-case/-/merge_requests/3)
![First CML report in GitLab](/img/GitLab_CML_report.png '=400')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this link as it won't load (to the public at least):

image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/CC @0x2b3bfa0 please fix :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This 404 seems to still be an issue. Maybe the URL to the MR is different or the GL project needs to be made public?

@github-actions
Copy link

github-actions bot commented Nov 18, 2022

Link Check Report

All 12 links passed!

Comment on lines 138 to 139
[![](/img/bitbucket_cloud_pr.png '=600')](https://bitbucket.org/iterative-ai/cml-base-case/pull-requests/2)
![First CML report in BitBucket](/img/bitbucket_cloud_pr.png '=600')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same.

image

Doesn't the link checker catch these?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/CC @0x2b3bfa0 please fix :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same

content/docs/config.md Outdated Show resolved Hide resolved
redirects-list.json Show resolved Hide resolved
Comment on lines 94 to 95
[![](/img/GitLab_CML_report.png '=400')](https://gitlab.com/iterative.ai/cml-base-case/-/merge_requests/3)
![First CML report in GitLab](/img/GitLab_CML_report.png '=400')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/CC @0x2b3bfa0 please fix :)

Comment on lines 138 to 139
[![](/img/bitbucket_cloud_pr.png '=600')](https://bitbucket.org/iterative-ai/cml-base-case/pull-requests/2)
![First CML report in BitBucket](/img/bitbucket_cloud_pr.png '=600')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/CC @0x2b3bfa0 please fix :)

content/docs/config.md Outdated Show resolved Hide resolved
content/docs/config.md Show resolved Hide resolved
content/docs/config.md Outdated Show resolved Hide resolved
content/docs/ref/index.md Outdated Show resolved Hide resolved
Comment on lines 2 to 14

CML provides a number of commands to help package the outputs of ML workflows
into a CML report. These may include numeric data or model performance
visualizations. Let's look at the typical sequence (after
[configuration](/doc/config)):

∞ `cml runner` launches a runner hosted by a cloud compute provider or
[on-premise](/doc/self-hosted-runners).

∞ `cml pr` commits a set of files to a new branch and create a pull request.

∞ `cml comment` posts a Markdown report as a comment on a commit or pull/merge
request.

This comment was marked as resolved.

Copy link
Contributor Author

@jorgeorpinel jorgeorpinel Nov 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is where I moved the list of selected commands from Usage, to have something similar to https://dvc.org/doc/command-reference#typical-dvc-workflow . Of course we don't need to have the same structure or strategies as DVC docs but this one also applies, I think? There's a typical workflow (to give readers an expectation of which commands they'll probably use most).

We could update the list though, but I need help with that.

Copy link
Contributor

@casperdcl casperdcl Nov 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I strongly object1 to inserting synopsis/verbose explanations/justification/problem-context into a "command reference". To me a "cmd ref" is meant to be:

  • closer to a "glossary" or even "index page" in a book
    • super concise, dry, to-the-point efficient
  • aimed at power-users who know exactly what they conceptually need
    • e.g. can't quite remember the spelling/vocab (e.g. reader is wondering "was the flag called --machine-type or --cloud-type?")
  • links to other more prosaic pages for fuller explanations (no inline explanations)
  • includes common error codes/errors/edge cases/FAQ help

And I think this should apply to all software docs sites with zero exceptions.

To be CML-specific, we need to be even more careful to keep "cmd ref" as a concise replacement for CLI --help because almost nobody will run CLI --help locally (CML is a CI tool, not a local tool)

Footnotes

  1. based on my personal experience battling xkcd#773 scenarios

This comment was marked as resolved.

This comment was marked as resolved.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey folks, my take on this. And I agree that it's better for us to apply same rules (or apply exception consciously for a reason that we understand).

Since here you've been discussing the index page, not actual command reference pages. I don't have a particularly strong opinion about it tbh. Practically, this page indeed can serve as an index page:

closer to a "glossary" or even "index page" in a book
super concise, dry, to-the-point efficient

even its name implies this.

So, it doesn't hurt to have a table with commands. Something similar to a cheat sheet. If project doesn't have UG it might make sense to have diagrams, etc that would map all existing commands into workflow, etc.

I just don't see how that can hurt.


If we talk about specific pages inside command reference (an actual command pages). A page should have enough information to understand the semantics of an operation and what to expect from it if I run it in different ways (e.g. dvc gc - it should be clear that it collects references, removes those that are not used. It should be clear that -c affects the local cache and remote, etc, etc). The way to achieve that may be a bit different, but I would say from the most important to less important:

  • options and their descriptions. In some case it's fine to repeat the same sentence or two from CLI, in a lot of cases that's not enough. We should be of course trying to optimize so that both are close as much as possible (means that ppl can read less, can visit web less, etc). I just don't believe it's realistic to rely on one-two sentences there.
  • examples - any example, especially if it comes from a practical common problem worth a lot
  • description - I would expect ppl to read this less. Arguably this can become a part of UG, or something else.

Command reference in my head maps to a man page, to pages like this - https://git-scm.com/docs/git-checkout

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some major misconceptions here:

self-contained UI manuals [...] clig.dev [...] terminal-based documentation

This is completely inapplicable to CML, which effectively does not have terminal-based docs.1 It is far more akin to Studio - users only interact with it online. Basically this:

apply exception consciously for a reason that we understand

Secondly and more importantly, why should we delete content where a reasonable end-user needs it most?

Footnotes

  1. no reasonable end-user has ever run cml ... --help

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inapplicable to CML, which effectively does not have terminal-based docs

Good point. Let's keep that in mind but still the web format allows us more flexibility (no need to keep refs super dry IMO).

@jorgeorpinel jorgeorpinel changed the title Usage -> Config Usage -> Config (maybe) + related updates Nov 18, 2022
@shcheklein shcheklein temporarily deployed to cml-dev-config-ref-mipoc3rfgb7 November 24, 2022 06:42 Inactive
@shcheklein shcheklein temporarily deployed to cml-dev-config-ref-mipoc3rfgb7 November 24, 2022 06:51 Inactive
@shcheklein shcheklein temporarily deployed to cml-dev-config-ref-mipoc3rfgb7 November 24, 2022 07:01 Inactive
@shcheklein shcheklein temporarily deployed to cml-dev-config-ref-mipoc3rfgb7 November 24, 2022 07:16 Inactive
@jorgeorpinel jorgeorpinel changed the title Usage -> Config (maybe) + related updates Usage -> User Guide (initial) Nov 25, 2022
@jorgeorpinel jorgeorpinel added the C: guide Content of /doc/user-guide label Nov 25, 2022
Copy link
Contributor Author

@jorgeorpinel jorgeorpinel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too many changes in this PR. Could we please split this?

The only change unrelated to the PR purpose was to the comment command, which has now been moved to #383. Everything else comes from the existing Usage page, being turned into a User Guide section now.

Recap of changes 👇🏼

Comment on lines 64 to 77
## Examples

### Post a text report

Write to your report using whatever method you prefer. For example, copy the
contents of a text file containing the results of ML model training:

```cli
$ cat results.txt >> report.md
```

### Post a graphic report

Display images using Markdown or HTML. Note that if an image is an output of
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@casperdcl casperdcl Dec 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please revert

Comment on lines 1 to +6
# Command Reference

CML provides a number of commands to help package the outputs of ML workflows
into a CML report, which may include numeric data or model performance
visualizations. Let's look at the typical sequence (after
[configuration](/doc/user-guide)):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we move it back?

@casperdcl don't we need some content in the cmd ref index though? Currently it's empty except for a Generic Options section. Seems a little bare. Imagine a user lands in this page as their entry point to the docs site (per analytics this does happen sometimes) or as one of the first pages visited. It should give some general info on cml commands as a whole, link to GS or Usage, etc., I think.

Granted, this could be addressed separately and/or we can repeat the text above in you prefer to also keep this general guidance in the Usage (now UG index) page. Lmk

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please revert

Comment on lines -1 to 3
# Using CML
# User Guide: Introduction

A GitLab, GitHub, or Bitbucket account is required. Familiarity with
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Repurposed (simplified) much of this doc as the new UG index page

Comment on lines +32 to +37
"label": "CML with DVC",
"slug": "cml-with-dvc"
},
{
"label": "Self-hosted Runners",
"slug": "self-hosted-runners"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved these 2 existing docs into the (new) UG section

redirects-list.json Show resolved Hide resolved
@jorgeorpinel jorgeorpinel requested review from shcheklein and removed request for casperdcl December 16, 2022 00:46
@shcheklein
Copy link
Member

@jorgeorpinel thanks for cleaning this up!

Now, @casperdcl @jorgeorpinel my biggest question is - how this page https://cml-dev-config-ref-ahaolkflhty.herokuapp.com/doc/user-guide is different from the three pages here https://cml-dev-config-ref-ahaolkflhty.herokuapp.com/doc/start/github (+gitlab and +bb).

I think we need to agree on that first if we are touching this structure at all (which I would not consider that critical tbh).

My 2cs on this:

  • Existing Get Started index should be dissolved into the Home page and into "What is CML?" (can be part of UG or a global page)
  • The new UG index page serves better to become a new GS index page. It's already a quick start to some extent and it deserves to be at the beginning of the flow. If people need more details they can read GH, GL, BB sections in GS.

@casperdcl
Copy link
Contributor

casperdcl commented Dec 19, 2022

how this page cml-dev-config-ref-ahaolkflhty.herokuapp.com/doc/user-guide is different from the three pages here cml-dev-config-ref-ahaolkflhty.herokuapp.com/doc/start/github (+gitlab and +bb).

@shcheklein my intended target audiences (implied by the cards at https://cml.dev/doc) are:

  • /doc/start pages: beginners to CI/CD
    • I want to change them to use tabs, but that's out of scope for this PR
  • /doc/usage: intermediate/experienced users of CI/CD but new to CML reports
  • /doc/cml-with-dvc: intermediate/experienced users of DVC but new to CML reports
  • /doc/self-hosted-runners: advanced users of CI/CD but new to CML runners (cloud and/or on-premise)

So I feel we're losing target audiences if we move pieces of content and/or dissolve pages.

To me 100% of the changes in this PR harm the funnel.

@shcheklein
Copy link
Member

Thanks @casperdcl

It's not enough though to state the expectation, it should be clear to people and they should be reading it in the way you expect them to read. This is my biggest concern with the current structure (even I'm getting lost from time to time as I mentioned a few times - which of the pages to take to copy paste? which way to install to pick, etc).

I would be surprised if anyone can understand that start is for beginners and usage is for more advanced. Both of them are "get started/quick start"-kinda material and are hard to differentiate. If we want to keep it this way - let's be very explicit about this, in the text and in the name.

Also, I still think it can be a single section, page that covers both. Starts with the assumption that people are familiar with CI/CD and provides more details below, referencing them explicitly.

^^ this is primarily about the landing page + install button + get started + usage.

I'm fine with DVC and self-hosted runners. Clearly here we can debate only if we need an extra layer or not. I don't have a strong opinion on that and totally fine with either. I doubt that change like this hurts anything. Though, in the current state of the project we can do w/o an extra layer.

@casperdcl
Copy link
Contributor

casperdcl commented Dec 19, 2022

@shcheklein I fully agree.

To me a gewd action plan might be:

  1. rename /doc/start to "Intro to CI/CD and CML reports"
  2. rename /doc/usage to "Intro to CML reports"

move some existing content from (1) to (2), then write some more appropriate content in (1).

About landing page + install, yes we should fix that too (but separate issues, right? Probably #250 and #404)

@shcheklein
Copy link
Member

Hmm, it sounds too complicated to me still.

Even if you read the names:

Intro to CI/CD and CML reports
Intro to CML reports

They overlap, which one should I read if I'm familiar with CI/CD more or less? It's already a sign that something is wrong here.

@shcheklein shcheklein temporarily deployed to cml-dev-config-ref-ahaolkflhty December 27, 2022 17:11 Inactive
@jorgeorpinel
Copy link
Contributor Author

jorgeorpinel commented Dec 27, 2022

how this page (UG: Intro) is different from the three pages here (GS) ?
touching this structure ... I would not consider that critical

Good Q but this is an existing problem (see current Usage page). Except that it's worst rn, since it also tries to cover configuration, command reference, etc. It's messy (originally intended as a cheat sheet, I think).

This PR essentially cleans that up, which is intended as a step in the right direction. For example, this allowed us to identify the clear overlap with the GS and have this discussion.

Get Started index should be dissolved...
UG index to become a new GS index page (If people need more details they can read GH, GL, BB pages)

Yep, that could work 👍🏼

intended target audiences

Nice intentions but the content does not currently reflect that. E.g. the Usage page is as introductory as the GS (as @shcheklein points out). Also, we have an official framework for docs, which is what I'm following.

About landing page + install, yes we should fix that too (but separate issues, right?

Agree. Let's address as a follow-up.

100% of the changes in this PR harm the funnel.

Confused by this 😕. We have been discussing these change in different places for some time. You even suggested some of them specifically from what I remember, @casperdcl (e.g. call the page "User Guide: Introduction").

Regardless, these changes are not major (just a stepping stone) and hardly hurt anything. I wish we could be more nimble and move fwd.

Should we set up a call?

jorgeorpinel added a commit that referenced this pull request Dec 27, 2022
These changes pertain to #382 .
jorgeorpinel added a commit that referenced this pull request Dec 27, 2022
@casperdcl casperdcl self-assigned this Feb 14, 2023
@jorgeorpinel jorgeorpinel dismissed casperdcl’s stale review February 20, 2023 18:41

Outdated, I think.

Copy link
Contributor Author

@jorgeorpinel jorgeorpinel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently I had some pending comments here. Let's see what they are 👇🏼

U: Nothing is still relevant.

content/docs/ref/comment.md Outdated Show resolved Hide resolved
content/docs/ref/comment.md Outdated Show resolved Hide resolved
content/docs/ref/comment.md Outdated Show resolved Hide resolved
content/docs/ref/comment.md Outdated Show resolved Hide resolved
content/docs/ref/comment.md Outdated Show resolved Hide resolved
content/docs/ref/comment.md Outdated Show resolved Hide resolved
@jorgeorpinel
Copy link
Contributor Author

jorgeorpinel commented Feb 24, 2023

I can no longer push to this repo but I've solved the conflicts in my branch. I set up a fork so this PR can be reopened in https://github.com/iterative/cml.dev/compare/master...jorgeorpinel:cml.dev:config--ref?expand=1 if needed. Thanks

@jorgeorpinel jorgeorpinel removed their assignment Feb 24, 2023
@shcheklein
Copy link
Member

@jorgeorpinel I gave the access back, please feel free to resolve and let's redeploy the branch to make the final decision on this.

@jorgeorpinel
Copy link
Contributor Author

Thanks @shcheklein . Done

@shcheklein shcheklein temporarily deployed to cml-dev-config-ref-uz0nzru4ibv March 8, 2023 04:50 Inactive
@github-actions
Copy link

github-actions bot commented Mar 8, 2023

Link Check Report

All 11 links passed!

@shcheklein
Copy link
Member

@jorgeorpinel I see the intention, and sorry for asking you to resolve the conflict. I've decide to close this PR, can't focus now on this, and I don't think it changes the dynamics for (now not so complicated) docs structure in CML. I think it's fine to have everything on the top level for now. If you feel there were some improvements / fixes besides the structure changes - please let's move the into a separate PR. Thanks for the effort!

@shcheklein shcheklein closed this Mar 9, 2023
@jorgeorpinel
Copy link
Contributor Author

No worries. There are some minor things. Extracted to #450.

@jorgeorpinel jorgeorpinel mentioned this pull request Mar 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C: guide Content of /doc/user-guide discussion documentation Markdown files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants