Skip to content

Commit

Permalink
README rewrite (+ CONTRIB updates) (#1816)
Browse files Browse the repository at this point in the history 
* readme: make intro = value prop +
+ fancy note

* readme: spave between VS and Code
per #1816 (review)

* readme: reorg existing content per
per iterative/dvc.org#2443 (comment)

* readme: add features list

* readme: extend/fix/link other parts of the structure

* readme: fix DVC icon

* readme: value prop edited from
https://docs.google.com/document/d/10THN0WwvPaDOAnmWd37xMOSWftJrUSIPlA2m0iHGDNI/edit#
per #1816 (comment)

* Update README.md

Co-authored-by: Dave Berenbaum <[email protected]>

* Update README.md

* readme: rewrap

* readme: add list of high-level features
from https://docs.google.com/document/d/10THN0WwvPaDOAnmWd37xMOSWftJrUSIPlA2m0iHGDNI/edit#

* readme: update features and add competitive advantages
based on https://docs.google.com/document/d/10THN0WwvPaDOAnmWd37xMOSWftJrUSIPlA2m0iHGDNI/edit#

* readme: mention that it's beta
per #1816 (comment)

* readme: small improvements to lists

* readme: better Quick start

* readme: populate Config section

* readme: move Useful commands before Config

* config: update settings descriptions (readme and package)

* contrib: copy edits

* readme: fix vs-code commands lnk

* contrib: kill link to wiki (outdated)

* readme: lint (BROKEN)

* Update README.md

Co-authored-by: mattseddon <[email protected]>

* Update README.md

Co-authored-by: mattseddon <[email protected]>

* fix linter-broken line

* contrib: simpler dev env steps

* readme: better intro wording
per #1816 (review)

* readme: plots view -> dashboard :/
per #1816 (comment)

* readme: support should go to Discord first
per #1816 (review)
and #1816 (review)

* readme: exp bookkeeping -> tracking
per #1816 (review)

* readme: clarify reproducibility feature
per #1816 (review)

* readme: more specific data mgmt use
per #1816 (review)

* readme: improve UI components list
per #1816 (review)

* readme: fix img hack
may not work until repo goes public

* add badges to README

* drop some badges

* preview banner

* update link to point to main

* preview gif

* readme: impro feats

* contrib: move note about DVC projects

* revert to changelog from main and add prettier rule

* Update README.md

Co-authored-by: mattseddon <[email protected]>

* readme: move animation after value prop paragraph

* Apply suggestions from code review

Co-authored-by: mattseddon <[email protected]>

* Update README.md

* Restyled by whitespace

* Restyled by prettier-markdown

* Update README.md

* contrib: valid md new line >:(

* readme: move images to extension/images and
and add screenshot of Get Started page

* readme: move images back to extension/docs/ foler
per #1816 (review)

* swap banner to png

* replace outdated walkthrough screenshot

* update style of inline dvc png so it does not look bonkers

* (fixup) use absolute links for the marketplace and in product

Co-authored-by: Dave Berenbaum <[email protected]>
Co-authored-by: mattseddon <[email protected]>
Co-authored-by: Matt Seddon <[email protected]>
Co-authored-by: Restyled.io <[email protected]>
  • Loading branch information
@jorgeorpinel @dberenbaum
@mattseddon @restyled-commits
5 people authored Jun 14, 2022
1 parent ab92427 commit 2f6b3cb
Show file tree
Hide file tree
Showing 8 changed files with 207 additions and 91 deletions.
1 change: 1 addition & 0 deletions .prettierignore
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
CHANGELOG.md
1 change: 1 addition & 0 deletions .vscode/settings.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
"appdirs",
"camelcase",
"chokidar",
"Codespaces",
"conda",
"datapoint",
"datapoints",
Expand Down
86 changes: 41 additions & 45 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
@@ -1,55 +1,49 @@
# Feedback
## Feedback

See something that should be changed? Want to request a new feature? Open
[a GitHub Issue!](https://github.com/iterative/vscode-dvc/issues)
See something that should be changed? Want to request a new feature? Open an
[issue on GitHub](https://github.com/iterative/vscode-dvc/issues)!

# Development Setup
## Pull requests

Ensure [Yarn](https://yarnpkg.com/) and
[Visual Studio Code](https://code.visualstudio.com) are installed.

It is recommended that you have a DVC project available to test the extension
against. We have provided [the demo project](#the-demo-project) as part of this
repo but feel free to use any DVC project that you have available.

To enable formatting on save in VS Code, install the `esbenp.prettier-vscode`
extension. This is optional, but highly recommended as PRs with improper format
will be blocked from merge until the issue is fixed.
You may also submit a change request to this repository directly
[from here](https://github.com/iterative/vscode-dvc/pulls). Contributions are
highly appreciated!

<a id='note'></a>
> 💡 **Tip**: To enable formatting on save in VS Code, install the
> `esbenp.prettier-vscode` extension. This is highly recommended as PRs with
> improper format will be blocked from merge until fixed.
Note: When using any project that relies on an isolated python environment
(conda, venv, etc) the
[ms-python.python](https://github.com/Microsoft/vscode-python) extension needs
to be installed into VS Code. It is used by this extension to locate and utilize
the required environment.
## Development environment

## Development Environment
The development environment allows contributors to test their changes to the
extension. The local repository source is built and loaded into the Extension
Development Host.

The development environment allows contributors to test out the extension. The
current repository source is built and loaded into the Extension Development
Host as an Extension.

Please be aware that having a separate (`.vsix`) version of the extension
installed will cause all kinds of chaos in your development environment.
First, ensure that [Yarn](https://yarnpkg.com/) and
[Visual Studio Code](https://code.visualstudio.com) are installed.

- Open the monorepo root as a project in VS Code
- Open this repository's root directory as a project in VS Code.

- Run `Tasks: Run Build Task` (Ctrl+Shift+b) to start the extension and webview
development servers (alternatively run `yarn dev-server` from the terminal).

> **Warning**: Having a separate (`.vsix`) version of the extension installed
> may cause all kinds of chaos in your development environment.
- Open the Extension Development Host, a child instance of VS Code with the
results of the dev servers installed as an extension, with `Start Debugging`
(f5).
Note: selecting the `Run Extension` option when running the debugger will
prevent all other extensions from being loaded into the VS code instance. This
will improve the performance of VS code but can cause certain DVC commands to
fail if the project uses an isolated python environment (see
[this note](#note)).
(f5).

- Open the demo or another DVC project in the Extension Development Host; VS
Code will remember the last project opened, so this step only has to be done
once.
> **Note**: using the `Run Extension` command when running the debugger will
> prevent all other extensions from loading into VS Code. This will improve
> the performance of VS Code but can cause certain DVC commands to fail if the
> project uses an isolated python environment (see [this warning](#warning)).
- Open a DVC project in the Extension Development Host. VS Code will remember
the last project opened, so this step only has to be done once.

> **Note**: We have provided a demo project as part of this repo, but feel
> free to use any DVC project that you have available.
## The demo project

Expand All @@ -75,9 +69,17 @@ encouraged to try other DVC repositories- especially real-world cases!
environment to be loaded.

- This will happen automatically IF the `ms-python.python` extension is
installed within the VS code instance that you are developing against.
installed within the VS Code instance that you are developing against.

<a id='warning'></a>

## React Component Development with Storybook
> **Warning**
> When using any project that relies on an isolated Python environment (`conda`,
> `venv`, etc.), Microsoft's
> [Python extension](https://github.com/Microsoft/vscode-python) is required.
> It's used by this extension to locate and utilize the required environment.
## React component development with Storybook

Start Storybook with `yarn storybook` in either the monorepo root or the
`webview` project, and you can develop the React components this plugin uses
Expand All @@ -87,9 +89,3 @@ There are some discrepancies between the Storybook environment and the
environment of a real VS Code extension, custom themes being one big one. Always
make sure to try out changed components in the full dev environment before
merging!

# Resources

Check out the
[Resources page](https://github.com/iterative/vscode-dvc/wiki/Resources) of our
GitHub wiki for a list of relevant docs and related projects.
188 changes: 153 additions & 35 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,31 +1,121 @@
![banner](https://raw.githubusercontent.com/iterative/vscode-dvc/main/extension/resources/banner.png)

# DVC Extension for Visual Studio Code

<!--- These will be broken until the repo is public --->
<!--- [![Continuous Integration](https://github.com/iterative/vscode-dvc/actions/workflows/continuous-integration.yml/badge.svg)](https://github.com/iterative/vscode-dvc/actions/workflows/continuous-integration.yml) --->
<!--- [![Cross-Platform Test](https://github.com/iterative/vscode-dvc/actions/workflows/cross-platform-test.yml/badge.svg)](https://github.com/iterative/vscode-dvc/actions/workflows/cross-platform-test.yml) --->
<!-- [![DVC CLI Output Test](https://github.com/iterative/vscode-dvc/actions/workflows/dvc-cli-output-test.yml/badge.svg)](https://github.com/iterative/vscode-dvc/actions/workflows/dvc-cli-output-test.yml) -->
[![Version](https://vsmarketplacebadge.apphb.com/version-short/iterative.dvc.svg)](https://https://marketplace.visualstudio.com/items?itemName=Iterative.dvc)
[![Installs](https://vsmarketplacebadge.apphb.com/installs/iterative.dvc.svg)](https://https://marketplace.visualstudio.com/items?itemName=Iterative.dvc)
[![Downloads](https://vsmarketplacebadge.apphb.com/downloads/iterative.dvc.svg)](https://https://marketplace.visualstudio.com/items?itemName=Iterative.dvc)
[![Ratings](https://vsmarketplacebadge.apphb.com/rating-short/iterative.dvc.svg)](https://https://marketplace.visualstudio.com/items?itemName=Iterative.dvc)

[![Continuous Integration](https://github.com/iterative/vscode-dvc/actions/workflows/continuous-integration.yml/badge.svg)](https://github.com/iterative/vscode-dvc/actions/workflows/continuous-integration.yml)
[![Maintainability](https://api.codeclimate.com/v1/badges/fb243c31ea059c0038b2/maintainability)](https://codeclimate.com/repos/608b5886f52398018b00264c/maintainability)
[![Test Coverage](https://api.codeclimate.com/v1/badges/fb243c31ea059c0038b2/test_coverage)](https://codeclimate.com/repos/608b5886f52398018b00264c/test_coverage)

A [Visual Studio Code](https://code.visualstudio.com/)
[extension](https://marketplace.visualstudio.com/VSCode) that aims to allow
users of all technical backgrounds to effectively use [DVC](https://dvc.org/).

## Current State

We want early adopter feedback! Please let us know what you like and don't like
about the extension. Feel free to reach out in this repository's
[issues](https://github.com/iterative/vscode-dvc/issues) or via any of the other
existing [support channels](https://dvc.org/support).

## Quick Start

- **Step 1.**
[Install a supported version of DVC on your system](https://dvc.org/doc/install)
- **Step 2.**
[Install the DVC extension for Visual Studio Code](https://code.visualstudio.com/docs/editor/extension-gallery).
- **Step 3.** See Walkthrough.
Run, compare, visualize, and track machine learning experiments right in VS
Code. This extension uses [DVC](https://dvc.org/), an open-source data
versioning and ML experiment management tool. No additional services or
databases are required.

![Extension Overview](https://raw.githubusercontent.com/iterative/vscode-dvc/main/extension/docs/overview.gif)

- **Experiment tracking**: Record training data, parameters, and metrics on top
of [Git](https://git-scm.com/). Navigate your experiments, compare their
results, and find the best ML models.
- **Visualization**: Plot performance data in a customizable dashboard including
one or more overlaid experiments.
- **Live tracking**: Capture and see metrics changing in real time.
- **Reproducibility**: Make sure that anyone can recover or confirm previous
experiments, and run new experiments based on their results.
- **Data Management**: Handle and version large datasets, files, and models
effectively right from VS Code.

> **Note**: We are in public beta and welcome early adopter feedback!
> Feel free to reach out [via Discord] or open [issues in GitHub].
>
> [issues in github]: https://github.com/iterative/vscode-dvc/issues
> [via discord]: https://discord.gg/BGCjJHvDHt
### Why prefer this extension?

- Enjoy the best developer experience with the first experiment tracking
interface for an IDE.
- No external servers, databases, subscriptions, etc. Data stays fully under
your control and your existing Git hosting is used to share and collaborate.
- Go beyond a simple metrics dashboard with complete ML experiments that include
metrics, code, and data. Powered by [DVC experiment
versioning][dvc experiments].
- Implement [data versioning] on top of your favorite cloud storage, such as
Amazon S3, Azure Blob Storage, Google Cloud Storage, NFS, etc.

[dvc experiments]:
https://dvc.org/doc/user-guide/experiment-management/experiments-overview
[data versioning]: https://dvc.org/doc/use-cases/versioning-data-and-model-files

## Quick start

1. [Install DVC](https://dvc.org/doc/install) on your system.
2. Install [this extension] in VS Code.
3. Follow the **Get Started** page that pops up!

<!-- prettier-ignore -->
> 💡 Feel free to try our [example DVC project] first! Opening it [with Github
> Codespaces] will include this extension automatically.
![Initial Get Started page](https://raw.githubusercontent.com/iterative/vscode-dvc/main/extension/docs/walkthroughs.png)

[this extension]:
https://marketplace.visualstudio.com/items?itemName=Iterative.dvc
[example dvc project]: https://github.com/iterative/example-get-started
[with github codespaces]:
https://docs.github.com/en/codespaces/getting-started/quickstart#creating-your-codespace

### Learn more about DVC (Data Version Control)

See the DVC documentation to Get Started with [Experiment Versioning] or [Data
Management]. For deeper learning, try our [free course]!

[experiment versioning]: https://dvc.org/doc/start/experiments
[data management]: https://dvc.org/doc/start/data-management
[free course]: https://learn.iterative.ai/

<!-- [learn more]: extension/resources/walkthrough/dvc-learn-more.md -->

## What you get

This extension augments VS Code in the following ways:

- Adds a brand-new new **[DVC View]** (<img
src="https://raw.githubusercontent.com/iterative/vscode-dvc/main/extension/docs/dvc.png"
alt="DVC logo" height="16"/> icon in the Activity Bar) with panels to
visualize and manage [DVC Experiments].
- Provides special _Editors_ to manage **[Experiments]** and display **[Plots]**
in IDE _Tabs_.
- Adds a **[DVC Tracked]** panel to the _Explorer_ view. This shows a tree of
the data files and directories tracked by DVC, including their state, and
options to [synchronize] them (from/to remote storage).
- Adds a **[DVC panel]** to the _Source Control_ view to display the [workspace
status]. You can [restore] or reset project versions (based on the current Git
`HEAD` commit) as well as manage new and existing data from here.
- Registers several **Commands** in the _[Command Palette]_ (see next section).
- Includes a DVC channel for the _Output_ panel (useful for
[debugging](#debugging)).

[dvc view]:
https://github.com/iterative/vscode-dvc/blob/main/extension/resources/walkthrough/view-container.md
[experiments]:
https://github.com/iterative/vscode-dvc/blob/main/extension/resources/walkthrough/experiments-table.md
[plots]:
https://github.com/iterative/vscode-dvc/blob/main/extension/resources/walkthrough/plots.md
[command palette]:
https://code.visualstudio.com/docs/getstarted/userinterface#_command-palette
[dvc tracked]:
https://github.com/iterative/vscode-dvc/blob/main/extension/resources/walkthrough/dvc-tracked-files.md
[synchronize]: https://dvc.org/doc/start/data-management#storing-and-sharing
[dvc panel]:
https://github.com/iterative/vscode-dvc/blob/main/extension/resources/walkthrough/dvc-tracked-files.md
[workspace status]:
https://dvc.org/doc/command-reference/status#local-workspace-status
[restore]: https://dvc.org/doc/start/data-management#switching-between-versions

## Useful commands

Expand All @@ -40,16 +130,46 @@ in one of the following commands:
| `DVC: Show Experiments` | Show an interactive version of DVC's `exp show` [command](https://dvc.org/doc/command-reference/exp/show). |
| `DVC: Show Plots` | Show an interactive version of DVC's `plots diff` [command](https://dvc.org/doc/command-reference/plots/diff). |

## Features
Learn more about the extension's [commands].

[commands]:
https://github.com/iterative/vscode-dvc/blob/main/extension/resources/walkthrough/command-palette.md

## Configuration

These are the VS Code [settings] available for the Extension:

- [Command Palette](extension/resources/walkthrough/command-palette.md)
- [Source Control Management](extension/resources/walkthrough/source-control-management.md)
- [Tracked Resources](extension/resources/walkthrough/tracked-explorer.md)
- [DVC View Container](extension/resources/walkthrough/view-container.md)
- [Experiments Table](extension/resources/walkthrough/experiments-table.md)
- [Plots](extension/resources/walkthrough/plots.md)
[settings]: https://code.visualstudio.com/docs/getstarted/settings

### Debugging
| **Option** | **Description** |
| -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| `dvc.dvcPath` | Path or shell command to the DVC binary. Required unless Microsoft's [Python extension] is installed and the `dvc` package found in its environment. |
| `dvc.pythonPath` | Path to the desired Python interpreter to use with DVC. Required when using a virtual environment. |
| `dvc.doNotShowWalkthroughAfterInstall` | Do not prompt to show the Get Started page after installing. Useful for pre-configured development environments |
| `dvc.doNotRecommendRedHatExtension` | Do not prompt to install the Red Hat YAML extension, which helps with DVC YAML schema validation (`dvc.yaml` and `.dvc` files). |
| `dvc.doNotShowCliUnavailable` | Do not warn when the workspace contains a DVC project but the DVC binary is unavailable. |
| `dvc.doNotShowUnableToFilter` | Do not warn before disabling auto-apply filters when these would result in too many experiments being selected. |

> **Note** that the `Setup The Workspace` command helps you set up the basic
> ones at the [Workspace level] (saved to `.vscode/setting.json`).
[python extension]:
https://marketplace.visualstudio.com/items?itemName=ms-python.python
[workspace level]:
https://code.visualstudio.com/docs/getstarted/settings#_workspace-settings

## Debugging

### The DVC Extension

Please see the DVC [channel] in the IDE's _Output_ panel to see the underlying
DVC commands being run, as well as their error output. Feel free to share this
with us [via Discord] or use it to report [issues in GitHub].

[channel]:
https://code.visualstudio.com/api/extension-capabilities/common-capabilities#output-channel

### Your DVC project

Due to the way DVC pipelines run scripts of any language from the command line,
users must debug pipeline scripts (e.g. `train.py`) standalone in whatever way
Expand All @@ -58,12 +178,10 @@ pipelines, and most scripts are capable of running outside of DVC.

## Contributing

See development docs and contributing guidelines in
[CONTRIBUTING.md](CONTRIBUTING.md)

## Explore
See the development and contributing guidelines in
[CONTRIBUTING.md](https://github.com/iterative/vscode-dvc/blob/main/CONTRIBUTING.md).

View [more resources](extension/resources/walkthrough/dvc-learn-more.md).
![GitHub Contributors Image](https://contrib.rocks/image?repo=iterative/vscode-dvc)

## Data and telemetry

Expand Down
0 extension/resources/overview.gif → extension/docs/overview.gif
View file
File renamed without changes
Binary file added extension/docs/walkthroughs.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 2f6b3cb

Please sign in to comment.