diff --git a/content/docs/sidebar.json b/content/docs/sidebar.json index 254cc8c4b3..c41c8831ac 100644 --- a/content/docs/sidebar.json +++ b/content/docs/sidebar.json @@ -525,7 +525,11 @@ "label": "Create a View", "slug": "create-view" }, - "visualize-experiments", + "view-settings", + { + "label": "Visualize and Compare", + "slug": "visualize-experiments" + }, "run-experiments", "teams" ] diff --git a/content/docs/start/index.md b/content/docs/start/index.md index ecb46dceb8..9f1951c318 100644 --- a/content/docs/start/index.md +++ b/content/docs/start/index.md @@ -76,3 +76,8 @@ by one in the next few pages: comparison across many ML experiments. Track your experiments with automatic versioning and checkpoint logging. Compare differences in parameters, metrics, code, and data. Apply, drop, roll back, resume, or share any experiment. + +**New!** Once you set up your DVC repository, you can also interact with it +using DVC Studio, the online UI for DVC. +[Here's a demo](https://studio.iterative.ai/team/Iterative/views/example-get-started-zde16i6c4g) +of how that looks like! diff --git a/content/docs/studio/create-view.md b/content/docs/studio/create-view.md index 93de45d2cc..f136115032 100644 --- a/content/docs/studio/create-view.md +++ b/content/docs/studio/create-view.md @@ -1,67 +1,70 @@ # Create a View -To create a new view for your repository, follow these steps. +You can create views from your Git repositories, just like the `Demo` view you +saw in the last section. To create a new view, follow these steps. -1. Sign into your [DVC Studio](https://studio.iterative.ai/) dashboard. - ![](https://static.iterative.ai/img/studio/login_home.png) +1. Sign in to your [DVC Studio](https://studio.iterative.ai/) dashboard using + GitHub, GitLab or Bitbucket. 2. Click on `Add a View`. All the organizations that you have access to will be - listed. ![](https://static.iterative.ai/img/studio/create_view.png) + listed. + +> To create views from your GitHub repositories, you must install the DVC Studio +> GitHub app. Refer to the section on +> [GitHub app installation](#dvc-studio-github-app-installation) for more +> details. 3. Open the organization whose repository you want to connect to. You can also - use the search bar to search directly for the repo which you want to connect - to. ![](https://static.iterative.ai/img/studio/select_repo.png) + use the search bar to directly look for a repository. -4. Select the Git repository for which you want to create a view. For this, - hover over the required repository and click `Connect`. + ![](https://static.iterative.ai/img/studio/select_repo_v2.png) -5. Once you click on `Connect`, you will be able to specify advanced settings - for the connection. - ![](https://static.iterative.ai/img/studio/view_settings.png) +4. Specify additional connection settings if required. -> In most scenarios, you can skip the advanced settings. Refer to the -> [Advanced Settings](#advanced-settings) section below for more details. + ![](https://static.iterative.ai/img/studio/view_settings_v2.png) -You should now see that a view has been added in your dashboard. +> In most scenarios, you can skip these additional View settings. Refer to the +> [View Settings](/doc/studio/view-settings) section for more details. -![](https://static.iterative.ai/img/studio/view_added.png) +You should now see that a View has been added in your dashboard. If your project requires any of the advanced settings, then remember to configure them by opening the View settings. Otherwise, your view may not work as expected. To go to View settings, click on the -![](https://static.iterative.ai/img/studio/view_open_settings_icon.png) icon in -the view. In the menu that opens up (highlighted in yellow in the screenshot -below), click on `Settings`. +![](https://static.iterative.ai/img/studio/view_open_settings_icon_v2.png) icon +in the view. In the menu that opens up, click on `Settings`. -![](https://static.iterative.ai/img/studio/view_open_settings.png) +![](https://static.iterative.ai/img/studio/view_open_settings_v2.png) -## Advanced Settings +## Share a view -You will need to specify advanced settings in the following scenarios. +You can share your views on the web. Simply click on the button labelled +`Private` next to the name of the view. In the menu that pops up, turn on +`Share to Web`. -> These settings can be skipped when creating the view -- they can be edited -> later. +![](https://static.iterative.ai/img/studio/view_share_v2.png) -- **Monorepo:** If you have connected to a - [monorepo](https://en.wikipedia.org/wiki/Monorepo), then specify the full path - to the root directory of the project that you want to connect. +You can change a shared (public) view and make it private by turning off +`Share to web`. -- **Data remotes:** The metrics and parameters that you want to include in the - view may be present in a data remote (cloud storage or another - location outside of the Git repo). If you want to include such data in your - views, then you will have to grant DVC Studio access to the data remote. +## DVC Studio GitHub app installation -- **Custom metrics and parameters:** DVC Studio automatically detects metrics, - plots, and parameters files specified in the project's `dvc.yaml`. If you want - to connect custom files, you can add them by clicking the `Add file` button. - Enter the full file path, and specify whether the file is for `Metrics` or - `Parameters`. +If you are using a GitHub account, you will have to install the DVC Studio +GitHub app on the GitHub repositories/organizations that you want to use with +DVC Studio. When you try to create a view, if the app is not already installed, +DVC Studio will not be able to fetch the list of your GitHub repositories. In +this case, you will be prompted to configure Git integrations as shown below. -## Share a view +![](https://static.iterative.ai/img/studio/configure_git_integrations.png) -You can share your views on the web. Simply click on the button labelled -`Private` next to the name of the view. In the menu that pops up, turn on -`Share to Web`. ![](https://static.iterative.ai/img/studio/view_share.png) +Click on the link to `Configure Git integrations settings`. This will open the +`Git integrations` section of your profile page. -You can change a shared (public) view and make it private by turning off -`Share to web`. +![](https://static.iterative.ai/img/studio/configure_github.png) + +Click on the `Configure` button, and you will be redirected to the GitHub page +to install and authorize the DVC Studio GitHub app. + +> Note that you do not have to authorize DVC Studio on all the repositories in +> your GitHub organization. You can limit access to only repositories that you +> want to use with DVC Studio. diff --git a/content/docs/studio/get-started.md b/content/docs/studio/get-started.md index 3057f3ee6f..0bb550b335 100644 --- a/content/docs/studio/get-started.md +++ b/content/docs/studio/get-started.md @@ -1,31 +1,38 @@ # Get Started with DVC Studio Here, we will walk you through a tutorial to use DVC Studio for collaboration on -your ML projects. You will need access to a Github, Gitlab or Bitbucket account +your ML projects. You will need access to a GitHub, GitLab or Bitbucket account which has access to the Git repositories you want to connect. DVC Studio creates -views from repositories when you connect to them. +[views](#what-is-a-view) from repositories when you connect to them. + +The following video provides you a quick overview of DVC Studio. + +https://www.youtube.com/watch?v=hKf4twg832g ## What is a view? A _view_ is an interactive representation of the experiments run in your ML -project. DVC Studio automatically identifies datasets, metrics and -hyperparameters in your ML experiments. You can also add custom files with the -required data (more on this later). Using this data, DVC Studio creates a -summary view. This includes tables of all the metrics across all your -experiments. You can also generate plots and compare experiments here. +project. DVC Studio identifies datasets, metrics and hyperparameters in your ML +experiments. These values can either be in DVC repositories or you can add +custom files with the required data. Refer to +[View settings](/doc/studio/view-settings) to understand the different ways in +which you can prepare your Git repository for use with DVC Studio. Using these +values, DVC Studio creates a View, which is a tabular presentation of all your +experiments along with their datasets, metrics ad hyperparameters. You can also +generate plots and compare experiments here. ## DVC Studio Views page In your browser, open . Sign in with your Github, -Gitlab, or Bitbucket account. +GitLab, or Bitbucket account. -![](https://static.iterative.ai/img/studio/login_home.png) _When you first +![](https://static.iterative.ai/img/studio/login_home_v2.png) _When you first login, an example view is already created for you to explore, and you can add more views._ -When you first login, you will find that there already exists a view connecting -to an example DVC project. Use this view to explore the features that DVC Studio -has to offer. +When you first login, you will find that there already exists a `Demo` view +connecting to an example DVC project. Use this view to explore the features that +DVC Studio has to offer. DVC Studio automatically identifies datasets, metrics and hyperparameters in your ML experiments. Each view on the dashboard displays the metrics. In the @@ -37,10 +44,10 @@ You can dive deep into all the experiments committed to the repo. For this, open the view by clicking the view name (in this case, `example-get-started`). A table will be generated as shown below. This includes metrics, hyperparameters -and information about the datasets. All the data is flattened and neatly +and information about the datasets. All these values are flattened and neatly presented for you to evaluate and compare the experiments. -![](https://static.iterative.ai/img/studio/view_components.png) +![](https://static.iterative.ai/img/studio/view_components_v2.png) This tabular display has the following components: diff --git a/content/docs/studio/index.md b/content/docs/studio/index.md index b3e63b51c4..ef7d96316c 100644 --- a/content/docs/studio/index.md +++ b/content/docs/studio/index.md @@ -1,7 +1,9 @@ # DVC Studio -[`DVC Studio`](https://studio.iterative.ai/) is a comprehensive and interactive -collaboration tool for your Machine Learning projects. +[`DVC Studio`](https://studio.iterative.ai/) is a web application that you can +[access online](https://studio.iterative.ai/) or even host on-prem. It works +with the data, metrics and hyperparameters that you add to your ML project +repositories. Using the power of leading open-source tools DVC, CML and Git, it enables you to seamlessly manage data and models, run and track experiments, and visualize and @@ -9,17 +11,17 @@ share results. Project website: https://studio.iterative.ai -![](https://static.iterative.ai/img/studio/main.png) _DVC Studio experiments +![](https://static.iterative.ai/img/studio/main_v2.png) _DVC Studio experiments dashboard_ Use DVC, CML and Studio to: -- **Visualize, collaborate, and do everything that a regular ML tracking tool - does.** +- Visualize, collaborate, and do everything that a regular ML tracking tool + does. -- **Keep your code, data and model connected at all times.** +- Keep your code, data and model connected at all times. -- **Use the power of Git to track and preserve all your experiments.** +- Use the power of Git to track and preserve all your experiments. -- **Run [CI/CD](https://en.wikipedia.org/wiki/CI/CD) for your ML projects on - cloud resources of your choice without any new tools.** +- Run experiments for your ML projects on cloud resources of your choice without + any new tools. diff --git a/content/docs/studio/run-experiments.md b/content/docs/studio/run-experiments.md index ee60c25a2e..eb778e436f 100644 --- a/content/docs/studio/run-experiments.md +++ b/content/docs/studio/run-experiments.md @@ -5,24 +5,59 @@ CI/CD set-up is used to run the experiments. For instance, [CML](https://dvc.org/doc/cml) Github Actions can be used to run ML experiments on each new commit. -> Note that you cannot run experiments on the demo view (`example-get-started`) -> that is provided to you by default. Once you create views for your ML project -> repositories, you can follow the instructions given below and run experiments -> directly from DVC Studio. +> Note that due to access restrictions, you cannot run experiments on the demo +> view (`example-get-started`) that is provided to you by default. Once you +> create views for your ML project repositories, you can follow the instructions +> given below to run experiments directly from DVC Studio. + +Watch this video for an overview of how you can run experiments from DVC Studio, +or read below for details. + +https://www.youtube.com/watch?v=nXJXR-zBvHQ To run experiments from DVC Studio, select the commit that you want to use and -click the `Run` button. A form will let you to specify all the changes that you -want to make to your experiment input files/dirs and parameters. +click the `Run` button. A form will let you specify all the changes that you +want to make to your experiment. On this form, there are 2 types of inputs that +you can change: + +1. **Input data files**: You can change datasets as well as metrics and model + files. The list of files that you can change will depend on your ML project. + For instance, in the `example-get-started` project, you can change the + `data.xml` file. If you select the + `Show all input parameters (including hidden)` option, then you can also + change the hidden files such as the `model.pkl` model file and the + `scores.json` metrics file. You can also choose not to change any input data + files if you only wish to change the values of one or more hyperparameters. +2. **Hyperparameters**: You can change the values of the hyperparameters that + are defined in your ML project. For instance, in the `example-get-started` + project, you can change `max_features` (the maximum number of features that + the model uses), `ngrams`, etc. You can also choose not to change any + hyperparameters if you only wish to change one or more input data files. + +The default values of the input data files and hyperparameters in this form are +extracted from your selected commit. + +![](https://static.iterative.ai/img/studio/cml_changes_v2.png) + +Once you have made all the required changes, enter your Git commit message and +description. Then, select the branch to commit to. You can commit to either the +base branch or a new branch. If you commit to a new branch, a Git pull request +will automatically be created from the new branch to the base branch. Now, click +on `Commit changes`. + +![](https://static.iterative.ai/img/studio/cml_commit_v2.png) -drawing +At this point, the new experiment appears in the view table. If you just +committed to a new branch, then a new pull request will also have been created +from the new branch to the base branch. -The default values in this form are extracted from your selected commit. For -instance, in the given example, `max_features` can be increased to 4000, -`ngrams` can be changed to 2, and so on. Enter your commit message and -description, select the branch to commit to (either the base branch or a new -branch), and click on `Commit changes`. +If your project is integrated with a CI/CD setup (e.g. GitHub Actions), the +CI/CD setup will get invoked. If this setup includes a model training process, +it will be triggered, which means that your ML experiment will run +automatically. The model training can happen on any cloud or Kubernetes. For +more details on how to set up CI/CD pipelines for your ML project, refer to +[CML](https://cml.dev). -Now, the CI/CD pipeline that you have set up (eg, Github Actions) will be -invoked to run the experiment. If you refresh the view (reload the page), the -experiment (i.e. the commit that you just pushed) along with its results -(metrics) will be included in the list of all commits. +Once the experiment completes, its metrics will be available in the view table. +You can then generate plots and trend charts for it, or compare it with the +other experiments. diff --git a/content/docs/studio/teams.md b/content/docs/studio/teams.md index 4b18b28af1..c54caa2a6c 100644 --- a/content/docs/studio/teams.md +++ b/content/docs/studio/teams.md @@ -6,31 +6,16 @@ define teams with one or more team members. The team members are also called collaborators, and you can assign different roles to them. The views that you create in your team's page will be accessible to all members of the team. -## Roles - -Team members can have different roles. - -- **View.** Users with the View access cannot edit team settings or create new - views within the team's page. They have read-only access to the views created - by other team members. -- **Edit.** In addition to accessing all the team's views, users with the Edit - role can also create new views for the team, edit the views' settings, and run - experiments. -- **Admin.** Admin users have full access to the team's views and settings. They - can do everything that viewers and editors can do. Additionally, they can add - (invite) and remove collaborators as well as change team settings such as - cloud credentials (data remotes). - ## Create a team To create a team, click on the drop down next to `Personal`. All the teams that you have created so far will be listed within "Teams" in the drop down menu. If you have not created any team so far, this list will be empty. Now, click on -`Create a team`. ![](https://static.iterative.ai/img/studio/team_create.png) +`Create a team`. ![](https://static.iterative.ai/img/studio/team_create_v2.png) You will be asked to enter the URL namespace for your team. Enter a unique name. The URL for your team will be formed using this name. -![](https://static.iterative.ai/img/studio/team_enter_name.png) +![](https://static.iterative.ai/img/studio/team_enter_name_v2.png) Then, click the `Create team` button on the top right corner. @@ -43,19 +28,35 @@ collaborators by accessing team settings later. If you wish to add collaborators now, enter their email addresses. An email invite will be sent to each invitee, and they will have to join using their -Github, Gitlab or Bitbucket account. +GitHub, GitLab or Bitbucket account. You can add multiple collaborators. Each collaborator can be assigned the Admin, -Edit, or View role. ![](https://static.iterative.ai/img/studio/team_roles.png) +Edit, or View role. Refer to the [Roles](#roles) section below for more details +about the roles. ![](https://static.iterative.ai/img/studio/team_roles_v2.png) Once you have added the people that you wish to add to your team, click on `Send Invites and Close` on the top right corner. +## Roles + +Team members can have different roles. + +- **View.** Users with the View access cannot edit team settings or create new + views within the team's page. They have read-only access to the views created + by other team members. +- **Edit.** In addition to accessing all the team's views, users with the Edit + role can also create new views for the team, edit the views' settings, and run + experiments. +- **Admin.** Admin users have full access to the team's views and settings. They + can do everything that viewers and editors can do. Additionally, they can add + (invite) and remove collaborators as well as change team settings such as + cloud credentials (data remotes). + ## Manage your team and its views Once you have created the team, the team's page opens up. -![](https://static.iterative.ai/img/studio/team_page.png) +![](https://static.iterative.ai/img/studio/team_page_v2.png) On this page, you can perform three types of tasks: @@ -69,4 +70,6 @@ On this page, you can perform three types of tasks: - **Change settings.** Finally, you can click on the `Settings` menu item to change the team name, add credentials for the data remotes, and delete the - team. ![](https://static.iterative.ai/img/studio/team_settings.png) + team. + + ![](https://static.iterative.ai/img/studio/team_settings_v2.png) diff --git a/content/docs/studio/view-settings.md b/content/docs/studio/view-settings.md new file mode 100644 index 0000000000..e73f0a44e8 --- /dev/null +++ b/content/docs/studio/view-settings.md @@ -0,0 +1,111 @@ +# Additional Settings for a View + +If you are creating a view for a DVC repo, and if the DVC repo is at the root of +the Git repository and does not reference remote/cloud storage, then you can +successfully visualize it without specifying additional settings. + +Alternatively, you could create views from: + +- Non-DVC repositories +- Project sub-directories in a monorepo +- Custom files in your repository or remote/cloud storage + +In each of these scenarios, you will need to specify advanced settings for DVC +Studio to be able to access the data required for visualization. Details are +given below. + +## Non-DVC repositories + +DVC Studio creates views by identifying datasets, metrics and hyperparameters +defined in your Git repositories. These details (datasets, metrics and +hyperparameters) are stored in your Git repositories as CSV, JSON or YAML files. +You can add these details to your Git repositories in two ways: + +1. **Set up DVC repositories**: You can use [DVC](https://dvc.org/) and Git to + version your code, data and models all within your Git repositories. By using + DVC, you can be sure not to bloat your repositories with large volumes of + data or huge models. These large assets reside in the cloud or other remote + storage locations. You will simply track their version info in Git. DVC also + enables you to + [share your data and model files](/doc/use-cases/sharing-data-and-model-files), + [create data registries](/doc/use-cases/data-registries), + [create data pipelines](/doc/start/data-pipelines), connect them with + [CML](/doc/cml) for CI/CD in machine learning, and so on. Find more about the + features and benefits of DVC [here](/doc/start). + + Refer to the [DVC documentation](https://dvc.org/doc) to initialize a DVC + repository. You can then connect to this DVC repository and create a view as + described [earlier](/doc/studio/create-view). DVC Studio automatically + detects metrics, plots, and hyperparameters files specified in the project's + `dvc.yaml`. Each time you push a commit to this DVC repository, your view + will reflect the new changes. + +2. **Specify custom files with your metrics and parameters**: If you are working + with a non-DVC repository, you can still create views for it provided that + metrics and hyperparameters are stored in CSV, JSON or YAML files. To + visualize such custom data, simply + [specify the custom files](#specifying-view-settings) to use, and DVC Studio + will efficiently generate tables and plots for your custom input. For + instance, if you have an ML project for which you generate and save metrics + either manually or using some ML tracking tools, then you can create a view + for this project by specifying the file (within your Git repo) which contains + your saved metrics. + + So as you can see, DVC Studio simply requires your metrics and + hyperparameters to be available in data files in your Git repositories. This + video further illustrates this concept. + + https://www.youtube.com/watch?v=5xM5az78Lrg + +## Monorepo + +Depending on how you have set up your Git repositories, your DVC repo (for which +you are trying to create the view) may not be in the root of your Git repo. +Instead, it could be in a sub-directory of a +[monorepo](https://en.wikipedia.org/wiki/Monorepo). If this is the case, you +will need to specify the full path to the sub-directory that you want to use +with your view. + +## Data remotes (cloud/remote storage) + +The metrics and parameters that you want to include in the view may also be +present in a data remote (cloud storage or another location outside the Git +repo). If you want to include such data in your views, then you will have to +grant DVC Studio access to the data remote. + +# Specifying View settings + +For any of the scenarios defined above, specify the additional settings as +described below. You can access these settings at any time after creating the +view. For this, click on the +![](https://static.iterative.ai/img/studio/view_open_settings_icon_v2.png) icon +in the view. In the menu that opens up, click on `Settings`. + +- **Custom metrics and parameters:** If you want to connect custom files, you + can add them by clicking the `Add file` button. Enter the full file path, and + specify whether the file is for `Metrics` or `Parameters`. + +- **Monorepo:** If you have connected to a + [monorepo](https://en.wikipedia.org/wiki/Monorepo), then specify the full path + to the sub-directory that contains the DVC repo for which you want to create + the view. + +- **Data remotes:** If you need to set up DVC data remotes for your view, you + will need to do it after your view has been created. First, create your view + without specifying the data remotes. Once your view is created, open the View + settings. Open the `Data remotes / cloud storage credentials` section. The + data remotes that are used in your DVC repo will be listed. Now, click on + `Add new credentials`. In the form that opens up, select the provider (Amazon + S3, GCP, etc.). Depending on the provider, you will be asked for more details + such as the credentials name, username, password etc. + + ![](https://static.iterative.ai/img/studio/s3_remote_settings.png) + + For details on what permissions are required, refer to the DVC documentation + on + [supported storage types](/doc/command-reference/remote/add#supported-storage-types). + + Note that DVC Studio uses the credentials only to read plots/metrics files if + they are not saved into Git. It does not access any other data in your remote + storage. And you do not need to provide the credentials if any DVC data remote + in not used in your Git repository. diff --git a/content/docs/studio/visualize-experiments.md b/content/docs/studio/visualize-experiments.md index b3f2dc87bd..cb978af573 100644 --- a/content/docs/studio/visualize-experiments.md +++ b/content/docs/studio/visualize-experiments.md @@ -15,21 +15,19 @@ are generated. To generate metric plots, select one or more experiments (represented by the commits), and click on the 'Show plots' button. -drawing - The plots will appear below the tabular display. If you have selected more than one experiment, you can use the plots to compare them. -drawing +![](https://static.iterative.ai/img/studio/plots_v2.png) ## Generate trend charts -Click on the Trends button to generate a plot of how the metrics changed over +Click on the `Trends` button to generate a plot of how the metrics changed over the course of the different experiments. For each metric, the trend charts show how the metric changed from one commit to another. You can include one or more branches in the trend chart. -drawing +![](https://static.iterative.ai/img/studio/trends_v2.png) ## Compare experiments @@ -37,4 +35,4 @@ To compare different experiments, select two experiments (represented by the commits), and click on the `Compare` button. The metrics, parameters and files in the selected experiments will be displayed side by side for easy comparison. -drawing +![](https://static.iterative.ai/img/studio/compare_v2.png) diff --git a/src/components/LayoutHeader/alert.tsx b/src/components/LayoutHeader/alert.tsx index 69ac432ee6..57e3e2bdab 100644 --- a/src/components/LayoutHeader/alert.tsx +++ b/src/components/LayoutHeader/alert.tsx @@ -7,11 +7,11 @@ import styles from './styles.module.css' const LayoutAlert: React.FC<{ collapsed: boolean }> = ({ collapsed }) => (
- - 🛡 + + {' '} - DVC for Enterprise - data access control & - compliance!{' '} + DVC Studio, the online UI + for DVC, is live!{' '}
)