Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd: add parameters for configuration of WebDAV remotes #1617

Merged
merged 22 commits into from
Aug 6, 2020
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
a00dd30
cmd: add parameters for configuration of WebDAV remotes
Jul 22, 2020
9864d50
Update content/docs/command-reference/remote/modify.md
jorgeorpinel Jul 23, 2020
6d7eeeb
cmd: add token authentication example to remote modify for WebDAV
Jul 23, 2020
127d00e
Update content/docs/command-reference/remote/modify.md
jorgeorpinel Aug 5, 2020
feb10a0
Update content/docs/command-reference/remote/modify.md
jorgeorpinel Aug 5, 2020
49e894d
Update content/docs/command-reference/remote/modify.md
jorgeorpinel Aug 5, 2020
4fa3d8b
Update content/docs/command-reference/remote/modify.md
jorgeorpinel Aug 5, 2020
8f0836c
Update content/docs/command-reference/remote/modify.md
iksnagreb Aug 5, 2020
b449189
Update modify.md
iksnagreb Aug 5, 2020
84b76dc
Update content/docs/command-reference/remote/modify.md
jorgeorpinel Aug 6, 2020
e70cd5b
Update content/docs/command-reference/remote/add.md
iksnagreb Aug 6, 2020
b76cec5
cmd: add explanation of WebDAV API location to 'remote add' command
Aug 6, 2020
b413b0e
cmd: clarify user/password, token and ask_password relation for WebDAV
iksnagreb Aug 6, 2020
2f72006
cmd: add ' quotes to token as it might contain special characters
iksnagreb Aug 6, 2020
d870b34
cmd: move 'user' order explanation after the example as suggested
iksnagreb Aug 6, 2020
31bccb5
cmd: add more explanation to WebDAV certificates parameters
iksnagreb Aug 6, 2020
098fe82
cmd: add 'url' parameter explanations to 'remote' modify for WebDAV
iksnagreb Aug 6, 2020
5eca211
Update content/docs/command-reference/remote/modify.md
jorgeorpinel Aug 6, 2020
5d3b9ec
Update content/docs/command-reference/remote/modify.md
jorgeorpinel Aug 6, 2020
99c7b8d
Update content/docs/command-reference/remote/modify.md
jorgeorpinel Aug 6, 2020
0dbbb27
Update content/docs/command-reference/remote/modify.md
jorgeorpinel Aug 6, 2020
08f888f
Update content/docs/command-reference/remote/modify.md
jorgeorpinel Aug 6, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions content/docs/command-reference/remote/add.md
Original file line number Diff line number Diff line change
Expand Up @@ -344,6 +344,18 @@ $ dvc remote add -d myremote https://example.com/path/to/dir

<details>

### Click for WebDAV

```dvc
$ dvc remote add -d myremote webdavs://example.com/path/to/dir
iksnagreb marked this conversation as resolved.
Show resolved Hide resolved
```

> See also `dvc remote modify` for a full list of WebDAV parameters.

</details>

<details>

### Click for local remote

A "local remote" is a directory in the machine's file system.
Expand Down
63 changes: 63 additions & 0 deletions content/docs/command-reference/remote/modify.md
Original file line number Diff line number Diff line change
Expand Up @@ -582,6 +582,69 @@ more information.

</details>

<details>

### Click for WebDAV

- `token` - token for WebDAV server, can be empty in case of using
`user/password` authentication.
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved

```dvc
$ dvc remote modify --local myremote token mytoken
Copy link
Contributor

@jorgeorpinel jorgeorpinel Aug 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it need quotes like other string params? I.e.:

Suggested change
$ dvc remote modify --local myremote token mytoken
$ dvc remote modify --local myremote token "<mytoken>"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know, thought it to be more like a variant of user, which has no quotes? But adding quotes should probably be fine. Are there guidelines what parameters are considered as strings/numerics/... and what formatting to apply to each?

Copy link
Contributor

@jorgeorpinel jorgeorpinel Aug 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's more of a command line parser thing. All strings can be wrapped with " but if they have special chars or spaces then you definitely need to wrap them. And in fact ' may be better... Idk. I'll just commit this!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, have you tried this? I assume you tested all the WebDAV params so you can probably tell or double check whether quotes are needed.

Copy link
Contributor

@jorgeorpinel jorgeorpinel Aug 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or if you can rec a quick WebDAV server software to install and run locally for testing, please share 🙂

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I think you are right, it is better to wrap this in '. There seems to be no standard defining the content and structure of tokens for WebDAV (at least it should be some kind of string). What I can tell for the service I am using (it is basically an onwcloud): it supports OAuth 2.0, probably bearer tokens, which might already contain some basic punctuation characters.

Copy link
Contributor

@jorgeorpinel jorgeorpinel Aug 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the tip. Resolving Leaving unresolved for future ref.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or if you can rec a quick WebDAV server software to install and run locally for testing, please share slightly_smiling_face

Check this link: https://doc.owncloud.com/server/admin_manual/installation/docker/#docker-compose-yaml-file

Copy-paste that into a docker-compose.yml file and then docker-compose up.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, thanks @skshetry! Will try this next time its needed.

```

- `user` - username for WebDAV server, can be empty in case of using `token`
authentication. The order in which DVC searches for username is:

1. `user` specified in one of the DVC configs;
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
2. `user` specified in the url (e.g. `webdav://[email protected]/path`)

```dvc
$ dvc remote modify --local myremote user myuser
```

- `password` - password for WebDAV server, can be empty in case of using `token`
authentication.

```dvc
$ dvc remote modify --local myremote password mypassword
```

> The username, password, and token (may) contain sensitive user info.
> Therefore, it's safer to add them with the `--local` option, so they're
> written to a Git-ignored config file.

- `ask_password` - ask each time for the password to use for `user/password`
authentication.
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved

```dvc
$ dvc remote modify myremote ask_password true
```

> Note that the `password` parameter takes precedence over `ask_password`. If
> `password` is specified, DVC will not prompt the user to enter a password
> for this remote.

- `cert_path` - path to certificate used for WebDAV server authentication.

```dvc
$ dvc remote modify myremote cert_path /path/to/cert
```

- `key_path` - path to private key to use to access a remote.

```dvc
$ dvc remote modify myremote key_path /path/to/key
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these alternatives to token and to user/password auth?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, according to the WebDAV client documentation (and that is also what I can infer from the implementation), these are additional parameters. But they are optional parameters, probably we should give this information.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to say they're optional. Most parameters except what's shown in the remote add reference are assumed to be optional. But ideally we should briefly explain what these are for, or link to some WebDAV documentation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added some explanation, is it enough? I don't know what I could link to, these are not really WebDAV specific options - they are just passed to the underlying requests API.

WebDAV seems to be the only remote having such options, even though HTTP is also based on requests it does not expose these option?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that was more than enough. Thanks @iksnagreb!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WebDAV seems to be the only remote having such options, even though HTTP is also based on requests it does not expose these option?

Good question. TBH Idk but maybe @efiop, who I always pick on for these questions 😬, does.


- `timeout` - connection timeout for WebDAV server.
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved

```dvc
$ dvc remote modify myremote timeout timeoutseconds
iksnagreb marked this conversation as resolved.
Show resolved Hide resolved
```

</details>

## Example: Customize an S3 remote

Let's first set up a _default_ S3 remote.
Expand Down