diff --git a/README.rst b/README.rst index 1738ee6..b50aee2 100644 --- a/README.rst +++ b/README.rst @@ -65,7 +65,7 @@ Here are a few things you might do with ``vault-cli``: ohsosecret $ # Load a secret into the environment variables: - $ vault-cli env --path mysecret -- env | grep MYSECRET + $ vault-cli env --envvar mysecret -- env | grep MYSECRET MYSECRET_MYKEY=ohsosecret $ # Load an ssh key into your ssh-agent: diff --git a/docs/howto/environment.rst b/docs/howto/environment.rst index 7f3cefa..87cf9cf 100644 --- a/docs/howto/environment.rst +++ b/docs/howto/environment.rst @@ -16,7 +16,7 @@ works: $ vault-cli set test/my_secret value=qwerty Done - $ vault-cli env --path test/my_secret -- bash -c 'echo $MY_SECRET_VALUE' + $ vault-cli env --envvar test/my_secret -- bash -c 'echo $MY_SECRET_VALUE' qwerty Environment variable naming @@ -68,29 +68,29 @@ Let's consider the vault contains only the following secret: This table maps input to output. Note that there will always be a single environment variable and its value will always be ``mysecret``. -+-------------+-----------------------+---------------------------+ -| ``--path`` | ``--omit-single-key`` | environment variable name | -+-------------+-----------------------+---------------------------+ -| ``a`` | False | ``A_B_C`` | -+-------------+-----------------------+---------------------------+ -| ``a`` | True | ``A_B`` | -+-------------+-----------------------+---------------------------+ -| ``a=D`` | False | ``D_B_C`` | -+-------------+-----------------------+---------------------------+ -| ``a=D`` | True | ``D_B`` | -+-------------+-----------------------+---------------------------+ -| ``a/b`` | False | ``B_C`` | -+-------------+-----------------------+---------------------------+ -| ``a/b`` | True | ``B`` | -+-------------+-----------------------+---------------------------+ -| ``a/b=D`` | False | ``D_C`` | -+-------------+-----------------------+---------------------------+ -| ``a/b=D`` | True | ``D`` | -+-------------+-----------------------+---------------------------+ -| ``a/b:c`` | True or False | ``C`` | -+-------------+-----------------------+---------------------------+ -| ``a/b:c=D`` | True or False | ``D`` | -+-------------+-----------------------+---------------------------+ ++---------------+-----------------------+---------------------------+ +| ``--envvar`` | ``--omit-single-key`` | environment variable name | ++---------------+-----------------------+---------------------------+ +| ``a`` | False | ``A_B_C`` | ++---------------+-----------------------+---------------------------+ +| ``a`` | True | ``A_B`` | ++---------------+-----------------------+---------------------------+ +| ``a=D`` | False | ``D_B_C`` | ++---------------+-----------------------+---------------------------+ +| ``a=D`` | True | ``D_B`` | ++---------------+-----------------------+---------------------------+ +| ``a/b`` | False | ``B_C`` | ++---------------+-----------------------+---------------------------+ +| ``a/b`` | True | ``B`` | ++---------------+-----------------------+---------------------------+ +| ``a/b=D`` | False | ``D_C`` | ++---------------+-----------------------+---------------------------+ +| ``a/b=D`` | True | ``D`` | ++---------------+-----------------------+---------------------------+ +| ``a/b:c`` | True or False | ``C`` | ++---------------+-----------------------+---------------------------+ +| ``a/b:c=D`` | True or False | ``D`` | ++---------------+-----------------------+---------------------------+ Recommended setup ----------------- @@ -112,7 +112,7 @@ Your call would look like: .. code:: console - $ vault-cli env --omit-single-key --path myapp -- myapp + $ vault-cli env --omit-single-key --envvar myapp -- myapp Ignoring errors --------------- @@ -123,7 +123,7 @@ even if it will be missing some secrets. .. code:: console - $ vault-cli env --path myapp --force -- myapp + $ vault-cli env --envvar myapp --force -- myapp .. warning:: diff --git a/docs/howto/read.rst b/docs/howto/read.rst index 67bc231..821169b 100644 --- a/docs/howto/read.rst +++ b/docs/howto/read.rst @@ -74,6 +74,11 @@ can write the secret to a specific file: with ways to write on ephemeral storage, and check your umask__ and the permissions of the created file. See :ref:`SystemD` for safe integration strategies. +.. note:: + + ``vault-cli env`` also lets you to write secrets to a file just before launching + an arbitrary command. + .. __: https://en.wikipedia.org/wiki/Umask diff --git a/docs/howto/ssh.rst b/docs/howto/ssh.rst index c85b236..ffe2982 100644 --- a/docs/howto/ssh.rst +++ b/docs/howto/ssh.rst @@ -38,5 +38,5 @@ If you need to have both ssh access and secrets as environment variables (see $ # If your key is not passphrase-protected $ vault-cli ssh --key path/to/ssh_private_key:value \ - -- vault-cli env --path myapp \ + -- vault-cli env --envvar myapp \ -- myapp_that_needs_secrets_and_ssh diff --git a/docs/howto/systemd.rst b/docs/howto/systemd.rst index a4da671..0142a58 100644 --- a/docs/howto/systemd.rst +++ b/docs/howto/systemd.rst @@ -58,7 +58,7 @@ launch the program through ``vault-cli env``. Let’s launch it as a one-off: .. code:: console - $ vault-cli env --path mysecret:value -- myprogram + $ vault-cli env --envvar mysecret:value -- myprogram This will make a variable named ``VALUE`` available to ``myprogram``. See the :ref:`vault-cli env ` dedicated page for more details on how you can @@ -85,7 +85,7 @@ We’ll create an override file that will change ExecStart to wrap it in # opens a new file for edition [Service] ExecStart= - ExecStart=vault-cli env --path mysecret:value=MYVAR -- myprogram --options + ExecStart=vault-cli env --envvar mysecret:value=MYVAR -- myprogram --options The empty ``ExecStart=`` tells SystemD to ignore the previous command to launch and only launch the following one. @@ -97,8 +97,8 @@ Save and quit the file. Load you new configuration file with: $ sudo systemctl daemon-reload $ sudo systemctl restart myprogram.service -Writing a single secret to a file before start ----------------------------------------------- +Writing secrets to files on the filesystem before start +------------------------------------------------------- In some cases, you will need to have a file in the filesystem that contains directly the secret. This is often the case with private keys. @@ -110,8 +110,8 @@ be written on disk. .. __: https://en.wikipedia.org/wiki/RAM_drive -In this case, we’ll also create a service override file, but this time, -we will be adding a command that launches before our main command: +In this case, we’ll also create a service override file. We'll add a wrapper +arount our program like before. .. code:: console @@ -119,7 +119,7 @@ we will be adding a command that launches before our main command: # opens a new file for edition [Service] TemporaryFileSystem=/private - ExecStartPre=vault-cli get mysecret --output=/private/path/to/secret/file + ExecStart=vault-cli env --file mysecret:key=/private/path/to/secret/file -- myprogram --options Save and quit the file. Load your new configuration file with: @@ -131,7 +131,7 @@ Save and quit the file. Load your new configuration file with: You will need to configure ``myprogram`` to look for your secret file at ``/private/path/to/secret/file``. -If you need several files, you can repeat the ``ExecStartPre`` line as +If you need several files, you can add more ``--file`` flags, as many times as needed. .. note:: @@ -143,6 +143,15 @@ many times as needed. Bake secrets into a complex configuration file ---------------------------------------------- +.. warning:: + + It's been reported__ that this approach doesn't work as intended. It's left + for inspiration, but as of today, ``ExecStartPre`` cannot write to the + private filesystem created by ``TemporaryFileSystem`` in way that ``ExecStart`` + can later read. Please refer to the ticket for workarounds. + + .. __: https://github.com/peopledoc/vault-cli/issues/185 + In some cases, the program you want to launch doesn’t accept configuration through environment but only through configuration files. You could be tempted to use the method above, but the configuration file diff --git a/docs/howto/upgrade.rst b/docs/howto/upgrade.rst index 69232bc..d917998 100644 --- a/docs/howto/upgrade.rst +++ b/docs/howto/upgrade.rst @@ -37,10 +37,10 @@ The following list shows how to update your commands: (old) vault get path/to/creds (new) vault get path/to/creds value - (old) vault env --path path/to/creds=FOO -- env # FOO=xxx - (new) vault env --path path/to/creds=FOO -- env # FOO_VALUE=xxx - (new) vault env --path path/to/creds:value=FOO -- env # FOO=xxx - (new) vault env --omit-single_key --path path/to/creds=FOO -- env # FOO=xxx + (old) vault env --envvar path/to/creds=FOO -- env # FOO=xxx + (new) vault env --envvar path/to/creds=FOO -- env # FOO_VALUE=xxx + (new) vault env --envvar path/to/creds:value=FOO -- env # FOO=xxx + (new) vault env --omit-single_key --envvar path/to/creds=FOO -- env # FOO=xxx The default output of ``vault get-all`` has also changed and is now flat by default (this behavior is controlled with the ``--flat/--no-flat`` diff --git a/docs/quickstart.rst b/docs/quickstart.rst index 361ffdc..4b42d69 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -208,7 +208,7 @@ Let's try it. First we'll launch the command ``env``, which prints the environme .. code:: console - $ vault-cli env --path demo -- env | tail -1 + $ vault-cli env --envvar demo -- env | tail -1 DEMO_BLAKE2_SECRET_KEY=du9dibieNg3lei0teidal9 As you can see, the secrets (or, here, the secret) under the path ``demo`` have been @@ -238,7 +238,7 @@ Ok, now for the real thing: .. code:: console - $ vault-cli env --path demo -- ./docs/quickstart_demo.py yay + $ vault-cli env --envvar demo -- ./docs/quickstart_demo.py yay 341c93333a9df726c57671891d6bbea1 **Yay!** diff --git a/tests/unit/test_cli.py b/tests/unit/test_cli.py index 8dd2559..9760fce 100644 --- a/tests/unit/test_cli.py +++ b/tests/unit/test_cli.py @@ -414,9 +414,9 @@ def test_env_filter_key(cli_runner, vault_with_token, mocker): cli.cli, [ "env", - "--path", + "--envvar", "foo/baz:user=MYNAME", - "--path", + "--envvar", "foo/baz:password", "--", "echo", @@ -444,6 +444,40 @@ def test_env_omit_single_key(cli_runner, vault_with_token, mocker): assert kwargs["environment"]["FOO_BAZ"] == "yo" +def test_env_file(cli_runner, vault_with_token, mocker, tmp_path): + mocker.patch("vault_cli.environment.exec_command") + + path = tmp_path / "foo" + vault_with_token.db = {"foo/bar": {"value": "yay"}} + cli_runner.invoke( + cli.cli, ["env", "--file", f"foo/bar:value={path}", "--", "echo", "yay"] + ) + assert path.read_text() == "yay\n" + + +def test_env_file_format_error(cli_runner, vault_with_token, mocker, tmp_path): + mocker.patch("vault_cli.environment.exec_command") + + vault_with_token.db = {"foo/bar": {"value": "yay"}} + result = cli_runner.invoke( + cli.cli, ["env", "--file", "foo/bar", "--", "echo", "yay"] + ) + assert result.exit_code != 0 + assert "expects both a vault path and a filesystem path" in result.output + + +def test_env_file_yaml(cli_runner, vault_with_token, mocker, tmp_path): + mocker.patch("vault_cli.environment.exec_command") + + path = tmp_path / "foo" + vault_with_token.db = {"foo/bar": {"value": "yay"}} + cli_runner.invoke( + cli.cli, + ["env", "--file", f"foo/bar={path}", "--", "echo", "yay"], + ) + assert path.read_text() == "---\nvalue: yay\n" + + def test_main(environ, mocker): mock_cli = mocker.patch("vault_cli.cli.cli") diff --git a/vault_cli/cli.py b/vault_cli/cli.py index 9dc3c54..8a68744 100644 --- a/vault_cli/cli.py +++ b/vault_cli/cli.py @@ -465,11 +465,27 @@ def delete(client_obj: client.VaultClientBase, name: str, key: Optional[str]) -> formerly deprecated. See command help for details. """, ) +@click.option( + "--file", + multiple=True, + help=""" + Write a secret from this path into a file on the filesystem. Expected format is + path/in/vault[:key]=/path/in/filesystem . This option is meant to be used when + your command can only read its inputs from a file and not from the environment (e.g. + secret keys, ...). It's highly recommended to only use the option when provided with + a secure private temporary filesystem. Writing to a physical disk should be avoided + when possible. If the secret a collection (object or array), it's dumped in YAML + format. + """, +) @click.option( "-o", "--omit-single-key/--no-omit-single-key", default=False, - help="When the secret has only one key, don't use that key to build the name of the environment variable", + help=""" + When the secret has only one key, don't use that key to build the name of the + environment variable. This option doesn't affect --file. + """, ) @click.option( "-f", @@ -483,6 +499,7 @@ def delete(client_obj: client.VaultClientBase, name: str, key: Optional[str]) -> def env( client_obj: client.VaultClientBase, envvar: Sequence[str], + file: Sequence[str], omit_single_key: bool, force: bool, command: Sequence[str], @@ -501,18 +518,19 @@ def env( By default the name is build upon the relative path to the parent of the given path (in parameter) and the name of the keys in the value mapping. Let's say that we have stored the mapping `{'username': 'me', 'password': 'xxx'}` at path `a/b/c` - Using `--path a/b` will inject the following environment variables: B_C_USERNAME and B_C_PASSWORD - Using `--path a/b/c` will inject the following environment variables: C_USERNAME and C_PASSWORD - Using `--path a/b/c:username` will only inject `USERNAME=me` in the environment. + Using `--envvar a/b` will inject the following environment variables: B_C_USERNAME and B_C_PASSWORD + Using `--envvar a/b/c` will inject the following environment variables: C_USERNAME and C_PASSWORD + Using `--envvar a/b/c:username` will only inject `USERNAME=me` in the environment. You can customize the variable names generation by appending `=SOME_PREFIX` to the path. In this case the part corresponding to the base path is replace by your prefix. - Using `--path a/b=FOO` will inject the following environment variables: FOO_C_USERNAME and FOO_C_PASSWORD - Using `--path a/b/c=FOO` will inject the following environment variables: FOO_USERNAME and FOO_PASSWORD - Using `--path a/b/c:username=FOO` will inject `FOO=me` in the environment. + Using `--envvar a/b=FOO` will inject the following environment variables: FOO_C_USERNAME and FOO_C_PASSWORD + Using `--envvar a/b/c=FOO` will inject the following environment variables: FOO_USERNAME and FOO_PASSWORD + Using `--envvar a/b/c:username=FOO` will inject `FOO=me` in the environment. """ envvars = list(envvar) or [] + files = list(file) or [] env_secrets = {} @@ -534,6 +552,22 @@ def env( env_secrets.update(env_updates) + for file in files: + path, key, filesystem_path = get_env_parts(file) + if not (path and filesystem_path): + raise click.BadOptionUsage( + "file", "--file expects both a vault path and a filesystem path" + ) + secret_obj = client_obj.get_secret(path=path, key=key or None) + + if not isinstance(secret_obj, (list, dict)): + secret = str(secret_obj).strip() + "\n" + else: + secret = yaml.safe_dump( + secret_obj, default_flow_style=False, explicit_start=True + ) + pathlib.Path(filesystem_path).write_text(secret) + if bool(client_obj.errors) and not force: raise click.ClickException("There was an error while reading the secrets.") environment.exec_command(command=command, environment=env_secrets)