certificates generated from vault have invalid "Not Before" field

[gites]

Describe the bug
After upgrading to 0.11.2 certificates generated from vault have invalid "Not Before" field set to Jan 1 00:00:00 1 GMT.

        Validity
            Not Before: Jan  1 00:00:00 1 GMT
            Not After : Nov 22 20:19:19 2018 GMT

To Reproduce
We have long-running Vault cluster starting from version ~0.4
Steps to reproduce the behavior:

1. Setup Vault 0.10.4 with pki secret engine (we upgraded from thath wersion)
2. Configure pki role:

{
  "allowed_domains": "example.com",
  "allow_subdomains": true,
  "allow_bare_domains": true,
  "max_ttl": "1080h",
  "ttl": "1080h",
  "key_bits": 4096,
  "generate_lease": true
}

3. Upgrade Vault to 0.11.2 or 0.11.3
4. Request new cert and key from pki endpoin

vault write  pki/issue/example common_name=example.com

Expected behavior
The Not Before: field should be set to the current date of certificate generation.

Environment:

• Vault Server Version (retrieve with vault status):
  Version 0.11.3

• Vault CLI Version (retrieve with vault version):
  Vault v0.11.3 ('fb601237bfbe4bc16ff679f642248ee8a86e627b')

• Server Operating System/Architecture:
  Linux/amd64

Vault server configuration file(s):

ui = true
disable_mlock = false
disable_cache = false
default_lease_ttl = "768h"
max_lease_ttl = "768h"
plugin_directory = "/etc/vault/plugins"

backend "consul" {
  address = "127.0.0.1:8500"

  path = "vault-dev/"
  scheme = "http"
  check_timeout = "5s"
  disable_registration = "false"
  service = "vault-dev"
  service_tags = "dev"

  max_parallel = "128"
  tls_skip_verify = "false"
}

listener "tcp" {
  address = "0.0.0.0:8200"
  tls_disable = "false"
  tls_cert_file="/etc/vault/ssl/vault-bundle.pem"
  tls_key_file="/etc/vault/ssl/vault-key.pem"
}

telemetry {
  statsd_address = "127.0.0.1:9466"
  disable_hostname = true
}

Additional context

This may be introduced by:

secrets/pki: Add configuration support for setting NotBefore [GH-5325]

Actual role as seen by Vault:

vault read pki/roles/dev-tls
Key                                   Value
---                                   -----
allow_any_name                        false
allow_bare_domains                    true
allow_glob_domains                    false
allow_ip_sans                         true
allow_localhost                       true
allow_subdomains                      true
allow_token_displayname               false
allowed_domains                       [example.com]
allowed_other_sans                    <nil>
allowed_serial_numbers                <nil>
allowed_uri_sans                      <nil>
basic_constraints_valid_for_non_ca    false
client_flag                           true
code_signing_flag                     false
country                               <nil>
email_protection_flag                 false
enforce_hostnames                     true
ext_key_usage                         <nil>
ext_key_usage_oids                    <nil>
generate_lease                        true
key_bits                              4096
key_type                              rsa
key_usage                             [DigitalSignature KeyAgreement KeyEncipherment]
locality                              <nil>
max_ttl                               1080h
no_store                              false
not_before_duration                   0s
organization                          <nil>
ou                                    <nil>
policy_identifiers                    <nil>
postal_code                           <nil>
province                              <nil>
require_cn                            false
server_flag                           true
street_address                        <nil>
ttl                                   1080h
use_csr_common_name                   true
use_csr_sans                          true

Note that not_before_duration is set to 0s.

[jefferai]

If you are able to try that branch ^ that would be great. It should fix the problem but confirmation from the reporter is always nice :-)

[gites]

Works like a charm:

        Validity
            Not Before: Oct  8 21:19:40 2018 GMT
            Not After : Nov 22 21:20:04 2018 GMT

[jefferai]

Thanks!

[shayangz]

Any recommendations on how to deal with this if we have upgraded to 0.11.3 in prod already? anything safer/easier than a downgrade path?

[jefferai]

Sure, you can build the binary yourself, or in less than a week there will be a beta.

[chrishoffman]

@shayangz You can also update your PKI roles to explicitly set 30s as the not_before_duration.

[jefferai]

Oh right, thanks Chris! Sometimes the easy things are easy to forget.

[shayangz]

btw, this issue also exists in pki/intermediate/generate/internal endpoint. Is the fix addresses this scenario as well @jefferai?

@chrishoffman, is there a tune option on the secret engine to enforce not_before_duration of 30s?

[jefferai]

It should be fixed there as well, yes.

No tune option, it's something specific to the PKI backend.

[mahmoudm]

This problem also happens for

pki/root/sign-intermediate

There doesn't currently appear to be a workaround for this. Is this planned for a hotfix? It is currently impacting us in production.

[jefferai]

Look four comments up (from yours) for the workaround

[mahmoudm]

I've already tried that -- it does not work for the endpoint I mentioned. I've confirmed that all my roles are set to not_before_duration = 30s.

[jefferai]

It should be fixed in master if you want to build from there.