Provide a callback to decrypt DB passwords

[twicksell]

Common requirement is to encrypt your db password and put it in your props file. It'd be nice for Boot to provide some kind of callback so that I can provide decryption logic. Perhaps consider making it flexible so we can use it for more than just the DB passwords. My initial thought was to register a custom spel function and create a property like
spring.datasource=#{decrypt('someCrazyEncryptedPassword')}

[philwebb]

There might already be hooks that you can use to achieve this. If you have a bean that implements ApplicationListener<ApplicationEvent> and is ordered just after ConfigFileApplicationListener you might be able to mess with the Environment to replace specific values.

Another option might be to create a custom PropertySourcesPlaceholderConfigurer that supports decryption.

[dsyer]

I foresee a couple of problems in the implementation of this feature. I like Phil's approach of postprocessing theEnvironment but you need to do the decryption to happen super early in ApplicationContext lifecycle terms (ideally before any beans are instantiated), so if your decryption callback requires any wiring (e.g. access to the Environment itself) then there is a chicken-and-egg issue. I'm not sure a vanilla Spring Boot app with a single ApplicationContext is the right way to solve that problem. The other (big) problem is how do you propose to secure the decryption? Presumably the properties can be encrypted safely (e.g. at build time), but at runtime where are you going to get the key(s)? Presumably you don't want to store the key alongside the decrypted value, so you have to rely on an OS environment variable or something?

I'll show you what we have already on Friday face to face if you can make it to the meetings.

[markpollack]

There were some requests for this via XD users, ATM the suggestion is using unix permissions on a yml profile variant file. There is also https://jira.spring.io/browse/SPR-10666 that has links to more possible approaches.

[dsyer]

Thanks for the link. UNIX permissions is the best (most secure) way to deal with local files. We are also working on a server-based solution (so the keys are never available to the client app). Jasypt seems like it doesn't provide much that you can't do with Spring Security Crypto (other than more flavours of encryption) - the principle is the same though: for client side local files you still need UNIX permissions to protect the key.

[twicksell]

In our situation we already have the keys secured via an external service and some ACLs. The problem I have is with injecting my decryption logic using this key into Boot's DataSourceProperties. I got what I wanted by extending and overriding DataSourceProperties to inject my decryption logic, but that felt pretty hacky. My thinking was that Boot could replicate that with a simple callback or configurer bean. Its single purpose, but handles the most common case of wanting to decrypt a DB password while leaving the implementation details of how to decrypt to the user. And because this is happening during refresh we'll have access to any beans in the context we need to assist with decryption.

[stephane-deraco]

Hi, I would appreciate this enhancement. What I have done to use encrypted values in application.yml is the following: I have created a component implementing PropertySourceLoader, with HIGHEST_PRECEDENCE for @Order.

In that component, I initialize Jasypt with the key coming from System.getenv("masterkey"). And then I implements load in which I return an instance of org.jasypt.spring31.properties.EncryptablePropertiesPropertySource.

It is working for me so far, but I sure would appreciate if this was included in Boot.

[dsyer]

That's great and thanks for the update.

If anyone else is interested in the evolution of this story, then we have some features in Spring Cloud (with a release coming soon). The config client is definitely an awesome addition to any Spring Boot application and it's pretty lightweight hopefully. It will give you decryption of local property sources with all the features I think you should need. (By the way, I would use an environment key that is hidden by default in the Spring Boot Actuator /env endpoint - one ending in "password" or "secret".)

I'm not opposed to using Jasypt, but Spring Cloud doesn't use it (there are Spring Security primitives that are close enough and simpler to operate hopefully). Spring Cloud also provides asymmetric key cryptography, which I think is a must have.

[stephane-deraco]

Thanks Dave, I'll take a look at Spring Cloud.
I didn't knew that environment variables ending in password or secret were hidden, thanks for the info!

Spring Boot is really a great project!

[hohwille]

Just to answer some of the concerns why this could be helpful. In general this is some sort of security by obscurity if in the end the application can decrypt the passwords. However, the question is more how to do a safe packaging and distribution of an application.
With spring boot you can easily externalize the configurations (./config/application.properties) and build portable deliverables. However, if you want automated deployments of the configurations per environment you might want to build config packages as RPM, deb, yum, etc.
With the suggested approach you could have the production settings in maven filters in a VCS where every developer can access but as the passwords are strongly encrypted the passwords are still secure.
Then only e.g. a keystore or some additional secret is required on the machine for the installation beside the packages build and deployed (via DevOps). So IMHO this is a nice feature.
As it seems Jasypt already solved this with spring boot but if there was official support designed by the spring makers this would be even better as I also fear the chicken-egg-problem when things get more complex.

[dsyer]

As I said above, I don't think Jasypt cuts it for me. Please take a look at the features in Spring Cloud Context (moved from Spring Cloud Config during the 1.0.x releases). All the encryption support is in a single, small jar with only Spring Boot dependencies. It would be dumb to duplicate that here.

[hohwille]

Fine. I will have a look. Thanks for the hint.

Could you also have a look at this and consider some rethinking about springs inner design:
ulisesbocchio/jasypt-spring-boot#7

I am pretty much aware that spring is a swiss army knife and people will use it in million different (and sometimes sick) ways. So regression is not an easy one, but I would be happy, if you could have a look and see if there is something wrong in spring that can be fixed. Thanks!

[dsyer]

@ulisesbocchio explains in that issue that it is his design decision not to cache decrypted values. I don't see what it has to do with Spring Boot yet. Please let us know separately if you find something that needs changing.

[hohwille]

I filed a separate issue for this so I am done here.

[hohwille]

Please take a look at the features in Spring Cloud Context

So you are talking about things like this:
https://github.com/spring-cloud/spring-cloud-commons/blob/master/spring-cloud-context/src/main/java/org/springframework/cloud/context/encrypt/EncryptorFactory.java

Sorry to say so but this is rather disappointing. Actually it would be better to allow to configure a custom decrytpor bean that is used for property values in the ENC(...) form. Then your current hardwired implementation can be a fallback if nothing else is configured.

[mminella]

For the record, there is a related Spring Core issue for similar functionality that's been around for a while: https://jira.spring.io/browse/SPR-10666

[arafique]

Hi,

Can anyone please help me with following issue

http://stackoverflow.com/questions/31824601/spring-boot-not-loading-propertysourceloader

Thanks

[ulisesbocchio]

@arafique For what I can tell the SpringFactoriesLoader mechanism for factory PropertySourceLoader is only used by PropertySourcesLoader which in turn is ONLY used to load the application properties. Those are either application.properties, application.yml, or application.yaml. So for your example just add a file application.properties to your classpath and you'll get the exception you are expecting.
What I don't know is why other property source import mechanisms such as using annotation @propertysource are not going through PropertySourcesLoader to take advantage of the factories loaders and override mechanism.

[DanieleTorino]

@stephane-deraco: How did you get Spring to use your PropertySourceLoader? I tried your approach, but I can't seem to get Spring Boot to use my component.

[stephane-deraco]

@DanieleTorino In fact, I have followed the advice given by @dsyer and took a look at Spring cloud client. However, I do not use Spring cloud.

I came to this solution inspired by https://github.com/spring-cloud/spring-cloud-commons/blob/cde7c7f3118382490c28776f66e0a56f248141fd/spring-cloud-context/src/main/java/org/springframework/cloud/bootstrap/encrypt/EnvironmentDecryptApplicationInitializer.java :

Create a class implementing ApplicationContextInitializer<ConfigurableApplicationContext> and annotated with @Configuration.

Initialize your encryptor with something like :

// Try to initialize the encryptor with master password
encryptor = new StandardPBEStringEncryptor();
final char[] password = // Get your pass here
((StandardPBEStringEncryptor) encryptor).setPasswordCharArray(password);
Arrays.fill(password, '\0');

And then write:

    @Override
    public void initialize(ConfigurableApplicationContext applicationContext) {
        ConfigurableEnvironment environment = applicationContext.getEnvironment();
        Map<String, Object> overrides = new LinkedHashMap<>();
        for (PropertySource<?> source : environment.getPropertySources()) {
            decrypt(source, overrides);
        }
        if (!overrides.isEmpty()) {
            environment.getPropertySources().addFirst(
                    new MapPropertySource("decrypted", overrides));
        }
    }

    private void decrypt(PropertySource<?> source, Map<String, Object> overrides) {
        if (source instanceof EnumerablePropertySource) {
            EnumerablePropertySource<?> enumerable = (EnumerablePropertySource<?>) source;
            for (String key : enumerable.getPropertyNames()) {
                String value = source.getProperty(key).toString();
                if (value.startsWith("{cipher}")) {
                    value = value.substring("{cipher}".length());
                    try {
                        value = encryptor.decrypt(value);
                        LOG.debug("Decrypted: key={}", key);
                    } catch (Exception e) {
                        LOG.warn(String.format("Cannot decrypt: key=%s", key), e);
                        // Set value to empty to avoid making a password out of the
                        // cipher text
                        value = "";
                    }
                    overrides.put(key, value);
                }
            }
        } else if (source instanceof CompositePropertySource) {
            for (PropertySource<?> nested : ((CompositePropertySource) source)
                    .getPropertySources()) {
                decrypt(nested, overrides);
            }
        }
    }

This is something that works for me, including on Yaml properties file.

[DanieleTorino]

@stephane-deraco: Thanks a lot. I got it working with your help, my complete solution can be found on stackoverflow.

[stephane-deraco]

@DanieleTorino You're welcome

[philwebb]

I'm going to close this because I think https://cloud.spring.io/spring-cloud-vault/ is the best way to solve this.

[PAX523]

I came to this solution inspired by https://github.com/spring-cloud/spring-cloud-commons/blob/cde7c7f3118382490c28776f66e0a56f248141fd/spring-cloud-context/src/main/java/org/springframework/cloud/bootstrap/encrypt/EnvironmentDecryptApplicationInitializer.java :

Create a class implementing ApplicationContextInitializer and annotated with @configuration.

...and if @Configuration doesn't do the trick then register your ApplicationContextInitializer before starting your Spring Boot application in main method via:

  public static void main(final String[] args) {
    final SpringApplication springApplication = new SpringApplication(MySpringBootService.class);
    springApplication.addInitializers(new ApplicationPropertiesDecryptor());
    springApplication.run(args);
  }