Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

End to end TLS SSL #16452

Merged
merged 94 commits into from
Oct 21, 2020
Merged

End to end TLS SSL #16452

merged 94 commits into from
Oct 21, 2020

Conversation

mnriem
Copy link
Contributor

@mnriem mnriem commented Oct 16, 2020

No description provided.

Copy link
Member

@JonathanGiles JonathanGiles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've only had a very brief few moments to look through this but I wanted to share the first round of feedback now rather than wait until a complete review could be completed.

@@ -63,6 +63,7 @@ org.apache.avro:avro-maven-plugin;1.9.2
org.apache.commons:commons-compress;1.20
org.apache.commons:commons-lang3;3.10
org.apache.httpcomponents:httpclient;4.5.12
org.apache.httpcomponents.client5:httpclient5;5.0.1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears as if this dependency is being used in a shaded fashion, alongside other libraries. I would be happier if we didn't need these dependencies at all though. From a quick look through the code it doesn't appear to be baked in deeply, mainly showing up it seems in the LegacyRestClient.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dependencies of the JCA provider are shaded to make sure the provider is self-contained and can be used in any Java application

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, understood. My comment was mainly focused on whether the httpclient is necessary in the first place? It doesn't appear to be from a cursory glance.

<rules>
<bannedDependencies>
<includes>
<include>org.apache.httpcomponents.client5:httpclient5:[5.0.1]</include> <!-- {x-include-update;org.apache.httpcomponents.client5:httpclient5;external_dependency} -->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a huge fan of allowing this dependency - would love to see this removed if at all possible.

<version>3.1.0</version> <!-- {x-version-update;org.apache.maven.plugins:maven-checkstyle-plugin;external_dependency} -->
<configuration>
<skip>true</skip>
</configuration>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't skip this check.

<version>3.1.12.2</version> <!-- {x-version-update;com.github.spotbugs:spotbugs-maven-plugin;external_dependency} -->
<configuration>
<skip>true</skip>
</configuration>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't skip this check.

<version>0.11.2</version> <!-- {x-version-update;org.revapi:revapi-maven-plugin;external_dependency} -->
<configuration>
<skip>true</skip>
</configuration>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't skip this check.

/**
* Stores the OAuth2 token base URL.
*/
private static final String OAUTH2_TOKEN_BASE_URL = "https://login.microsoftonline.com/";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this intended to work against clouds other than Azure Public Cloud? If so this would need to be configurable.

JsonConverter converter = JsonConverterFactory.createJsonConverter();
SecretBundle secretBundle = (SecretBundle) converter.fromJson(body, SecretBundle.class);
try {
KeyStore keyStore = KeyStore.getInstance("PKCS12");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certificates downloaded from Key Vault through the get secret API might be in PKCS12 or PEM format. These can be differentiated by the by the contentType of the secret bundle.

Copy link
Member

@vcolin7 vcolin7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many of my comments and suggestions can be pushed to a later release, but there are a few that I think should be at least discussed before we go ahead with merging.

Additionally, there are still a couple missing changes to make to files involved with CI to properly set everything up for the release. Please refer this PR and this PR. The first one shows how to make an exclusion for some README issues and how to include a new library on the code coverage runs. The second one shows how to add checkstyle suppressions (when they are justified) and how to include a new package on the build pipeline (you have already performed this step).

Finally, I think we should consider refactoring some classes into a models package and possibly a couple others in the future.

Comment on lines +57 to +101
KeyVaultJcaProvider provider = new KeyVaultJcaProvider();
Security.addProvider(provider);

KeyStore ks = KeyStore.getInstance("AzureKeyVault");
KeyVaultLoadStoreParameter parameter = new KeyVaultLoadStoreParameter(
System.getProperty("azure.keyvault.uri"),
System.getProperty("azure.tenant.id"),
System.getProperty("azure.client.id"),
System.getProperty("azure.client.secret"));
ks.load(parameter);

SSLContext sslContext = SSLContexts
.custom()
.loadTrustMaterial(ks, new TrustSelfSignedStrategy())
.build();

SSLConnectionSocketFactory sslSocketFactory = SSLConnectionSocketFactoryBuilder
.create()
.setSslContext(sslContext)
.setHostnameVerifier((hostname, session) -> {
return true;
})
.build();

PoolingHttpClientConnectionManager cm = PoolingHttpClientConnectionManagerBuilder
.create()
.setSSLSocketFactory(sslSocketFactory)
.build();

String result = null;

try ( CloseableHttpClient client = HttpClients.custom().setConnectionManager(cm).build()) {
HttpGet httpGet = new HttpGet("https://localhost:8766");
HttpClientResponseHandler<String> responseHandler = (ClassicHttpResponse response) -> {
int status = response.getCode();
String result1 = "Not success";
if (status == 204) {
result1 = "Success";
}
return result1;
};
result = client.execute(httpGet, responseHandler);
} catch (IOException ioe) {
ioe.printStackTrace();
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit; Small details.

Suggested change
KeyVaultJcaProvider provider = new KeyVaultJcaProvider();
Security.addProvider(provider);
KeyStore ks = KeyStore.getInstance("AzureKeyVault");
KeyVaultLoadStoreParameter parameter = new KeyVaultLoadStoreParameter(
System.getProperty("azure.keyvault.uri"),
System.getProperty("azure.tenant.id"),
System.getProperty("azure.client.id"),
System.getProperty("azure.client.secret"));
ks.load(parameter);
SSLContext sslContext = SSLContexts
.custom()
.loadTrustMaterial(ks, new TrustSelfSignedStrategy())
.build();
SSLConnectionSocketFactory sslSocketFactory = SSLConnectionSocketFactoryBuilder
.create()
.setSslContext(sslContext)
.setHostnameVerifier((hostname, session) -> {
return true;
})
.build();
PoolingHttpClientConnectionManager cm = PoolingHttpClientConnectionManagerBuilder
.create()
.setSSLSocketFactory(sslSocketFactory)
.build();
String result = null;
try ( CloseableHttpClient client = HttpClients.custom().setConnectionManager(cm).build()) {
HttpGet httpGet = new HttpGet("https://localhost:8766");
HttpClientResponseHandler<String> responseHandler = (ClassicHttpResponse response) -> {
int status = response.getCode();
String result1 = "Not success";
if (status == 204) {
result1 = "Success";
}
return result1;
};
result = client.execute(httpGet, responseHandler);
} catch (IOException ioe) {
ioe.printStackTrace();
}
KeyVaultJcaProvider provider = new KeyVaultJcaProvider();
Security.addProvider(provider);
KeyStore ks = KeyStore.getInstance("AzureKeyVault");
KeyVaultLoadStoreParameter parameter = new KeyVaultLoadStoreParameter(
System.getProperty("azure.keyvault.uri"),
System.getProperty("azure.tenant.id"),
System.getProperty("azure.client.id"),
System.getProperty("azure.client.secret"));
ks.load(parameter);
SSLContext sslContext = SSLContexts
.custom()
.loadTrustMaterial(ks, new TrustSelfSignedStrategy())
.build();
SSLConnectionSocketFactory sslSocketFactory = SSLConnectionSocketFactoryBuilder
.create()
.setSslContext(sslContext)
.setHostnameVerifier((hostname, session) -> {
return true;
})
.build();
PoolingHttpClientConnectionManager cm = PoolingHttpClientConnectionManagerBuilder
.create()
.setSSLSocketFactory(sslSocketFactory)
.build();
String result = null;
try (CloseableHttpClient client = HttpClients.custom().setConnectionManager(cm).build()) {
HttpGet httpGet = new HttpGet("https://localhost:8766");
HttpClientResponseHandler<String> responseHandler = (ClassicHttpResponse response) -> {
int status = response.getCode();
String result1 = "Not success";
if (status == 204) {
result1 = "Success";
}
return result1;
};
result = client.execute(httpGet, responseHandler);
} catch (IOException ioe) {
ioe.printStackTrace();
}

<!-- END: Empty Java Doc & Sources -->
<plugin>
<groupId>org.revapi</groupId>
<artifactId>revapi-maven-plugin</artifactId>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed the Key Vault Secrets Spring Boot Starter does not make use of the Revapi plugin. Should we consider adding it there?

import static org.springframework.core.Ordered.LOWEST_PRECEDENCE;

@Order(LOWEST_PRECEDENCE)
public class KeyVaultCertificatesEnvironmentPostProcessor implements EnvironmentPostProcessor {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that the all classes that are part of the Key Vault Secrets Spring Boot Starter live in the azure-spring-boot folder under the package com.azure.spring.keyvault, should this be there as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chenrujun @saragluna Can you answer this question?

Comment on lines +1 to +3
server.ssl.key-store-password=doesnotmatter
server.ssl.key-password=doesnotmatter
server.ssl.trust-store-password=doesnotmatter
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can use dummy-value or something else instead (I don't have a better suggestion at the moment). As it stands right now, it is a little ambiguous if what does not matter is the value or the property at all.

Suggested change
server.ssl.key-store-password=doesnotmatter
server.ssl.key-password=doesnotmatter
server.ssl.trust-store-password=doesnotmatter
server.ssl.key-store-password=dummy-value
server.ssl.key-password=dummy-value
server.ssl.trust-store-password=dummy-value

Copy link
Member

@vcolin7 vcolin7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving based on conversations with @JonathanGiles and @mnriem. Suggested changes and improvements will target future releases.

@chenrujun chenrujun dismissed JonathanGiles’s stale review October 21, 2020 01:53

Suggested changes and improvements will target future releases.

* Stores the OAuth2 managed identity URL.
*/
private static final String OAUTH2_MANAGED_IDENTITY_TOKEN_URL
= "http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will work on IMDS supported Azure Platforms only.
For e.g This won't work in App Service.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, the case of App Service has been handled separately. And according to this doc, requesting access tokens via Azure resources VM Extension Endpoint is also not encouraged. Thus are there any other cases that are not covered? Thanks


StringBuilder url = new StringBuilder();
url.append(System.getenv("MSI_ENDPOINT"))
.append("?api-version=2017-09-01")
Copy link
Member

@g2vinay g2vinay Oct 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a newer 2019 API version available, which uses IDENTITY_ENDPOINT and IDENTITY_HEADER env vars.

Copy link
Member

@yiliuTo yiliuTo Mar 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @g2vinay , I found the following information that when using Linux Consumption, 2017 API version is required. Thus is it a better choice that we remain use of 2017 API version to support more cases?
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
azure-spring All azure-spring related issues KeyVault
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants