Skip to content

Commit

Permalink
Fallback to FileInputStream if PemReader fails to read certificates
Browse files Browse the repository at this point in the history
  • Loading branch information
@yashaswaj
yashaswaj committed Aug 3, 2020
1 parent 88db105 commit f7fe758
Show file tree
Hide file tree
Showing 4 changed files with 85 additions and 40 deletions.
54 changes: 28 additions & 26 deletions presto-docs/src/main/sphinx/connector/hive.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -294,45 +294,47 @@ Property Name Description
Thrift Metastore Configuration Properties
-----------------------------------------

============================================================= ============================================================ ============
Property Name Description Default
============================================================= ============================================================ ============
``hive.metastore.uri`` The URI(s) of the Hive metastore to connect to using the
Thrift protocol. If multiple URIs are provided, the first
URI is used by default, and the rest of the URIs are
fallback metastores. This property is required.
Example: ``thrift://192.0.2.3:9083`` or
``thrift://192.0.2.3:9083,thrift://192.0.2.4:9083``
=============================================================== ============================================================ ============
Property Name Description Default
=============================================================== ============================================================ ============
``hive.metastore.uri`` The URI(s) of the Hive metastore to connect to using the
Thrift protocol. If multiple URIs are provided, the first
URI is used by default, and the rest of the URIs are
fallback metastores. This property is required.
Example: ``thrift://192.0.2.3:9083`` or
``thrift://192.0.2.3:9083,thrift://192.0.2.4:9083``

``hive.metastore.username`` The username Presto uses to access the Hive metastore.
``hive.metastore.username`` The username Presto uses to access the Hive metastore.

``hive.metastore.authentication.type`` Hive metastore authentication type.
Possible values are ``NONE`` or ``KERBEROS``
(defaults to ``NONE``).
``hive.metastore.authentication.type`` Hive metastore authentication type.
Possible values are ``NONE`` or ``KERBEROS``
(defaults to ``NONE``).

``hive.metastore.thrift.impersonation.enabled`` Enable Hive metastore end user impersonation.
``hive.metastore.thrift.impersonation.enabled`` Enable Hive metastore end user impersonation.

``hive.metastore.thrift.delegation-token.cache-ttl`` Time to live delegation token cache for metastore. ``1h``
``hive.metastore.thrift.delegation-token.cache-ttl`` Time to live delegation token cache for metastore. ``1h``

``hive.metastore.thrift.delegation-token.cache-maximum-size`` Delegation token cache maximum size. 1,000
``hive.metastore.thrift.delegation-token.cache-maximum-size`` Delegation token cache maximum size. 1,000

``hive.metastore.thrift.client.ssl.enabled`` Use SSL when connecting to metastore. ``false``
``hive.metastore.thrift.client.ssl.enabled`` Use SSL when connecting to metastore. ``false``

``hive.metastore.thrift.client.ssl.key`` Path to PEM private key and client certificate (key store).
``hive.metastore.thrift.client.ssl.key`` Path to private key and client certificate (key store).

``hive.metastore.thrift.client.ssl.key-password`` Password for the PEM private key.
``hive.metastore.thrift.client.ssl.key-password`` Password for the private key.

``hive.metastore.thrift.client.ssl.trust-certificate`` Path to the PEM server certificate chain (trust store).
Required when SSL is enabled.
``hive.metastore.thrift.client.ssl.trust-certificate`` Path to the server certificate chain (trust store).
Required when SSL is enabled.

``hive.metastore.service.principal`` The Kerberos principal of the Hive metastore service.
``hive.metastore.thrift.client.ssl.trust-certificate-password`` Password for the trust store

``hive.metastore.client.principal`` The Kerberos principal that Presto uses when connecting
to the Hive metastore service.
``hive.metastore.service.principal`` The Kerberos principal of the Hive metastore service.

``hive.metastore.client.keytab`` Hive metastore client keytab location.
``hive.metastore.client.principal`` The Kerberos principal that Presto uses when connecting
to the Hive metastore service.

============================================================= ============================================================ ============
``hive.metastore.client.keytab`` Hive metastore client keytab location.

=============================================================== ============================================================ ============

AWS Glue Catalog Configuration Properties
-----------------------------------------
Expand Down
50 changes: 38 additions & 12 deletions ...n/java/io/prestosql/plugin/hive/metastore/thrift/DefaultThriftMetastoreClientFactory.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,9 @@
import javax.security.auth.x500.X500Principal;

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.security.GeneralSecurityException;
import java.security.KeyStore;
import java.security.cert.Certificate;
Expand Down Expand Up @@ -80,7 +82,8 @@ public DefaultThriftMetastoreClientFactory(
config.isTlsEnabled(),
Optional.ofNullable(config.getKeystorePath()),
Optional.ofNullable(config.getKeystorePassword()),
config.getTruststorePath()),
config.getTruststorePath(),
Optional.ofNullable(config.getTruststorePassword())),
Optional.ofNullable(config.getSocksProxy()),
config.getMetastoreTimeout(),
metastoreAuthentication,
Expand Down Expand Up @@ -111,7 +114,8 @@ private static Optional<SSLContext> buildSslContext(
boolean tlsEnabled,
Optional<File> keyStorePath,
Optional<String> keyStorePassword,
File trustStorePath)
File trustStorePath,
Optional<String> trustStorePassword)
{
if (!tlsEnabled) {
return Optional.empty();
Expand All @@ -120,16 +124,27 @@ private static Optional<SSLContext> buildSslContext(
try {
// load KeyStore if configured and get KeyManagers
KeyManager[] keyManagers = null;
KeyStore keyStore = null;
char[] keyManagerPassword = new char[0];
if (keyStorePath.isPresent()) {
KeyStore keyStore = PemReader.loadKeyStore(keyStorePath.get(), keyStorePath.get(), keyStorePassword);
try {
keyStore = PemReader.loadKeyStore(keyStorePath.get(), keyStorePath.get(), keyStorePassword);
}
catch (IOException | GeneralSecurityException e) {
keyManagerPassword = keyStorePassword.map(String::toCharArray).orElse(null);
keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
try (InputStream in = new FileInputStream(keyStorePath.get())) {
keyStore.load(in, keyManagerPassword);
}
}
validateCertificates(keyStore);
KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
keyManagerFactory.init(keyStore, new char[0]);
keyManagerFactory.init(keyStore, keyManagerPassword);
keyManagers = keyManagerFactory.getKeyManagers();
}

// load TrustStore
KeyStore trustStore = loadTrustStore(trustStorePath);
KeyStore trustStore = loadTrustStore(trustStorePath, trustStorePassword);

// create TrustManagerFactory
TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
Expand All @@ -151,16 +166,27 @@ private static Optional<SSLContext> buildSslContext(
}
}

private static KeyStore loadTrustStore(File trustStorePath)
private static KeyStore loadTrustStore(File trustStorePath, Optional<String> trustStorePassword)
throws IOException, GeneralSecurityException
{
List<X509Certificate> certificateChain = PemReader.readCertificateChain(trustStorePath);

KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
trustStore.load(null, null);
for (X509Certificate certificate : certificateChain) {
X500Principal principal = certificate.getSubjectX500Principal();
trustStore.setCertificateEntry(principal.getName(), certificate);
try {
// attempt to read the trust store as a PEM file
List<X509Certificate> certificateChain = PemReader.readCertificateChain(trustStorePath);
if (!certificateChain.isEmpty()) {
trustStore.load(null, null);
for (X509Certificate certificate : certificateChain) {
X500Principal principal = certificate.getSubjectX500Principal();
trustStore.setCertificateEntry(principal.getName(), certificate);
}
return trustStore;
}
}
catch (IOException | GeneralSecurityException e) {
}

try (InputStream in = new FileInputStream(trustStorePath)) {
trustStore.load(in, trustStorePassword.map(String::toCharArray).orElse(null));
}
return trustStore;
}
Expand Down
18 changes: 16 additions & 2 deletions ...o-hive/src/main/java/io/prestosql/plugin/hive/metastore/thrift/ThriftMetastoreConfig.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ public class ThriftMetastoreConfig
private File keystorePath;
private String keystorePassword;
private File truststorePath;
private String trustStorePassword;

@NotNull
public Duration getMetastoreTimeout()
Expand Down Expand Up @@ -231,7 +232,7 @@ public File getKeystorePath()
}

@Config("hive.metastore.thrift.client.ssl.key")
@ConfigDescription("Path to the PEM key store")
@ConfigDescription("Path to the key store")
public ThriftMetastoreConfig setKeystorePath(File keystorePath)
{
this.keystorePath = keystorePath;
Expand All @@ -258,13 +259,26 @@ public File getTruststorePath()
}

@Config("hive.metastore.thrift.client.ssl.trust-certificate")
@ConfigDescription("Path to the PEM trust store")
@ConfigDescription("Path to the trust store")
public ThriftMetastoreConfig setTruststorePath(File truststorePath)
{
this.truststorePath = truststorePath;
return this;
}

public String getTruststorePassword()
{
return trustStorePassword;
}

@Config("hive.metastore.thrift.client.ssl.trust-certificate-password")
@ConfigDescription("Password for the trust store")
public ThriftMetastoreConfig setTruststorePassword(String trustStorePassword)
{
this.trustStorePassword = trustStorePassword;
return this;
}

@AssertTrue(message = "Trust store must be provided when TLS is enabled")
public boolean isTruststorePathValid()
{
Expand Down
3 changes: 3 additions & 0 deletions ...ve/src/test/java/io/prestosql/plugin/hive/metastore/thrift/TestThriftMetastoreConfig.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ public void testDefaults()
.setKeystorePath(null)
.setKeystorePassword(null)
.setTruststorePath(null)
.setTruststorePassword(null)
.setImpersonationEnabled(false)
.setDelegationTokenCacheTtl(new Duration(1, HOURS))
.setDelegationTokenCacheMaximumSize(1000)
Expand All @@ -74,6 +75,7 @@ public void testExplicitPropertyMappings()
.put("hive.metastore.thrift.client.ssl.key", keystoreFile.toString())
.put("hive.metastore.thrift.client.ssl.key-password", "keystore-password")
.put("hive.metastore.thrift.client.ssl.trust-certificate", truststoreFile.toString())
.put("hive.metastore.thrift.client.ssl.trust-certificate-password", "truststore-password")
.put("hive.metastore.thrift.impersonation.enabled", "true")
.put("hive.metastore.thrift.delegation-token.cache-ttl", "1d")
.put("hive.metastore.thrift.delegation-token.cache-maximum-size", "9999")
Expand All @@ -93,6 +95,7 @@ public void testExplicitPropertyMappings()
.setKeystorePath(keystoreFile.toFile())
.setKeystorePassword("keystore-password")
.setTruststorePath(truststoreFile.toFile())
.setTruststorePassword("truststore-password")
.setImpersonationEnabled(true)
.setDelegationTokenCacheTtl(new Duration(1, DAYS))
.setDelegationTokenCacheMaximumSize(9999)
Expand Down

0 comments on commit f7fe758

Please sign in to comment.