Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#5157]fix: User can specify the krb5 conf file path for Kerberos enabled Hadoop catalog #5165

Closed
wants to merge 1 commit into from
Closed

[#5157]fix: User can specify the krb5 conf file path for Kerberos enabled Hadoop catalog #5165

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions ...main/java/org/apache/gravitino/catalog/hadoop/authentication/kerberos/KerberosClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,12 @@ public String login(String keytabFilePath) throws IOException {
Preconditions.checkArgument(
principalComponents.size() == 2, "The principal has the wrong format");

// Set Kerberos krb file
String krbFilePath = kerberosConfig.getKrb5Conf();
Preconditions.checkArgument(
StringUtils.isNotBlank(krbFilePath), "The Kerberos krb file can't be blank");
System.setProperty("java.security.krb5.conf", krbFilePath);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this method class loader isolated? I mean what if I set java.security.krb5.conf in catalogA, does it take effect in catalogB

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The configuration is separated for each catalog, this mean the setting of catalogA can not affect catalogB.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean if we set the value in catalogA, and in catalogB, could we still get the value if catalogB does not set it?


// Login
UserGroupInformation.setConfiguration(hadoopConf);
UserGroupInformation.loginUserFromKeytab(catalogPrincipal, keytabFilePath);
Expand Down
22 changes: 22 additions & 0 deletions ...main/java/org/apache/gravitino/catalog/hadoop/authentication/kerberos/KerberosConfig.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ public class KerberosConfig extends AuthenticationConfig {
public static final String FETCH_TIMEOUT_SEC_KEY =
"authentication.kerberos.keytab-fetch-timeout-sec";

public static final String KRB5_CONF_KEY = "java.security.krb5.conf";

public static final ConfigEntry<String> PRINCIPAL_ENTRY =
new ConfigBuilder(PRINCIPAL_KEY)
.doc("The principal of the Kerberos connection")
Expand Down Expand Up @@ -70,6 +72,14 @@ public class KerberosConfig extends AuthenticationConfig {
.checkValue(value -> value > 0, ConfigConstants.POSITIVE_NUMBER_ERROR_MSG)
.createWithDefault(2);

public static final ConfigEntry<String> KRB5_CONF_ENTRY =
new ConfigBuilder(KRB5_CONF_KEY)
.doc("Kerberos krb file for configuration of Kerberos.")
.version(ConfigConstants.VERSION_0_7_0)
.stringConf()
.checkValue(StringUtils::isNotBlank, ConfigConstants.NOT_BLANK_ERROR_MSG)
.createWithDefault("/etc/krb5.conf");

public KerberosConfig(Map<String, String> properties) {
super(properties);
loadFromMap(properties, k -> true);
Expand All @@ -91,6 +101,10 @@ public int getFetchTimeoutSec() {
return get(FETCH_TIMEOUT_SEC_ENTRY);
}

public String getKrb5Conf() {
return get(KRB5_CONF_ENTRY);
}

public static final Map<String, PropertyEntry<?>> KERBEROS_PROPERTY_ENTRIES =
new ImmutableMap.Builder<String, PropertyEntry<?>>()
.put(
Expand Down Expand Up @@ -125,5 +139,13 @@ public int getFetchTimeoutSec() {
false /* immutable */,
60 /* defaultValue */,
false /* hidden */))
.put(
KRB5_CONF_KEY,
PropertyEntry.stringOptionalPropertyEntry(
KRB5_CONF_KEY,
"The Kerberos krb file for the catalog",
false /* immutable */,
"/etc/krb5.conf" /* defaultValue */,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the default value is /etc/krb5.conf, what if the file is not found?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the file is not exist, "Can't get Kerberos realm" error message will be shown. This behavior is same as before.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

false /* hidden */))
.build();
}
3 changes: 3 additions & 0 deletions docs/hadoop-catalog.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ Besides the [common catalog properties](./gravitino-server-config.md#gravitino-c
| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating |
| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1 |
| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple` | No | 0.5.1 |
| `java.security.krb5.conf` | Kerberos krb file for configuration of Kerberos. | /etc/krb5.conf | No | 0.7.0-incubating |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me think if there is a better name for it, java.security.krb5.conf is so long.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about krb5.conf.path?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM about the name changing. That mean I should create a new PR for it, is it correct?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That mean I should create a new PR for it?

No, if it's okay, please do it in this PR.

| `authentication.kerberos.principal` | The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1 |
| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1 |
| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 | No | 0.5.1 |
Expand Down Expand Up @@ -67,6 +68,7 @@ The Hadoop catalog supports creating, updating, deleting, and listing schema.
| `location` | The storage location managed by Hadoop schema. | (none) | No | 0.5.0 |
| `authentication.impersonation-enable` | Whether to enable impersonation for this schema of the Hadoop catalog. | The parent(catalog) value | No | 0.6.0-incubating |
| `authentication.type` | The type of authentication for this schema of Hadoop catalog , currently we only support `kerberos`, `simple`. | The parent(catalog) value | No | 0.6.0-incubating |
| `java.security.krb5.conf` | Kerberos krb5 config file for this schema. | The parent(catalog) value | No | 0.7.0-incubating |
| `authentication.kerberos.principal` | The principal of the Kerberos authentication for this schema. | The parent(catalog) value | No | 0.6.0-incubating |
| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication for this scheam. | The parent(catalog) value | No | 0.6.0-incubating |

Expand All @@ -86,6 +88,7 @@ Refer to [Schema operation](./manage-fileset-metadata-using-gravitino.md#schema-
|----------------------------------------------------|--------------------------------------------------------------------------------------------------------|--------------------------|----------|-----------------|
| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog fileset. | The parent(schema) value | No | 0.6.0 |
| `authentication.type` | The type of authentication for Hadoop catalog fileset, currently we only support `kerberos`, `simple`. | The parent(schema) value | No | 0.6.0 |
| `java.security.krb5.conf` | Kerberos krb5 config file for this fileset. | The parent(schema) value | No | 0.7.0-incubating |
| `authentication.kerberos.principal` | The principal of the Kerberos authentication for the fileset. | The parent(schema) value | No | 0.6.0 |
| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication for the fileset. | The parent(schema) value | No | 0.6.0 |

Expand Down
Loading