Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add metric headers #1503

Merged
merged 27 commits into from
Oct 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
49b7c1b
add metrics for user credential.
zhumin8 Aug 27, 2024
4b4cfa8
metrics for sa.
zhumin8 Sep 10, 2024
ea04051
metrics for ImpersonatedCredentials
zhumin8 Sep 10, 2024
62bcfe5
metrics for compute engine credentials.
zhumin8 Sep 12, 2024
a9b6327
remove redundant util method. fix ImpersonatedCredentials overiding e…
zhumin8 Sep 12, 2024
e05558f
create a CredentialType enum class. rm redundant getter from UserCred…
zhumin8 Sep 13, 2024
973218d
some test cleanups.
zhumin8 Sep 13, 2024
3222930
Merge branch 'main' into metrics
zhumin8 Sep 13, 2024
adca794
Merge branch 'main' into metrics
zhumin8 Sep 16, 2024
1c8dc27
Merge branch 'main' into metrics
zhumin8 Sep 25, 2024
8742cc7
change name CredentialType to CredentialTypeForMetrics.
zhumin8 Sep 27, 2024
dd90f83
add request type untracked.
zhumin8 Sep 27, 2024
fa2f3f5
add javadoc comment, rename for readability, rename UNKNOWN to DO_NOT…
zhumin8 Sep 27, 2024
58b6072
extract constants from metrics string.
zhumin8 Sep 30, 2024
cb6bb84
add case for request type unspecified and credenty type do not send.
zhumin8 Sep 30, 2024
225a42b
simplify getGoogleCredentialsMetricsHeader logic and deal with additi…
zhumin8 Sep 30, 2024
8fe4d71
extract helper function setMetricsHeader. Add tests. Add mockito-inline.
zhumin8 Sep 30, 2024
6c50e3c
remove mockito-inline.
zhumin8 Sep 30, 2024
d0b40f7
add comments. change DO_NOT_SEND label.
zhumin8 Sep 30, 2024
35e63b2
fix typo in name.
zhumin8 Oct 1, 2024
226e3e3
remove setMetricsCredentialType()
zhumin8 Oct 1, 2024
5242f34
review feedback: refactor, introduce shouldSendMetricsHeader flag.
zhumin8 Oct 1, 2024
daca76f
lint fix.
zhumin8 Oct 1, 2024
d1f66ee
Merge branch 'main' into metrics
zhumin8 Oct 1, 2024
670dd23
update test check condition to be consistent.
zhumin8 Oct 1, 2024
36cb40d
Merge branch 'main' into metrics
zhumin8 Oct 2, 2024
3df3035
Merge branch 'main' into metrics
zhumin8 Oct 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions credentials/java/com/google/auth/CredentialTypeForMetrics.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
/*
* Copyright 2024 Google LLC
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are
* met:
*
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following disclaimer
* in the documentation and/or other materials provided with the
* distribution.
*
* * Neither the name of Google LLC nor the names of its
* contributors may be used to endorse or promote products derived from
* this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

package com.google.auth;

/**
* Defines the different types of credentials that can be used for metrics.
*
* <p>Each credential type is associated with a label that is used for reporting purposes. Add new
* enum constant only when corresponding configs established.
*
* <p>Credentials with type {@code CredentialTypeForMetrics.DO_NOT_SEND} is default value for
* credential implementations that do not set type specifically. It is not expected to send metrics.
*
* <p>
*
* @see #getLabel()
*/
public enum CredentialTypeForMetrics {
USER_CREDENTIALS("u"),
SERVICE_ACCOUNT_CREDENTIALS_AT("sa"),
SERVICE_ACCOUNT_CREDENTIALS_JWT("jwt"),
VM_CREDENTIALS("mds"),
IMPERSONATED_CREDENTIALS("imp"),
DO_NOT_SEND("dns");

private String label;

private CredentialTypeForMetrics(String label) {
this.label = label;
}

public String getLabel() {
return label;
}
}
12 changes: 12 additions & 0 deletions credentials/java/com/google/auth/Credentials.java
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,18 @@ public String getUniverseDomain() throws IOException {
return GOOGLE_DEFAULT_UNIVERSE;
}

/**
* Gets the credential type used for internal metrics header.
*
* <p>The default is {@code CredentialTypeForMetrics.DO_NOT_SEND}. For a credential that is
* established to track for metrics, this default should be overridden.
*
* @return a enum value for credential type
*/
public CredentialTypeForMetrics getMetricsCredentialType() {
return CredentialTypeForMetrics.DO_NOT_SEND;
}

/**
* Get the current request metadata, refreshing tokens if required.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,12 @@
import com.google.api.client.http.HttpStatusCodes;
import com.google.api.client.json.JsonObjectParser;
import com.google.api.client.util.GenericData;
import com.google.auth.CredentialTypeForMetrics;
import com.google.auth.Credentials;
import com.google.auth.Retryable;
import com.google.auth.ServiceAccountSigner;
import com.google.auth.http.HttpTransportFactory;
import com.google.auth.oauth2.MetricsUtils.RequestType;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Joiner;
import com.google.common.base.MoreObjects.ToStringHelper;
Expand Down Expand Up @@ -133,7 +135,6 @@ public class ComputeEngineCredentials extends GoogleCredentials
*/
private ComputeEngineCredentials(ComputeEngineCredentials.Builder builder) {
super(builder);

this.transportFactory =
firstNonNull(
builder.getHttpTransportFactory(),
Expand All @@ -153,6 +154,11 @@ private ComputeEngineCredentials(ComputeEngineCredentials.Builder builder) {
}
}

@Override
public CredentialTypeForMetrics getMetricsCredentialType() {
return CredentialTypeForMetrics.VM_CREDENTIALS;
}

/** Clones the compute engine account with the specified scopes. */
@Override
public GoogleCredentials createScoped(Collection<String> newScopes) {
Expand Down Expand Up @@ -234,7 +240,8 @@ public String getUniverseDomain() throws IOException {
}

private String getUniverseDomainFromMetadata() throws IOException {
HttpResponse response = getMetadataResponse(getUniverseDomainUrl());
HttpResponse response =
getMetadataResponse(getUniverseDomainUrl(), RequestType.UNTRACKED, false);
int statusCode = response.getStatusCode();
if (statusCode == HttpStatusCodes.STATUS_CODE_NOT_FOUND) {
return Credentials.GOOGLE_DEFAULT_UNIVERSE;
Expand All @@ -260,7 +267,8 @@ private String getUniverseDomainFromMetadata() throws IOException {
/** Refresh the access token by getting it from the GCE metadata server */
@Override
public AccessToken refreshAccessToken() throws IOException {
HttpResponse response = getMetadataResponse(createTokenUrlWithScopes());
HttpResponse response =
getMetadataResponse(createTokenUrlWithScopes(), RequestType.ACCESS_TOKEN_REQUEST, true);
int statusCode = response.getStatusCode();
if (statusCode == HttpStatusCodes.STATUS_CODE_NOT_FOUND) {
throw new IOException(
Expand Down Expand Up @@ -325,7 +333,8 @@ public IdToken idTokenWithAudience(String targetAudience, List<IdTokenProvider.O
}
}
documentUrl.set("audience", targetAudience);
HttpResponse response = getMetadataResponse(documentUrl.toString());
HttpResponse response =
getMetadataResponse(documentUrl.toString(), RequestType.ID_TOKEN_REQUEST, true);
InputStream content = response.getContent();
if (content == null) {
throw new IOException("Empty content from metadata token server request.");
Expand All @@ -334,13 +343,21 @@ public IdToken idTokenWithAudience(String targetAudience, List<IdTokenProvider.O
return IdToken.create(rawToken);
}

private HttpResponse getMetadataResponse(String url) throws IOException {
private HttpResponse getMetadataResponse(
String url, RequestType requestType, boolean shouldSendMetricsHeader) throws IOException {
GenericUrl genericUrl = new GenericUrl(url);
HttpRequest request =
transportFactory.create().createRequestFactory().buildGetRequest(genericUrl);
JsonObjectParser parser = new JsonObjectParser(OAuth2Utils.JSON_FACTORY);
request.setParser(parser);
request.getHeaders().set(METADATA_FLAVOR, GOOGLE);
// do not send metric header for getUniverseDomain and getAccount
if (shouldSendMetricsHeader) {
MetricsUtils.setMetricsHeader(
request,
MetricsUtils.getGoogleCredentialsMetricsHeader(requestType, getMetricsCredentialType()));
}

request.setThrowExceptionOnExecuteError(false);
HttpResponse response;
try {
Expand Down Expand Up @@ -440,7 +457,10 @@ private static boolean pingComputeEngineMetadata(
transportFactory.create().createRequestFactory().buildGetRequest(tokenUrl);
request.setConnectTimeout(COMPUTE_PING_CONNECTION_TIMEOUT_MS);
request.getHeaders().set(METADATA_FLAVOR, GOOGLE);

MetricsUtils.setMetricsHeader(
request,
MetricsUtils.getGoogleCredentialsMetricsHeader(
RequestType.METADATA_SERVER_PING, CredentialTypeForMetrics.DO_NOT_SEND));
HttpResponse response = request.execute();
try {
// Internet providers can return a generic response to all requests, so it is necessary
Expand Down Expand Up @@ -588,7 +608,8 @@ public byte[] sign(byte[] toSign) {
}

private String getDefaultServiceAccount() throws IOException {
HttpResponse response = getMetadataResponse(getServiceAccountsUrl());
HttpResponse response =
getMetadataResponse(getServiceAccountsUrl(), RequestType.UNTRACKED, false);
int statusCode = response.getStatusCode();
if (statusCode == HttpStatusCodes.STATUS_CODE_NOT_FOUND) {
throw new IOException(
Expand Down
11 changes: 10 additions & 1 deletion oauth2_http/java/com/google/auth/oauth2/IamUtils.java
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,11 @@
import com.google.api.client.json.JsonObjectParser;
import com.google.api.client.util.ExponentialBackOff;
import com.google.api.client.util.GenericData;
import com.google.auth.CredentialTypeForMetrics;
import com.google.auth.Credentials;
import com.google.auth.ServiceAccountSigner;
import com.google.auth.http.HttpCredentialsAdapter;
import com.google.auth.oauth2.MetricsUtils.RequestType;
import com.google.common.io.BaseEncoding;
import java.io.IOException;
import java.io.InputStream;
Expand Down Expand Up @@ -178,6 +180,7 @@ private static String getSignature(
* @param transport transport used for building the HTTP request
* @param targetAudience the audience the issued ID token should include
* @param additionalFields additional fields to send in the IAM call
* @param credentialTypeForMetrics credential type for credential making this call
* @return IdToken issed to the serviceAccount
* @throws IOException if the IdToken cannot be issued.
* @see <a
Expand All @@ -189,7 +192,8 @@ static IdToken getIdToken(
HttpTransport transport,
String targetAudience,
boolean includeEmail,
Map<String, ?> additionalFields)
Map<String, ?> additionalFields,
CredentialTypeForMetrics credentialTypeForMetrics)
throws IOException {

String idTokenUrl = String.format(ID_TOKEN_URL_FORMAT, serviceAccountEmail);
Expand All @@ -211,6 +215,11 @@ static IdToken getIdToken(
request.setParser(parser);
request.setThrowExceptionOnExecuteError(false);

MetricsUtils.setMetricsHeader(
request,
MetricsUtils.getGoogleCredentialsMetricsHeader(
RequestType.ID_TOKEN_REQUEST, credentialTypeForMetrics));

HttpResponse response = request.execute();
int statusCode = response.getStatusCode();
if (statusCode >= 400 && statusCode < HttpStatusCodes.STATUS_CODE_SERVER_ERROR) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,11 @@
import com.google.api.client.http.json.JsonHttpContent;
import com.google.api.client.json.JsonObjectParser;
import com.google.api.client.util.GenericData;
import com.google.auth.CredentialTypeForMetrics;
import com.google.auth.ServiceAccountSigner;
import com.google.auth.http.HttpCredentialsAdapter;
import com.google.auth.http.HttpTransportFactory;
import com.google.auth.oauth2.MetricsUtils.RequestType;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.MoreObjects;
import com.google.common.collect.ImmutableMap;
Expand Down Expand Up @@ -406,7 +408,7 @@ static ImpersonatedCredentials fromJson(
.setSourceCredentials(sourceCredentials)
.setTargetPrincipal(targetPrincipal)
.setDelegates(delegates)
.setScopes(new ArrayList<String>())
.setScopes(new ArrayList<>())
.setLifetime(DEFAULT_LIFETIME_IN_SECONDS)
.setHttpTransportFactory(transportFactory)
.setQuotaProjectId(quotaProjectId)
Expand All @@ -431,6 +433,11 @@ public GoogleCredentials createScoped(Collection<String> scopes) {
.build();
}

@Override
public CredentialTypeForMetrics getMetricsCredentialType() {
return CredentialTypeForMetrics.IMPERSONATED_CREDENTIALS;
}

/**
* Clones the impersonated credentials with a new calendar.
*
Expand Down Expand Up @@ -508,6 +515,10 @@ public AccessToken refreshAccessToken() throws IOException {
HttpRequest request = requestFactory.buildPostRequest(url, requestContent);
adapter.initialize(request);
request.setParser(parser);
MetricsUtils.setMetricsHeader(
request,
MetricsUtils.getGoogleCredentialsMetricsHeader(
RequestType.ACCESS_TOKEN_REQUEST, getMetricsCredentialType()));

HttpResponse response = null;
try {
Expand Down Expand Up @@ -557,7 +568,8 @@ public IdToken idTokenWithAudience(String targetAudience, List<IdTokenProvider.O
transportFactory.create(),
targetAudience,
includeEmail,
ImmutableMap.of("delegates", this.delegates));
ImmutableMap.of("delegates", this.delegates),
getMetricsCredentialType());
}

@Override
Expand Down
49 changes: 49 additions & 0 deletions oauth2_http/java/com/google/auth/oauth2/MetricsUtils.java
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,16 @@

package com.google.auth.oauth2;

import com.google.api.client.http.HttpRequest;
import com.google.auth.CredentialTypeForMetrics;
import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;

class MetricsUtils {
static final String API_CLIENT_HEADER = "x-goog-api-client";
static final String CRED_TYPE = "cred-type";
static final String AUTH_REQUEST_TYPE = "auth-request-type";
private static final String authLibraryVersion = getAuthLibraryVersion();
private static final String javaLanguageVersion = System.getProperty("java.version");

Expand Down Expand Up @@ -67,4 +71,49 @@ private static String getAuthLibraryVersion() {
}
return version;
}

public enum RequestType {
ACCESS_TOKEN_REQUEST("at"),
ID_TOKEN_REQUEST("it"),
METADATA_SERVER_PING("mds"),
UNTRACKED("untracked");

private String label;

private RequestType(String label) {
this.label = label;
}

public String getLabel() {
return label;
}
}

/**
* Formulates metrics header string. Header string takes format: “gl-java/JAVA_VERSION
* auth/LIB_VERSION auth-request-type/REQUEST_TYPE cred-type/CREDENTIAL_TYPE”. "auth-request-type"
* and "cred-type" can be omitted.
*
* @param requestType Auth request type to be specified in metrics, omit when {@code
* RequestType.UNTRACKED}
* @param credentialTypeForMetrics Credential type to be included in metrics string, omit when
* {@code CredentialTypeForMetrics.DO_NOT_SEND}
* @return metrics header string to send
*/
static String getGoogleCredentialsMetricsHeader(
lqiu96 marked this conversation as resolved.
Show resolved Hide resolved
RequestType requestType, CredentialTypeForMetrics credentialTypeForMetrics) {
StringBuilder stringBuilder =
new StringBuilder(MetricsUtils.getLanguageAndAuthLibraryVersions());
if (requestType != RequestType.UNTRACKED) {
stringBuilder.append(String.format(" %s/%s", AUTH_REQUEST_TYPE, requestType.getLabel()));
}
if (credentialTypeForMetrics != CredentialTypeForMetrics.DO_NOT_SEND) {
stringBuilder.append(String.format(" %s/%s", CRED_TYPE, credentialTypeForMetrics.getLabel()));
}
return stringBuilder.toString();
}

static void setMetricsHeader(HttpRequest request, String metricsHeader) {
request.getHeaders().set(MetricsUtils.API_CLIENT_HEADER, metricsHeader);
}
}
Loading