Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add metric headers #1503

Merged
merged 27 commits into from
Oct 2, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
49b7c1b
add metrics for user credential.
zhumin8 Aug 27, 2024
4b4cfa8
metrics for sa.
zhumin8 Sep 10, 2024
ea04051
metrics for ImpersonatedCredentials
zhumin8 Sep 10, 2024
62bcfe5
metrics for compute engine credentials.
zhumin8 Sep 12, 2024
a9b6327
remove redundant util method. fix ImpersonatedCredentials overiding e…
zhumin8 Sep 12, 2024
e05558f
create a CredentialType enum class. rm redundant getter from UserCred…
zhumin8 Sep 13, 2024
973218d
some test cleanups.
zhumin8 Sep 13, 2024
3222930
Merge branch 'main' into metrics
zhumin8 Sep 13, 2024
adca794
Merge branch 'main' into metrics
zhumin8 Sep 16, 2024
1c8dc27
Merge branch 'main' into metrics
zhumin8 Sep 25, 2024
8742cc7
change name CredentialType to CredentialTypeForMetrics.
zhumin8 Sep 27, 2024
dd90f83
add request type untracked.
zhumin8 Sep 27, 2024
fa2f3f5
add javadoc comment, rename for readability, rename UNKNOWN to DO_NOT…
zhumin8 Sep 27, 2024
58b6072
extract constants from metrics string.
zhumin8 Sep 30, 2024
cb6bb84
add case for request type unspecified and credenty type do not send.
zhumin8 Sep 30, 2024
225a42b
simplify getGoogleCredentialsMetricsHeader logic and deal with additi…
zhumin8 Sep 30, 2024
8fe4d71
extract helper function setMetricsHeader. Add tests. Add mockito-inline.
zhumin8 Sep 30, 2024
6c50e3c
remove mockito-inline.
zhumin8 Sep 30, 2024
d0b40f7
add comments. change DO_NOT_SEND label.
zhumin8 Sep 30, 2024
35e63b2
fix typo in name.
zhumin8 Oct 1, 2024
226e3e3
remove setMetricsCredentialType()
zhumin8 Oct 1, 2024
5242f34
review feedback: refactor, introduce shouldSendMetricsHeader flag.
zhumin8 Oct 1, 2024
daca76f
lint fix.
zhumin8 Oct 1, 2024
d1f66ee
Merge branch 'main' into metrics
zhumin8 Oct 1, 2024
670dd23
update test check condition to be consistent.
zhumin8 Oct 1, 2024
36cb40d
Merge branch 'main' into metrics
zhumin8 Oct 2, 2024
3df3035
Merge branch 'main' into metrics
zhumin8 Oct 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions credentials/java/com/google/auth/CredentialType.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
/*
* Copyright 2024 Google LLC
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are
* met:
*
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following disclaimer
* in the documentation and/or other materials provided with the
* distribution.
*
* * Neither the name of Google LLC nor the names of its
* contributors may be used to endorse or promote products derived from
* this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

package com.google.auth;

public enum CredentialType {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since these are strictly defined by our metrics implementation, can we make that more clear everywhere?

For instance we might call this "CredentialNameForMetrics" (or something better than that).

That way we know that we can't add enum items without a definition that is established elsewhere, and there won't be Credentials that have a CredentialType of Unknown (when we actually do know the type, we just don't want to score it for metrics.)

USER_CREDENTIALS("u"),
SERVICE_ACCOUNT_CREDENTIALS_AT("sa"),
SERVICE_ACCOUNT_CREDENTIALS_JWT("jwt"),
VM_CREDENTIALS("mds"),
IMPERSONATED_CREDENTIALS("imp"),
UNKNOWN("unknown");

private String label;

private CredentialType(String label) {
this.label = label;
}

public String getLabel() {
return label;
}
}
10 changes: 10 additions & 0 deletions credentials/java/com/google/auth/Credentials.java
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ public abstract class Credentials implements Serializable {

public static final String GOOGLE_DEFAULT_UNIVERSE = "googleapis.com";

private CredentialType credentialType = CredentialType.UNKNOWN;
lqiu96 marked this conversation as resolved.
Show resolved Hide resolved

/**
* A constant string name describing the authentication technology.
*
Expand All @@ -70,6 +72,14 @@ public String getUniverseDomain() throws IOException {
return GOOGLE_DEFAULT_UNIVERSE;
}

public CredentialType getCredentialType() {
return this.credentialType;
}

public void setCredentialType(CredentialType credentialType) {
this.credentialType = credentialType;
}
lqiu96 marked this conversation as resolved.
Show resolved Hide resolved
lqiu96 marked this conversation as resolved.
Show resolved Hide resolved

/**
* Get the current request metadata, refreshing tokens if required.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,12 @@
import com.google.api.client.http.HttpStatusCodes;
import com.google.api.client.json.JsonObjectParser;
import com.google.api.client.util.GenericData;
import com.google.auth.CredentialType;
import com.google.auth.Credentials;
import com.google.auth.Retryable;
import com.google.auth.ServiceAccountSigner;
import com.google.auth.http.HttpTransportFactory;
import com.google.auth.oauth2.MetricsUtils.RequestType;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Joiner;
import com.google.common.base.MoreObjects.ToStringHelper;
Expand Down Expand Up @@ -133,7 +135,7 @@ public class ComputeEngineCredentials extends GoogleCredentials
*/
private ComputeEngineCredentials(ComputeEngineCredentials.Builder builder) {
super(builder);

this.setCredentialType(CredentialType.VM_CREDENTIALS);
this.transportFactory =
firstNonNull(
builder.getHttpTransportFactory(),
Expand Down Expand Up @@ -234,7 +236,7 @@ public String getUniverseDomain() throws IOException {
}

private String getUniverseDomainFromMetadata() throws IOException {
HttpResponse response = getMetadataResponse(getUniverseDomainUrl());
HttpResponse response = getMetadataResponse(getUniverseDomainUrl(), RequestType.UNSPECIFIED);
int statusCode = response.getStatusCode();
if (statusCode == HttpStatusCodes.STATUS_CODE_NOT_FOUND) {
return Credentials.GOOGLE_DEFAULT_UNIVERSE;
Expand All @@ -260,7 +262,8 @@ private String getUniverseDomainFromMetadata() throws IOException {
/** Refresh the access token by getting it from the GCE metadata server */
@Override
public AccessToken refreshAccessToken() throws IOException {
HttpResponse response = getMetadataResponse(createTokenUrlWithScopes());
HttpResponse response =
getMetadataResponse(createTokenUrlWithScopes(), RequestType.ACCESS_TOKEN_REQUEST);
int statusCode = response.getStatusCode();
if (statusCode == HttpStatusCodes.STATUS_CODE_NOT_FOUND) {
throw new IOException(
Expand Down Expand Up @@ -325,7 +328,8 @@ public IdToken idTokenWithAudience(String targetAudience, List<IdTokenProvider.O
}
}
documentUrl.set("audience", targetAudience);
HttpResponse response = getMetadataResponse(documentUrl.toString());
HttpResponse response =
getMetadataResponse(documentUrl.toString(), RequestType.ID_TOKEN_REQUEST);
InputStream content = response.getContent();
if (content == null) {
throw new IOException("Empty content from metadata token server request.");
Expand All @@ -334,13 +338,22 @@ public IdToken idTokenWithAudience(String targetAudience, List<IdTokenProvider.O
return IdToken.create(rawToken);
}

private HttpResponse getMetadataResponse(String url) throws IOException {
private HttpResponse getMetadataResponse(String url, RequestType requestType) throws IOException {
zhumin8 marked this conversation as resolved.
Show resolved Hide resolved
GenericUrl genericUrl = new GenericUrl(url);
HttpRequest request =
transportFactory.create().createRequestFactory().buildGetRequest(genericUrl);
JsonObjectParser parser = new JsonObjectParser(OAuth2Utils.JSON_FACTORY);
request.setParser(parser);
request.getHeaders().set(METADATA_FLAVOR, GOOGLE);
// do not send metric header for getUniverseDomain and getAccount
if (requestType != RequestType.UNSPECIFIED) {
lqiu96 marked this conversation as resolved.
Show resolved Hide resolved
request
.getHeaders()
.set(
MetricsUtils.API_CLIENT_HEADER,
MetricsUtils.getGoogleCredentialsMetricsHeader(requestType, getCredentialType()));
}

request.setThrowExceptionOnExecuteError(false);
HttpResponse response;
try {
Expand Down Expand Up @@ -440,7 +453,12 @@ private static boolean pingComputeEngineMetadata(
transportFactory.create().createRequestFactory().buildGetRequest(tokenUrl);
request.setConnectTimeout(COMPUTE_PING_CONNECTION_TIMEOUT_MS);
request.getHeaders().set(METADATA_FLAVOR, GOOGLE);

request
.getHeaders()
.set(
MetricsUtils.API_CLIENT_HEADER,
MetricsUtils.getGoogleCredentialsMetricsHeader(
RequestType.METADATA_SERVER_PIN, CredentialType.UNKNOWN));
HttpResponse response = request.execute();
try {
// Internet providers can return a generic response to all requests, so it is necessary
Expand Down Expand Up @@ -588,7 +606,7 @@ public byte[] sign(byte[] toSign) {
}

private String getDefaultServiceAccount() throws IOException {
HttpResponse response = getMetadataResponse(getServiceAccountsUrl());
HttpResponse response = getMetadataResponse(getServiceAccountsUrl(), RequestType.UNSPECIFIED);
int statusCode = response.getStatusCode();
if (statusCode == HttpStatusCodes.STATUS_CODE_NOT_FOUND) {
throw new IOException(
Expand Down
13 changes: 12 additions & 1 deletion oauth2_http/java/com/google/auth/oauth2/IamUtils.java
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,11 @@
import com.google.api.client.json.JsonObjectParser;
import com.google.api.client.util.ExponentialBackOff;
import com.google.api.client.util.GenericData;
import com.google.auth.CredentialType;
import com.google.auth.Credentials;
import com.google.auth.ServiceAccountSigner;
import com.google.auth.http.HttpCredentialsAdapter;
import com.google.auth.oauth2.MetricsUtils.RequestType;
import com.google.common.io.BaseEncoding;
import java.io.IOException;
import java.io.InputStream;
Expand Down Expand Up @@ -178,6 +180,7 @@ private static String getSignature(
* @param transport transport used for building the HTTP request
* @param targetAudience the audience the issued ID token should include
* @param additionalFields additional fields to send in the IAM call
* @param credentialType credential type for credential making this call
* @return IdToken issed to the serviceAccount
* @throws IOException if the IdToken cannot be issued.
* @see <a
Expand All @@ -189,7 +192,8 @@ static IdToken getIdToken(
HttpTransport transport,
String targetAudience,
boolean includeEmail,
Map<String, ?> additionalFields)
Map<String, ?> additionalFields,
CredentialType credentialType)
throws IOException {

String idTokenUrl = String.format(ID_TOKEN_URL_FORMAT, serviceAccountEmail);
Expand All @@ -211,6 +215,13 @@ static IdToken getIdToken(
request.setParser(parser);
request.setThrowExceptionOnExecuteError(false);

request
.getHeaders()
.set(
MetricsUtils.API_CLIENT_HEADER,
MetricsUtils.getGoogleCredentialsMetricsHeader(
RequestType.ID_TOKEN_REQUEST, credentialType));

HttpResponse response = request.execute();
int statusCode = response.getStatusCode();
if (statusCode >= 400 && statusCode < HttpStatusCodes.STATUS_CODE_SERVER_ERROR) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,11 @@
import com.google.api.client.http.json.JsonHttpContent;
import com.google.api.client.json.JsonObjectParser;
import com.google.api.client.util.GenericData;
import com.google.auth.CredentialType;
import com.google.auth.ServiceAccountSigner;
import com.google.auth.http.HttpCredentialsAdapter;
import com.google.auth.http.HttpTransportFactory;
import com.google.auth.oauth2.MetricsUtils.RequestType;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.MoreObjects;
import com.google.common.collect.ImmutableMap;
Expand Down Expand Up @@ -452,6 +454,7 @@ public ImpersonatedCredentials createWithCustomCalendar(Calendar calendar) {

private ImpersonatedCredentials(Builder builder) {
super(builder);
this.setCredentialType(CredentialType.IMPERSONATED_CREDENTIALS);
this.sourceCredentials = builder.getSourceCredentials();
this.targetPrincipal = builder.getTargetPrincipal();
this.delegates = builder.getDelegates();
Expand Down Expand Up @@ -508,6 +511,12 @@ public AccessToken refreshAccessToken() throws IOException {
HttpRequest request = requestFactory.buildPostRequest(url, requestContent);
adapter.initialize(request);
request.setParser(parser);
request
.getHeaders()
.set(
MetricsUtils.API_CLIENT_HEADER,
MetricsUtils.getGoogleCredentialsMetricsHeader(
RequestType.ACCESS_TOKEN_REQUEST, getCredentialType()));
zhumin8 marked this conversation as resolved.
Show resolved Hide resolved

HttpResponse response = null;
try {
Expand Down Expand Up @@ -557,7 +566,8 @@ public IdToken idTokenWithAudience(String targetAudience, List<IdTokenProvider.O
transportFactory.create(),
targetAudience,
includeEmail,
ImmutableMap.of("delegates", this.delegates));
ImmutableMap.of("delegates", this.delegates),
getCredentialType());
}

@Override
Expand Down
41 changes: 41 additions & 0 deletions oauth2_http/java/com/google/auth/oauth2/MetricsUtils.java
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@

package com.google.auth.oauth2;

import com.google.auth.CredentialType;
import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;
Expand Down Expand Up @@ -67,4 +68,44 @@ private static String getAuthLibraryVersion() {
}
return version;
}

public enum RequestType {
ACCESS_TOKEN_REQUEST("at"),
ID_TOKEN_REQUEST("it"),
METADATA_SERVER_PIN("mds"),
zhumin8 marked this conversation as resolved.
Show resolved Hide resolved
UNSPECIFIED("unspecified");

private String label;

private RequestType(String label) {
this.label = label;
}

public String getLabel() {
return label;
}
}

static String getGoogleCredentialsMetricsHeader(
lqiu96 marked this conversation as resolved.
Show resolved Hide resolved
RequestType requestType, CredentialType credentialType) {
if (requestType == RequestType.UNSPECIFIED) {
return String.format(
"%s %s/%s",
MetricsUtils.getLanguageAndAuthLibraryVersions(), "cred-type", credentialType.getLabel());
}
if (credentialType == CredentialType.UNKNOWN) {
return String.format(
"%s %s/%s",
MetricsUtils.getLanguageAndAuthLibraryVersions(),
"auth-request-type",
requestType.getLabel());
}
return String.format(
"%s %s/%s %s/%s",
MetricsUtils.getLanguageAndAuthLibraryVersions(),
"auth-request-type",
zhumin8 marked this conversation as resolved.
Show resolved Hide resolved
requestType.getLabel(),
"cred-type",
credentialType.getLabel());
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,13 @@
import com.google.api.client.util.GenericData;
import com.google.api.client.util.Joiner;
import com.google.api.client.util.Preconditions;
import com.google.auth.CredentialType;
import com.google.auth.Credentials;
import com.google.auth.RequestMetadataCallback;
import com.google.auth.ServiceAccountSigner;
import com.google.auth.http.AuthHttpConstants;
import com.google.auth.http.HttpTransportFactory;
import com.google.auth.oauth2.MetricsUtils.RequestType;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.MoreObjects.ToStringHelper;
import com.google.common.collect.ImmutableList;
Expand Down Expand Up @@ -505,6 +507,12 @@ public AccessToken refreshAccessToken() throws IOException {
HttpRequestFactory requestFactory = transportFactory.create().createRequestFactory();
HttpRequest request = requestFactory.buildPostRequest(new GenericUrl(tokenServerUri), content);

request
.getHeaders()
.set(
MetricsUtils.API_CLIENT_HEADER,
MetricsUtils.getGoogleCredentialsMetricsHeader(
RequestType.ACCESS_TOKEN_REQUEST, getCredentialType()));
if (this.defaultRetriesEnabled) {
request.setNumberOfRetries(OAuth2Utils.DEFAULT_NUMBER_OF_RETRIES);
} else {
Expand Down Expand Up @@ -586,6 +594,14 @@ private IdToken getIdTokenOauthEndpoint(String targetAudience) throws IOExceptio
UrlEncodedContent content = new UrlEncodedContent(tokenRequest);

HttpRequest request = buildIdTokenRequest(tokenServerUri, transportFactory, content);
// add metric header
request
.getHeaders()
.set(
MetricsUtils.API_CLIENT_HEADER,
MetricsUtils.getGoogleCredentialsMetricsHeader(
RequestType.ID_TOKEN_REQUEST, getCredentialType()));

HttpResponse httpResponse = executeRequest(request);

GenericData responseData = httpResponse.parseAs(GenericData.class);
Expand Down Expand Up @@ -1013,9 +1029,12 @@ private Map<String, List<String>> getRequestMetadataForGdu(URI uri) throws IOExc
// configured then use scopes to get access token.
if ((!createScopedRequired() && !useJwtAccessWithScope)
|| isConfiguredForDomainWideDelegation()) {
// assertion token flow
this.setCredentialType(CredentialType.SERVICE_ACCOUNT_CREDENTIALS_AT);
return super.getRequestMetadata(uri);
}

// self-signed JWT flow
this.setCredentialType(CredentialType.SERVICE_ACCOUNT_CREDENTIALS_JWT);
return getRequestMetadataWithSelfSignedJwt(uri);
}

Expand Down
Loading