Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(datasource/docker): Add support for Google Application Default Credentials #23903

Merged
merged 32 commits into from
Sep 21, 2023
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
c2f0d28
Added support for Google Application Default Credentials
kvanzuijlen Aug 16, 2023
bb6cd70
PR feedback
kvanzuijlen Aug 17, 2023
72ff9dd
Ran prettier
kvanzuijlen Aug 17, 2023
595b659
Made regex more strict
kvanzuijlen Aug 18, 2023
3dd9a9f
Prettier
kvanzuijlen Aug 18, 2023
efac06a
Merge branch 'main' into google-adc-support
kvanzuijlen Aug 21, 2023
c1419e1
Fix for workload identity after testing against real repo
kvanzuijlen Aug 23, 2023
99d8195
Merge branch 'main' into google-adc-support
kvanzuijlen Aug 30, 2023
08fab55
Merge branch 'main' into google-adc-support
rarkins Aug 31, 2023
73199fb
Fixed pnpm lockfile
kvanzuijlen Aug 31, 2023
6ebcb5e
Merge remote-tracking branch 'origin/google-adc-support' into google-…
kvanzuijlen Aug 31, 2023
e8df7ae
Merge branch 'main' into google-adc-support
kvanzuijlen Aug 31, 2023
9043cf2
Merge branch 'main' into google-adc-support
kvanzuijlen Sep 1, 2023
761e583
Merge branch 'main' into google-adc-support
rarkins Sep 1, 2023
969d3df
Fixed pnpm lockfile + processed PR changes
kvanzuijlen Sep 4, 2023
9d91f2e
Merge branch 'main' into google-adc-support
kvanzuijlen Sep 4, 2023
373f905
Ran prettier
kvanzuijlen Sep 4, 2023
d61f190
Small improvement to implementation
kvanzuijlen Sep 5, 2023
dada3b2
Wrote tests
kvanzuijlen Sep 5, 2023
9f80158
Merge branch 'main' into google-adc-support
kvanzuijlen Sep 5, 2023
7af03c6
Merge branch 'main' into google-adc-support
kvanzuijlen Sep 12, 2023
03e7238
Rewrote tests
kvanzuijlen Sep 12, 2023
1b22bf4
Merge branch 'main' into google-adc-support
kvanzuijlen Sep 19, 2023
16b8ef7
Merge branch 'main' into google-adc-support
rarkins Sep 20, 2023
ffbc365
Ran prettier
kvanzuijlen Sep 20, 2023
e5a105f
Merge branch 'main' into google-adc-support
rarkins Sep 20, 2023
1f6d7d4
Merge branch 'main' into google-adc-support
kvanzuijlen Sep 20, 2023
82b2cf2
Merge branch 'main' into google-adc-support
kvanzuijlen Sep 20, 2023
c88db10
Apply suggestions from code review
kvanzuijlen Sep 21, 2023
bb20264
Ran prettier
kvanzuijlen Sep 21, 2023
9a0c31f
Update lib/modules/datasource/docker/index.spec.ts
viceice Sep 21, 2023
0bdd0b5
Update lib/modules/datasource/docker/index.spec.ts
viceice Sep 21, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/usage/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,13 @@ Renovate can authenticate with AWS ECR using AWS access key id & secret as the u

#### Google Container Registry / Google Artifact Registry

##### Using Application Default Credentials / Workload Identity

Just configure [ADC](https://cloud.google.com/docs/authentication/provide-credentials-adc) /
[Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) as normal and _don't_
provide a username, password or token. Renovate will automatically retrieve the credentials using the
google-auth-library.

##### Using long-lived service account credentials

To access the Google Container Registry (deprecated) or the Google Artifact Registry, use the JSON service account with `Basic` authentication, and use the:
Expand Down
18 changes: 18 additions & 0 deletions lib/modules/datasource/docker/common.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ import {
} from '../../../util/url';
import { api as dockerVersioning } from '../../versioning/docker';
import { ecrRegex, getECRAuthToken } from './ecr';
import { getGoogleAccessToken, googleRegex } from './google';
import type { OciHelmConfig } from './schema';
import type { RegistryRepository } from './types';

Expand Down Expand Up @@ -99,6 +100,23 @@ export async function getAuthHeaders(
if (auth) {
opts.headers = { authorization: `Basic ${auth}` };
}
} else if (
googleRegex.test(registryHost) &&
typeof opts.username === 'undefined' &&
typeof opts.password === 'undefined' &&
typeof opts.token === 'undefined'
) {
logger.trace(
{ registryHost, dockerRepository },
`Using google auth for Docker registry`
);
const accessToken = await getGoogleAccessToken();
if (accessToken) {
const auth = Buffer.from(
`${'oauth2accesstoken'}:${accessToken}`
).toString('base64');
opts.headers = { authorization: `Basic ${auth}` };
}
} else if (opts.username && opts.password) {
logger.trace(
{ registryHost, dockerRepository },
Expand Down
24 changes: 24 additions & 0 deletions lib/modules/datasource/docker/google.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import { GoogleAuth } from 'google-auth-library';
import { logger } from '../../../logger';
import { regEx } from '../../../util/regex';
import { addSecretForSanitizing } from '../../../util/sanitize';

export const googleRegex = regEx(
/(((eu|us|asia)\.)?gcr\.io|[a-z0-9-]+-docker\.pkg\.dev)/
);

export async function getGoogleAccessToken(): Promise<string | null> {
const googleAuth: GoogleAuth = new GoogleAuth({
scopes: 'https://www.googleapis.com/auth/cloud-platform',
});
const accessToken = await googleAuth.getAccessToken();
if (accessToken) {
// sanitize token
addSecretForSanitizing(accessToken);
return accessToken;
}
logger.warn(
'Could not retrieve access token using google-auth-library getAccessToken'
);
return null;
}
219 changes: 219 additions & 0 deletions lib/modules/datasource/docker/index.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import {
GetAuthorizationTokenCommandOutput,
} from '@aws-sdk/client-ecr';
import { mockClient } from 'aws-sdk-client-mock';
import * as _googleAuth from 'google-auth-library';
import { mockDeep } from 'jest-mock-extended';
import { getDigest, getPkgReleases } from '..';
import { range } from '../../../../lib/util/range';
Expand All @@ -14,14 +15,18 @@ import * as _hostRules from '../../../util/host-rules';
import { DockerDatasource } from '.';

const hostRules = mocked(_hostRules);
const googleAuth = mocked(_googleAuth);

jest.mock('../../../util/host-rules', () => mockDeep());
jest.mock('google-auth-library');

const ecrMock = mockClient(ECRClient);

const baseUrl = 'https://index.docker.io/v2';
const authUrl = 'https://auth.docker.io';
const amazonUrl = 'https://123456789.dkr.ecr.us-east-1.amazonaws.com/v2';
const gcrUrl = 'https://eu.gcr.io/v2';
const garUrl = 'https://europe-docker.pkg.dev/v2';
const dockerHubUrl = 'https://hub.docker.com/v2/repositories';

function mockEcrAuthResolve(
Expand Down Expand Up @@ -353,6 +358,220 @@ describe('modules/datasource/docker/index', () => {
expect(res).toBeNull();
});

it('supports Google ADC authentication for gcr', async () => {
httpMock
.scope(gcrUrl)
.get('/')
.reply(401, '', {
'www-authenticate': 'Basic realm="My Private Docker Registry Server"',
})
.head('/some-project/some-package/manifests/some-tag')
.matchHeader(
'authorization',
'Basic b2F1dGgyYWNjZXNzdG9rZW46c29tZS10b2tlbg=='
)
.reply(200, '', { 'docker-content-digest': 'some-digest' });

googleAuth.GoogleAuth.mockImplementationOnce(
jest.fn().mockImplementationOnce(() => ({
getAccessToken: jest.fn().mockResolvedValue('some-token'),
}))
);

hostRules.find.mockReturnValue({});
const res = await getDigest(
{
datasource: 'docker',
packageName: 'eu.gcr.io/some-project/some-package',
},
'some-tag'
);
expect(res).toBe('some-digest');
expect(googleAuth.GoogleAuth).toHaveBeenCalledTimes(1);
});

it('supports Google ADC authentication for gar', async () => {
httpMock
.scope(garUrl)
.get('/')
.reply(401, '', {
'www-authenticate': 'Basic realm="My Private Docker Registry Server"',
})
.head('/some-project/some-repo/some-package/manifests/some-tag')
.matchHeader(
'authorization',
'Basic b2F1dGgyYWNjZXNzdG9rZW46c29tZS10b2tlbg=='
)
.reply(200, '', { 'docker-content-digest': 'some-digest' });

googleAuth.GoogleAuth.mockImplementationOnce(
jest.fn().mockImplementationOnce(() => ({
getAccessToken: jest.fn().mockResolvedValue('some-token'),
}))
);

hostRules.find.mockReturnValue({});
const res = await getDigest(
{
datasource: 'docker',
packageName:
'europe-docker.pkg.dev/some-project/some-repo/some-package',
},
'some-tag'
);
expect(res).toBe('some-digest');
expect(googleAuth.GoogleAuth).toHaveBeenCalledTimes(1);
});

it('supports basic authentication for gcr', async () => {
httpMock
.scope(gcrUrl)
.get('/')
.reply(401, '', {
'www-authenticate': 'Basic realm="My Private Docker Registry Server"',
})
.head('/some-project/some-package/manifests/some-tag')
.matchHeader(
'authorization',
'Basic c29tZS11c2VybmFtZTpzb21lLXBhc3N3b3Jk'
)
.reply(200, '', { 'docker-content-digest': 'some-digest' });

googleAuth.GoogleAuth.mockImplementationOnce(
jest.fn().mockImplementationOnce(() => ({
getAccessToken: jest.fn().mockResolvedValue('some-token'),
}))
);

const res = await getDigest(
{
datasource: 'docker',
packageName: 'eu.gcr.io/some-project/some-package',
},
'some-tag'
);
expect(res).toBe('some-digest');
expect(googleAuth.GoogleAuth).toHaveBeenCalledTimes(0);
});

it('supports basic authentication for gar', async () => {
httpMock
.scope(garUrl)
.get('/')
.reply(401, '', {
'www-authenticate': 'Basic realm="My Private Docker Registry Server"',
})
.head('/some-project/some-repo/some-package/manifests/some-tag')
.matchHeader(
'authorization',
'Basic c29tZS11c2VybmFtZTpzb21lLXBhc3N3b3Jk'
)
.reply(200, '', { 'docker-content-digest': 'some-digest' });

googleAuth.GoogleAuth.mockImplementationOnce(
jest.fn().mockImplementationOnce(() => ({
getAccessToken: jest.fn().mockResolvedValue('some-token'),
}))
);

const res = await getDigest(
{
datasource: 'docker',
packageName:
'europe-docker.pkg.dev/some-project/some-repo/some-package',
},
'some-tag'
);
expect(res).toBe('some-digest');
expect(googleAuth.GoogleAuth).toHaveBeenCalledTimes(0);
});

it('supports public gcr', async () => {
httpMock
.scope(gcrUrl)
.get('/')
.reply(200, '', {
})
.head('/some-project/some-package/manifests/some-tag')
.reply(200, '', { 'docker-content-digest': 'some-digest' });

hostRules.find.mockReturnValue({});
const res = await getDigest(
{
datasource: 'docker',
packageName: 'eu.gcr.io/some-project/some-package',
},
'some-tag'
);
expect(res).toBe('some-digest');
expect(googleAuth.GoogleAuth).toHaveBeenCalledTimes(0);
});

it('supports public gar', async () => {
httpMock
.scope(garUrl)
.get('/')
.reply(200, '', {
})
.head('/some-project/some-repo/some-package/manifests/some-tag')
.reply(200, '', { 'docker-content-digest': 'some-digest' });

hostRules.find.mockReturnValue({});
const res = await getDigest(
{
datasource: 'docker',
packageName:
'europe-docker.pkg.dev/some-project/some-repo/some-package',
},
'some-tag'
);
expect(res).toBe('some-digest');
expect(googleAuth.GoogleAuth).toHaveBeenCalledTimes(0);
});

it('continues without token if Google ADC fails for gcr', async () => {
hostRules.find.mockReturnValue({});
httpMock.scope(gcrUrl).get('/').reply(401, '', {
'www-authenticate': 'Basic realm="My Private Docker Registry Server"',
});
googleAuth.GoogleAuth.mockImplementationOnce(
jest.fn().mockImplementationOnce(() => ({
getAccessToken: jest.fn().mockResolvedValue(undefined),
}))
);
const res = await getDigest(
{
datasource: 'docker',
packageName: 'eu.gcr.io/some-project/some-package',
},
'some-tag'
);
expect(res).toBeNull();
expect(googleAuth.GoogleAuth).toHaveBeenCalledTimes(1);
});

it('continues without token if Google ADC fails for gar', async () => {
hostRules.find.mockReturnValue({});
httpMock.scope(garUrl).get('/').reply(401, '', {
'www-authenticate': 'Basic realm="My Private Docker Registry Server"',
});
googleAuth.GoogleAuth.mockImplementationOnce(
jest.fn().mockImplementationOnce(() => ({
getAccessToken: jest.fn().mockRejectedValue('some-error'),
}))
);
const res = await getDigest(
{
datasource: 'docker',
packageName:
'europe-docker.pkg.dev/some-project/some-repo/some-package',
},
'some-tag'
);
expect(res).toBeNull();
expect(googleAuth.GoogleAuth).toHaveBeenCalledTimes(1);
});

it('continues without token, when no header is present', async () => {
httpMock
.scope(baseUrl)
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,7 @@
"glob": "10.3.4",
"global-agent": "3.0.0",
"good-enough-parser": "1.1.23",
"google-auth-library": "9.0.0",
"got": "11.8.6",
"graph-data-structure": "3.3.0",
"handlebars": "4.7.8",
Expand Down
Loading