Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add proxy support for registry cache #246

Merged
merged 4 commits into from
Nov 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/usage/registry-cache/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,10 @@ The `providerConfig.caches[].secretReferenceName` is the name of the reference f
> [!NOTE]
> It is only possible to provide one set of credentials for one private upstream registry.

The `providerConfig.caches[].proxy.httpProxy` field represents the proxy server for HTTP connections which is used by the registry cache. It must include an `https://` or `http://` scheme.

The `providerConfig.caches[].proxy.httpsProxy` field represents the proxy server for HTTPS connections which is used by the registry cache. It must include an `https://` or `http://` scheme.

## Garbage Collection

When the registry cache receives a request for an image that is not present in its local store, it fetches the image from the upstream, returns it to the client and stores the image in the local store. The registry cache runs a scheduler that deletes images when their time to live (ttl) expires. When adding an image to the local store, the registry cache also adds a time to live for the image. The ttl defaults to `168h` (7 days) and is configurable. The garbage collection can be disabled by setting the ttl to `0s`. Requesting an image from the registry cache does not extend the time to live of the image. Hence, an image is always garbage collected from the registry cache store when its ttl expires.
Expand Down
57 changes: 57 additions & 0 deletions hack/api-reference/registry.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,49 @@ Defaults to 168h (7 days).</p>
</tr>
</tbody>
</table>
<h3 id="registry.extensions.gardener.cloud/v1alpha3.Proxy">Proxy
</h3>
<p>
(<em>Appears on:</em>
<a href="#registry.extensions.gardener.cloud/v1alpha3.RegistryCache">RegistryCache</a>)
</p>
<p>
<p>Proxy contains settings for a proxy used in the registry cache.</p>
</p>
<table>
<thead>
<tr>
<th>Field</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<code>httpProxy</code></br>
<em>
string
</em>
</td>
<td>
<em>(Optional)</em>
<p>HTTPProxy field represents the proxy server for HTTP connections which is used by the registry cache.</p>
</td>
</tr>
<tr>
<td>
<code>httpsProxy</code></br>
<em>
string
</em>
</td>
<td>
<em>(Optional)</em>
<p>HTTPSProxy field represents the proxy server for HTTPS connections which is used by the registry cache.</p>
</td>
</tr>
</tbody>
</table>
<h3 id="registry.extensions.gardener.cloud/v1alpha3.RegistryCache">RegistryCache
</h3>
<p>
Expand Down Expand Up @@ -129,6 +172,20 @@ string
<p>SecretReferenceName is the name of the reference for the Secret containing the upstream registry credentials.</p>
</td>
</tr>
<tr>
<td>
<code>proxy</code></br>
<em>
<a href="#registry.extensions.gardener.cloud/v1alpha3.Proxy">
Proxy
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>Proxy contains settings for a proxy used in the registry cache.</p>
</td>
</tr>
</tbody>
</table>
<h3 id="registry.extensions.gardener.cloud/v1alpha3.RegistryCacheStatus">RegistryCacheStatus
Expand Down
10 changes: 10 additions & 0 deletions pkg/apis/registry/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ type RegistryCache struct {
GarbageCollection *GarbageCollection
// SecretReferenceName is the name of the reference for the Secret containing the upstream registry credentials
SecretReferenceName *string
// Proxy contains settings for a proxy used in the registry cache.
Proxy *Proxy
}

// Volume contains settings for the registry cache volume.
Expand All @@ -58,6 +60,14 @@ type GarbageCollection struct {
TTL metav1.Duration
}

// Proxy contains settings for a proxy used in the registry cache.
type Proxy struct {
// HTTPProxy field represents the proxy server for HTTP connections which is used by the registry cache.
HTTPProxy *string
// HTTPSProxy field represents the proxy server for HTTPS connections which is used by the registry cache.
HTTPSProxy *string
}

var (
// DefaultTTL is the default time to live of a blob in the cache.
DefaultTTL = metav1.Duration{Duration: 7 * 24 * time.Hour}
Expand Down
13 changes: 13 additions & 0 deletions pkg/apis/registry/v1alpha3/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ type RegistryCache struct {
// SecretReferenceName is the name of the reference for the Secret containing the upstream registry credentials.
// +optional
SecretReferenceName *string `json:"secretReferenceName,omitempty"`
// Proxy contains settings for a proxy used in the registry cache.
// +optional
Proxy *Proxy `json:"proxy,omitempty"`
}

// Volume contains settings for the registry cache volume.
Expand All @@ -66,6 +69,16 @@ type GarbageCollection struct {
TTL metav1.Duration `json:"ttl"`
}

// Proxy contains settings for a proxy used in the registry cache.
type Proxy struct {
// HTTPProxy field represents the proxy server for HTTP connections which is used by the registry cache.
// +optional
HTTPProxy *string `json:"httpProxy,omitempty"`
// HTTPSProxy field represents the proxy server for HTTPS connections which is used by the registry cache.
// +optional
HTTPSProxy *string `json:"httpsProxy,omitempty"`
}

var (
// DefaultTTL is the default time to live of a blob in the cache.
DefaultTTL = metav1.Duration{Duration: 7 * 24 * time.Hour}
Expand Down
34 changes: 34 additions & 0 deletions pkg/apis/registry/v1alpha3/zz_generated.conversion.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

31 changes: 31 additions & 0 deletions pkg/apis/registry/v1alpha3/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions pkg/apis/registry/validation/validation.go
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,14 @@ func validateRegistryCache(cache registry.RegistryCache, fldPath *field.Path) fi
allErrs = append(allErrs, field.Invalid(fldPath.Child("garbageCollection").Child("ttl"), ttl.Duration.String(), "ttl must be a non-negative duration"))
}
}
if cache.Proxy != nil {
if cache.Proxy.HTTPProxy != nil {
allErrs = append(allErrs, ValidateURL(fldPath.Child("proxy").Child("httpProxy"), *cache.Proxy.HTTPProxy)...)
}
if cache.Proxy.HTTPSProxy != nil {
allErrs = append(allErrs, ValidateURL(fldPath.Child("proxy").Child("httpsProxy"), *cache.Proxy.HTTPSProxy)...)
}
}

return allErrs
}
Expand Down
39 changes: 39 additions & 0 deletions pkg/apis/registry/validation/validation_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,10 @@ var _ = Describe("Validation", func() {
api.RegistryCache{
Upstream: "my-registry.io:5000",
RemoteURL: ptr.To("http://my-registry.io:5000"),
Proxy: &api.Proxy{
HTTPProxy: ptr.To("http://127.0.0.1"),
HTTPSProxy: ptr.To("https://127.0.0.1:1234"),
},
},
api.RegistryCache{
Upstream: "quay.io",
Expand Down Expand Up @@ -224,6 +228,41 @@ var _ = Describe("Validation", func() {
})),
))
})

It("should deny invalid proxy config", func() {
registryConfig.Caches[0].Proxy = &api.Proxy{
HTTPProxy: ptr.To("10.10.10.10"),
HTTPSProxy: nil,
}
registryConfig.Caches = append(registryConfig.Caches,
api.RegistryCache{
Upstream: "my-registry.io",
Proxy: &api.Proxy{
HTTPProxy: nil,
HTTPSProxy: ptr.To("http://foo!bar"),
},
},
api.RegistryCache{
Upstream: "my-registry2.io",
Proxy: &api.Proxy{
HTTPProxy: nil,
HTTPSProxy: nil,
},
},
)
Expect(ValidateRegistryConfig(registryConfig, fldPath)).To(ConsistOf(
PointTo(MatchFields(IgnoreExtras, Fields{
"Type": Equal(field.ErrorTypeInvalid),
"Field": Equal("providerConfig.caches[0].proxy.httpProxy"),
"BadValue": Equal("10.10.10.10"),
})),
PointTo(MatchFields(IgnoreExtras, Fields{
"Type": Equal(field.ErrorTypeInvalid),
"Field": Equal("providerConfig.caches[1].proxy.httpsProxy"),
"BadValue": Equal("http://foo!bar"),
})),
))
})
})

Describe("#ValidateRegistryConfigUpdate", func() {
Expand Down
31 changes: 31 additions & 0 deletions pkg/apis/registry/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 15 additions & 0 deletions pkg/component/registrycaches/registry_caches.go
Original file line number Diff line number Diff line change
Expand Up @@ -380,6 +380,21 @@ func (r *registryCaches) computeResourcesDataForRegistryCache(ctx context.Contex
},
}

if cache.Proxy != nil {
if cache.Proxy.HTTPProxy != nil {
statefulSet.Spec.Template.Spec.Containers[0].Env = append(statefulSet.Spec.Template.Spec.Containers[0].Env, corev1.EnvVar{
Name: "HTTP_PROXY",
Value: *cache.Proxy.HTTPProxy,
})
}
if cache.Proxy.HTTPSProxy != nil {
statefulSet.Spec.Template.Spec.Containers[0].Env = append(statefulSet.Spec.Template.Spec.Containers[0].Env, corev1.EnvVar{
Name: "HTTPS_PROXY",
Value: *cache.Proxy.HTTPSProxy,
})
}
}

var vpa *vpaautoscalingv1.VerticalPodAutoscaler
if r.values.VPAEnabled {
updateMode := vpaautoscalingv1.UpdateModeAuto
Expand Down
Loading