Skip to content

Commit

Permalink
Merge pull request #1226 from exdx/fix/catalog-polling-spec
Browse files Browse the repository at this point in the history 
fix: refactor catalog polling to use updateStrategy field
  • Loading branch information
@openshift-merge-robot
openshift-merge-robot authored Jan 14, 2020
2 parents 2187d3e + d75ca3b commit ffb7589
Show file tree
Hide file tree
Showing 13 changed files with 231 additions and 98 deletions.
26 changes: 16 additions & 10 deletions deploy/chart/templates/0000_50_olm_06-catalogsource.crd.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -97,16 +97,6 @@ spec:
a registry-server with. Only used when SourceType = SourceTypeGrpc.
If present, the address field is ignored.
type: string
poll:
description: Poll is used to determine the time interval between checks
of the latest catalog source version. The catalog operator polls to
see if a new version of the catalog source is available. If available,
the latest image is pulled and gRPC traffic is directed to the latest
catalog source.
type: object
properties:
interval:
type: string
publisher:
type: string
secrets:
Expand All @@ -119,6 +109,22 @@ spec:
sourceType:
description: SourceType is the type of source
type: string
updateStrategy:
description: UpdateStrategy defines how updated catalog source images
can be discovered Consists of an interval that defines polling duration
and an embedded strategy type
type: object
properties:
registryPoll:
type: object
properties:
interval:
description: Interval is used to determine the time interval
between checks of the latest catalog source version. The catalog
operator polls to see if a new version of the catalog source
is available. If available, the latest image is pulled and
gRPC traffic is directed to the latest catalog source.
type: string
status:
type: object
properties:
Expand Down
14 changes: 11 additions & 3 deletions doc/design/catalog-polling.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ This is the latest version of the X operator in the catalog source. Lets say Ope
to include the v2.0 version of the X operator and pushed to the same `master` tag. With catalog polling enabled, OLM will pull down the newer version
of the catalog source image and route service traffic to the newer pod. The existing subscription will seamlessly create the v2.0 operator and remove the old v1.0 one.

Each type of check for an updated catalog source is called an `updateStrategy`. Only one `updateStrategy` is supported at a time.
`registryPoll` is a type of `updateStrategy` that checks an image registry for an updated version of the same tag. The `interval` defines
the amount of time between each successive poll.

## Example Spec
Here is an example catalog source that polls `quay.io/my-catalogs/my-catalog:master` every 45 minutes to see if the image has been updated.

Expand All @@ -23,8 +27,9 @@ spec:
displayName: CatalogSource Test
sourceType: grpc
image: quay.io/my-catalogs/my-catalog:master
poll:
interval: 45m
updateStrategy:
registryPoll:
interval: 2m
```
It is required for the catalog source to be sourceType grpc and be backed by an image for polling to work.
Expand All @@ -33,4 +38,7 @@ It is required for the catalog source to be sourceType grpc and be backed by an
* The polling sequence is not instantaneous - it can take up to 15 minutes from each poll for the new catalog source pod to be deployed
into the cluster. It may take longer for larger clusters.
* Because OLM pulls down the image every poll interval and starts the pod, to see if its updated, the updated catalog pod must be able to be
scheduled onto the cluster. If the cluster is at absolutely maximum capacity, without autoscaling enabled, this feature may not work.
scheduled onto the cluster. If the cluster is at absolutely maximum capacity, without autoscaling enabled, this feature may not work.
* OLM checks to see whether the container ImageID has changed between the old and new catalog source image when determining if an upgrade
is in order. It does not actually parse the image content itself to check for later CSVs. If there is a bad upgrade to the catalog source image,
simply overwrite the tag with another version and it will be pulled down, or delete and recreate the catalog source.
26 changes: 16 additions & 10 deletions manifests/0000_50_olm_06-catalogsource.crd.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -97,16 +97,6 @@ spec:
a registry-server with. Only used when SourceType = SourceTypeGrpc.
If present, the address field is ignored.
type: string
poll:
description: Poll is used to determine the time interval between checks
of the latest catalog source version. The catalog operator polls to
see if a new version of the catalog source is available. If available,
the latest image is pulled and gRPC traffic is directed to the latest
catalog source.
type: object
properties:
interval:
type: string
publisher:
type: string
secrets:
Expand All @@ -119,6 +109,22 @@ spec:
sourceType:
description: SourceType is the type of source
type: string
updateStrategy:
description: UpdateStrategy defines how updated catalog source images
can be discovered Consists of an interval that defines polling duration
and an embedded strategy type
type: object
properties:
registryPoll:
type: object
properties:
interval:
description: Interval is used to determine the time interval
between checks of the latest catalog source version. The catalog
operator polls to see if a new version of the catalog source
is available. If available, the latest image is pulled and
gRPC traffic is directed to the latest catalog source.
type: string
status:
type: object
properties:
Expand Down
20 changes: 14 additions & 6 deletions pkg/api/apis/operators/catalogsource_types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,10 @@ type CatalogSourceSpec struct {
// +Optional
Image string

// Poll is used to determine the time interval between checks of the latest catalog source version.
// The catalog operator polls to see if a new version of the catalog source is available.
// If available, the latest image is pulled and gRPC traffic is directed to the latest catalog source.
// UpdateStrategy defines how updated catalog source images can be discovered
// Consists of an interval that defines polling duration and an embedded strategy type
// +Optional
Poll *Poll
UpdateStrategy *UpdateStrategy

// Secrets represent set of secrets that can be used to access the contents of the catalog.
// It is best to keep this list small, since each will need to be tried for every catalog entry.
Expand All @@ -65,8 +64,17 @@ type CatalogSourceSpec struct {
Icon Icon
}

type Poll struct {
Interval metav1.Duration
// UpdateStrategy holds all the different types of catalog source update strategies
// Currently only registry polling strategy is implemented
type UpdateStrategy struct {
*RegistryPoll
}

type RegistryPoll struct {
// Interval is used to determine the time interval between checks of the latest catalog source version.
// The catalog operator polls to see if a new version of the catalog source is available.
// If available, the latest image is pulled and gRPC traffic is directed to the latest catalog source.
Interval *metav1.Duration
}

type RegistryServiceStatus struct {
Expand Down
45 changes: 27 additions & 18 deletions pkg/api/apis/operators/v1alpha1/catalogsource_types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,10 @@ type CatalogSourceSpec struct {
// +Optional
Image string `json:"image,omitempty"`

// Poll is used to determine the time interval between checks of the latest catalog source version.
// The catalog operator polls to see if a new version of the catalog source is available.
// If available, the latest image is pulled and gRPC traffic is directed to the latest catalog source.
// UpdateStrategy defines how updated catalog source images can be discovered
// Consists of an interval that defines polling duration and an embedded strategy type
// +Optional
Poll *Poll `json:"poll,omitempty"`
UpdateStrategy *UpdateStrategy `json:"updateStrategy,omitempty"`

// Secrets represent set of secrets that can be used to access the contents of the catalog.
// It is best to keep this list small, since each will need to be tried for every catalog entry.
Expand All @@ -73,8 +72,17 @@ type CatalogSourceSpec struct {
Icon Icon `json:"icon,omitempty"`
}

type Poll struct {
Interval metav1.Duration `json:"interval,omitempty"`
// UpdateStrategy holds all the different types of catalog source update strategies
// Currently only registry polling strategy is implemented
type UpdateStrategy struct {
*RegistryPoll `json:"registryPoll,omitempty"`
}

type RegistryPoll struct {
// Interval is used to determine the time interval between checks of the latest catalog source version.
// The catalog operator polls to see if a new version of the catalog source is available.
// If available, the latest image is pulled and gRPC traffic is directed to the latest catalog source.
Interval *metav1.Duration `json:"interval,omitempty"`
}

type RegistryServiceStatus struct {
Expand Down Expand Up @@ -157,28 +165,26 @@ func (c *CatalogSource) SetLastUpdateTime() {
}

// Check if it is time to update based on polling setting
func (c *CatalogSource) ReadyToUpdate() bool {
if !c.PollingEnabled() {
func (c *CatalogSource) Update() bool {
if !c.Poll() {
return false
}
interval := c.Spec.Poll.Interval.Duration
logrus.WithField("CatalogSource", c.Name).Infof("polling interval %v", interval)
interval := c.Spec.UpdateStrategy.Interval.Duration
latest := c.Status.LatestImageRegistryPoll
if latest == nil {
logrus.WithField("CatalogSource", c.Name).Infof("latest poll %v", latest)
logrus.WithField("CatalogSource", c.Name).Debugf("latest poll %v", latest)
} else {
logrus.WithField("CatalogSource", c.Name).Infof("latest poll %v", *c.Status.LatestImageRegistryPoll)
logrus.WithField("CatalogSource", c.Name).Debugf("latest poll %v", *c.Status.LatestImageRegistryPoll)
}


logrus.WithField("CatalogSource", c.Name).Infof("polling interval is zero %t", c.Status.LatestImageRegistryPoll.IsZero())
if c.Status.LatestImageRegistryPoll.IsZero() {
logrus.WithField("CatalogSource", c.Name).Infof("creation interval plus interval before now %t", c.CreationTimestamp.Add(interval).Before(time.Now()))
logrus.WithField("CatalogSource", c.Name).Debugf("creation timestamp plus interval before now %t", c.CreationTimestamp.Add(interval).Before(time.Now()))
if c.CreationTimestamp.Add(interval).Before(time.Now()) {
return true
}
} else {
logrus.WithField("CatalogSource", c.Name).Infof("latest poll plus interval before now %t", c.Status.LatestImageRegistryPoll.Add(interval).Before(time.Now()))
logrus.WithField("CatalogSource", c.Name).Debugf("latest poll plus interval before now %t", c.Status.LatestImageRegistryPoll.Add(interval).Before(time.Now()))
if c.Status.LatestImageRegistryPoll.Add(interval).Before(time.Now()) {
return true
}
Expand All @@ -187,10 +193,13 @@ func (c *CatalogSource) ReadyToUpdate() bool {
return false
}

// CatalogPollingEnabled determines whether the polling feature is enabled on the particular catalog source
func (c *CatalogSource) PollingEnabled() bool {
// Poll determines whether the polling feature is enabled on the particular catalog source
func (c *CatalogSource) Poll() bool {
if c.Spec.UpdateStrategy == nil {
return false
}
// if polling interval is zero polling will not be done
if c.Spec.Poll == nil {
if c.Spec.UpdateStrategy.RegistryPoll == nil {
return false
}
// if catalog source is not backed by an image polling will not be done
Expand Down
35 changes: 25 additions & 10 deletions pkg/api/apis/operators/v1alpha1/types_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ func TestOwnsCRD(t *testing.T) {
}
}

func TestCatalogSource_ReadyToUpdate(t *testing.T) {
func TestCatalogSource_Update(t *testing.T) {
var table = []struct {
description string
catsrc CatalogSource
Expand All @@ -117,7 +117,11 @@ func TestCatalogSource_ReadyToUpdate(t *testing.T) {
catsrc: CatalogSource{
ObjectMeta: metav1.ObjectMeta{CreationTimestamp: metav1.Time{Time: time.Now()}},
Spec: CatalogSourceSpec{
Poll: &Poll{Interval: metav1.Duration{Duration: 1 * time.Second}},
UpdateStrategy: &UpdateStrategy{
RegistryPoll: &RegistryPoll{
Interval: &metav1.Duration{Duration: 1 * time.Second},
},
},
Image: "mycatsrcimage",
SourceType: SourceTypeGrpc},
},
Expand All @@ -129,7 +133,11 @@ func TestCatalogSource_ReadyToUpdate(t *testing.T) {
catsrc: CatalogSource{
ObjectMeta: metav1.ObjectMeta{CreationTimestamp: metav1.Time{Time: time.Now()}},
Spec: CatalogSourceSpec{
Poll: &Poll{Interval: metav1.Duration{Duration: 1 * time.Second}},
UpdateStrategy: &UpdateStrategy{
RegistryPoll: &RegistryPoll{
Interval: &metav1.Duration{Duration: 1 * time.Second},
},
},
Image: "mycatsrcimage",
SourceType: SourceTypeGrpc,
},
Expand All @@ -143,7 +151,11 @@ func TestCatalogSource_ReadyToUpdate(t *testing.T) {
catsrc: CatalogSource{
ObjectMeta: metav1.ObjectMeta{CreationTimestamp: metav1.Time{Time: time.Now()}},
Spec: CatalogSourceSpec{
Poll: &Poll{Interval: metav1.Duration{Duration: 1 * time.Second}},
UpdateStrategy: &UpdateStrategy{
RegistryPoll: &RegistryPoll{
Interval: &metav1.Duration{Duration: 1 * time.Second},
},
},
Image: "mycatsrcimage",
SourceType: SourceTypeGrpc,
},
Expand All @@ -156,11 +168,11 @@ func TestCatalogSource_ReadyToUpdate(t *testing.T) {

for i, tt := range table {
time.Sleep(table[i].sleep)
require.Equal(t, tt.result, table[i].catsrc.ReadyToUpdate(), table[i].description)
require.Equal(t, tt.result, table[i].catsrc.Update(), table[i].description)
}
}

func TestCatalogSource_PollingEnabled(t *testing.T) {
func TestCatalogSource_Poll(t *testing.T) {
var table = []struct {
description string
catsrc CatalogSource
Expand All @@ -182,14 +194,17 @@ func TestCatalogSource_PollingEnabled(t *testing.T) {
catsrc: CatalogSource{Spec: CatalogSourceSpec{
Image: "my-image",
SourceType: SourceTypeGrpc,
Poll: &Poll{Interval: metav1.Duration{
Duration: 1 * time.Minute,
}}},
UpdateStrategy: &UpdateStrategy{
RegistryPoll: &RegistryPoll{
Interval: &metav1.Duration{Duration: 1 * time.Second},
},
},
},
},
result: true,
},
}
for i, tt := range table {
require.Equal(t, tt.result, table[i].catsrc.PollingEnabled(), table[i].description)
require.Equal(t, tt.result, table[i].catsrc.Poll(), table[i].description)
}
}
Loading

0 comments on commit ffb7589

Please sign in to comment.