⚠️ Serve catalog over HTTPS

[trgeiger]

This adds the ability to use HTTPS on the catalog server.

If the TLS cert and key filenames are provided via the relevant flags, then the server will be set to use HTTPS. If neither or only one of those options are provided, the server will use HTTP.

This also changes the http-external-address flag to just external-address, with the user no longer prepending the address with http or https.

• Should I add a check for if the user does have http or https in the address, and adjust it to the proper one depending on the presence of the key/cert?
• Should we add a duplicate flag --http-external-address for backwards compatibility and just map it to the new flag if used?

This PR also includes an overhaul of the organization of the manifests. Following the example of rukpak, cert-manager is used in the Makefile build targets so HTTPS is enabled in kind, but cert-manager is not a required dependency of catalogd. Instead, it's applied as an overlay which we target in the Makefile.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 48.84%. Comparing base (250e348) to head (e7133a6).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #263   +/-   ##
=======================================
  Coverage   48.84%   48.84%           
=======================================
  Files           8        8           
  Lines         434      434           
=======================================
  Hits          212      212           
  Misses        201      201           
  Partials       21       21           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[everettraven]

Out of scope for this PR, but seeing this I don't think we need these permissions on pods anymore. we don't use a pod to unpack the catalog contents anymore and probably forgot to come back and cleanup these permissions. I'll create an issue for this.

[joelanford]

As I mentioned in Slack, I didn't realize how complex the signature for this function would be when I suggested it. At this point, it seems like we might as well go one step further and setup the entire server as part of this function.

I think just adding the storeDir would give us all the info we need to do that?

Also some nit suggestions on naming and order of parameters.

• Put the manager parameter first. That aligns with other functions that configure/add stuff to the manager.
• Group storeDir and externalAddr together, since they are used to build the storage struct.
• Use certFile and keyFile (instead of cert and key to make it more clear that they are the filenames of the cert and key, not the cert and key themselves)
• Group listenAddr, certFile, and keyFile together since they are used to build the listener

Putting all that together, WDYT of this?

func AddCatalogServerToManager(mgr ctrl.Manager, storeDir, externalAddr, listenAddr, certFile, keyFile string) error

[everettraven]

If we are concerned about the size of the function signature, it can probably be reduced to something like:

type CatalogServerConfig struct {
...
}

func AddCatalogServerToManager(mgr ctrl.Manager, cfg CatalogServerConfig) (net.Listener, string, error) { 
... 
}

[everettraven]

LGTM - thanks @trgeiger . I'll hold merging for a bit to allow @joelanford to take another pass

[trgeiger]

Thanks, Bryce. Should we adjust anything in the readme's quickstart instructions or anywhere else in docs?

[everettraven]

Thanks, Bryce. Should we adjust anything in the readme's quickstart instructions or anywhere else in docs?

Ah, great question (and good catch). We should probably update the quickstart instructions for port-forwarding and the curl command for fetching catalog contents to reflect the new state with an HTTPS based server as opposed to HTTP. I think this information is also present in https://github.com/operator-framework/catalogd/blob/main/docs/fetching-catalog-contents.md

[trgeiger]

How should we handle changing the instructions, given that HTTP is still supported if no TLS keys are provided, and the cert-manager is now an overlay. Do we just add that information as caveats to the instructions? Should I detail pulling down the cert and providing it to curl? etc.

[everettraven]

How should we handle changing the instructions, given that HTTP is still supported if no TLS keys are provided, and the cert-manager is now an overlay. Do we just add that information as caveats to the instructions? Should I detail pulling down the cert and providing it to curl? etc.

I'm not too familiar with how the overlays work, but IIUC the default deployment (i.e what is released) is going to be including the cert-manager resources and enable TLS (please correct me if I am wrong). If that is the default I would document whatever the default is.

I would not document pulling down the cert and providing it to curl and instead document with curl -k to ignore the certs. We can include a disclaimer in the docs that this is only for demonstration purposes and that if they want to trust the certificates to follow some other documentation on how to do that (I'm sure there is some documentation on this that already exists)

[trgeiger]

I made a couple small edits to README.md and https://github.com/operator-framework/catalogd/blob/main/docs/fetching-catalog-contents.md, let me know what you think

[everettraven]

A couple nits on the docs edits but I won't hold this PR on those. Great work @trgeiger !

[trgeiger]

Those were quick fixes, just made em so no need to hold off for another issue/pr for those.