[messagebus, bridge] Remove messagebus cross-cluster dependency

[geropl]

Description

This PR contains two complementary changes:

1. remove the messagebus shovel config, and all non-local exchanges (gitpod.ws) and use local exchanges instead (gitpod.ws.local)
2. change ws-manager-bridge so that bridges that are non-governing:
   a. still don't write updates to the DB,
   b. but do the same derivation of updates,
   c. and distributes those over their cluster's local messagebus

Note: This PR can be reviewed commit by commit!

Pain points

This polling and hashing feel quite expensive, especially in scenarios where things go haywire already (a lot of workspaces hanging in preparing). But I see no way around it until we pushed image-builder into workspace clusters (#7845). 🤷 Once that's done we should be able to just delete PreparingUpdateEmulator 🗑️ .

Deployment

This PR is meant to be a drop-in replacement for the current setup, which means:

behavior before and after should be identical (order of updates, state persisted to DB before update sent, except for cross-cluster stuff, etc.)
all currently running workspaces should behave as usual, e.g. keep running smoothly
rolling this out will require a messagebus restart, but apart from that behave like a regular deployment

ToDo:

• think about how to properly test this, ideally before on staging
• create a cleanup PR for the ops repo https://github.com/gitpod-io/ops/pull/666
• solve the "imagebuild in different cluster" case
  • w.r.t. instance updates: PreparingUpdateEmulator
  • w.r.t. image build logs solved in: Stream imagebuild logs directly from pods #7899

Related Issue(s)

Fixes #7468

How to test

cluster-local works as before

• start a workspace with devtools open, and watch the websocket messages that keep flowing in
• note the onInstaceUpdate notifications
• note how the workspace starts as expected

cross-cluster

We'll test this on staging as the setup is already there, and an alternative setup would eat too much time at the moment.

Release Notes

messagebus: remove cross-cluster dependency

Documentation

[codecov]

Codecov Report

Merging #7523 (3eacd33) into main (42d529c) will decrease coverage by 1.13%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #7523      +/-   ##
==========================================
- Coverage   12.31%   11.17%   -1.14%     
==========================================
  Files          20       18       -2     
  Lines        1161      993     -168     
==========================================
- Hits          143      111      -32     
+ Misses       1014      880     -134     
+ Partials        4        2       -2     

Flag	Coverage Δ
components-gitpod-cli-app	11.17% <ø> (ø)
components-local-app-app-darwin-amd64	?
components-local-app-app-darwin-arm64	?
components-local-app-app-linux-amd64	?
components-local-app-app-linux-arm64	?
components-local-app-app-windows-386	?
components-local-app-app-windows-amd64	?
components-local-app-app-windows-arm64	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
components/local-app/pkg/auth/auth.go
components/local-app/pkg/auth/pkce.go

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 42d529c...3eacd33. Read the comment docs.

[roboquat]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please ask for approval from geropl and additionally assign mrsimonemms after the PR has been reviewed.
You can assign the PR to them by writing /assign @mrsimonemms in a comment when ready.

Associated issue: #7468

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS
• components/gitpod-db/OWNERS
• components/gitpod-messagebus/OWNERS
• components/server/OWNERS
• components/ws-manager-bridge/OWNERS
• installer/OWNERS
• installer/pkg/components/ws-manager-bridge/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[geropl]

/hold

Don't merge before tomorrows deployment 🙂

Now: don't merge when I'm not prepared to monitor deployment. 🕵️‍♂️

[JanKoehnlein]

LGTM codewise. Let's merge and test it on staging

[geropl]

/unhold