S3 configuration is not removed when relation is removed.

[Abuelodelanada]

Bug Description

When a relation between a coodinator (say Mimir) and S3 is removed, worker's charm config is not updated (by removing S3 config section) and they still in active/idle status instead of blocked + a meaningful message.

To Reproduce

1. Deploy a coordinator
2. Deploy a recommended amount of workers
3. Deploy S3 integrator
4. Relate coordinator to workers
5. Relate coordinator to S3
6. Verify everything is active/idle
7. Remove relation between S3 and coordinator
8. S3 config are still in worker's config and workers still in active/idle

Environment

.

Relevant log output

.

Additional context

No response

[PietroPasotti]

Huh.
This is probably because we never got around to implementing the 'shut down the workers when the coordinator becomes incoherent' behaviour, which was originally designed.

What's probably going on is:
coordinator runs __init__, realizes it is incoherent, refuses to process the event further.
the coordinator never updates the worker configs
the worker keeps executing (but the workload, presumably, goes down)
the worker only notices in the next update-status that the workload is down (assuming we have pebble checks implemented)

Likely, the solution is to not just return from the coordinator if incoherent, but stop the workers by dropping all config

[PietroPasotti]

I would expect the coordinator not to be still active/idle though. if it is, there's something else that's wrong