Disabled partitions implementation

[ztlpn]

• Shut down disabled partitions in controller_backend
• Skip disabled partitions in balancers
• Forbid partition replica reconfiguration commands for disabled topics/partitions
• Return appropriate error codes for kafka requests related to disabled partitions

Backports Required

• none - not a bug fix
• none - this is a backport
• none - issue does not exist in previous branches
• none - papercut/not impactful enough to backport
• v23.2.x
• v23.1.x
• v22.3.x

Release Notes

Features

• Added the ability do disable whole topics and specific topic partitions. Disabled partitions are ignored by Redpanda and producing/consuming to them is impossible. This is useful when only a handful of partitions prevent the cluster from becoming healthy (e.g. if the data in them is corrupted in a way that causes Redpanda to crash).

[mmaslankaprv]

nit: maybe we can add a method bool is_disabled(const model::ntp&) const in metadata_cache as i see this patter repeating all over the PR

[ztlpn]

It is a small optimization - we query the disabled set for the topic once and then use it, instead of looking up the topic disabled set for each partition.

[rockwotj]

We also need to disable reads and writes in the transform subsystem.

[ztlpn]

@rockwotj Can you give me some code pointers? Where should I insert those checks?

[vbotbuildovich]

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/41814#018c1366-e570-4c0b-ae04-d9def320f523

[rockwotj]

I only reviewed the last two commits, as I'm missing some context on the other commits here, but the transform changes LGTM.