Future of NACK

[Jarema]

This discussion is meant to align how the future of NACK should look like, and if there are NATS community members willing to contribute to make it happen.

Current state of NACK

NACK, while working fine with the set of features it has, is also missing some important ones that are mentioned quite often:

1. No reconcile loop Add periodic checking of stream/consumers #95
2. No support for KV and Object Store
3. Sometimes not clear defaults, or inheriting different defaults from the Server
4. Challenges in codebase with NACK working in a really declarative mode

New opportunities

As nats 2.11 will get Pedantic Mode for consumers and streams, and jsm will fully support it, it will become significantly easier to handle changes via any tooling that manages NATS resources in declarative way.

Scope

The proposed scope for changes is as follows:

• add reconcile loop that checks streams and consumers state
• support streams, consumers, kv and object store
• add support for ownership via metadata
• cleanup / rewrite of the NACK codebase if it makes things much easier thanks to both Pedantic Mode and how we interact and test k8s layer.

Please share your feedback!

[mweibel]

That would be great. As it is now, I'm not sure I can run NACK on a production cluster because it doesn't support reconciliation.

It would also be great to add admission webhooks for a few cases.
For example:

# Create a stream.
$ kubectl apply -f - <<EOF
apiVersion: jetstream.nats.io/v1beta2
kind: Stream
metadata:
  name: mystream
spec:
  name: mystream
  subjects: ["orders.*"]
  storage: file
  maxAge: 1h
  replicas: 1
EOF

# create same stream with another metadata.name
$ kubectl apply -f - <<EOF
apiVersion: jetstream.nats.io/v1beta2
kind: Stream
metadata:
  name: mystream2
spec:
  name: mystream
  subjects: ["orders.*"]
  storage: file
  maxAge: 1h
  replicas: 1
EOF

# delete one of them
$ kubectl delete stream mystream2

When you do this, you'll see that stream mystream has been deleted even though another Stream is still referencing it.

Using reconciliation the stream would get created again. However, in this case it might be more beneficial to deny creating another stream referencing the same stream name using an admission webhook.

The same applies to consumers and probably other custom resources of NACK too.

More of a general remark: I wonder if it makes sense to be able to specify the spec.name when there's already metadata.name. Same for consumers: does it make sense to be able to specify spec.durableName when metadata.name is already given by kubernetes?