Event to Topic relationship

[ormsbee]

In openedx/edx-platform#32249, we had a discussion about the notifications app that veered into message bus topic/event mapping questions that deserved its own separate conversation.

@robrap wrote:

Thanks @ormsbee. Without my getting lost in notifications design considerations, using existing events, new events, and the event bus all makes sense to me.

I've certainly read that only one service should be responsible for writing to any given topic. I'm not sure if the following question applies to this use case, but is this a useful rule or not? If it comes up, we'd need to determine if it makes sense to have a shared topic that any service could write to and is used as a single pipe into some other service, or if we have a service that subscribes to all of the topics that include some event, like USER_NOTIFICATION, and either responds immediately or writes it to a new consolidated topic. This is just an implementation detail about how to use the event bus, and isn't about whether or not to use the event bus.

@ormsbee replied:

I've certainly read that only one service should be responsible for writing to any given topic.

I think it's sane as long as it's either one writer with potentially many consumers, or many writers with one consumer. For auditing purposes, things like PII would also make sense to have many writers, one consumer.

If it comes up, we'd need to determine if it makes sense to have a shared topic that any service could write to and is used as a single pipe into some other service, or if we have a service that subscribes to all of the topics that include some event, like USER_NOTIFICATION, and either responds immediately or writes it to a new consolidated topic.

I would favor having a single topic for all user notifications to go to, partitioned by a hash of the user.

Apologies, but I haven't been following event type -> topic mapping discussions. I had thought that it would be based on the event types themselves, and not the sending or receiving apps. I don't want to derail this ticket into that discussion, but do you have a link to the ticket where that was hashed out? I vaguely recall skimming something a while back, but I can't seem to find it.

@robrap replied:

• When producing, you produce an event (with an event type) to a topic. Unfortunately, it is possible to produce an event to the wrong topic, but no one is likely to listen to it, so it will just die out once fixed.
• When consuming, the management command consumes from a topic. All event types on the topic will be converted to signals, and only those signals with receiver code will be handled. See https://openedx.atlassian.net/wiki/spaces/AC/pages/3508699151/How+to+start+using+the+Event+Bus#Consuming-the-signal.
  I wouldn't be able to point to specific issues around this except the recent [Discovery] Handle multiple lifecycle events on single event bus topic #78, which was around how to implement the capability of sending multiple event types to a single topic.

I think it's sane as long as it's either one writer with potentially many consumers, or many writers with one consumer.

I imagine that the many writers to one consumer pattern would have a definite initial consumer and purpose in mind. However, I'm not sure that a second consumer might not come along and say "Hey, that's the perfect set of events that I want for my own purpose." Would it be an issue for there to be a second consumer?

If we want to allow many producers producing to a single topic, I think the important point would be that both producers and consumers of that topic should have this shared understanding from the start, and it should be well documented. In other words, it would not be ok for a service to see an event that it could also fire, and just start firing it into a previously existing topic.