-
Notifications
You must be signed in to change notification settings - Fork 21
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: move how to add new implementation doc from atlassian
- Loading branch information
1 parent
7dfa048
commit f91067d
Showing
1 changed file
with
31 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
How to add a new concrete implementation of the event bus | ||
========================================================= | ||
|
||
Context | ||
------- | ||
|
||
Though the initial Open edX event bus was implemented using Kafka, the ultimate goal is for maintainers to be able to swap out different technologies like Redis or Pulsar. See `How to start using the Event bus <https://openedx.atlassian.net/wiki/spaces/AC/pages/3508699151>`_ for more information. Existing Concrete Implementations | ||
|
||
- `Kafka <https://github.com/openedx/event-bus-kafka>`_ | ||
- `Redis Streams <https://github.com/openedx/event-bus-redis>`_ | ||
|
||
Producing | ||
--------- | ||
|
||
There should be a producer class that inherits from `EventBusProducer <https://github.com/openedx/openedx-events/blob/cbb59f124ed84afacb9ec99baa82a86381370dcc/openedx_events/event_bus/__init__.py#L66>`_ in openedx-events. | ||
|
||
The defined ``send`` method is meant to be called from within a signal receiver in the producing service. | ||
|
||
Consuming | ||
--------- | ||
|
||
At a high level, the consumer should be a process that takes the signals and events from the broker and emits the signal with the event. There should be a consumer class that inherits from `EventBusConsumer <https://github.com/openedx/openedx-events/blob/06635f3642cee4020d6787df68bba694bd1233fe/openedx_events/event_bus/__init__.py#L127>`_ in openedx-events. | ||
|
||
The consumer class then needs to implement ``consume_indefinitely`` loop, which will stay running and listen to events as they come in. | ||
|
||
If you are doing this within Django, one thing to be aware of is Django will not do any of the automatic connection cleanup that it usually does per request. This means that if your database restarts while the ``consume_indefinitely`` loop is running, Django may try to hold on to a defunct connection. To address this, we have included an `utility function <../../openedx_events/tooling.py#L323>`_ in openedx-events which needs to called before processing any signal. Checkout `consumer.py <https://github.com/openedx/event-bus-redis/blob/main/edx_event_bus_redis/internal/consumer.py>`_ in event bus redis implementation. We’re also resetting the RequestCache in the utility function, and there may be later, more comprehensive changes.. | ||
|
||
Abstraction tickets | ||
------------------- | ||
|
||
The known remaining work for a fully abstracted event bus is captured in the `Abstraction tickets <https://github.com/orgs/edx/projects/11/views/4?filterQuery=label%3Aevent-bus+-status%3ADone+abstraction>`_ |