Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MVP proposal #28

Open
wants to merge 80 commits into
base: master
Choose a base branch
from
Open

MVP proposal #28

wants to merge 80 commits into from

Conversation

lsfera
Copy link
Contributor

@lsfera lsfera commented Aug 1, 2024

This PR covers all basic usage I was able to think about.
Messages are always stored in PostgreSql jsonb data type, hence publisher process is constrained on json serialization.

Consuming messages is instead more flexible since we are able to read data as string, object, or stream.
The former require subscriber(s) to subscribe publication(s) through one of the prefixed ConsumesRaw( String<>|Object<>| Strings | Objects ) methods.
The generics version require the RawUrn attributes to decorate the POCO class( no body required here, since it's nothing but a placeholder). RawUrn Urn property will be used to constraint publication creation, providing the row filter where clause(*).
Since we need to conform to a common standard shared by producer and consumer, the only supported NamingPolicy for this use case is the AttributeNamingPolicy - must be used on publication side.

The plural versions ConsumesRawStrings and ConsumesRawObjects won't provide any filter on publication, thus providing a single message sink for all published data.

Last but not least the Consumes<> subscription method provides the hook to the generics IHandle interface.
jsonb message data are read as stream and desarialised as T using System.Text.Json.JsonSerializer through the (mandatory) JsonSerializerContext.

Message table creation is enforced at subscription time ( but can also be assured at publication time) and support a few customization on column names/size see related tests.

(*) Remark Publication and subscription are managed by subscriber process.

@lsfera lsfera mentioned this pull request Aug 1, 2024
@oskardudycz
Copy link
Collaborator

@lsfera, could you expand the requirement for URN mapping requirement? I think that I'm not fully following that part. Some examples showing the most basic and recommended way would help. I wouldn't like to have forced users to add marker attributes.

Also, would it be possible to break this PR into smaller bits? It'd be easier to follow and understand the whole reasoning.

@lsfera
Copy link
Contributor Author

lsfera commented Aug 19, 2024

@lsfera, could you expand the requirement for URN mapping requirement? I think that I'm not fully following that part. Some examples showing the most basic and recommended way would help. I wouldn't like to have forced users to add marker attributes.

URN mapping is required only if you want to limit interception of outboxed data to a narrower set.
In this case we provide several options.
You can:

  1. Decorate a class with one or more RawRoutedByUrnAttribute instances, either implementing the IMessageHandler<string> or the IMessageHandler<object> from IMessageHandler interface and binding through the ConsumeRaw[String|Object] accordingly.
    Publisher and subscriber must conform to URN type mapping;
  2. Decorate a class with one or more RawRoutedByStringAttribute instances, either implementing the IMessageHandler<string> or the IMessageHandler<object> from IMessageHandler interface and binding through the ConsumeRaw[String|Object] accordingly.
    Publisher and subscriber must conform to FQN type mapping;

None of the above requires deserialization from subscriber nor A POCO class - just a placeholder to enable attribute decoration.

In case you want to deserialize your object - e.g.: in process transformation - you need to provide specific decoration via the MessageRoutedByUrnAttribute and use the Consumes<> method.
As long as pub/sub agree on the INamingConvention you are free to use either FQNNamingPolicy or AttributeNamingPolicy.

I provided testing for almost any aspect discussed - they use the default (concise) configuration,
For a more verbose and complete options set see SubscriberWorker demo.

Let me know if I was clear enough.

Also, would it be possible to break this PR into smaller bits? It'd be easier to follow and understand the whole reasoning.

This is a bit harder.
If we agree on smaller chunks of work with shorter feedback loop, I can work on feature branches and provide more modular PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants