[v4.0.0] - Massive Simplification and Full Open Telemetry Integration

[houseofcat]

When I first started this project, visibility with consumers was my main contention at work. Prometheus was the primary acceptable solution in 2017/2018 other than APMs.

The landscape has vastly changed. Some of my designs are hard to work around being flexible using the IMetricsProvider. For the last couple of versions I have gotten OpenTelemetry to start working inside the contraints of IMetricsProvider but its messy bulky and I don't like it.

OpenTelemetry is an accepted standard now. I am switching to this through out the HouseofCat.RabbitMQ library because that's what I was working around in the past, there wasn't a standard.

I apologize to anyone who is still using Prometheus. Please stay on 3.2.x. It's very stable as is but I won't be supporting that core library using non-OpenTelemetry as there should not be a need to interchange with flavors of metrics going forward. In fact, Prometheus and OpenTelemetry are compatible with a OpenTelemetry to Prometheus exporter.

Issues Need Solving

1. Using Trace/Spans for Workflows is a must.
2. Publishing needs to be incorporated into OpenTelemetry.
   a. Need to incorporate proper Activity tagging and Activity Kind Producer.
3. Consumer needs to be incorporated into OpenTelemetry.
   a. Need to incorporate proper Activity tagging and Activity Kind Consumer.
4. Topologer needs to be incorporated into OpenTelemetry.
   a. Need to incorporate proper Activity tagging and Activity Kind Internal.
5. There needs to be ways for Developers to extend, alter, create their own spans in Workflows. A lot will be auto-wired but I will not strangle engineers to integrate how they want to or even go above and beyond what I propose.

Elephant in the Room

There are some uncomfortable contract changes I will most likely make in order to "align" with not just OpenTelemetry labeling but perhaps how we wrap payloads. The end result should be a much more pleasant and streamlined experience no matter what I choose to do. Continuing the initiative to roll out guides and examples.

Changes So Far

• Letter is being renamed Message.
  • Feedback I got was many people don't like it.
  • MessageId generation is automatic for convenience.
• LetterMetadata is being renamed Metadata.
• Metadata.Id renamed to Metadata.PayloadId to help indicate it's purpose with a comment indicating it's the identifier for your inner payload data.
• Headers have changed to be of type X-RD-* prefix.
• OpenTelemetry header traceparent is now baked into RabbitMQ headers.
• RoutingOptions is removed. They are now properties on Message.
• Envelope is removed. Properties on Message.
• BuildState has no auto-deserialization/statekey required.
• there is data and the ContentType now matches.
• All GlobalConsumerOptions / PipelineOptions are all gone.
  • It's now just ConsumerOptions and properties associated for how the consumer is used inside your application.
• Ability to disable Queue creation on startup for Consumers.
• All generics have reverted back to simplification of property and fields. Will make mutating/extending different but understand the trade off is simplification.
• ReceivedData is now ReceivedMessage.
• ReceivedMessage now will only auto deserialize ReceivedMessage.Message if it's ObjectType is IMessage.
  • The ContentType is application/json or application/msgpack (all read from the headers).
  • The data content length must also be greater than 0.
  • Should they fail to deserialize the flag FailedToDeserialize will be marked as true but will continue the flow of execution without throwing an exception.
• References to data byte[] now align with body byte[] which the RabbitMQ.Client uses.
• Cutting back on overlocking with SemaphoreSlim.
  • Removed from ChannelHost/ConnectionHost on using the Channel/Connection.
• IModel Channel is now directly accessible from IChannelHost in stead of a function.
  • Assignments are controlled by the Host objects themselves, so how they mutate is naturally way more single threaded. Especially with the AutoRecovery changes.
• IMetricsProvider is gone. OpenTelemetry is the path forward.
• ConsumerWorkflow will auto generate a RootSpan or ChildActiveSpan based on Telemetry traceparent header provided with your messages.
• ConsumerWorkflow deserialization/BuildState step no longer deserializes the inner payload. This means that you can put whatever you like in Workflows and use your first custom step to manage what to do with ReceivedMessage.Data as you see fit.
• All or nearly 100% of all byte[] are now ReadOnlyMemory<byte>.

[houseofcat]

#54