Outbox messages are store with defferent "EndpointName" and "ActualEndpointName"

[RomanOlegovich]

I expect "ActualEndpointName" equals "EndpointName" in the outbox table, but it's defferent (see screanschort).

Here outbound configuration

                        .AddOutbound<NotifyPaymentEvent>(endpoint => endpoint
                            .ProduceTo(nameof(NotifyPaymentEvent))
                            .WithKafkaKey<NotifyPaymentEvent>(envelope => envelope.Message.Id)
                            .Configure(producerConfig)
                            .SerializeAsJsonUsingNewtonsoft()
                            .ProduceToOutbox())

This seems to be the cause of the error:

fail: Silverback.Messaging.Outbound.TransactionalOutbox.OutboxWorker[1077]
      Failed to produce the message stored in the outbox. | endpointName: StudentUserPhoneDuplicateEvent, messageType: *project-name*.DrivingStartingEvent, *project-name*, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null, messageId: 1e563aad-8b99-4daa-a981-f667d507bb8a, unused1: (null), unused2: (null)
      System.InvalidOperationException: No endpoint with name 'StudentUserPhoneDuplicateEvent' could be found for a message of type '*project-name*.DrivingStartingEvent'.
         at Silverback.Messaging.Outbound.TransactionalOutbox.OutboxWorker.GetTargetEndpoint(Type messageType, String endpointName, IServiceProvider serviceProvider)
         at Silverback.Messaging.Outbound.TransactionalOutbox.OutboxWorker.ProcessMessageAsync(OutboxStoredMessage message, ConcurrentBag`1 failedMessages, IOutboxReader outboxReader, IServiceProvider serviceProvider)

[BEagle1984]

It looks like a possible bug, yes.

Can you please provide a full project that reproduce the issue? It's hard to figure it out without seeing the entire configuration.

[RomanOlegovich]

I wrote a message but forgot to send it.
To be more clarify.
StudentPhoneDuplicateEvent and NotifyPaymentEvent are generated in one CronJob, which do:

1. Generate list of StudentPhoneDuplicateEvent 
            await SyncStudents() => {
                foreach (var addEvent in addEvents)
                    await publisher.PublishAsync(addEvent);
                outBoxDbContext.SaveChanges();
            }

2.
Generate list of NotifyPaymentEvent 
            await SyncPayments() => {
                foreach (var addEvent in addEvents)
                    await publisher.PublishAsync(addEvent);
                outBoxDbContext.SaveChanges();
            }

[RomanOlegovich]

I can't provide full project, I'm not owner.
I can cut out all the pieces of code related to Silverback and paste here 😃.

If you need a full project that reproduce the issue, it will take time

[BEagle1984]

It doesn't have to be the same project. Anything I can run to reproduce the issue would be enough. If I can't debug it is gonna be difficult to fix it.

[davimorao]

Hi, I encountered the same problem in my tests.

When sending messages of different types with ICommandPublisher or IEventPublisher in the same process, the endpoint of the first message will be replicated for the next messages in the EndpointName field when calling DbContext.SaveChangesAsync() to persist the information.

Follow the steps to reproduce:

Code:
await _command.ExecuteAsync(new ExecuteSampleCommand { Guid = Guid.NewGuid(), TransactionId = Guid.NewGuid(), Sample = new ExecuteSampleCommand.SampleDto { Name = "Test", SampleStatus = ESampleStatus.SampleStatusTwo, InternalId = 1 } });
await kafkaManagementDbContext.SaveChangesAsync();

await _command.ExecuteAsync(new ExecuteSampleCommand2 { Guid = Guid.NewGuid(), TransactionId = Guid.NewGuid(), Sample = new ExecuteSampleCommand2.SampleDto { Name = "Test", InternalId = 1 } });
await kafkaManagementDbContext.SaveChangesAsync();

The error occurs according to the code above, and will also occur if the commit in the DbContext after sending the first and second message

Database

[BEagle1984]

Can you provide a working project that reproduces the issue?
I'd be glad to debug it.

[davimorao]

Here's the sample project, let me know if I need more info, thanks.

https://github.com/davimorao/poc-silverback-outbox

[alefcarlos]

@BEagle1984 are you good ?

Any news about that issue ? Or any advice how could we workaround that behaviour?

[BEagle1984]

I unfortunately didn't have time to debug this issue yet.

An obvious workaround would be to not use the outbox but I guess it's not an option for you.

[alefcarlos]

@BEagle1984 , dont you use outbox in production ?

[BEagle1984]

Not me personally, as I don't have the need in my current projects. But some colleagues do and I never heard of this issue. Maybe different version, different circumstances, etc.

[cookie-bytes]

In OutboxProduceStrategyImplementation I think trying to cache the IProducer is causing the issue.
If I change "ProduceAsync" to the following the endpoint is configured correctly in the outbox table.

        public Task ProduceAsync(IOutboundEnvelope envelope)
        {
            Check.NotNull(envelope, nameof(envelope));

            _logger.LogWrittenToOutbox(envelope);

            if (_producer == null || _producer.Endpoint.Name != envelope.Endpoint.Name)
            {
                _producer = _outboundQueueBroker.GetProducer(envelope.Endpoint);
            }
            return _producer.ProduceAsync(envelope);
        }

Does this fix sound reasonable?

Thanks

[BEagle1984]

I'm very sorry, I didn't have time yet to look deeply into the issue.

Your explanation makes sense and the fix seems indeed reasonable. Pack it into a PR. 🙂

Thanks a lot! 😎

[cookie-bytes]

@BEagle1984 I have created a PR, but the pipeline is failing on a test. I can't get this test to fail locally. Any ideas?

https://github.com/BEagle1984/silverback/pull/188/checks?check_run_id=11459889363

[BEagle1984]

It's ok, it's probably flaky on the pipeline. I'm trying to make the tests more reliable but most of the improvements have been implemented for the upcoming big release and not back ported to the current one (in master). I'm checking your pr.

[BEagle1984]

I'm still trying to figure out what happened with the test but I uploaded a version (v4.1.1-beta.1) to nuget.org which includes this PR.
Do you guys have the possibility to test it?

[cookie-bytes]

Yes, we will update to the new package tomorrow and start our tests. Thank you

[davimorao]

Here's the sample project, let me know if I need more info, thanks.

https://github.com/davimorao/poc-silverback-outbox

Hello everyone, I did a test regarding the scenario of this example and messages of different types were stored, sent and consumed correctly.

[alefcarlos]

@BEagle1984 when are you planning to release the final version of 4.1.1 ? Currently its 4.1.1-beta.1.

[BEagle1984]

I will release this weekend or at the beginning of next week. Did you try the beta @alefcarlos ?

[alefcarlos]

@BEagle1984, @davimorao and I work together rs

We tried the beta and everything worked good!

@cookie-bytes did you try that?

[cookie-bytes]

Yes, tested and working as expected. It has not been through performance testing though.

[BEagle1984]

Published v4.1.1 including the fix