Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High Performance StringSetAsync with response processing. #1164

Closed
ghetze opened this issue Jun 20, 2019 · 1 comment
Closed

High Performance StringSetAsync with response processing. #1164

ghetze opened this issue Jun 20, 2019 · 1 comment

Comments

@ghetze
Copy link

ghetze commented Jun 20, 2019

Hi.
I'm new to redis, and currently I am making some tests to see the performance when sending lots of records no a redis sentinel configuration.(1 master with one slave and 3 sentinels wich monitors the master).
The idea is to send 30 000 000 records to a redis db as fast as possible and then to wait for the result to see how many of them crashed or not.
The code I', using is :

private void TestPerformance()
        {
            int failedRecords = 0;
            for (int i = 1; i <= 30_000_000; i++)
            {
                try
                {
                    Task result = DB.StringSetAsync(i.ToString(), i);
                    result.ConfigureAwait(false);
                    result.ContinueWith(task =>
                    {
                        if (task.IsFaulted)
                            logger.Error("Task faulted on"+i);
                        if (task.Exception != null)
                            logger.Error("An exception occurred :" + task.Exception.InnerException.Message);
                        Interlocked.Increment(ref failedRecords);
                    });
                }
                catch (Exception ex)
                {
                    logger.Error("Error:" + ex.Message);
                }
            }
        }

Is there a faster way to send messages and after sending them to wait for the responses ?

NickCraver added a commit that referenced this issue Jan 18, 2022
In #1854 we switched to a `Thread` here to prevent the pile-up case, however under concurrent load this incurs a 10-11% penalty given backlogs aren't all that uncommon (due to the lock contention case). In .NET 6.0+, let's use the thread pool growth semantics to handle the stall case and use the cheaper `Task.Run()` approach.

Running the benchmarks from #1164 I was experimenting with on lock perf, I noticed a large perf sink in thread creation as a result of lock contention. For that benchmark as an example (high concurrent load) we're talking about a ~93 second -> ~84 second run reduction and similar wins in concurrent load tests.
@NickCraver
Copy link
Collaborator

Looks like this was resolved in App-vNext/Polly#659 - tidying up here!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants