Skip to content
This repository has been archived by the owner on Jun 23, 2022. It is now read-only.

perf(slog): Use a special thread pool for slog write callback #568

Merged
merged 6 commits into from
Jul 24, 2020

Conversation

foreverneverer
Copy link
Contributor

@foreverneverer foreverneverer commented Jul 22, 2020

Write callback tasks of 'Shared-log' and 'Private log' are both in the thread pool THREAD_POOL_DEFAULT. However, when data disk (aka plog disk) has a high load(for example, when pegasus is running background compaction), the plog's callback will spend a long time flushing the data to disk, thus results in that shared log callback being blocked, which has a heavy impact on user writes latency.

This patch split the thread pools of 'shared log' and 'private log' callback, and add new thread pool to serve the shared log's callback, which achieve obvious optimization. See the result as follow (5 replica server, 3*30thread ycsb client, value_length=1KB):

Before:

P99

选区_308

P999

选区_309

After:

P99

选区_306

P999

选区_307

NOTE:

levy5307
levy5307 previously approved these changes Jul 22, 2020
@acelyc111 acelyc111 changed the title refactor: change the slog write callback pool perf(slog): Use a special thread pool for slog write callback Jul 23, 2020
acelyc111
acelyc111 previously approved these changes Jul 23, 2020
@foreverneverer foreverneverer dismissed stale reviews from acelyc111 and levy5307 via e7184c5 July 23, 2020 03:52
@neverchanje neverchanje added the type/performance Issues or PRs related to performacne label Jul 23, 2020
acelyc111
acelyc111 previously approved these changes Jul 23, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
2.2.0 type/config-change PR that made modification on configs, which should be noted in release note. type/performance Issues or PRs related to performacne
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants