Skip to content

feat: split read request thread pool - #782

Merged
Smityz merged 5 commits into
apache:masterfrom
Smityz:split-proc-formal
Aug 13, 2021
Merged

Smityz merged 5 commits into
apache:masterfrom
Smityz:split-proc-formal

Conversation

@Smityz

@Smityz Smityz commented Jul 16, 2021 •

Copy link
Copy Markdown
Contributor

Split read request into 2 threads pools.
get and ttluse one threads pool, and other read requests use another one

Related PR: XiaoMi/rdsn#856
Benchmark: benchmark.pdf

@acelyc111

Copy link
Copy Markdown
Member

@Smityz What's your motivation to do this work?

@Smityz Smityz added the type/config-change Added or modified configuration that should be noted on release note of new version. label Aug 9, 2021
@Smityz

Smityz commented Aug 9, 2021 •

Copy link
Copy Markdown
Contributor Author

@Smityz What's your motivation to do this work?

Slow query tasks like multiget/scan should be separated from the thread pool, otherwise get request will be blocked by queueing up

@neverchanje

neverchanje commented Aug 9, 2021 •

Copy link
Copy Markdown
Contributor

Imo separation of different types of operations into multiple pools doesn't essentially solve the problem... It only mitigates some cases...

Think of a complex workload that consists of huge single key get, and small span of multiget. How do y suppose to adjust the thread pools? Will you assign a set of threads separately for each operation?

@foreverneverer

Copy link
Copy Markdown
Contributor

Imo separation of different types of operations into multiple pools doesn't essentially solve the problem... It only mitigates some cases...

Think of a complex workload that consists of huge single key get, and small span of multiget. How do y suppose to adjust the thread pools? Will you assign a set of threads separately for each operation?

Yeah, right. Actually we have discussed the problem for different case. However, since the benchmark has showed the improvement, at least it doesn't become worse, we decide apply it and verify its online effect.

scan query and point query is different type for exec time in most case, so seperate them seem to be appropriate in theory.

@Smityz
Smityz merged commit bed7f4d into apache:master Aug 13, 2021
@hycdong hycdong mentioned this pull request Sep 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type/config-change Added or modified configuration that should be noted on release note of new version.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants