Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

net/http: use copyBufPool in transferWriter.doBodyCopy() #57205

Closed
wants to merge 1 commit into from
Closed

net/http: use copyBufPool in transferWriter.doBodyCopy() #57205

wants to merge 1 commit into from

Conversation

mmatczuk
Copy link

@mmatczuk mmatczuk commented Dec 9, 2022 •
edited
Loading

This is a followup to CL 14177. It applies copyBufPool optimization to
transferWriter.doBodyCopy(). The function is used every time Request or
Response is written.

Without this patch for every Request and Response processed, if there is
a body, we need to allocate and GC a 32k buffer. This is quickly causing
GC pressure.

Fixes #57202

@gopherbot
Copy link
Contributor

This PR (HEAD: ba09cda) has been imported to Gerrit for code review.

Please visit https://go-review.googlesource.com/c/go/+/456435 to see it.

Tip: You can toggle comments from me using the comments slash command (e.g. /comments off)
See the Wiki page for more info

@gopherbot
Copy link
Contributor

Message from Gopher Robot:

Patch Set 1:

Congratulations on opening your first change. Thank you for your contribution!

Next steps:
A maintainer will review your change and provide feedback. See
https://go.dev/doc/contribute#review for more info and tips to get your
patch through code review.

Most changes in the Go project go through a few rounds of revision. This can be
surprising to people new to the project. The careful, iterative review process
is our way of helping mentor contributors and ensuring that their contributions
have a lasting impact.

During May-July and Nov-Jan the Go project is in a code freeze, during which
little code gets reviewed or merged. If a reviewer responds with a comment like
R=go1.11 or adds a tag like "wait-release", it means that this CL will be
reviewed as part of the next development cycle. See https://go.dev/s/release
for more details.

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@mmatczuk mmatczuk mentioned this pull request Dec 9, 2022
Closed
@gopherbot
Copy link
Contributor

Message from Damien Neil:

Patch Set 1:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from qiulaidongfeng:

Patch Set 1: Code-Review+1 Run-TryBot+1

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Gopher Robot:

Patch Set 1:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Gopher Robot:

Patch Set 1:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from [email protected]:

Patch Set 1:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Gopher Robot:

Patch Set 1: TryBot-Result-1

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from qiulaidongfeng:

Patch Set 1: -Run-TryBot

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@mmatczuk mmatczuk force-pushed the http_transfer_use_copy_buffer branch from ba09cda to 2e69bbe Compare November 2, 2023 09:19
@gopherbot
Copy link
Contributor

This PR (HEAD: 2e69bbe) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/456435.

Important tips:

  • Don't comment on this PR. All discussion takes place in Gerrit.
  • You need a Gmail or other Google account to log in to Gerrit.
  • To change your code in response to feedback:
    • Push a new commit to the branch used by your GitHub PR.
    • A new "patch set" will then appear in Gerrit.
    • Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
    • Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
    • Multiple commits in the PR will be squashed by GerritBot.
  • The title and description of the GitHub PR are used to construct the final commit message.
    • Edit these as needed via the GitHub web interface (not via Gerrit or git).
    • You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
  • See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

@gopherbot
Copy link
Contributor

Message from [email protected]:

Patch Set 2:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@mmatczuk mmatczuk force-pushed the http_transfer_use_copy_buffer branch from 2e69bbe to d02a78b Compare November 2, 2023 09:44
@gopherbot
Copy link
Contributor

This PR (HEAD: d02a78b) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/456435.

Important tips:

  • Don't comment on this PR. All discussion takes place in Gerrit.
  • You need a Gmail or other Google account to log in to Gerrit.
  • To change your code in response to feedback:
    • Push a new commit to the branch used by your GitHub PR.
    • A new "patch set" will then appear in Gerrit.
    • Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
    • Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
    • Multiple commits in the PR will be squashed by GerritBot.
  • The title and description of the GitHub PR are used to construct the final commit message.
    • Edit these as needed via the GitHub web interface (not via Gerrit or git).
    • You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
  • See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

@gopherbot
Copy link
Contributor

Message from qiulaidongfeng:

Patch Set 3: Code-Review+1 Run-TryBot+1

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Gopher Robot:

Patch Set 3:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Gopher Robot:

Patch Set 3: TryBot-Result+1

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from qiulaidongfeng:

Patch Set 3: -Run-TryBot

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Damien Neil:

Patch Set 3: Commit-Queue+1

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Go LUCI:

Patch Set 3:

Dry run: CV is trying the patch.

Bot data: {"action":"start","triggered_at":"2023-11-06T21:37:05Z","revision":"8acf506ea83aacccc18c87e2bb13debee573dbc5"}

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Damien Neil:

Patch Set 3: Code-Review+2

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Damien Neil:

Patch Set 3: -Commit-Queue

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Go LUCI:

Patch Set 3:

This CL has passed the run

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Go LUCI:

Patch Set 3: LUCI-TryBot-Result+1

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Bryan Mills:

Patch Set 3: Code-Review+1

(2 comments)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@mmatczuk mmatczuk force-pushed the http_transfer_use_copy_buffer branch from d02a78b to c167574 Compare November 8, 2023 09:33
@gopherbot
Copy link
Contributor

This PR (HEAD: c167574) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/456435.

Important tips:

  • Don't comment on this PR. All discussion takes place in Gerrit.
  • You need a Gmail or other Google account to log in to Gerrit.
  • To change your code in response to feedback:
    • Push a new commit to the branch used by your GitHub PR.
    • A new "patch set" will then appear in Gerrit.
    • Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
    • Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
    • Multiple commits in the PR will be squashed by GerritBot.
  • The title and description of the GitHub PR are used to construct the final commit message.
    • Edit these as needed via the GitHub web interface (not via Gerrit or git).
    • You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
  • See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

@gopherbot
Copy link
Contributor

Message from Bryan Mills:

Patch Set 4: Code-Review+1

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Michał Matczuk:

Patch Set 4:

(3 comments)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Bryan Mills:

Patch Set 4:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@mmatczuk
net/http: use copyBufPool in transferWriter.doBodyCopy()
908573c 
This is a followup to CL 14177. It applies copyBufPool optimization to
transferWriter.doBodyCopy(). The function is used every time Request or
Response is written.

Without this patch for every Request and Response processed, if there is
a body, we need to allocate and GC a 32k buffer. This is quickly causing
GC pressure.

Fixes golang#57202
@mmatczuk mmatczuk force-pushed the http_transfer_use_copy_buffer branch from c167574 to 908573c Compare November 9, 2023 09:37
@gopherbot
Copy link
Contributor

Message from Michał Matczuk:

Patch Set 5:

(2 comments)

Please don’t reply on this GitHub thread. Visit golang.org/cl/456435.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

This PR (HEAD: 908573c) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/456435.

Important tips:

  • Don't comment on this PR. All discussion takes place in Gerrit.
  • You need a Gmail or other Google account to log in to Gerrit.
  • To change your code in response to feedback:
    • Push a new commit to the branch used by your GitHub PR.
    • A new "patch set" will then appear in Gerrit.
    • Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
    • Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
    • Multiple commits in the PR will be squashed by GerritBot.
  • The title and description of the GitHub PR are used to construct the final commit message.
    • Edit these as needed via the GitHub web interface (not via Gerrit or git).
    • You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
  • See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

gopherbot pushed a commit that referenced this pull request Nov 9, 2023
@mmatczuk @neild
net/http: use copyBufPool in transferWriter.doBodyCopy()
96eeb45 
This is a followup to CL 14177. It applies copyBufPool optimization to
transferWriter.doBodyCopy(). The function is used every time Request or
Response is written.

Without this patch for every Request and Response processed, if there is
a body, we need to allocate and GC a 32k buffer. This is quickly causing
GC pressure.

Fixes #57202

Change-Id: I4c30e1737726ac8d9937846106efd02effbae300
GitHub-Last-Rev: 908573c
GitHub-Pull-Request: #57205
Reviewed-on: https://go-review.googlesource.com/c/go/+/456435
Reviewed-by: Damien Neil <[email protected]>
Reviewed-by: Bryan Mills <[email protected]>
TryBot-Result: Gopher Robot <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: qiulaidongfeng <[email protected]>
@gopherbot
Copy link
Contributor

This PR is being closed because golang.org/cl/456435 has been merged.

@gopherbot gopherbot closed this Nov 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

net/http: transferWriter.doBodyCopy() calls io.Copy() resulting in many buffer allocations
2 participants
@mmatczuk @gopherbot