Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Paddle Inference] Decrease peak memory #53930

Merged
merged 2 commits into from
May 19, 2023
Merged

[Paddle Inference] Decrease peak memory #53930

merged 2 commits into from
May 19, 2023

Conversation

zhoutianzi666
Copy link
Contributor

@zhoutianzi666 zhoutianzi666 commented May 18, 2023 •
edited
Loading

PR types

Others

PR changes

Others

Description

  • Decrease peak memory
  • 时刻保持一个簇里的DenseTensor共享一个buffer
  • 旧的内存共享：每个簇里只有一个DenseTensor，因而时时刻刻每个簇里只有一个底层的buffer。
  • 新的内存共享：每个簇里有m 个 DenseTensor，并且我们没有时时刻刻保证这m个DenseTensor共享一个buffer，尽管稳定下来后肯定会共享一个buffer。
  • 因此在新的内存共享下面我们要保证： 一个簇里面的m个 DenseTensor应该要 时时刻刻 共享 唯一的 一块buffer。
  • 这能很大程度上减少峰值显存。
  • vae模型测试：峰值内存：16.7G -> 14.3G
  • 另外一个模型测试：6650 MB ->6115 MB

@paddle-bot
Copy link

paddle-bot bot commented May 18, 2023

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@paddle-bot
Copy link

paddle-bot bot commented May 18, 2023

❌ The PR is not created using PR's template. You can refer to this Demo.
Please use PR's template, it helps save our maintainers' time so that more developers get helped.

jiweibo
jiweibo approved these changes May 19, 2023
View reviewed changes
Copy link
Contributor

@jiweibo jiweibo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jiweibo jiweibo merged commit eb193d8 into PaddlePaddle:develop May 19, 2023
@iamsonderr iamsonderr mentioned this pull request Jul 6, 2023
Merged
@zhoutianzi666 zhoutianzi666 changed the title [Paddle-TRT] Decrease peak memory [Paddle Inference] Decrease peak memory Aug 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jiweibo jiweibo jiweibo approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@zhoutianzi666 @jiweibo