Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

内存消耗随pass数增长,可能存在内存管理问题 #310

Closed
jueljust opened this issue Nov 2, 2016 · 8 comments
Closed

内存消耗随pass数增长,可能存在内存管理问题 #310

jueljust opened this issue Nov 2, 2016 · 8 comments
Assignees
Labels

Comments

@jueljust
Copy link

jueljust commented Nov 2, 2016

demo/image_classification

pass=0 mem%=24.4% mapped: 8370660K writeable/private: 8036476K shared: 0K
pass=6 mem%=37.4%

如果不是,请提供内存消耗与pass配置数据的关系,方便计算需要的资源

测试运行在local模式下

@reyoung
Copy link
Collaborator

reyoung commented Nov 2, 2016

@jueljust 麻烦问下,这个mem%是怎么计算出来的呢?

@jueljust
Copy link
Author

jueljust commented Nov 2, 2016

@reyoung top显示的进程MEM%

mapped数据是 pmap -d {PID} | grep mapped 出来的结果

@gangliao
Copy link
Contributor

gangliao commented Nov 2, 2016

#311

@reyoung
Copy link
Collaborator

reyoung commented Nov 2, 2016

@jueljust Are you using CACHE_PASS_IN_MEM in your dataprovider?

@jueljust
Copy link
Author

jueljust commented Nov 2, 2016

@reyoung 我用的就是demo的image_classification 没有修改任何代码和配置
搜代码没搜到cache配置,应该就是用默认配置

原文
./image_classification/image_provider.py:61:@Provider(init_hook=hook)

没有显示指定cache配置

@reyoung
Copy link
Collaborator

reyoung commented Nov 2, 2016

@jueljust 那你是使用的GPU --use_gpu=1?

@jueljust
Copy link
Author

jueljust commented Nov 2, 2016

@reyoung --use_gpu=1 被注释,未显式配置--use_cpu
机器上没有独立显卡

paddle train \
--config=$config \
--dot_period=10 \
--log_period=100 \
--test_all_data_in_one_period=1 \
# --use_gpu=1 \
--trainer_count=1 \
--num_passes=200 \
--save_dir=$output \
2>&1 | tee $log

reyoung added a commit to reyoung/Paddle that referenced this issue Nov 2, 2016
* Forget to finishTestPeriod in testOnePeriod.
* Fix PaddlePaddle#318
* Protentially related to PaddlePaddle#310, PaddlePaddle#311
@reyoung reyoung added this to the Make Paddle Easier to Core. milestone Nov 10, 2016
@reyoung reyoung removed this from the Paddle Usability milestone May 3, 2017
@qingqing01
Copy link
Contributor

close via ##323

Meiyim pushed a commit to Meiyim/Paddle that referenced this issue May 21, 2021
…_mem_opt

Remove deprecated fluid.memory_optimize
thisjiang pushed a commit to thisjiang/Paddle that referenced this issue Oct 28, 2021
PaddlePaddle#310)

* add x86 schedule injective, enhance simplify(min/max/mod/var...), enhance vectorize loop

* add buffer align and fix ci problems

* fix code styles
gglin001 added a commit to graphcore/Paddle-fork that referenced this issue Mar 17, 2022
* fix linking with paddle_ipu
lizexu123 pushed a commit to lizexu123/Paddle that referenced this issue Feb 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants