demo/sentiment$ ./train.sh error

[hdulbj]

lbj@ubuntu:~/work/Paddle-master/demo/sentiment$ ./train.sh
I0910 20:52:11.219410 26997 Util.cpp:144] commandline: /usr/local/bin/../opt/paddle/bin/paddle_trainer --config=trainer_config.py --save_dir=./model_output --job=train --use_gpu=false --trainer_count=4 --num_passes=10 --log_period=10 --dot_period=20 --show_parameter_stats_period=100 --test_all_data_in_one_period=1
I0910 20:52:11.220432 26997 Util.cpp:113] Calling runInitFunctions
I0910 20:52:11.221349 26997 Util.cpp:126] Call runInitFunctions done.
[INFO 2016-09-10 20:52:12,141 networks.py:1122] The input order is [word, label]
[INFO 2016-09-10 20:52:12,142 networks.py:1129] The output order is [cost_0]
I0910 20:52:12.241542 26997 Trainer.cpp:169] trainer mode: Normal
I0910 20:52:13.312613 26997 PyDataProvider2.cpp:219] loading dataprovider dataprovider::process
[INFO 2016-09-10 20:52:13,584 dataprovider.py:22] dict len : 101745
I0910 20:52:13.779626 26997 PyDataProvider2.cpp:219] loading dataprovider dataprovider::process
[INFO 2016-09-10 20:52:13,780 dataprovider.py:22] dict len : 101745
I0910 20:52:13.781951 26997 GradientMachine.cpp:134] Initing parameters..
I0910 20:52:14.922147 26997 GradientMachine.cpp:141] Init parameters done.
I0910 20:52:25.065448 27001 ThreadLocal.cpp:39] thread use undeterministic rand seed:27002
I0910 20:52:25.857774 26999 ThreadLocal.cpp:39] thread use undeterministic rand seed:27000
I0910 20:52:26.466917 26998 ThreadLocal.cpp:39] thread use undeterministic rand seed:26999
I0910 20:52:26.739195 27000 ThreadLocal.cpp:39] thread use undeterministic rand seed:27001
/usr/local/bin/paddle: line 81: 26997 Killed ${DEBUGGER} $MYDIR/../opt/paddle/bin/paddle_trainer ${@:2}

[qingqing01]

This problem is similar to #63, please try that solutions. You can set pool_size as follows in dataprovider.py to reduce data cache memory.

@provider(init_hook=hook, pool_size=1024)

[hdulbj]

I did what you advised ,but it still has problems
lbj@ubuntu:~/work/Paddle-master/demo/sentiment$ ./train.sh
I0912 07:26:11.295408 4219 Util.cpp:144] commandline: /usr/local/bin/../opt/paddle/bin/paddle_trainer --config=trainer_config.py --save_dir=./model_output --job=train --use_gpu=false --trainer_count=4 --num_passes=10 --log_period=10 --dot_period=20 --show_parameter_stats_period=100 --test_all_data_in_one_period=1
I0912 07:26:11.296186 4219 Util.cpp:113] Calling runInitFunctions
I0912 07:26:11.296751 4219 Util.cpp:126] Call runInitFunctions done.
[INFO 2016-09-12 07:26:12,275 networks.py:1122] The input order is [word, label]
[INFO 2016-09-12 07:26:12,275 networks.py:1129] The output order is [cost_0]
I0912 07:26:12.353705 4219 Trainer.cpp:169] trainer mode: Normal
I0912 07:26:13.966311 4219 PyDataProvider2.cpp:219] loading dataprovider dataprovider::process
[INFO 2016-09-12 07:26:14,002 dataprovider.py:22] dict len : 101745
I0912 07:26:14.200013 4219 PyDataProvider2.cpp:219] loading dataprovider dataprovider::process
[INFO 2016-09-12 07:26:14,218 dataprovider.py:22] dict len : 101745
I0912 07:26:14.218703 4219 GradientMachine.cpp:134] Initing parameters..
I0912 07:26:15.289685 4219 GradientMachine.cpp:141] Init parameters done.
I0912 07:26:19.343603 4220 ThreadLocal.cpp:39] thread use undeterministic rand seed:4221
I0912 07:26:19.514636 4221 ThreadLocal.cpp:39] thread use undeterministic rand seed:4222
I0912 07:26:19.536335 4222 ThreadLocal.cpp:39] thread use undeterministic rand seed:4223
I0912 07:26:19.704776 4223 ThreadLocal.cpp:39] thread use undeterministic rand seed:4224

I0912 07:29:23.252763 4219 TrainerInternal.cpp:162] Batch=10 samples=1280 AvgCost=0.692175 CurrentCost=0.692175 Eval: classification_error_evaluator=0.485156 CurrentEval: classification_error_evaluator=0.485156

I0912 07:32:37.542174 4219 TrainerInternal.cpp:162] Batch=20 samples=2560 AvgCost=0.666661 CurrentCost=0.641147 Eval: classification_error_evaluator=0.401172 CurrentEval: classification_error_evaluator=0.317188

I0912 07:35:50.129344 4219 TrainerInternal.cpp:162] Batch=30 samples=3840 AvgCost=0.626617 CurrentCost=0.54653 Eval: classification_error_evaluator=0.342448 CurrentEval: classification_error_evaluator=0.225

I0912 07:38:51.197893 4219 TrainerInternal.cpp:162] Batch=40 samples=5120 AvgCost=0.585538 CurrentCost=0.462299 Eval: classification_error_evaluator=0.307422 CurrentEval: classification_error_evaluator=0.202344

I0912 07:41:56.019923 4219 TrainerInternal.cpp:162] Batch=50 samples=6400 AvgCost=0.551691 CurrentCost=0.416303 Eval: classification_error_evaluator=0.282031 CurrentEval: classification_error_evaluator=0.180469

I0912 07:44:51.390555 4219 TrainerInternal.cpp:162] Batch=60 samples=7680 AvgCost=0.525094 CurrentCost=0.392109 Eval: classification_error_evaluator=0.264844 CurrentEval: classification_error_evaluator=0.178906

I0912 07:47:44.667140 4219 TrainerInternal.cpp:162] Batch=70 samples=8960 AvgCost=0.502319 CurrentCost=0.365668 Eval: classification_error_evaluator=0.250112 CurrentEval: classification_error_evaluator=0.161719
/usr/local/bin/paddle: line 81: 4219 Killed ${DEBUGGER} $MYDIR/../opt/paddle/bin/paddle_trainer ${@:2}

[qingqing01]

@hdulbj please check your available memory size. If it is not enough, you can try to reduce dictionary size or batch size.