Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

all passes‘ cost is 0 besides the first pass #2581

Closed
Damon-wyg opened this issue Jun 23, 2017 · 1 comment
Closed

all passes‘ cost is 0 besides the first pass #2581

Damon-wyg opened this issue Jun 23, 2017 · 1 comment
Assignees

Comments

@Damon-wyg
Copy link

Damon-wyg commented Jun 23, 2017

I train a dnn model with cluster paddle,but all pass cost is zero,checked the data which is ok, and local running can get normal cost。As follows:
[RIG_NOTICE] log_time=1498196443 cost=0.006403 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498196793 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498197141 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498197488 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498197834 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498198182 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498198529 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498198878 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498199227 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498199577 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498199925 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498200272 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498200619 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498200967 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498201316 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498201665 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498202013 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498202360 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498202704 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498203052 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498203401 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498203757 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498204118 cost=0.000000 job_id=job.38811.instances
[RIG_NOTICE] log_time=1498204478 cost=0.000000 job_id=job.38811.instances
don't understand why~~

@Damon-wyg
Copy link
Author

I've resolved it, which is caused by that wrong train data list path was writed. fixed in the trainer_config will be ok.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants