Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

test_loss.test_l2_loss has fixed seed that can mask flakiness #11699

Closed
szha opened this issue Jul 13, 2018 · 2 comments · Fixed by #11995
Closed

test_loss.test_l2_loss has fixed seed that can mask flakiness #11699

szha opened this issue Jul 13, 2018 · 2 comments · Fixed by #11995
Labels
Flaky Test

Comments

@szha
Copy link
Member

szha commented Jul 13, 2018

The unit test in title have been using fixed seed to mask flakiness. Suggested action:

  1. Evaluate whether the test is flaky without fixed seed. If not, remove seed. Else move to 2
  2. If test is flaky, determine whether it's an actual uncaught edge case. If so, fix the operator. Else move to 3
  3. If numerical instability is inevitable, adjust tolerance level appropriately.
@apeforest
Copy link
Contributor

Thanks for filing this issue. We will investigate this Flaky test

@haojin2
Copy link
Contributor

haojin2 commented Aug 2, 2018

Fix delivered in #11995

@eric-haibin-lin eric-haibin-lin mentioned this issue Aug 4, 2018
Merged
7 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
Flaky Test
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove fixed seed for test_l1_loss and test_l2_loss eric-haibin-lin/mxnet
3 participants
@szha @apeforest @haojin2