Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Re-enabling randomized test_operator/test_operator_gpu.test_dot #11888

Merged
merged 1 commit into from
Jul 25, 2018

Conversation

haojin2
Copy link
Contributor

@haojin2 haojin2 commented Jul 25, 2018

Description

Getting rid of the fixed seed for test_operator/test_operator_gpu.test_dot as the flakiness cannot be reproduced.

Checklist

Essentials

  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage:
  • Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
  • Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
  • Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
  • Code is well-documented:
  • For user-facing API changes, API doc string has been updated.
  • For new C++ functions in header files, their functionalities and arguments are documented.
  • For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
  • Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
  • To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

  • Get rid of the fixed seed for test_operator/test_operator_gpu.test_dot

Comments

Can pass more than 10000 times on both CPU & GPU:

MXNET_TEST_COUNT=10000 nosetests -s --verbose tests/python/unittest/test_operator.py:test_dot
/home/ubuntu/anaconda3/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py:46: DeprecationWarning: OpenSSL.rand is deprecated - you should use os.urandom instead
  import OpenSSL.SSL
/home/ubuntu/anaconda3/lib/python3.6/site-packages/nose/util.py:453: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
  inspect.getargspec(func)
[INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=194063305 to reproduce.
test_operator.test_dot ... ok

----------------------------------------------------------------------
Ran 1 test in 9446.096s

OK
MXNET_TEST_COUNT=10000 nosetests -s --verbose tests/python/gpu/test_operator_gpu.py:test_dot
/home/ubuntu/anaconda3/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py:46: DeprecationWarning: OpenSSL.rand is deprecated - you should use os.urandom instead
  import OpenSSL.SSL
/home/ubuntu/anaconda3/lib/python3.6/site-packages/nose/util.py:453: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
  inspect.getargspec(func)
[INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=2632151 to reproduce.
test_operator_gpu.test_dot ... ok

----------------------------------------------------------------------
Ran 1 test in 36.951s

OK

@haojin2 haojin2 changed the title Re-enabling test_operator/test_operator_gpu.test_dot Re-enabling randomized test_operator/test_operator_gpu.test_dot Jul 25, 2018
@haojin2
Copy link
Contributor Author

haojin2 commented Jul 25, 2018

@marcoabreu

@haojin2
Copy link
Contributor Author

haojin2 commented Jul 25, 2018

Problem of fixed seed tracked at #11714

# Seed set because the test is not robust enough to operate on random data
@with_seed(1234)
# @haojin2: Getting rid of fixed seed as flakiness could not be reproduced,
# tracked at https://github.com/apache/incubator-mxnet/issues/11714
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to remove that comment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's still keep it here, in case if it fails on some other platforms, or we detect any regression on this operator in the future. What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me

@haojin2
Copy link
Contributor Author

haojin2 commented Jul 25, 2018

@marcoabreu Checks passed, please merge when you feel comfortable to do so.

@marcoabreu marcoabreu merged commit 7cd01ff into apache:master Jul 25, 2018
@haojin2 haojin2 deleted the test_dot_not_flaky branch July 25, 2018 23:47
XinYao1994 pushed a commit to XinYao1994/incubator-mxnet that referenced this pull request Aug 29, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants