Garbage ctx.dev_mask seen in Clojure CPU Integration CI run

[DickJC123]

Description

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-16671/2/pipeline

  what():  [23:30:23] src/engine/threaded_engine_perdevice.cc:131: Check failed: ctx.dev_mask() == Context::kGPU (1140850888 vs. 2) : 

Error Message

terminate called after throwing an instance of 'dmlc::Error'

what():  [23:30:23] src/engine/threaded_engine_perdevice.cc:131: Check failed: ctx.dev_mask() == Context::kGPU (1140850888 vs. 2) : 

Stack trace:

[bt] (0) /tmp/mxnet6219749271079940772/mxnet-scala(dmlc::LogMessageFatal::~LogMessageFatal()+0x32) [0x7fc43f08c972]
[bt] (1) /tmp/mxnet6219749271079940772/libmxnet.so(mxnet::engine::ThreadedEnginePerDevice::PushToExecute(mxnet::engine::OprBlock*, bool)+0x280) [0x7fc3173d2950]
[bt] (2) /tmp/mxnet6219749271079940772/libmxnet.so(mxnet::engine::ThreadedEngine::Push(mxnet::engine::Opr*, mxnet::Context, int, bool)+0x41b) [0x7fc3173c006b]
[bt] (3) /tmp/mxnet6219749271079940772/libmxnet.so(mxnet::engine::ThreadedEngine::PushAsync(std::function<void (mxnet::RunContext, mxnet::engine::CallbackOnComplete)>, mxnet::Context, std::vector<mxnet::engine::Var*, std::allocator<mxnet::engine::Var*> > const&, std::vector<mxnet::engine::Var*, std::allocator<mxnet::engine::Var*> > const&, mxnet::FnProperty, int, char const*, bool)+0xd8) [0x7fc3173bcaf8]
[bt] (4) /tmp/mxnet6219749271079940772/libmxnet.so(mxnet::engine::ThreadedEngine::BulkFlush()+0x481) [0x7fc3173bbb31]
[bt] (5) /tmp/mxnet6219749271079940772/libmxnet.so(mxnet::engine::ThreadedEngine::Push(mxnet::engine::Opr*, mxnet::Context, int, bool)+0x42) [0x7fc3173bfc92]
[bt] (6) /tmp/mxnet6219749271079940772/libmxnet.so(mxnet::engine::ThreadedEngine::PushAsync(std::function<void (mxnet::RunContext, mxnet::engine::CallbackOnComplete)>, mxnet::Context, std::vector<mxnet::engine::Var*, std::allocator<mxnet::engine::Var*> > const&, std::vector<mxnet::engine::Var*, std::allocator<mxnet::engine::Var*> > const&, mxnet::FnProperty, int, char const*, bool)+0xd8) [0x7fc3173bcaf8]
[bt] (7) /tmp/mxnet6219749271079940772/libmxnet.so(mxnet::engine::ThreadedEngine::BulkFlush()+0x481) [0x7fc3173bbb31]
[bt] (8) /tmp/mxnet6219749271079940772/libmxnet.so(mxnet::engine::ThreadedEngine::Push(mxnet::engine::Opr*, mxnet::Context, int, bool)+0x42) [0x7fc3173bfc92]

Tests failed.

To Reproduce

(If you developed your own code, please provide a short script that reproduces the error. For existing examples, please provide link.)

Steps to reproduce

(Paste the commands you ran that produced the error.)

What have you tried to solve it?

Environment

We recommend using our script for collecting the diagnositc information. Run the following command and paste the outputs below:

curl --retry 10 -s https://raw.githubusercontent.com/dmlc/gluon-nlp/master/tools/diagnose.py | python

# paste outputs here

[samskalicky]

@DickJC123 can this be closed now that #16671 is merged?
@zachgk assign @DickJC123

[pribadihcr]

any solution?. got the same error

apache-mxnet-src-1.7.0-incubating/src/engine/threaded_engine_perdevice.cc:131: Check failed: ctx.dev_mask() == Context::kGPU (1373401408 vs. 2) :