Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

R-package. Bug in early stop callback function for maximize = TRUE #4587

Closed
srijoyray opened this issue Jan 8, 2017 · 1 comment
Closed

R-package. Bug in early stop callback function for maximize = TRUE #4587

srijoyray opened this issue Jan 8, 2017 · 1 comment

Comments

@srijoyray
Copy link

There is a bug in the early stop callback function #3938 when eval.metric needs to be maximized. The mx.callback.early.stop as soon as the evaluation increases even when the "maximize" is set to TRUE.

The line 138 of R-package/R/callback.R must be changed from

if (result$value > mx.best.score | (maximize == TRUE & result$value < mx.best.score)) {

to

if ((maximize == FALSE & result$value > mx.best.score) | (maximize == TRUE & result$value < mx.best.score)) {
@thirdwing thirdwing self-assigned this Apr 16, 2017
@MarcBarakat MarcBarakat mentioned this issue Apr 20, 2017
Merged
@thirdwing
Copy link
Contributor

Fixed in #4587.

cjolivier01 pushed a commit to cjolivier01/mxnet that referenced this issue May 30, 2017
fixing the early stop for maximize = T (apache#5915)
8d2b046 
close apache#4587
piiswrong pushed a commit that referenced this issue Jun 5, 2017
@cjolivier01 @piiswrong
batchnorm specify channel axis and performance optimizations for batc…
788c280 
…hnorm (#6411)

* Add channel_axis to batch norm, performance improvements

* rearrange tests a bit

* rearrange tests a bit

* CR changes

* cpp package link issue

* Fix: MSVC wants all parallel omp to be int

* CR comments, expand legal negative axes

* lint

* lint

* Fix download link (#6431)

* Fix Download Button

* Small fix

* Add release note (#6434)

* Fixing tutorials. (#6436)

Most of the fixes should be self evident. For tutorial on pre-trained models, one of the images doesn't exist anymore so selected a new one. Long-term, we should put such images on web-data repo but alas, some other day.

For Handwritten digit tutorial, we are missing couple of imports in the test_utils.py that was recently created.

Note that: for pre-trained model tutorial, we get a softmax_label warning and the probability scores are not really probabilities. Will deal with that issue in another PR.

Testing:
I've tried to test all the notebooks with this change and things look fine.

* Formatting fixes (#6433)

* Formatting fixes

* lint fixed

* fix

* doc bash 2-5, for pack, unpack, pack_img and unpack_img (#6140)

* doc bash for pack, unpack, pack_img and unpack_img

* Add comments for labels could be 1d list

* Update recordio.py

* Update recordio.py

* Update recordio.py

fixing text

* Update recordio.py

fixing text

* remove empty line

* Improve style (#6445)

* Correction (#6444)

* CSVIter example correction

* fix

* Update documentation for MXNetDataIter in io.py (#6000) (#6113)

* Update documentation for MXNetDataIter in io.py (#6000)

* [DOC] Respond to feedback (#6113)

* Fix minor issues with api pages. (#6410)

1. In the notes section for ndarray, references did not seem clear enough to be referring to mxnet.ndarray or numpy.ndarray. Added the package names as prefixes to make it more obvious.
2. "share the same C++ operator source codes" => "share the same code". Since we don't really need to throw in more details than required.
3. Other relatively  minor language changes which will be obvious from the diff.

Note that I'm relatively not sure about the need for 1/ since it makes things more verbose. Let me know if it unnecessary and I'll remove it.

* fixing the early stop for maximize = T  (#5915)

close #4587

* Update documentation for mxnet.ndarray.GridGenerator. (#6430)

* Update documentation for mxnet.ndarray.GridGenerator.

Thanks @Lyken17 for #6147

* Fix lint error.

* Minor fix.

* Remove the example.

* Update documentation for deconvolution operation. (#6184)

* Update documentation for deconvolution operation.

* Add examples.

* Remove the example.

* skip lines that have %matplotlib (#6451)

* Fixing some more broken links before v0.10 release (#6449)

* close #4838 (#6452)

* Fix linear regression (#6432)

* Fix Linear Regression Tutorial

* Small fix

* Pre-trained model tutorial fixes. (#6453)

Before the change on running the tutorial for the first time: "UserWarning: Data provided by label_shapes don't match names specified by label_names ([] vs. ['softmax_label'])". It also showed probability of >>1 due to incorrect usage of np.argsort().

* Nightly test tutorial (#6447)

* Add tutorial test

* Fix pylint

* Small fix

* fix member variable name: make them end with underline (#6438)

* [R] captcha example (#6443)

* skip lines that have %matplotlib (#6459)

* Fix cudnn_deconv not guarding no_bias (#6456)

* Fixing up issues in install guide (#6463)

* Fixing copy code functionality for bash command (#6465)

* Residual unroll (#6397)

* residual unroll

* unroll for residual cell

* merge_outputs fix

* Linear regression Tutorial link (#6468)

* Fixing a link in the linear regression tutorial.

The link was initally going to mxnet-test.readthedocs.io. Changed it to mxnet.io/api.

* More appropriate language.

* bump up version number for release (#6462)

* bump up version number for release

* update version for scala/R/backend

* [R][DOC] update R installation guide (#6457)

* Use sphinx==1.3.5 in Dockerfile.doc (#6470)

changed PR name

* Add 0.10 release info to README.md and NEWS.md (#6471)

@nswamy wants to merge it immediately, so i'm going to do it now. I also changed the PR title.

* fix batchNorm cpp example (#6454)

* Update im2rec.py (#6473)

Updated Line 107 of 'im2rec.py'. Read an image as binary.

* Change Interface of  NDArray & TBlob for DLPack Compatible (#6345)

* Change Interface of NDArray & TBlob for DLPack Compatible

Fix for cudnn operator

Fix cpp tests

* Update nnvm

* Fix for MKL mem

* Fix for windows macro

* Bump up version number to 0.10.1

* Update NDArray Save&Load

* trigger update

* Add test for legacy data load

* Use LegacyTShapeLoad

* trigger update

* Update tensor_blob.h

* change 'channel_axis' parameter to 'axis'

* Change DEFAULT_CHANNEL_AXIS to DEFAULT_AXIS

* wait for dlpack PR to go through

* Trigger build
Guneet-Dhillon pushed a commit to Guneet-Dhillon/mxnet that referenced this issue Sep 13, 2017
@thirdwing
fixing the early stop for maximize = T (apache#5915)
4cb57d6 
close apache#4587
Guneet-Dhillon pushed a commit to Guneet-Dhillon/mxnet that referenced this issue Sep 13, 2017
@cjolivier01 @piiswrong
batchnorm specify channel axis and performance optimizations for batc…
c54afc3 
…hnorm (apache#6411)

* Add channel_axis to batch norm, performance improvements

* rearrange tests a bit

* rearrange tests a bit

* CR changes

* cpp package link issue

* Fix: MSVC wants all parallel omp to be int

* CR comments, expand legal negative axes

* lint

* lint

* Fix download link (apache#6431)

* Fix Download Button

* Small fix

* Add release note (apache#6434)

* Fixing tutorials. (apache#6436)

Most of the fixes should be self evident. For tutorial on pre-trained models, one of the images doesn't exist anymore so selected a new one. Long-term, we should put such images on web-data repo but alas, some other day.

For Handwritten digit tutorial, we are missing couple of imports in the test_utils.py that was recently created.

Note that: for pre-trained model tutorial, we get a softmax_label warning and the probability scores are not really probabilities. Will deal with that issue in another PR.

Testing:
I've tried to test all the notebooks with this change and things look fine.

* Formatting fixes (apache#6433)

* Formatting fixes

* lint fixed

* fix

* doc bash 2-5, for pack, unpack, pack_img and unpack_img (apache#6140)

* doc bash for pack, unpack, pack_img and unpack_img

* Add comments for labels could be 1d list

* Update recordio.py

* Update recordio.py

* Update recordio.py

fixing text

* Update recordio.py

fixing text

* remove empty line

* Improve style (apache#6445)

* Correction (apache#6444)

* CSVIter example correction

* fix

* Update documentation for MXNetDataIter in io.py (apache#6000) (apache#6113)

* Update documentation for MXNetDataIter in io.py (apache#6000)

* [DOC] Respond to feedback (apache#6113)

* Fix minor issues with api pages. (apache#6410)

1. In the notes section for ndarray, references did not seem clear enough to be referring to mxnet.ndarray or numpy.ndarray. Added the package names as prefixes to make it more obvious.
2. "share the same C++ operator source codes" => "share the same code". Since we don't really need to throw in more details than required.
3. Other relatively  minor language changes which will be obvious from the diff.

Note that I'm relatively not sure about the need for 1/ since it makes things more verbose. Let me know if it unnecessary and I'll remove it.

* fixing the early stop for maximize = T  (apache#5915)

close apache#4587

* Update documentation for mxnet.ndarray.GridGenerator. (apache#6430)

* Update documentation for mxnet.ndarray.GridGenerator.

Thanks @Lyken17 for apache#6147

* Fix lint error.

* Minor fix.

* Remove the example.

* Update documentation for deconvolution operation. (apache#6184)

* Update documentation for deconvolution operation.

* Add examples.

* Remove the example.

* skip lines that have %matplotlib (apache#6451)

* Fixing some more broken links before v0.10 release (apache#6449)

* close apache#4838 (apache#6452)

* Fix linear regression (apache#6432)

* Fix Linear Regression Tutorial

* Small fix

* Pre-trained model tutorial fixes. (apache#6453)

Before the change on running the tutorial for the first time: "UserWarning: Data provided by label_shapes don't match names specified by label_names ([] vs. ['softmax_label'])". It also showed probability of >>1 due to incorrect usage of np.argsort().

* Nightly test tutorial (apache#6447)

* Add tutorial test

* Fix pylint

* Small fix

* fix member variable name: make them end with underline (apache#6438)

* [R] captcha example (apache#6443)

* skip lines that have %matplotlib (apache#6459)

* Fix cudnn_deconv not guarding no_bias (apache#6456)

* Fixing up issues in install guide (apache#6463)

* Fixing copy code functionality for bash command (apache#6465)

* Residual unroll (apache#6397)

* residual unroll

* unroll for residual cell

* merge_outputs fix

* Linear regression Tutorial link (apache#6468)

* Fixing a link in the linear regression tutorial.

The link was initally going to mxnet-test.readthedocs.io. Changed it to mxnet.io/api.

* More appropriate language.

* bump up version number for release (apache#6462)

* bump up version number for release

* update version for scala/R/backend

* [R][DOC] update R installation guide (apache#6457)

* Use sphinx==1.3.5 in Dockerfile.doc (apache#6470)

changed PR name

* Add 0.10 release info to README.md and NEWS.md (apache#6471)

@nswamy wants to merge it immediately, so i'm going to do it now. I also changed the PR title.

* fix batchNorm cpp example (apache#6454)

* Update im2rec.py (apache#6473)

Updated Line 107 of 'im2rec.py'. Read an image as binary.

* Change Interface of  NDArray & TBlob for DLPack Compatible (apache#6345)

* Change Interface of NDArray & TBlob for DLPack Compatible

Fix for cudnn operator

Fix cpp tests

* Update nnvm

* Fix for MKL mem

* Fix for windows macro

* Bump up version number to 0.10.1

* Update NDArray Save&Load

* trigger update

* Add test for legacy data load

* Use LegacyTShapeLoad

* trigger update

* Update tensor_blob.h

* change 'channel_axis' parameter to 'axis'

* Change DEFAULT_CHANNEL_AXIS to DEFAULT_AXIS

* wait for dlpack PR to go through

* Trigger build
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@thirdwing @srijoyray