[Fix] Some minor fixes for AMLC Tutorial #1355

sxjscience · 2020-09-07T23:06:08Z

Description

Some minor doc + API improvements.

Checklist

Essentials

PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
Changes are complete (i.e. I finished coding on this PR)
All changes have test coverage
Code is well-documented

Changes

Feature1, tests, (and when applicable, API doc)
Feature2, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

cc @dmlc/gluon-nlp-team

update update update

src/gluonnlp/utils/misc.py

codecov · 2020-09-07T23:26:33Z

Codecov Report

Merging #1355 into master will increase coverage by 10.91%.
The diff coverage is 80.00%.

@@             Coverage Diff             @@
##           master    #1355       +/-   ##
===========================================
+ Coverage   70.70%   81.61%   +10.91%     
===========================================
  Files         105       52       -53     
  Lines       12445     6882     -5563     
===========================================
- Hits         8799     5617     -3182     
+ Misses       3646     1265     -2381

Impacted Files	Coverage Δ
src/gluonnlp/models/bart.py	`70.62% <ø> (-15.63%)`	⬇️
src/gluonnlp/models/transformer.py	`98.51% <ø> (-0.43%)`	⬇️
src/gluonnlp/utils/preprocessing.py	`100.00% <ø> (ø)`
src/gluonnlp/utils/parameter.py	`86.53% <37.50%> (-8.92%)`	⬇️
src/gluonnlp/models/electra.py	`57.89% <50.00%> (-18.56%)`	⬇️
src/gluonnlp/utils/misc.py	`51.08% <85.71%> (-6.83%)`	⬇️
src/gluonnlp/layers.py	`86.93% <100.00%> (+0.15%)`	⬆️
src/gluonnlp/models/__init__.py	`67.74% <100.00%> (-28.93%)`	⬇️
src/gluonnlp/models/albert.py	`87.80% <100.00%> (-7.67%)`	⬇️
src/gluonnlp/models/bert.py	`87.67% <100.00%> (-7.20%)`	⬇️
... and 69 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4c2867d...9ef422f. Read the comment docs.

szha

LGTM

leezu · 2020-09-08T16:23:26Z

src/gluonnlp/utils/misc.py

@@ -651,3 +651,24 @@ def init_comm(backend, gpus):
            logging.info('GPU communication supported by KVStore')

    return store, num_workers, rank, local_rank, is_master_node, ctx_l
+
+
+def get_mxnet_recommended_ctx():


When committing to this function as part of 1.0 API, should we choose a more descriptive name? I think "recommended" is a bit unclear, as the function doesn't really know which context will work best for the particular setting. Also you're returning a list of ctx, not ctx.

Do you have any suggestions about the name?

Or get_mxnet_all_ctx()?

get_mxnet_gpu_ctx_list_fallback_cpu?

This looks too long.

Should we still use get_mxnet_visible_ctx ?

* Fix BERT fp16 bugs, add test (dmlc#1270) * Fix fp16 bug: not passing dtype to TransformerEncoderLayer * Re-hybridize after casting & add BERT test * Skip fp16 test if CPU ctx * remove debugging messages Co-authored-by: root <[email protected]> * [Fix][SageMaker] Make sure that the installation works in SageMaker (dmlc#1348) * Fasttext to 0.9.1 * Update setup.py * [CI] Add Codecov and Test Logs (dmlc#1349) * [Fix] Some minor fixes for AMLC Tutorial (dmlc#1355) * update update update update * Update test_utils_misc.py * update * update * Update test_layers.py * Update misc.py * Update mobilebert.py * add in_units and in_channels * Update __init__.py * Update mobilebert.py * Update README.md * fix test case * fix * Update test_utils_misc.py * fix bug * [FEATURE] gpt2 generation scripts (dmlc#1354) * remove prev_len in hybrid_forward parameters * update * sample * update * add gpt2_1558M * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update Co-authored-by: Hu <[email protected]> * [Fix] Minor fix for AMLC Tutorial - QA (dmlc#1359) * update Update README.md update try to use dataclasses * Update squad_utils.py * Update preprocessing.py * Update squad_utils.py * Update run_squad.py * [Log Message Improvement] Improve nlp process (dmlc#1362) * Update learn_subword.py * Update learn_subword.py * Update learn_subword.py * Update apply_subword.py * Set default ctx in conftest (dmlc#1363) * Fix the correctness of the Horovod support on squad (dmlc#1353) * revise squad * tiny fix * fix total_norm logging * shuffle before and after splitting * make pre_shuffle_seed fixed * fix flags * remove do_pre_shuffle * remove inside_split_shuffle Co-authored-by: Ubuntu <[email protected]> * [CI][BUGFIX] Custom Step for Uploading Code Coverage in Pull Request Event (dmlc#1364) * [FEATURE]Generation script improvement (dmlc#1365) * update * update * update * update * update * udpate * update * update * update * update Co-authored-by: Hu <[email protected]> * [Website][CI] Build Website without Warnings + Add Workflow for Building Website (dmlc#1327) * [Website] Documentation warnings Fixed + Create Makefile [Website] Documentation bug fix [Website] Bug fix [Website] Build without model_zoo [Website] Fix notebook * [Website][CI] Add workflow for building website * [CI] Add more dependencies * [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [Website] Add more dependencies * [Website][CI] Add Compile notebook step + Preview website * [CI] Add shell script for compiling notebooks * [CI] Add permission for shell script * [Website] Update * [Website] Update * [CI] Add uploading build artifacts * [CI] Update * [CI] Update Indentation * [CI] Remove some dependencies * [BUGFIX] Fix URL encoding (dmlc#1370) * [FEATURE]Update readme of nmt (dmlc#1373) * update * update * update * update * update * update * update * update Co-authored-by: Hu <[email protected]> * [CI] Improve website building workflow (dmlc#1377) * BERT pretraining (dmlc#1376) * bert * update * address comments * update * [Fix][Docker] Fix the docker image + Fix pretrain_corpus document. (dmlc#1378) * update * Update ubuntu18.04-devel-gpu.Dockerfile * fix the docker image * Update README.md * Update ubuntu18.04-devel-gpu.Dockerfile * Update README.md * fix readme * Add CPU DockerFile * update * update * Update ubuntu18.04-devel-gpu.Dockerfile * update * prepare to add TVM to docker * try to update * Update ubuntu18.04-devel-gpu.Dockerfile * Update ubuntu18.04-devel-gpu.Dockerfile * Update install_openmpi.sh * update * Create install_llvm.sh * Update ubuntu18.04-base-gpu.Dockerfile * Update ubuntu18.04-base-gpu.Dockerfile * Update run_squad2_albert_base.sh * Update prepare_squad.py * Update prepare_squad.py * Update prepare_squad.py * fix * Update README.md * update * update * Update README.md * Update README.md * Update ubuntu18.04-devel-gpu.Dockerfile * update * Update README.md * fix * Update ubuntu18.04-base-cpu.Dockerfile * update * add tvm to lazy import * update * Update README.md * update * Update README.md * Update run_squad2_albert_base.sh * update * update * update * update * update * Update README.md * Update install_ubuntu18.04_core.sh * update * update * update * fix * Update README.md * Update run_batch_squad.sh * update * Update run_batch_squad.sh * Update run_batch_squad.sh * update * Update README.md * fix * Update gluon_nlp_job.sh * update * Update README.md * Update README.md * Update README.md * update * Update README.md * update * Update install_python_packages.sh * Update install_llvm.sh * Update install_python_packages.sh * Update install_llvm.sh * update * Update install_ubuntu18.04_core.sh * fix * Update submit-job.py * Update submit-job.py * Update README.md * Update README.md * Update prepare_gutenberg.py * Delete gluon_nlp_cpu_job.sh * Update prepare_gutenberg.py * Update prepare_gutenberg.py * Update prepare_gutenberg.py * Update conf.py * update * Update generate_commands.py * fix readme * use os.link for hard link * Update README.md * Update README.md * Update gluon_nlp_job.sh * Update __init__.py * Update benchmark_utils.py * try to use multi-stage build * Update benchmark_utils.py * multi-stage build * Update README.md * Update README.md * update * Update submit-job.py * fix documentation * fix * update * Update test.sh * Update test.sh * Update test.sh * Update test.sh * Update README.md * Update test.sh * fix * Update README.md * Update gluon_nlp_job.sh * [Website] Add AMLC Tutorial to Website (dmlc#1379) * [Website] Add AMLC Tutorial * [Website] Add tsv encoding * [Website] Add model zoo * [Website] Update Makefile * [Website] Update Makefile * [Website] Update Makefile * [Website] Update compile_notebooks.sh * [Website] Update Makefile * [Website] Add title to generation * [Website] Update workflow * update * [Website] Update model_zoo.rst * [Website] Update model_zoo.rst * [BUGFIX] Fix Codecov (dmlc#1391) * Update coveragerc * Update coveragerc * Update coveragerc * Update workflow * Update workflow * update * update Co-authored-by: MoisesHer <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: Xingjian Shi <[email protected]> Co-authored-by: barry-jin <[email protected]> Co-authored-by: ht <[email protected]> Co-authored-by: Hu <[email protected]> Co-authored-by: Leonard Lausen <[email protected]> Co-authored-by: Ubuntu <[email protected]> Co-authored-by: Ziyue Huang <[email protected]>

sxjscience added 2 commits September 7, 2020 16:00

update

3de5959

update update update

Update test_utils_misc.py

3e8517e

sxjscience requested a review from a team as a code owner September 7, 2020 23:06

szha reviewed Sep 7, 2020

View reviewed changes

src/gluonnlp/utils/misc.py Outdated Show resolved Hide resolved

update

25a3367

szha approved these changes Sep 8, 2020

View reviewed changes

sxjscience added 5 commits September 7, 2020 19:26

update

c467db4

Update test_layers.py

5bc7572

Update misc.py

4ac3f64

Update mobilebert.py

2e9c22c

add in_units and in_channels

c225a96

sxjscience mentioned this pull request Sep 8, 2020

[FEATURE] gpt2 generation scripts #1354

Merged

8 tasks

sxjscience added 2 commits September 7, 2020 22:19

Update __init__.py

a94cbdc

Update mobilebert.py

697e7e7

szha approved these changes Sep 8, 2020

View reviewed changes

Update README.md

d2af847

leezu reviewed Sep 8, 2020

View reviewed changes

sxjscience added 5 commits September 8, 2020 10:26

fix test case

52ea6fd

fix

a7c2a78

Update test_utils_misc.py

833aa9f

fix bug

03be4c1

Merge remote-tracking branch 'upstream/master' into text_prediction

9ef422f

sxjscience merged commit 0223686 into dmlc:master Sep 8, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Fix] Some minor fixes for AMLC Tutorial #1355

[Fix] Some minor fixes for AMLC Tutorial #1355

sxjscience commented Sep 7, 2020

codecov bot commented Sep 7, 2020 •

edited

Loading

szha left a comment

leezu Sep 8, 2020

sxjscience Sep 8, 2020

sxjscience Sep 8, 2020

leezu Sep 8, 2020

sxjscience Sep 8, 2020

sxjscience Sep 8, 2020 •

edited

Loading

[Fix] Some minor fixes for AMLC Tutorial #1355

[Fix] Some minor fixes for AMLC Tutorial #1355

Conversation

sxjscience commented Sep 7, 2020

Description

Checklist

Essentials

Changes

Comments

codecov bot commented Sep 7, 2020 • edited Loading

Codecov Report

szha left a comment

Choose a reason for hiding this comment

leezu Sep 8, 2020

Choose a reason for hiding this comment

sxjscience Sep 8, 2020

Choose a reason for hiding this comment

sxjscience Sep 8, 2020

Choose a reason for hiding this comment

leezu Sep 8, 2020

Choose a reason for hiding this comment

sxjscience Sep 8, 2020

Choose a reason for hiding this comment

sxjscience Sep 8, 2020 • edited Loading

Choose a reason for hiding this comment

codecov bot commented Sep 7, 2020 •

edited

Loading

sxjscience Sep 8, 2020 •

edited

Loading