Skip to content
This repository has been archived by the owner on Jan 15, 2024. It is now read-only.

[Log Message Improvement] Improve nlp process #1362

Merged
merged 4 commits into from
Sep 14, 2020
Merged

[Log Message Improvement] Improve nlp process #1362

merged 4 commits into from
Sep 14, 2020

Conversation

sxjscience
Copy link
Member

Description

Tell the users where the models will be saved into.

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

Changes

  • Feature1, tests, (and when applicable, API doc)
  • Feature2, tests, (and when applicable, API doc)

Comments

  • If this change is a backward incompatible change, why must this change be made.
  • Interesting edge cases to note here

cc @dmlc/gluon-nlp-team

@sxjscience sxjscience requested a review from a team as a code owner September 14, 2020 03:07
@codecov
Copy link

codecov bot commented Sep 14, 2020 •
edited
Loading

Codecov Report

Merging #1362 into master will increase coverage by 10.83%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff             @@
##           master    #1362       +/-   ##
===========================================
+ Coverage   70.88%   81.71%   +10.83%     
===========================================
  Files         105       52       -53     
  Lines       12499     6898     -5601     
===========================================
- Hits         8860     5637     -3223     
+ Misses       3639     1261     -2378
Impacted Files Coverage Δ
src/gluonnlp/models/xlmr.py 58.82% <0.00%> (-29.42%) ⬇️
src/gluonnlp/models/__init__.py 67.74% <0.00%> (-29.04%) ⬇️
src/gluonnlp/models/electra.py 57.89% <0.00%> (-18.56%) ⬇️
src/gluonnlp/models/bart.py 70.62% <0.00%> (-15.63%) ⬇️
src/gluonnlp/utils/config.py 61.53% <0.00%> (-15.39%) ⬇️
src/gluonnlp/models/roberta.py 84.38% <0.00%> (-9.29%) ⬇️
src/gluonnlp/utils/misc.py 51.08% <0.00%> (-8.67%) ⬇️
src/gluonnlp/models/albert.py 87.80% <0.00%> (-7.67%) ⬇️
src/gluonnlp/models/gpt2.py 90.94% <0.00%> (-7.32%) ⬇️
src/gluonnlp/models/bert.py 87.67% <0.00%> (-7.20%) ⬇️
... and 57 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 19ab08d...29f9b40. Read the comment docs.

@sxjscience sxjscience merged commit 0e86648 into dmlc:master Sep 14, 2020
zheyuye added a commit to zheyuye/gluon-nlp that referenced this pull request Oct 20, 2020
@zheyuye @MoisesHer
@sxjscience @barry-jin @hutao965 @leezu @ZiyueHuang
Keep up with others (#2)
23d9d73 
* Fix BERT fp16 bugs, add test (dmlc#1270)

* Fix fp16 bug: not passing dtype to TransformerEncoderLayer

* Re-hybridize after casting & add BERT test

* Skip fp16 test if CPU ctx

* remove debugging messages

Co-authored-by: root <[email protected]>

* [Fix][SageMaker] Make sure that the installation works in SageMaker (dmlc#1348)

* Fasttext to 0.9.1

* Update setup.py

* [CI] Add Codecov and Test Logs (dmlc#1349)

* [Fix] Some minor fixes for AMLC Tutorial (dmlc#1355)

* update

update

update

update

* Update test_utils_misc.py

* update

* update

* Update test_layers.py

* Update misc.py

* Update mobilebert.py

* add in_units and in_channels

* Update __init__.py

* Update mobilebert.py

* Update README.md

* fix test case

* fix

* Update test_utils_misc.py

* fix bug

* [FEATURE] gpt2 generation scripts (dmlc#1354)

* remove prev_len in hybrid_forward parameters

* update

* sample

* update

* add gpt2_1558M

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

Co-authored-by: Hu <[email protected]>

* [Fix] Minor fix for AMLC Tutorial - QA (dmlc#1359)

* update

Update README.md

update

try to use dataclasses

* Update squad_utils.py

* Update preprocessing.py

* Update squad_utils.py

* Update run_squad.py

* [Log Message Improvement] Improve nlp process (dmlc#1362)

* Update learn_subword.py

* Update learn_subword.py

* Update learn_subword.py

* Update apply_subword.py

* Set default ctx in conftest (dmlc#1363)

* Fix the correctness of the Horovod support on squad (dmlc#1353)

* revise squad

* tiny fix

* fix total_norm logging

* shuffle before and after splitting

* make pre_shuffle_seed fixed

* fix flags

* remove do_pre_shuffle

* remove inside_split_shuffle

Co-authored-by: Ubuntu <[email protected]>

* [CI][BUGFIX] Custom Step for Uploading Code Coverage in Pull Request Event (dmlc#1364)

* [FEATURE]Generation script improvement (dmlc#1365)

* update

* update

* update

* update

* update

* udpate

* update

* update

* update

* update

Co-authored-by: Hu <[email protected]>

* [Website][CI] Build Website without Warnings + Add Workflow for Building Website  (dmlc#1327)

* [Website] Documentation warnings Fixed + Create Makefile

[Website] Documentation bug fix

[Website] Bug fix

[Website] Build without model_zoo

[Website] Fix notebook

* [Website][CI] Add workflow for building website

* [CI] Add more dependencies

* [CI] Update buildwebsite.yml

[CI] Update buildwebsite.yml

* [CI] Update buildwebsite.yml

* [CI] Update buildwebsite.yml

* [CI] Update buildwebsite.yml

* [CI] Update buildwebsite.yml

* [CI] Update buildwebsite.yml

* [CI] Update buildwebsite.yml

* [CI] Update buildwebsite.yml

* [CI] Update buildwebsite.yml

[CI] Update buildwebsite.yml

[CI] Update buildwebsite.yml

[CI] Update buildwebsite.yml

[CI] Update buildwebsite.yml

[CI] Update buildwebsite.yml

[CI] Update buildwebsite.yml

[CI] Update buildwebsite.yml

[CI] Update buildwebsite.yml

* [CI] Update buildwebsite.yml

* [CI] Update buildwebsite.yml

* [Website] Add more dependencies

* [Website][CI] Add Compile notebook step + Preview website

* [CI] Add shell script for compiling notebooks

* [CI] Add permission for shell script

* [Website] Update

* [Website] Update

* [CI] Add uploading build artifacts

* [CI] Update

* [CI] Update Indentation

* [CI] Remove some dependencies

* [BUGFIX] Fix URL encoding (dmlc#1370)

* [FEATURE]Update readme of nmt (dmlc#1373)

* update

* update

* update

* update

* update

* update

* update

* update

Co-authored-by: Hu <[email protected]>

* [CI] Improve website building workflow (dmlc#1377)

* BERT pretraining (dmlc#1376)

* bert

* update

* address comments

* update

* [Fix][Docker] Fix the docker image + Fix pretrain_corpus document. (dmlc#1378)

* update

* Update ubuntu18.04-devel-gpu.Dockerfile

* fix the docker image

* Update README.md

* Update ubuntu18.04-devel-gpu.Dockerfile

* Update README.md

* fix readme

* Add CPU DockerFile

* update

* update

* Update ubuntu18.04-devel-gpu.Dockerfile

* update

* prepare to add TVM to docker

* try to update

* Update ubuntu18.04-devel-gpu.Dockerfile

* Update ubuntu18.04-devel-gpu.Dockerfile

* Update install_openmpi.sh

* update

* Create install_llvm.sh

* Update ubuntu18.04-base-gpu.Dockerfile

* Update ubuntu18.04-base-gpu.Dockerfile

* Update run_squad2_albert_base.sh

* Update prepare_squad.py

* Update prepare_squad.py

* Update prepare_squad.py

* fix

* Update README.md

* update

* update

* Update README.md

* Update README.md

* Update ubuntu18.04-devel-gpu.Dockerfile

* update

* Update README.md

* fix

* Update ubuntu18.04-base-cpu.Dockerfile

* update

* add tvm to lazy import

* update

* Update README.md

* update

* Update README.md

* Update run_squad2_albert_base.sh

* update

* update

* update

* update

* update

* Update README.md

* Update install_ubuntu18.04_core.sh

* update

* update

* update

* fix

* Update README.md

* Update run_batch_squad.sh

* update

* Update run_batch_squad.sh

* Update run_batch_squad.sh

* update

* Update README.md

* fix

* Update gluon_nlp_job.sh

* update

* Update README.md

* Update README.md

* Update README.md

* update

* Update README.md

* update

* Update install_python_packages.sh

* Update install_llvm.sh

* Update install_python_packages.sh

* Update install_llvm.sh

* update

* Update install_ubuntu18.04_core.sh

* fix

* Update submit-job.py

* Update submit-job.py

* Update README.md

* Update README.md

* Update prepare_gutenberg.py

* Delete gluon_nlp_cpu_job.sh

* Update prepare_gutenberg.py

* Update prepare_gutenberg.py

* Update prepare_gutenberg.py

* Update conf.py

* update

* Update generate_commands.py

* fix readme

* use os.link for hard link

* Update README.md

* Update README.md

* Update gluon_nlp_job.sh

* Update __init__.py

* Update benchmark_utils.py

* try to use multi-stage build

* Update benchmark_utils.py

* multi-stage build

* Update README.md

* Update README.md

* update

* Update submit-job.py

* fix documentation

* fix

* update

* Update test.sh

* Update test.sh

* Update test.sh

* Update test.sh

* Update README.md

* Update test.sh

* fix

* Update README.md

* Update gluon_nlp_job.sh

* [Website] Add AMLC Tutorial to Website (dmlc#1379)

* [Website] Add AMLC Tutorial

* [Website] Add tsv encoding

* [Website] Add model zoo

* [Website] Update Makefile

* [Website] Update Makefile

* [Website] Update Makefile

* [Website] Update compile_notebooks.sh

* [Website] Update Makefile

* [Website] Add title to generation

* [Website] Update workflow

* update

* [Website] Update model_zoo.rst

* [Website] Update model_zoo.rst

* [BUGFIX] Fix Codecov (dmlc#1391)

* Update coveragerc

* Update coveragerc

* Update coveragerc

* Update workflow

* Update workflow

* update

* update

Co-authored-by: MoisesHer <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: Xingjian Shi <[email protected]>
Co-authored-by: barry-jin <[email protected]>
Co-authored-by: ht <[email protected]>
Co-authored-by: Hu <[email protected]>
Co-authored-by: Leonard Lausen <[email protected]>
Co-authored-by: Ubuntu <[email protected]>
Co-authored-by: Ziyue Huang <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@szha szha szha approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sxjscience @szha