[BUGFIX] Freeze boto3 #1370

barry-jin · 2020-09-21T23:02:28Z

Description

Fix issue #1369

Checklist

Essentials

PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
Changes are complete (i.e. I finished coding on this PR)
All changes have test coverage
Code is well-documented

Changes

Feature1, tests, (and when applicable, API doc)
Feature2, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

cc @dmlc/gluon-nlp-team

github-actions · 2020-09-21T23:23:48Z

The documentation website for preview: http://gluon-nlp-dev.s3-accelerate.amazonaws.com/PR#1370/freeze-boto3/index.html

codecov · 2020-09-21T23:25:13Z

Codecov Report

Merging #1370 into master will decrease coverage by 10.92%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master    #1370       +/-   ##
===========================================
- Coverage   81.68%   70.76%   -10.93%     
===========================================
  Files          52      105       +53     
  Lines        6906    12518     +5612     
===========================================
+ Hits         5641     8858     +3217     
- Misses       1265     3660     +2395

Impacted Files	Coverage Δ
src/gluonnlp/data/loading.py	`78.11% <0.00%> (-5.29%)`	⬇️
tests/test_models.py	`100.00% <0.00%> (ø)`
scripts/processing/clean_tok_mono_corpus.py	`0.00% <0.00%> (ø)`
tests/test_data_vocab.py	`99.03% <0.00%> (ø)`
...ripts/datasets/question_answering/prepare_squad.py	`0.00% <0.00%> (ø)`
scripts/processing/apply_subword.py	`88.69% <0.00%> (ø)`
tests/test_data_batchify.py	`100.00% <0.00%> (ø)`
tests/test_initializer.py	`92.59% <0.00%> (ø)`
tests/test_models_roberta.py	`100.00% <0.00%> (ø)`
...ts/datasets/question_answering/prepare_searchqa.py	`0.00% <0.00%> (ø)`
... and 60 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 56de303...918f233. Read the comment docs.

leezu · 2020-09-21T23:58:27Z

setup.py

-            'boto3',
+            'boto3==1.14.32',


How about specifying an acceptable arrange, as this affects all users installing with [extra]

awscli and boto3 have the same dependency botocore. botocore 1.18.0 has removed doc files and migrated to awscli >= 1.18.140. Since we installed awscliduring building the docker file and installed boto3 when boot up an AWS batch job. So, to solve docevents missing issue, we can either keep boto3 and awscli all in old version or update awscli by updating the docker and take boto3 in newest version.

github-actions · 2020-09-22T00:07:27Z

The documentation website for preview: http://gluon-nlp-dev.s3-accelerate.amazonaws.com/PR#1370/freeze-boto3/index.html

github-actions · 2020-09-22T22:07:10Z

The documentation website for preview: http://gluon-nlp-dev.s3-accelerate.amazonaws.com/PR#1370/freeze-boto3/index.html

barry-jin · 2020-09-22T22:45:29Z

The documentation website for preview: http://gluon-nlp-dev.s3-accelerate.amazonaws.com/PR#1370/freeze-boto3/index.html

The documentation website for preview: http://gluon-nlp-dev.s3-accelerate.amazonaws.com/PR%231370/freeze-boto3/index.html

szha · 2020-09-22T22:57:46Z

the url looks weird. if there's trouble in handling # character, we can do without it.

github-actions · 2020-09-22T23:20:24Z

The documentation website for preview: http://gluon-nlp-dev.s3-accelerate.amazonaws.com/PR#1370/freeze-boto3/index.html

github-actions · 2020-09-22T23:21:12Z

The documentation website for preview: http://gluon-nlp-dev.s3-accelerate.amazonaws.com/PR#1370/freeze-boto3/index.html

* Fix BERT fp16 bugs, add test (dmlc#1270) * Fix fp16 bug: not passing dtype to TransformerEncoderLayer * Re-hybridize after casting & add BERT test * Skip fp16 test if CPU ctx * remove debugging messages Co-authored-by: root <[email protected]> * [Fix][SageMaker] Make sure that the installation works in SageMaker (dmlc#1348) * Fasttext to 0.9.1 * Update setup.py * [CI] Add Codecov and Test Logs (dmlc#1349) * [Fix] Some minor fixes for AMLC Tutorial (dmlc#1355) * update update update update * Update test_utils_misc.py * update * update * Update test_layers.py * Update misc.py * Update mobilebert.py * add in_units and in_channels * Update __init__.py * Update mobilebert.py * Update README.md * fix test case * fix * Update test_utils_misc.py * fix bug * [FEATURE] gpt2 generation scripts (dmlc#1354) * remove prev_len in hybrid_forward parameters * update * sample * update * add gpt2_1558M * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update Co-authored-by: Hu <[email protected]> * [Fix] Minor fix for AMLC Tutorial - QA (dmlc#1359) * update Update README.md update try to use dataclasses * Update squad_utils.py * Update preprocessing.py * Update squad_utils.py * Update run_squad.py * [Log Message Improvement] Improve nlp process (dmlc#1362) * Update learn_subword.py * Update learn_subword.py * Update learn_subword.py * Update apply_subword.py * Set default ctx in conftest (dmlc#1363) * Fix the correctness of the Horovod support on squad (dmlc#1353) * revise squad * tiny fix * fix total_norm logging * shuffle before and after splitting * make pre_shuffle_seed fixed * fix flags * remove do_pre_shuffle * remove inside_split_shuffle Co-authored-by: Ubuntu <[email protected]> * [CI][BUGFIX] Custom Step for Uploading Code Coverage in Pull Request Event (dmlc#1364) * [FEATURE]Generation script improvement (dmlc#1365) * update * update * update * update * update * udpate * update * update * update * update Co-authored-by: Hu <[email protected]> * [Website][CI] Build Website without Warnings + Add Workflow for Building Website (dmlc#1327) * [Website] Documentation warnings Fixed + Create Makefile [Website] Documentation bug fix [Website] Bug fix [Website] Build without model_zoo [Website] Fix notebook * [Website][CI] Add workflow for building website * [CI] Add more dependencies * [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [CI] Update buildwebsite.yml * [Website] Add more dependencies * [Website][CI] Add Compile notebook step + Preview website * [CI] Add shell script for compiling notebooks * [CI] Add permission for shell script * [Website] Update * [Website] Update * [CI] Add uploading build artifacts * [CI] Update * [CI] Update Indentation * [CI] Remove some dependencies * [BUGFIX] Fix URL encoding (dmlc#1370) * [FEATURE]Update readme of nmt (dmlc#1373) * update * update * update * update * update * update * update * update Co-authored-by: Hu <[email protected]> * [CI] Improve website building workflow (dmlc#1377) * BERT pretraining (dmlc#1376) * bert * update * address comments * update * [Fix][Docker] Fix the docker image + Fix pretrain_corpus document. (dmlc#1378) * update * Update ubuntu18.04-devel-gpu.Dockerfile * fix the docker image * Update README.md * Update ubuntu18.04-devel-gpu.Dockerfile * Update README.md * fix readme * Add CPU DockerFile * update * update * Update ubuntu18.04-devel-gpu.Dockerfile * update * prepare to add TVM to docker * try to update * Update ubuntu18.04-devel-gpu.Dockerfile * Update ubuntu18.04-devel-gpu.Dockerfile * Update install_openmpi.sh * update * Create install_llvm.sh * Update ubuntu18.04-base-gpu.Dockerfile * Update ubuntu18.04-base-gpu.Dockerfile * Update run_squad2_albert_base.sh * Update prepare_squad.py * Update prepare_squad.py * Update prepare_squad.py * fix * Update README.md * update * update * Update README.md * Update README.md * Update ubuntu18.04-devel-gpu.Dockerfile * update * Update README.md * fix * Update ubuntu18.04-base-cpu.Dockerfile * update * add tvm to lazy import * update * Update README.md * update * Update README.md * Update run_squad2_albert_base.sh * update * update * update * update * update * Update README.md * Update install_ubuntu18.04_core.sh * update * update * update * fix * Update README.md * Update run_batch_squad.sh * update * Update run_batch_squad.sh * Update run_batch_squad.sh * update * Update README.md * fix * Update gluon_nlp_job.sh * update * Update README.md * Update README.md * Update README.md * update * Update README.md * update * Update install_python_packages.sh * Update install_llvm.sh * Update install_python_packages.sh * Update install_llvm.sh * update * Update install_ubuntu18.04_core.sh * fix * Update submit-job.py * Update submit-job.py * Update README.md * Update README.md * Update prepare_gutenberg.py * Delete gluon_nlp_cpu_job.sh * Update prepare_gutenberg.py * Update prepare_gutenberg.py * Update prepare_gutenberg.py * Update conf.py * update * Update generate_commands.py * fix readme * use os.link for hard link * Update README.md * Update README.md * Update gluon_nlp_job.sh * Update __init__.py * Update benchmark_utils.py * try to use multi-stage build * Update benchmark_utils.py * multi-stage build * Update README.md * Update README.md * update * Update submit-job.py * fix documentation * fix * update * Update test.sh * Update test.sh * Update test.sh * Update test.sh * Update README.md * Update test.sh * fix * Update README.md * Update gluon_nlp_job.sh * [Website] Add AMLC Tutorial to Website (dmlc#1379) * [Website] Add AMLC Tutorial * [Website] Add tsv encoding * [Website] Add model zoo * [Website] Update Makefile * [Website] Update Makefile * [Website] Update Makefile * [Website] Update compile_notebooks.sh * [Website] Update Makefile * [Website] Add title to generation * [Website] Update workflow * update * [Website] Update model_zoo.rst * [Website] Update model_zoo.rst * [BUGFIX] Fix Codecov (dmlc#1391) * Update coveragerc * Update coveragerc * Update coveragerc * Update workflow * Update workflow * update * update Co-authored-by: MoisesHer <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: Xingjian Shi <[email protected]> Co-authored-by: barry-jin <[email protected]> Co-authored-by: ht <[email protected]> Co-authored-by: Hu <[email protected]> Co-authored-by: Leonard Lausen <[email protected]> Co-authored-by: Ubuntu <[email protected]> Co-authored-by: Ziyue Huang <[email protected]>

[BUGFIX] Freeze boto3

3988da5

barry-jin requested a review from a team as a code owner September 21, 2020 23:02

szha approved these changes Sep 21, 2020

View reviewed changes

[CI] Encoding # in URL

0530212

leezu reviewed Sep 21, 2020

View reviewed changes

[CI] Update Docker

d75f55e

barry-jin added 2 commits September 22, 2020 16:00

Remove #

59dd01c

Remove #

918f233

leezu merged commit d6e63f3 into dmlc:master Sep 23, 2020

barry-jin mentioned this pull request Sep 23, 2020

[CI] Cannot import name 'docevents' from boto3 #1369

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUGFIX] Freeze boto3 #1370

[BUGFIX] Freeze boto3 #1370

barry-jin commented Sep 21, 2020

github-actions bot commented Sep 21, 2020

codecov bot commented Sep 21, 2020 •

edited

Loading

leezu Sep 21, 2020

barry-jin Sep 22, 2020 •

edited

Loading

github-actions bot commented Sep 22, 2020

github-actions bot commented Sep 22, 2020

barry-jin commented Sep 22, 2020

szha commented Sep 22, 2020

github-actions bot commented Sep 22, 2020

github-actions bot commented Sep 22, 2020

[BUGFIX] Freeze boto3 #1370

[BUGFIX] Freeze boto3 #1370

Conversation

barry-jin commented Sep 21, 2020

Description

Checklist

Essentials

Changes

Comments

github-actions bot commented Sep 21, 2020

codecov bot commented Sep 21, 2020 • edited Loading

Codecov Report

leezu Sep 21, 2020

Choose a reason for hiding this comment

barry-jin Sep 22, 2020 • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Sep 22, 2020

github-actions bot commented Sep 22, 2020

barry-jin commented Sep 22, 2020

szha commented Sep 22, 2020

github-actions bot commented Sep 22, 2020

github-actions bot commented Sep 22, 2020

codecov bot commented Sep 21, 2020 •

edited

Loading

barry-jin Sep 22, 2020 •

edited

Loading