Skip to content
This repository has been archived by the owner on Jan 15, 2024. It is now read-only.

Commit

Permalink
[CI] Use Cuda 10.0 MXNet build (#991)
Browse files Browse the repository at this point in the history
* Update MXNet dependency to cu100-1.6.0b20191027 for CPU and GPU

Nightly release pipelines for cu101-mkl is currently blocked.
Recent CPU builds also failed.

* Simplify Dockerfile

* Fix lint
  • Loading branch information
leezu authored and szha committed Oct 28, 2019
1 parent dfd4c6d commit bfa5503
Show file tree
Hide file tree
Showing 6 changed files with 11 additions and 8 deletions.
1 change: 0 additions & 1 deletion ci/batch/docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,4 @@ FROM nvidia/cuda:10.1-cudnn7-devel-ubuntu18.04
ENV PATH /opt/conda/bin:$PATH
RUN git clone https://github.com/dmlc/gluon-nlp
WORKDIR gluon-nlp
RUN /bin/bash -c 'CONDA_ENVS_PATH=$PWD/conda CONDA_PKGS_DIRS=$PWD/conda/pkgs conda init bash && source /root/.bashrc && conda env update --prune -p conda/gpu/py3 -f env/gpu/py3.yml && source activate ./conda/gpu/py3 && pip install -v -e . && pip install awscli && python -m spacy download en && python -m spacy download de && python -m nltk.downloader all'
ADD gluon_nlp_job.sh .
2 changes: 1 addition & 1 deletion env/cpu/py3-master.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ dependencies:
- scipy=1.3.1
- pip:
- pylint-quotes==0.2.1
- mxnet-mkl>=1.6.0b20191006
- mxnet-cu100>=1.6.0b20191027
- sacremoses
- sentencepiece<0.2
- sphinx-autodoc-typehints==1.7.0
2 changes: 1 addition & 1 deletion env/docker/py3.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ dependencies:
- scikit-learn=0.21.3
- pip:
- pylint-quotes<0.2
- mxnet-cu101mkl>=1.6.0b20191006
- mxnet-cu100>=1.6.0b20191027
- sacremoses
- sentencepiece<0.2
- https://github.com/szha/mx-theme/tarball/master
Expand Down
2 changes: 1 addition & 1 deletion env/gpu/py3-master.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ dependencies:
- scipy=1.3.1
- pip:
- pylint-quotes<0.2
- mxnet-cu101mkl>=1.6.0b20191006
- mxnet-cu100>=1.6.0b20191027
- sacremoses
- sentencepiece<0.2
- https://github.com/szha/mx-theme/tarball/master
Expand Down
2 changes: 1 addition & 1 deletion scripts/parsing/parser/biaffine_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -322,7 +322,7 @@ def flatten_numpy(arr):
return arc_accuracy, rel_accuracy, overall_accuracy, outputs
return outputs

def save_parameters(self, filename):
def save_parameters(self, filename): # pylint: disable=arguments-differ
"""Save model
Parameters
Expand Down
10 changes: 7 additions & 3 deletions src/gluonnlp/model/train/cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,15 +69,19 @@ def __init__(self, lm_model, vocab_size, window, theta, lambdas, **kwargs):
with self.name_scope():
self.lm_model = lm_model

def save_parameters(self, filename):
def save_parameters(self, filename, deduplicate=False):
"""Save parameters to file.
filename : str
Path to file.
deduplicate : bool, default False
If True, save shared parameters only once. Otherwise, if a Block
contains multiple sub-blocks that share parameters, each of the
shared parameters will be separately saved for every sub-block.
"""
self.lm_model.save_parameters(filename)
self.lm_model.save_parameters(filename, deduplicate=deduplicate)

def load_parameters(self, filename, ctx=mx.cpu()): # pylint: disable=arguments-differ
def load_parameters(self, filename, ctx=mx.cpu()): # pylint: disable=arguments-differ
"""Load parameters from file.
filename : str
Expand Down

0 comments on commit bfa5503

Please sign in to comment.