fixing anaconda link in dockerfile #1120

gramhagen · 2020-06-09T18:49:42Z

Description

Fix anaconda link in Dockerfile

Related Issues

#1116

Checklist:

I have followed the contribution guidelines and code style for this project.
I have added tests covering my contributions.
I have updated the documentation accordingly.
This PR is being made to staging and not master.

r-yanyo · 2020-06-10T06:21:51Z

docker/Dockerfile

@@ -90,7 +90,6 @@ WORKDIR ${HOME}
 COPY --from=base ${HOME} .

 # Setup Conda environment
-ENV PATH="${HOME}/conda/bin:${PATH}"


This change happend python: not found error when I built full environment.

Oops. We need this line after since I'm pulling from Nvidia base

miguelgfierro

LGTM

r-yanyo · 2020-06-10T13:13:12Z

This is not about #1120, but docker build still failed at the following problem.
This may be related to aksnzhy/xlearn#211 .

I faced this problem before when I built environment without docker.
I installed cmake and, I fixed this error.

It seems to install cmake, but I don't know why this error happend.
https://github.com/microsoft/recommenders/blob/master/docker/Dockerfile#L20

If this problem should not be handled in this PR, Please ignore.

#22 378.0 Failed to build xlearn
#22 378.0 Installing collected packages: applicationinsights, azureml-telemetry, ruamel.yaml.clib, ruamel.yaml, azure-common, oauthlib, requests-oauthlib, isodate, msrest, PyJWT, adal, msrestazure, azure-mgmt-storage, jeepney, SecretStorage, contextlib2, websocket-client, docker, jmespath, pathspec, azure-graphrbac, azure-mgmt-keyvault, azure-mgmt-authorization, azure-mgmt-containerregistry, jsonpickle, azure-mgmt-resource, pyasn1, ndg-httpsclient, azureml-core, azureml-train-restclients-hyperdrive, azureml-train-core, azureml-train, azureml-dataprep-native, distro, dotnetcore2, fusepy, azureml-dataprep, azureml-pipeline-core, azureml-pipeline-steps, azureml-pipeline, azureml-widgets, azureml-contrib-notebook, azureml-tensorboard, azureml-sdk, azure-core, azure-storage-blob, portalocker, azure-nspkg, azure-cli-nspkg, azure-cli-telemetry, humanfriendly, msal, pkginfo, argcomplete, tabulate, colorama, knack, bcrypt, pynacl, paramiko, msal-extensions, azure-mgmt-core, azure-cli-core, azure-mgmt-cosmosdb, toml, regex, appdirs, typed-ast, black, patsy, statsmodels, category-encoders, pymongo, networkx, hyperopt, idna, zope.interface, zope.event, greenlet, gevent, Werkzeug, itsdangerous, flask, locustio, memory-profiler, nbconvert, pydocumentdb, pymanopt, xlearn, sentencepiece, sacremoses, filelock, docutils, botocore, s3transfer, boto3, tokenizers, transformers, tensorflow-estimator, h5py, keras-applications, termcolor, wrapt, protobuf, google-pasta, markdown, grpcio, absl-py, tensorboard, astor, opt-einsum, gast, keras-preprocessing, tensorflow-gpu, configparser, databricks-cli, nvidia-ml-py3, coverage, json-tricks, simplejson, PythonWebHDFS, schema, nni
#22 378.0   Attempting uninstall: idna
#22 378.0     Found existing installation: idna 2.9
#22 378.0     Uninstalling idna-2.9:
#22 378.0       Successfully uninstalled idna-2.9
#22 378.0   Attempting uninstall: nbconvert
#22 378.0     Found existing installation: nbconvert 5.6.1
#22 378.0     Uninstalling nbconvert-5.6.1:
#22 378.0       Successfully uninstalled nbconvert-5.6.1
#22 378.0     Running setup.py install for xlearn: started
#22 378.0     Running setup.py install for xlearn: finished with status 'error'
#22 378.0 Pip subprocess error:
#22 378.0   ERROR: Command errored out with exit status 1:
#22 378.0    command: /root/conda/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-uucssjaq/xlearn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-uucssjaq/xlearn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-1w__yva7
#22 378.0        cwd: /tmp/pip-install-uucssjaq/xlearn/
#22 378.0   Complete output (51 lines):
#22 378.0   /root/conda/lib/python3.6/site-packages/setuptools/dist.py:454: UserWarning: Normalizing '0.40.a1' to '0.40a1'
#22 378.0     warnings.warn(tmpl.format(**locals()))
#22 378.0   running bdist_wheel
#22 378.0   running build
#22 378.0   running build_py
#22 378.0   Traceback (most recent call last):
#22 378.0     File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 37, in silent_call
#22 378.0       subprocess.check_output(cmd, stderr=shut_up)
#22 378.0     File "/root/conda/lib/python3.6/subprocess.py", line 356, in check_output
#22 378.0       **kwargs).stdout
#22 378.0     File "/root/conda/lib/python3.6/subprocess.py", line 423, in run
#22 378.0       with Popen(*popenargs, **kwargs) as process:
#22 378.0     File "/root/conda/lib/python3.6/subprocess.py", line 729, in __init__
#22 378.0       restore_signals, start_new_session)
#22 378.0     File "/root/conda/lib/python3.6/subprocess.py", line 1364, in _execute_child
#22 378.0       raise child_exception_type(errno_num, err_msg, err_filename)
#22 378.0   FileNotFoundError: [Errno 2] No such file or directory: 'cmake': 'cmake'
#22 378.0
#22 378.0   During handling of the above exception, another exception occurred:
#22 378.0
#22 378.0   Traceback (most recent call last):
#22 378.0     File "<string>", line 1, in <module>
#22 378.0     File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 126, in <module>
#22 378.0       url='https://github.com/aksnzhy/xlearn')
#22 378.0     File "/root/conda/lib/python3.6/site-packages/setuptools/__init__.py", line 161, in setup
#22 378.0       return distutils.core.setup(**attrs)
#22 378.0     File "/root/conda/lib/python3.6/distutils/core.py", line 148, in setup
#22 378.0       dist.run_commands()
#22 378.0     File "/root/conda/lib/python3.6/distutils/dist.py", line 955, in run_commands
#22 378.0       self.run_command(cmd)
#22 378.0     File "/root/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
#22 378.0       cmd_obj.run()
#22 378.0     File "/root/conda/lib/python3.6/site-packages/wheel/bdist_wheel.py", line 223, in run
#22 378.0       self.run_command('build')
#22 378.0     File "/root/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
#22 378.0       self.distribution.run_command(command)
#22 378.0     File "/root/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
#22 378.0       cmd_obj.run()
#22 378.0     File "/root/conda/lib/python3.6/distutils/command/build.py", line 135, in run
#22 378.0       self.run_command(cmd_name)
#22 378.0     File "/root/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
#22 378.0       self.distribution.run_command(command)
#22 378.0     File "/root/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
#22 378.0       cmd_obj.run()
#22 378.0     File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 104, in run
#22 378.0       compile_cpp()
#22 378.0     File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 75, in compile_cpp
#22 378.0       silent_call(cmake_cmd, raise_error=True, error_msg='Please install CMake first')
#22 378.0     File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 41, in silent_call
#22 378.0       raise Exception(error_msg);
#22 378.0   Exception: Please install CMake first
#22 378.0   ----------------------------------------
#22 378.0   ERROR: Failed building wheel for xlearn
#22 378.0 ERROR: azureml-core 1.0.69 has requirement ruamel.yaml<=0.15.89,>=0.15.35, but you'll have ruamel-yaml 0.16.10 which is incompatible.
#22 378.0 ERROR: nni 1.5 has requirement hyperopt==0.1.2, but you'll have hyperopt 0.1.1 which is incompatible.
#22 378.0 ERROR: nni 1.5 has requirement scikit-learn<0.22,>=0.20, but you'll have scikit-learn 0.22.1 which is incompatible.
#22 378.0     ERROR: Command errored out with exit status 1:
#22 378.0      command: /root/conda/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-uucssjaq/xlearn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-uucssjaq/xlearn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-_2lbp1d3/install-record.txt --single-version-externally-managed --compile --install-headers /root/conda/include/python3.6m/xlearn
#22 378.0          cwd: /tmp/pip-install-uucssjaq/xlearn/
#22 378.0     Complete output (55 lines):
#22 378.0     /root/conda/lib/python3.6/site-packages/setuptools/dist.py:454: UserWarning: Normalizing '0.40.a1' to '0.40a1'
#22 378.0       warnings.warn(tmpl.format(**locals()))
#22 378.0     running install
#22 378.0     running build
#22 378.0     running build_py
#22 378.0     Traceback (most recent call last):
#22 378.0       File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 37, in silent_call
#22 378.0         subprocess.check_output(cmd, stderr=shut_up)
#22 378.0       File "/root/conda/lib/python3.6/subprocess.py", line 356, in check_output
#22 378.0         **kwargs).stdout
#22 378.0       File "/root/conda/lib/python3.6/subprocess.py", line 423, in run
#22 378.0         with Popen(*popenargs, **kwargs) as process:
#22 378.0       File "/root/conda/lib/python3.6/subprocess.py", line 729, in __init__
#22 378.0         restore_signals, start_new_session)
#22 378.0       File "/root/conda/lib/python3.6/subprocess.py", line 1364, in _execute_child
#22 378.0         raise child_exception_type(errno_num, err_msg, err_filename)
#22 378.0     FileNotFoundError: [Errno 2] No such file or directory: 'cmake': 'cmake'
#22 378.0
#22 378.0     During handling of the above exception, another exception occurred:
#22 378.0
#22 378.0     Traceback (most recent call last):
#22 378.0       File "<string>", line 1, in <module>
#22 378.0       File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 126, in <module>
#22 378.0         url='https://github.com/aksnzhy/xlearn')
#22 378.0       File "/root/conda/lib/python3.6/site-packages/setuptools/__init__.py", line 161, in setup
#22 378.0         return distutils.core.setup(**attrs)
#22 378.0       File "/root/conda/lib/python3.6/distutils/core.py", line 148, in setup
#22 378.0         dist.run_commands()
#22 378.0       File "/root/conda/lib/python3.6/distutils/dist.py", line 955, in run_commands
#22 378.0         self.run_command(cmd)
#22 378.0       File "/root/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
#22 378.0         cmd_obj.run()
#22 378.0       File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 93, in run
#22 378.0         install.run(self)
#22 378.0       File "/root/conda/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
#22 378.0         return orig.install.run(self)
#22 378.0       File "/root/conda/lib/python3.6/distutils/command/install.py", line 545, in run
#22 378.0         self.run_command('build')
#22 378.0       File "/root/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
#22 378.0         self.distribution.run_command(command)
#22 378.0       File "/root/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
#22 378.0         cmd_obj.run()
#22 378.0       File "/root/conda/lib/python3.6/distutils/command/build.py", line 135, in run
#22 378.0         self.run_command(cmd_name)
#22 378.0       File "/root/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
#22 378.0         self.distribution.run_command(command)
#22 378.0       File "/root/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
#22 378.0         cmd_obj.run()
#22 378.0       File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 104, in run
#22 378.0         compile_cpp()
#22 378.0       File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 75, in compile_cpp
#22 378.0         silent_call(cmake_cmd, raise_error=True, error_msg='Please install CMake first')
#22 378.0       File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 41, in silent_call
#22 378.0         raise Exception(error_msg);
#22 378.0     Exception: Please install CMake first
#22 378.0     ----------------------------------------
#22 378.0 ERROR: Command errored out with exit status 1: /root/conda/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-uucssjaq/xlearn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-uucssjaq/xlearn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-_2lbp1d3/install-record.txt --single-version-externally-managed --compile --install-headers /root/conda/include/python3.6m/xlearn Check the logs for full command output.
#22 378.0
#22 378.0
#22 378.0 CondaEnvException: Pip failed
#22 378.0
#22 378.0
------
failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [/bin/sh -c conda env update -f base.yaml &&     conda clean -fay &&     python -m ipykernel install --user --name 'python3' --display-name 'python3']: runc did not terminate sucessfully

gramhagen · 2020-06-10T15:01:42Z

hmm, odd. yeah. do you mind creating a separate issue for this?
tagging @yueguoguo he's more familiar with xlearn and cmake dependency. I think there was a previous issue around this?

fixing anaconda link in dockerfile

d8e69b1

gramhagen requested a review from yueguoguo June 9, 2020 18:49

gramhagen requested review from anargyri, loomlike and miguelgfierro as code owners June 9, 2020 18:49

r-yanyo reviewed Jun 10, 2020

View reviewed changes

miguelgfierro approved these changes Jun 10, 2020

View reviewed changes

Revert conda path removal

e39f518

r-yanyo mentioned this pull request Jun 11, 2020

[BUG] Docker build failed at 'Failed to build xlearn' #1121

Closed

anargyri approved these changes Jun 11, 2020

View reviewed changes

gramhagen merged commit c6093e3 into staging Jun 11, 2020

miguelgfierro deleted the scgraham/docker_fix branch June 11, 2020 10:22

gramhagen mentioned this pull request Jul 14, 2020

[ASK] Docker build failed at "RUN python" #1116

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fixing anaconda link in dockerfile #1120

fixing anaconda link in dockerfile #1120

gramhagen commented Jun 9, 2020

r-yanyo Jun 10, 2020

gramhagen Jun 10, 2020

miguelgfierro left a comment

r-yanyo commented Jun 10, 2020 •

edited

Loading

gramhagen commented Jun 10, 2020

fixing anaconda link in dockerfile #1120

fixing anaconda link in dockerfile #1120

Conversation

gramhagen commented Jun 9, 2020

Description

Related Issues

Checklist:

r-yanyo Jun 10, 2020

Choose a reason for hiding this comment

gramhagen Jun 10, 2020

Choose a reason for hiding this comment

miguelgfierro left a comment

Choose a reason for hiding this comment

r-yanyo commented Jun 10, 2020 • edited Loading

gramhagen commented Jun 10, 2020

r-yanyo commented Jun 10, 2020 •

edited

Loading