Auto pip install

[jiakunw]

I automated pip install but you have to manually set environmental variable.

[jiakunw]

The code I wrote ran successfully, but the later part did not becasue the cluster did not have cython installed. When I attempted to install cython I got permission errors:

[jiakunw]

This time I added requirement.txt. When I ran python setup.py install I got the following errors:


I also tried running pip.install mirage but I got some other errors:

[jiazhihao]

This time I added requirement.txt. When I ran python setup.py install I got the following errors:

Make sure you compile on a GPU node (instead of the catalyst-cluster login node), since there is no GPUs on the login node. You can do so by running salloc --nodes 1 --gres=gpu:1 and then ssh to the assigned node, which should be one of catalyst-0-9 ... catalyst-0-15.

I also tried running pip.install mirage but I got some other errors:

I am not 100% how to resolve this issue.

[jiakunw]

I fixed some bugs but _cython is not in mirage

[jiazhihao]

The issue is that setup.py does not find mirage/_cython, which is at mirage/python/mirage/_cython. I have pushed a change to setup.py to fix the dir path issue. Can you pull the changes in the main branch and try again?

[jiakunw]

I tried running the commands maually using the unmodified verison of setup.py as specified in INSTALL.md, and I got the same error message. Using pdb I realized the error message came from python return cythonize(ret, compiler_directives={"language_level" : 3}) could you please take a look at config_cython() ?

[jiakunw]

does that mean it worked?

I also have some questions about it. cuDNN and CUDA is not available as a Python package on PyPI. cuDNN is a GPU-accelerated library for deep neural networks provided by NVIDIA, and it should be installed via conda or directly from NVIDIA's website, not from PyPI. So I removed cuda and cudnn version from requirements.txt, but they are both installed, and it seemed working. Does that mean it's done or I should fix them??

[jiazhihao]

@jiakunw Thanks for the PR. One quick comment: you may not want to include your personal vscode config in this PR: vscode/sftp.json.

[jiakunw]

fixed!

[jiazhihao]

The PR looks great overall. One last thing: you can use git rm to remove the sftp.json file and run script/format.sh to auto-format your changes.

[jiakunw]

script/format.sh

scripts/format.sh: line 72: /Users/jackiexu/Development/mirage/.tools/clang-format-15-master-1d7ec53d: cannot execute binary file: Exec format error``` I got this error when I ran it

[jiazhihao]

I can correct format in another PR. Can you use git rm to remove your json file from this PR.

[jiakunw]

I removed it yesterday and there was no json file in the repo

[jiazhihao]

It is still there in this PR: https://github.com/mirage-project/mirage/pull/20/files. Let me push a fix to address this issue.

[jiakunw]

I re-removed it, it should be good for now :)

[jiazhihao]

merged. Great work!