Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mirgate Terminal Setup Steps into Developer Guide #5133

Merged
merged 9 commits into from
Jul 25, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Before we create web pages according to documents, environments need to be set.
```
conda create --name py36 python=3.6
pip install mkdocs==0.16.3
cd analytics-zoo
cd BigDL
docs/gen_site.py -p -s -m 8080
```
Then choose correct proxy and open the link of your ip address and port number.
Expand Down
31 changes: 31 additions & 0 deletions docs/readthedocs/source/doc/UserGuide/develop.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,9 +89,40 @@ export PYTHONPATH=BigDL/python/dllib/src:BigDL/python/nano/src:BigDL/python/orca
export PYTHONPATH=BigDL/python/dist/conf/spark-bigdl.conf:$PYTHONPATH
```

- Install and add `tflibs` to `TF_LIBS_PATH`:
```bash
# Install bigdl-tf and bigdl-math
pip install bigdl-tf bigdl-math

# Configure TF_LIBS_PATH
export TF_LIBS_PATH=$(python -c 'import site; print(site.getsitepackages()[0])')/bigdl/share/tflibs
```


The above environment variables should be available when running or debugging code in the IDE. When running applications in PyCharm, you can add runtime environment variables by clicking __Run__ -> __Edit Configurations__; then in the __Run/Debug Configurations__ panel, you can add necessary environment variables to your applications.


#### **1.3 Terminal Setup**

Besides setting the environment variables mentioned above manually for Linux users, we also provide a solution to set them with a script:

```bash
# Install bigdl-tf and bigdl-math
pip install bigdl-tf bigdl-math

cd BigDL/python/friesian
source dev/prepare_env.sh
```

You can verify the BigDL environment by running the following example.

```bash
python BigDL/python/dllib/examples/autograd/custom.py
```

Note that this approach will only work temporarily for this terminal.


### **2. Scala**

#### **2.1 Build**
Expand Down
8 changes: 5 additions & 3 deletions python/orca/developer_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,22 +34,24 @@ After running the above code there will be a `BigDl/dist` directory consisting o
```bash
pip install pyspark==2.4.3 # ./make-dist.sh build against spark 2.4.3 by default
pip install ray==1.9.2
pip install aiohttp==3.7.4 # dependencis introduced by ray, latest version has api changes
pip install aioredis==1.3.1 # dependencis introduced by ray, latest version has api changes
pip install aiohttp==3.7.4 # dependencies introduced by ray, latest version has api changes
pip install aioredis==1.3.1 # dependencies introduced by ray, latest version has api changes
pip install pytest pyarrow pandas
pip install bigdl-math bigdl-tf # dependencies introduced by bigdl-orca
pip3 install torch==1.10.1+cpu torchvision==0.11.2+cpu torchaudio==0.10.1+cpu -f https://download.pytorch.org/whl/cpu/torch_stable.html
```

2. Set a few environment variables

```bash
cd BigDL/python/orca
source dev/test/prepare_env.sh
source dev/prepare_env.sh
```

3. Run tests to verify environment

```bash
cd BigDL/python/orca
pytest test/bigdl/orca/learn/ray/pytorch/test_estimator_pyspark_backend.py
```

Expand Down