Updated install instructions for FreeBSD 12.2

[sam-harry]

I have written an updated version of the FreeBSD install guide, but I don't see a way to make a pull request for a wiki edit, so I've simply added it here.

The install instructions for 11.4 require a work around for installing the cryptography package from ports. This is not required in FreeBSD 12.2 as the cryptography package can be built by pip during the installation process.

Additionally, some changes were made to make the guide easier to read. The pkg install requirements were simplified to a single line. The build from ports instruction were moved to the bottom and it was specified that they are for 11.4-RELEASE users.

Lastly, the python version included in the instructions was upgraded to 3.8 and clvm_rs to 0.1.19

Thank you @perrin4869 for the testing and correction discussed below.

Suggested wiki page starts below this line

---

Building Chia from Source for FreeBSD

Tested on FreeBSD 12.2-RELEASE

Unfortunately, the cyptography package fails to install via pip in 11.4-RELEASE. It is possible to work around this by manually installing an older version of cryptography from ports (instructions below), but using 12.2-RELEASE or higher is recommended.

---

Upgrading Existing Chia Installs

If you're upgrading from a previously built chia installation, exit from your previous venv environment (deactivate), create a new directory in which to place the latest Chia (e.g. mkdir ~/chia-1.3.1 && cd ~/chia-1.3.1), clone the latest repo (git clone https://github.com/Chia-Network/chia-blockchain.git -b latest), enter it and create a new Python virtual environment within it (python3 -m venv venv). Now, activate the newest environment (. venv/bin/activate), upgrade pip (pip install --upgrade pip). Now you may skip down to the clvm_rs install section and begin there.

---

Why This Manual Installation?

Currently the only way to ensure Chia builds on FreeBSD is to do it from the source. By following these instructions to the letter, you should have no problem building the latest Chia from source on a FreeBSD 12.2.

Notes on TrueNAS

If you had been using NFS or Samba sharing to expose your plots to a harvester on another OS, such as Linux, you can instead build Chia within a jail (see the TrueNAS manual for 'jails'), expose your plot directories to it and run the harvester within. In my experience, it provides lower-latency and more reliable access to the plots since the disks are direct-attached and not being provided through an extra few layers of network protocols.

If you are using a fresh jail created by the TrueNAS web GUI you can access the jail shell by running iocage console JAIL_NAME from the TrueNAS host which will login to the jail as root by default. Althernatively, you may install openssh and setup a ssh key to login. The jail shell CLI provided by the TrueNAS GUI allows copy and pasting so you can easily paste your public-key into /root/.ssh/authorized_keys && chmod -R 700 /root/.ssh.

Other Notes

These instructions will have you building both chia-blockchain and clvm_rs from github source.

These instructions assume a fresh FreeBSD 12 installation!

Discouraged?

Following the instructions in this document will result in a working Chia CLI build on FreeBSD 12 if you follow step-by-step starting from a vanilla FreeBSD installation. Is something broken? Compare the commands you typed, accessible in your bash shell history, and match them with each command in this document. If you feel you've messed something up, do the following:

# if you have (venv) in your shell prompt, type deactivate
deactivate
# remove the chia-blockchain directory which will contain clvm_rs and the Python venv
rm -rf chia-blockchain
# ... now start again!

Pre-requisite package installation

If starting the build again after a failure and you have not re-installed FreeBSD, don't just skip this package installation section! You may have missed one or more software packages critical to the build.

Run the following to install the requried packages. Bash is required sh will not suffice for the build scripts. The development tools be used to build clvm_rs as well as the cryptography dependancy.

pkg install bash python38 py38-maturin py38-sqlite3 git rust gcc autotools pkgconf gmake cmake pybind11

Optional packages

If you are ssh'ing into the machine you might want to use 'screen' so that processes will continue even if you logout. For more information: https://www.freebsd.org/cgi/man.cgi?query=screen. 'tmux' is also a great alternative especially if you use iTerm2 on macOS as it supports native tabs and windows with the '-CC' CLI option.

# optional packages
pkg install screen tmux

Switch to bash

Make sure you change the shell for your chia-blockchain user. If you are logged in as root and will run chia as a non-root user, run this as it appears below. If you are logged in as your non-root user, you can omit the username because you are already that user. Logout and back in again to load the shell.

chsh -s /usr/local/bin/bash NONROOT_USERNAME

Repo Cloning and Virtual Environment (venv) Activation

# Clone the latest chia-blockchain repository, via HTTP:
git clone https://github.com/Chia-Network/chia-blockchain.git -b latest
# or with SSH:
git clone [email protected]:Chia-Network/chia-blockchain.git -b latest
# Note: you can specify the branch by adding "--branch <version>" like: git clone http://github.com/Chia-Network/chia-blockchain.git --branch 1.3.1

Create a virtual environment directory 'venv' from within the 'chia-blockchain' directory and activate it before proceeding

cd chia-blockchain
python3.8 -m venv venv
source venv/bin/activate

You are now in the virtual environment that Python (and so chia) will use. You should have a "(venv)" prefix to your terminal prompt to confirm the venv is working.

Upgrade pip:

pip install --upgrade pip

Install wheel:

pip install wheel

clvm_rs

Build and install the current version of clvm_rs (0.1.19).

git clone http://github.com/Chia-Network/clvm_rs.git --branch 0.1.19
cd clvm_rs/wheel
maturin develop --release
pip install git+https://github.com/Chia-Network/clvm@use_clvm_rs

clvm_rs 0.1.19 is now installed in your virtual environment.

Building Chia Itself

Change directories back to the chia-blockchain directory and run

bash install.sh

Once done, run:

chia init

UPnP

NOTE: if you need to disable UPnP - a protocol which automatically sets up port-forwarding on routers using NAT which is a typical setup at any residence with broadband - set "enable_upnp: False" in config.yaml. You can use the one-liner below or do it yourself.

sed -i .bak 's/enable_upnp: True/enable_upnp: False' ~/.chia/mainnet/config/config.yaml

While you don't absolutely need port 8444 forwarded to your Chia node, it is advised that you do so that other peers may connect to you instead of you solely connecting to them. For the average at-home farmer it is advised you do not disable UPnP unless you absolutely know what you're doing or have another node on your local network already using the port and are planning to Farm on Many Machines).

---

Installing cryptography from ports for 11.4-RELEASE

If using 11.4-RELEASE is required, e.g. runing FreeNAS 11, then you must manually install cyptography into your venv. This must be done after activating the vitrual environement but before running maturin develop --release. This can be achieved by loggin in as root building the port.

portsnap fetch
portsnap extract

cd /usr/ports/security/py-cryptography
# Instruct 'make' that the SSL library is openssl.
# Also force the Python version in case the port tries for a higher one
echo "DEFAULT_VERSIONS+=ssl=openssl python=3.8 python3=3.8" >> /etc/make.conf

make

then logging in as your chia user and copying the site-package to your virtual environment

# assumes that the chia user has chia-blockchain installed in the home directory
export VIRTUAL_ENV=/home/chia/chia-blockchain/venv

cp -R /usr/ports/security/py-cryptography/work-py38/stage/usr/local/lib/python3.8/site-packages/cryptography ${VIRTUAL_ENV}/lib/python3.8/site-packages/cryptography
cp -R /usr/ports/security/py-cryptography/work-py38/stage/usr/local/lib/python3.8/site-packages/cryptography-3.3.2-py3.8.egg-info ${VIRTUAL_ENV}/lib/python3.8/site-packages/cryptography-3.3.2-py3.8.egg-info

and lastely modifying the cryptography dependance in chia-blockchain/setup.py which may be done with sed from the chia-blockchain directory

sed -i '' 's|cryptography==3.4.7|cryptography==3.3.2|g' setup.py

now you are ready to build clvm_rs and continue the install.

---

Installed and Ready to Farm!

That's it! Provided the instructions were followed to the T, and the build is a fresh FreeBSD 12.2, either hardware or TrueNAS jailed, you should be good to go! Now go to town with chia start node or whatever floats your boat.

More details can be found in the Chia Quick Start Guide.

[perrin4869]

Version 1.3.1 is failing to build [email protected]:

💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo didn't build a cdylib. Did you miss crate-type = ["cdylib"] in the lib section of your Cargo.toml?

Any ideas how to fix this?

[perrin4869]

The solution is to replace cd clvm_rs -> cd clvm_rs/wheel

[cross]

I had this problem, and this worked. But, why is that needed in some cases, but not others? Should the maturin always be run in wheel?

[sam-harry]

Looks like I made a mistake in my writeup, @cross. Thank you for the correction @perrin4869

[perrin4869]

It's not a mistake, the instructions were right for chia 1.2.x which use clvm_rs <= 0.1.14

[perrin4869]

Successfully upgraded to 1.3.1 on FreeBSD jail 12.3 using these instructions, just need to checkout clvm_rs tag 0.1.19 and run maturin on the wheel directory instead of the root directory

[sam-harry]

Thank you for the testing and confirmation. I've updated my write up above to reflect it.

[loppefaaret]

@sam-harry Lovely write-up - we have had to lock down the wiki cause of potential attack vectors, but i will forward this to our doc writer, and see if we cant get that into the actual wiki
Thank you.

[perrin4869]

Update for 1.3.4:

clvm_rs was replaced by chia_rs and clvm_tools_rs. If upgrading from a previous version, run pip uninstall clvm_rs.
The new instructions to install the rust wheels are:

git clone http://github.com/Chia-Network/chia_rs.git
git clone https://github.com/Chia-Network/clvm_tools_rs
cd chia_rs/wheel
git checkout 0.1.1
maturin develop --release
cd ../../clvm_tools_rs
git checkout 0.1.7
maturin develop --release

[cross]

This works for me on FreeBSD 12.2 for chia 1.3.1 through 1.3.3. However, when I try to build 1.3.4 following the above process, bash install.sh eventually fails saying:
ERROR: Could not find a version that satisfies the requirement clvm-tools-rs==0.1.7 (from chia-blockchain) (from versions: none) ERROR: No matching distribution found for clvm-tools-rs==0.1.7

Any idea what needs to be tweaked to get past that? A newer branch of clvm_rs (after 0.1.19)?

[perrin4869]

Please see my comment above
#8763 (comment)

[perrin4869]

In FreeNAS 12.3 I've been able to run chia using python39, the instructions are the same, only replacing python38 -> python39 where applicable

[cross]

So, this may exist elsewhere re FreeBSD 13, but. I'm trying to build chia-blockchain 1.8.0 on FreeBSD 13.2. Following the above, I get to @perrin4869 update to chia_rs/clvm_tools_rs, but I get an error. Not knowing rust or wheel, I'm not sure how to address.
(branch/version taken from the CHANGELOG and setup.py in chia-blockchain 1.8.0 sources)

% git clone http://github.com/Chia-Network/chia_rs.git --branch 0.2.7
% git clone https://github.com/Chia-Network/clvm_tools_rs --branch 0.1.30
% cd chia_rs/wheel
% maturin develop --release
💥 maturin failed
  Caused by: Cargo metadata failed. Does your crate compile with `cargo build`?
  Caused by: `cargo metadata` exited with an error: error: failed to parse manifest at `/usr/home/cross2/src/chia-blockchain/chia_rs/Cargo.toml`

Caused by:
  namespaced features with the `dep:` prefix are only allowed on the nightly channel and requires the `-Z namespaced-features` flag on the command-line

% cargo build
error: failed to parse manifest at `/usr/home/cross2/src/chia-blockchain/chia_rs/Cargo.toml`

Caused by:
  namespaced features with the `dep:` prefix are only allowed on the nightly channel and requires the `-Z namespaced-features` flag on the command-line
%

[cross]

So, I manually removed the dep: prefix from the feature listed in chia_rs/Cargo.toml and from the two features in chia_rs/chia-protocol/Cargo.toml. This allowed me to build. But, I have no idea what any of that means, so don't know if what I did was sensible.

[perrin4869]

Did you upgrade cargo?

(venv) [root@chia ~/chia-blockchain]# cargo --version
cargo 1.68.2

[cross]

I did not. I'm seeing

% cargo --version
cargo 1.58.0

If upgrading it was/is the right answer, how would I do that?

[perrin4869]

seems you need at least 1.60.0 https://blog.rust-lang.org/2022/04/07/Rust-1.60.0.html 😉

try this:

pkg update
pkg upgrade

[cross]

Thanks. Upgrading rust to 1.68.2 has let chia_rs build without changes.
When I started this recent process, I was wondering "what packages should I update?" Rust didn't come to mind for whatever reason. I guess I should've started with the "install these packages" list, and upgraded them all.
If we ever get this into a document rather than an issue, we can try to make that more clear. ;-)

[lifros]

Successfully installed chia 2.1.4 on Freebsd 13.2
These are my steps:

sudo pkg install -y git-lite cmake gmake gmp python py39-sqlite3 pybind11 rust

git clone --branch latest https://github.com/Chia-Network/chia-blockchain.git
cd chia-blockchain
python3 -m venv venv
source venv/bin/activate.csh

pip install --upgrade pip

git clone --branch 0.2.15 https://github.com/Chia-Network/chia_rs.git
cd chia_rs/wheel
pip install .

cd ../..

git clone --branch 0.1.39 https://github.com/Chia-Network/clvm_tools_rs.git
cd clvm_tools_rs
pip install .

cd ..

sh ./install.sh

chia init

Everything seems to work but I have a problem with compressed plots.
I got this error message: RuntimeError: Harvester does not support compressed plots.

I setup correctly the config.yaml file.
Another harvester with Debian Linux works correctly with these compressed plots.

Any idea?

Thnaks

[BrandtH22]

Hey, unfortunately I have not tested on FreeBSD so my experience here is very limited.
The first thing i would double check are those config files and that the level of compression is supported by the chia version (2.1.4 supports bladebit up to C7).

If all that checks out then we may need to dig deeper into the code 🤔

If possible, once we have confirmed everything is working properly would you be willing/able to submit a PR updating our install instructions for freeBSD here? https://github.com/Chia-Network/chia-docs/blob/main/docs/getting-started/installation.md?plain=1#L432

[lifros]

I compared the config files with the Linux based harvester where compressed plots works with no problem and it is identical.

The installation of chia blockchain on Freebsd 13.2 went smoothly and did not report any errors or warnings.

It would take someone who knows how blockchain works and give some advice on why this error message: RuntimeError: Harvester does not support compressed plots.

[BrandtH22]

Thank you for the information, our dev team is looking more into the installation on freeBSD and we will update here as more information becomes available 👍

[lifros]

Ok, then I'll wait to update the installation istructions until we have a fully functional setup?

[BrandtH22]

That sounds like a plan, thank you!

[cross]

Did any more come of this? I see the FreeBSD instructions on the docs site are still quite old. And if there was a root cause and explanation of whether or not FreeBSD can harvest compressed plots I want that info too. :-)

[BrandtH22]

Hey @cross , there is no more information than what is available in this thread at this time. To summarize:

• while we do not directly support FreeBSD we are happy to work with the community to make that support possible
• we have updated the chia codebase to fix the install issue which was identified on freeBSD
• we have not identified the issue leading to the compressed plots error message though this could be that the config was not setup to harvest compressed plots.

If you decide to dive more into the freeBSD install instructions or possibilities as to why the compressed plot error occurs let us know if any questions arise and feel free to submit any prs or provide recommendations for prs as they are identified.

Thank you and make it an excellent day,
Brandt H