Cross Compiler CMake Usage Guide with rsynced Raspberry Pi 32 bit OS

Cross-Compiler CMake Usage Guide : Rsynced Raspberry Pi 32-bit OSes Sysroot

This guide documents the complete steps to create rootfs/sysroot so that you can create a cross compile environment for any Raspberry Pi(running 32-bit OS) on a Linux machine using rsync from the Raspberry Pi hardware you're already running. After that we will cross-compile a working Software Binaries (Hello-World CMAKE example in this case) with CMAKE using only the Raspberry Pi GCC Toolchains available within our project.

Note

Our Cross-Compiler toolchains also works out-of-the-box on any Linux distro via WSL2 on Windows 10 Machines. 💯

 

 

Prerequisites:

A. Hardware:

• Host [PC/Laptop]: Any x86/x86_64 AMD/Intel machine
• Target [Raspberry Pi]: Raspberry Pi any variant/module

B. Software:

• Host: Any Linux machine (Our Cross-Compiler toolchains also works out-of-the-box on any Linux distro via WSL2 on Windows 10 Machines. 💯)
• Target: Any Raspberry Pi 32-bit Buster (Debian 10) and above OS (Raspberry Pi OS Bookworm Tested)

C. Others:

• Networking: Your Target Machine (Raspberry Pi) and Host Machine (where you cross-compiling) both MUST have Internet Access, and MUST be on SAME Network to follow these instructions.

 

Steps/Settings for Target Machine (Raspberry Pi):

1. Start from Scratch (Optional)

Important

If you just brought a new Raspberry Pi or wanted to start from scratch just follow along. Otherwise, if you already has your Raspberry Pi setup, running, and Network Ready, then just skip to step 2.

Note

This section assume you have atleast 10GB SDcard for installing Raspberry Pi OS Bookworm OS and a Laptop/PC for uploading it.

1.1. Download Softwares & Prepare the SD card

1. Download the latest version of Raspberry Pi OS (Bookworm) from here on your laptop/pc.
2. You will be needing an image writer to write the downloaded OS into the SD card (micro SD card in our case). So download the open-source "win32 disk imager" from here, OR you can also use Balena Etcher instead.
3. Insert the SD card into the laptop/pc and run the image writer. Once open, browse and select the downloaded Raspberry Pi OS image file. Select the correct device, that is the drive representing the SD card.

Caution

If the drive (or device) selected is different from the SD card then the other selected drive will become corrupted. SO BE CAREFUL!

4. Once the write is complete, eject the SD card and insert it into the Raspberry Pi and turn it on. It should start booting up.
5. Please remember that after booting the Pi, there might be situations when the user credentials like the "username" and password will be asked. Raspberry Pi comes with a default username pi and password raspberry and so use it whenever it is being asked.

1.2 Set up Network

Now the you have your Raspberry Pi up and Running, its time to connect it your network with one of following ways:

1. If you have Monitor: Connect it to your raspberry pi along with a keyboard and mouse to navigate, and follow this guide.
2. If you don't have Monitor: Follow this guide
3. Any other way

2. Set up SSH

• If you have Monitor: On the Raspberry Pi terminal, type: sudo raspi-config and menu should pop up on your terminal. To enable SSH, go to: Interfacing Options -> SSH -> Yes, and Click OK to enable it. Choose Finish finally and exit.

• If you don't have Monitor: After setting up the network, if you don't have monitor or you operating it remotely. Then, enable SSH by just taking out your SD card, and hook it your computer, and simply create an empty file called ssh in the /boot/partition path inside SD card. Now insert back SD card into the Raspberry Pi.

3. Open Terminal

• From another Laptop/PC using SSH: To connect to your Pi from a different computer, copy and paste the following command into the terminal window:

Warning

• Change 192.168.1.47 in following commands with the IP-address of your Raspberry Pi present on the same network.
• Change pi with the SSH username for your Raspberry Pi.
• After executing this command below use the Raspberry Pi's SSH password.

ssh [email protected] 

It will ask for password, and if not changed, it is default (raspberry), and so use it whenever it is being asked.

Note

It is possible to configure your Raspberry Pi to allow access from another computer without needing to provide a password each time you connect. For more details, see here.

• On Raspberry Pi directly with a Monitor: Just search "Terminal" and click on it.

4. Enable Development Sources

You need to edit your sources list to enable development sources. To do this, enter the following command into pi terminal:

sudo nano /etc/apt/sources.list

In the nano text editor, uncomment the following lines by removing the # character from following line:

deb-src http://raspbian.raspberrypi.com/raspbian/ bookworm main contrib non-free rpi

When done, press CTRL+O and then ENTER to quit.

5. Update the system

Run the following commands in terminal to update the system

sudo apt update
sudo apt dist-upgrade
sudo apt install rsync

6. Enable rsync with elevated rights

Later in this guide, we will be using the rsync command to sync files between the Host PC/Laptop and the Raspberry Pi. For some of these files, root rights (i.e. sudo) is required internally.

You can do this with a single terminal command as follows:

echo "$USER ALL=NOPASSWD:$(which rsync)" | sudo tee --append /etc/sudoers

That's it. Now rsync should be setup to run with sudo if needed.

7. Install the important Development Packages

Run the following commands in Raspberry Pi terminal to install the required packages:

sudo apt install build-essential cmake unzip pkg-config gfortran gcc g++ gperf flex texinfo gawk bison openssl pigz libncurses-dev autoconf automake tar figlet

Important

Make sure to install additional packages based on software you cross-compiling with CMAKE.

8. Setup Important Symlinks as follows: (Important)

Our toolchains requires few additional symbolic links to work properly. Therefore, to create all required symbolic link reliably, we need to download SSymlinker bash script as follows:

wget https://raw.githubusercontent.com/abhiTronix/raspberry-pi-cross-compilers/refs/heads/master/utils/SSymlinker

Once it is downloaded, you just need to make it executable, and then run it for each path manually using the following commands:

sudo chmod +x SSymlinker
./SSymlinker -s /usr/include/arm-linux-gnueabihf/asm -d /usr/include
./SSymlinker -s /usr/include/arm-linux-gnueabihf/gnu -d /usr/include
./SSymlinker -s /usr/include/arm-linux-gnueabihf/bits -d /usr/include
./SSymlinker -s /usr/include/arm-linux-gnueabihf/sys -d /usr/include
./SSymlinker -s /usr/include/arm-linux-gnueabihf/openssl -d /usr/include
./SSymlinker -s /usr/lib/arm-linux-gnueabihf/crtn.o -d /usr/lib/crtn.o
./SSymlinker -s /usr/lib/arm-linux-gnueabihf/crt1.o -d /usr/lib/crt1.o
./SSymlinker -s /usr/lib/arm-linux-gnueabihf/crti.o -d /usr/lib/crti.o

That's it for Raspberry Pi setup.

 

 

Steps/Settings for Host Machine (PC/Laptop)

Now Raspberry Pi Side all setup, Let's focus on commands for our Host Machine, i.e. PC/Laptop, where you going to cross-compile software binaries (Hello world application in this case) for your Raspberry Pi.

Important

Make sure your Raspberry Pi and this Host machine (where you cross-compiling) MUST be on the SAME Network.

1. Update the Host Machine

First of all, Run the following commands to update your system and install important dependencies:

sudo apt update
sudo apt dist-upgrade
sudo apt install build-essential cmake unzip gfortran 
sudo apt install gcc git bison python gperf pkg-config gdb-multiarch wget rsync libgmp-dev libmpfr-dev
sudo apt install g++ gperf flex texinfo gawk bison openssl pigz libncurses-dev autoconf automake tar figlet

2. Setting up the directory structure

You can use these following commands to create "cmake-test" to use as workspace for the project:

sudo mkdir ~/cmake-test
sudo mkdir ~/cmake-test/tools
sudo mkdir ~/cmake-test/build
sudo mkdir ~/cmake-test/rootfs
sudo mkdir ~/cmake-test/rootfs/usr
sudo mkdir ~/cmake-test/rootfs/opt
sudo chown -R 1000:1000 ~/cmake-test
cd ~/cmake-test

Note

Ensure the last command should have changed your current directory to ~/cmake-test. If not, run the last line again to make sure you are inside it, as the next steps assume you're running your commands from this directory.

3. Download & Extract the Precompiled Cross-Compiler

Let's first change into tools directory for downloading our Precompiled Cross-compiler with the following command:

cd ~/cmake-test/tools

Caution

Ensure the last command should have changed your current directory to ~/cmake-test/tools now. If not, run it again.

A. Copy Binary URL:

Copy URL from one of following Precompiled Compressed Base (or Pre-installed) Toolchain version (for maximum compatibility) based on your Raspberry Pi Variant and OS you installed on it from below:

Warning

The Stretch (Debian Version 9) 32-bit/64-bit toolchains are no longer supported!

Raspberry Pi Board	Buster 32-bit OS (Debian Version 10)	Bullseye 32-bit OS (Debian Version 11)	Bookworm 32-bit OS (Debian Version 12)
Raspberry Pi - Zero W/WH/2W & 1 Model A/B/A+/B+	8.3.0	10.2.0	12.2.0
Raspberry Pi - 2/3 Model A/B	8.3.0	10.2.0	12.2.0
Raspberry Pi - 3 Model A+/B+ & 4 Model 400/B & 5 & Compute 3/3-lite/3+/4/4S	8.3.0	10.2.0	12.2.0

Tip

You can also use the latest cross-compiler binaries instead. But they are not tested or recommended.

B. Download Binary

After that, paste your copied URL and run the following command to download the Base Cross-compiler:

wget <Copied Binary URL goes here> #for e.g. => wget https://sourceforge.net/projects/raspberry-pi-cross-compilers/files/Raspberry%20Pi%20GCC%20Cross-Compiler%20Toolchains/Bookworm/GCC%2012.2.0/Raspberry%20Pi%202%2C%203/cross-gcc-12.2.0-pi_2-3.tar.gz 

C. Extract Binary:

Once it is downloaded, we can extract it using the following command:

tar xf cross-gcc-*.tar.gz

4. Sync Raspberry Pi rootfs (Most Important)

First, let's move back into the cmake-test folder as needed for the next sections:

cd ~/cmake-test

Now, we need to sync up our rootfs folder with the system files from the Raspberry Pi. We will be using rsync that let us sync (i.e. copy) files from the Raspberry Pi with appropriate permission onto your Host Machine, potentially saving you a lot of time.

To do this, enter the following commands one by one into your terminal:

Warning

• Change 192.168.1.47 in following commands with the IP-address of your Raspberry Pi present on the same network.
• Change pi with the default SSH username you used for your Raspberry Pi.
• After executing each command below use the Raspberry Pi's default SSH password.

• Command 1: rsync -avz --rsync-path="sudo rsync" --delete [email protected]:/lib rootfs
• Command 2: rsync -avz --rsync-path="sudo rsync" --delete [email protected]:/usr/include rootfs/usr
• Command 3: rsync -avz --rsync-path="sudo rsync" --delete [email protected]:/usr/lib rootfs/usr
• Command 4: rsync -avz --rsync-path="sudo rsync" --delete [email protected]:/opt rootfs/opt

Caution

Double check after each of the above commands that all the files have been copied to ~/cmake-test/rootfs folder. There will be an information message if there were any issues.

5. Fix symbolic links

The files we copied in the previous step still have symbolic links pointing to the file system on the Raspberry Pi. We need to alter this so that they become relative links from the new rootfs directory on the host machine. We can do this with a python script, which we can download as follows:

wget https://raw.githubusercontent.com/abhiTronix/rpi_rootfs/master/scripts/sysroot-relativelinks.py

Once it is downloaded, you just need to make it executable and run it, using the following commands:

sudo chmod +x sysroot-relativelinks.py
./sysroot-relativelinks.py rootfs

 

 

Build Dummy Hello-World Project

First, let's move back into the cmake-test folder as needed for the next sections:

cd ~/cmake-test

Now we need to create suitable files for our CMAKE project.

1. Create C++ main.cpp file:

Let’s try to compile and run a C++17 code that uses an if block with init-statement (the example is a bit simple, but will show you how to compile C++17 programs):

Note

This C++ program demonstrates the use of an if statement with an init-statement, a C++17 feature that allows declaring a variable within the if condition, limiting its scope to the if-else block.

// Include necessary header files
#include <iostream>  // For input/output operations

// Using the standard namespace (Note: generally not recommended in larger projects)
using namespace std;

int main() {
    // Demonstrating an if statement with an init-statement (C++17 feature)
    // The init-statement allows you to declare and initialize a variable
    // that is scoped to the if-else block
    if (int a = 5; a < 8) {
        // This block executes if 'a' is less than 8
        cout << "Local variable a is < 8\n";
    } else {
        // This block executes if 'a' is greater than or equal to 8
        // Note: In this case, this block will never execute because 'a' is always 5
        cout << "Local variable a is >= 8\n";
    }

    // Return 0 to indicate successful program execution
    return 0;
}

2. Create PI.cmake CMAKE File

Let's create a typical cross-compiling toolchain that has content such as:

Caution

Replace Cross-Compiler Toolchain's absolute path with yours in following CMAKE file before saving it.

# Enable verbose output for debugging
set(CMAKE_VERBOSE_MAKEFILE ON)

# Set the target system details
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR arm)

# Set the path to the cross-compilation toolchain
# WARNING: Change this path to match your toolchain location
set(tools $ENV{HOME}/cmake-test/tools/cross-pi-gcc-12.2.0-1)

# Set the path to the target system's root filesystem
set(rootfs_dir $ENV{HOME}/cmake-test/rootfs)

# Configure the root path for finding libraries and headers
set(CMAKE_FIND_ROOT_PATH ${rootfs_dir})
set(CMAKE_SYSROOT ${rootfs_dir})

# Set the target architecture
set(CMAKE_LIBRARY_ARCHITECTURE arm-linux-gnueabihf)

# Configure flags for linking and compiling
set(common_flags "-fPIC -Wl,-rpath-link,${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE} -L${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE}")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${common_flags}")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${common_flags}")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${common_flags}")

# Set the prefix for the cross-compiler binaries
set(BIN_PREFIX ${tools}/bin/arm-linux-gnueabihf)

# Configure the cross-compiler tools
set(CMAKE_C_COMPILER ${BIN_PREFIX}-gcc)
set(CMAKE_CXX_COMPILER ${BIN_PREFIX}-g++)
set(CMAKE_Fortran_COMPILER ${BIN_PREFIX}-gfortran)

# Configure additional cross-compiler tools
set(CMAKE_LINKER ${BIN_PREFIX}-ld CACHE STRING "Set the cross-compiler tool LD" FORCE)
set(CMAKE_AR ${BIN_PREFIX}-ar CACHE STRING "Set the cross-compiler tool AR" FORCE)
set(CMAKE_NM ${BIN_PREFIX}-nm CACHE STRING "Set the cross-compiler tool NM" FORCE)
set(CMAKE_OBJCOPY ${BIN_PREFIX}-objcopy CACHE STRING "Set the cross-compiler tool OBJCOPY" FORCE)
set(CMAKE_OBJDUMP ${BIN_PREFIX}-objdump CACHE STRING "Set the cross-compiler tool OBJDUMP" FORCE)
set(CMAKE_RANLIB ${BIN_PREFIX}-ranlib CACHE STRING "Set the cross-compiler tool RANLIB" FORCE)
set(CMAKE_STRIP ${BIN_PREFIX}-strip CACHE STRING "Set the cross-compiler tool STRIP" FORCE)

# Configure the behavior for finding programs, libraries, and include files
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

3. Create CMakeLists.txt File

Finally create a suitable project name (for e.g. cmake_hello_world_test) in CMakeLists.txt file:

# Specify the minimum version of CMake required
cmake_minimum_required(VERSION 3.10)

# Define the project name
project(cmake_hello_world_test)

# Enable verbose output during the build process
# This is useful for debugging build issues
set(CMAKE_VERBOSE_MAKEFILE ON)

# Set the C++ standard to C++17
# This ensures that C++17 features are available for the project
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

# Define the executable target
# This creates an executable named 'cmake_hello_world_test' from the 'main.cpp' source file
add_executable(cmake_hello_world_test main.cpp)

# Optional: Add include directories if needed
# include_directories(include)

# Optional: Link libraries if needed
# target_link_libraries(cmake_hello_world_test some_library)

# Optional: Set compile options if needed
# target_compile_options(cmake_hello_world_test PRIVATE -Wall -Wextra)

4. Configure CMAKE Project

Let's move into the build directory for further steps, as we don't want to build within that source directory as its crowded, so we will access it from within this this directory:

cd ~/cmake-test/build

Caution

Ensure you are still in the ~/cmake-test/build directory.

Finally, Now we can configure our Hello-World CMAKE Project as follows:

cmake -DCMAKE_TOOLCHAIN_FILE=~/cmake-test/PI.cmake  -DCMAKE_BUILD_TYPE=Debug ..

Important

If you get error like CMake Error at ...... (file): file failed to open for writing (No such file or directory). Then it is a permission issue and can be fixed using command:

sudo chown -R 1000:1000 ~/cmake-test

5. Finally Build Project

Our build has been configured now, and it is time to actually build the source files, and run the following command:

make -j$(nproc)

Note

The -j$(nproc) option indicates that the job should be spread into multiple threads and run in parallel on available cores.

and your compiled binary cmake_hello_world_test will be available at ~/cmake-test/build.

6. Copy our Cross-compiled Binary [Optional]

Let's copy our built cross-compiled binary back to our Raspberry Pi's home directory:

Warning

• Change 192.168.1.47 in following commands with the IP-address of your Raspberry Pi present on the same network.
• Change pi with the default SSH username you used for your Raspberry Pi.
• After executing this command below use the Raspberry Pi's SSH password.

rsync -avz ~/cmake-test/build/cmake_hello_world_test --delete [email protected]:~/cmake_hello_world_test

 

 

Steps for Target Machine (Raspberry Pi) [Optional]

After copying our cross-compiled binary cmake_hello_world_test, lets try to run it on our Raspberry Pi:

Run/Test our Cross-compiled Binary [Optional]

Let's move into the Raspberry Pi's home directory where we copied our cross-compiled binary cmake_hello_world_test, and then run it:

# move into home directory
cd ~/

# run the cross compiled binary
./cmake_hello_world_test

Check output and it should look something as follows:

$ ./cmake_hello_world_test
Local variable a is < 8

The Local variable a is < 8 output indicates the cross-compilation and if statement with an init-statement (C++17 feature) worked correctly.