MIRA
Cross Compiling


Contents

Basics

Cross compiling can become interesting if one wants to run MIRA on a device that is hardly capable of compiling MIRA on its own (e.g. due to CPU or MEMORY limitation). Cross compile can greatly help in this situation since it makes it possible to build binaries for a certain architecture (the target system) on a build system with different architecture.

The basic principle consists of 5 basic steps:

Note
This tutorial concentrates on cross compiling MIRA for a Raspberry Pi (ARM). You need to transform the paths, user names and IP addresses in this tutorial to match your system setup.
This tutorial assumes that you use
cd /home
as working directory
Tested Raspberry Pi versions are Zero W (0), 2 and 3.
Tested Raspbian versions are Wheezy, Stretch and Buster.
Tested GCC version are 6.3, 6.5 and 7.3. 8.3 is highly experimental.

Prepare MIRA system requirements

Since MIRA should run on the target system, all the system requirements have to be met on that system. If a standard linux distribution is used on the target system (e.g. Raspian (which is a Debian) on the Raspberry Pi) chances are high that all system requirements can be found in the default repositories.

On a Raspberry Pi with Raspian, all that needs to be done is to execute the following command:

sudo apt-get install libboost-all-dev libqt4-dev libqtwebkit-dev libogre-1.9-dev libopencv-dev libsqlite3-dev cmake libsvn-dev libxml2-dev libv4l-dev libssl1.0-dev libxrandr-dev zlib1g-dev libbz2-dev

Please note that this command installs the development versions of the libraries. Although it is not required to have header files of the libraries on the target system it simplifies the following steps a lot.

If not all the required libraries can be found in the default repositories they need to built e.g. from source code.

CMake for newer versions of Boost

Since on newer versions of Raspbian there are incompatible versions of cmake and boost in the default repositories, a newer version of cmake has to be downloaded manually:

wget https://cmake.org/files/v3.15/cmake-3.15.2-Linux-x86_64.sh
chmod +x cmake-3.15.2-Linux-x86_64.sh
./cmake-3.15.2-Linux-x86_64.sh

To use it by default, the PATH variable has to be modified:

export PATH=/home/cmake-3.15.2-Linux-x86_64/bin:$PATH

Setup cross compiling toolchain

Since the build system needs to generate binaries for a different architecture, a special compiler is required on the build system.

Setup cross compiling toolchain

This is quite simple for the Raspberry Pi for Raspbian Wheezy since there are preconfigured and prebuilt compilers available. Therefore, the following command can be used to obtain a toolchain for the Raspberry Pi:

git clone https://github.com/raspberrypi/tools

CrosstoolNG for newer versions of Raspbian (>= Stretch)

For newer versions of Raspbian, a toolchain has to be created manually. This can be done with CrosstoolNG and preconfigured config files for it.

First, the requirements for CrosstoolNG have to be installed:

sudo apt-get install gcc gperf bison flex texinfo help2man make libncurses5-dev autoconf automake libtool libtool-bin gawk

To download and build it, the following commands needs to be executed:

wget http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.24.0.tar.bz2
tar -xjvf crosstool-ng-1.24.0.tar.bz2
cd crosstool-ng-1.24.0
mkdir install
./configure --prefix=/home/crosstool-ng-1.24.0/install
make
make install

To use it, the PATH variable has to be modified:

export PATH=$PATH:/home/crosstool-ng-1.24.0/install/bin

Create toolchain with CrosstoolNG

For convenience, download a preconfigured config file:

mkdir gcc-linaro-7.3-arm-rpi3-linux-gnueabihf
cd gcc-linaro-7.3-arm-rpi3-linux-gnueabihf
wget ftp://nikr-synas.informatik.tu-ilmenau.de/NIKR-Downloads-anonym/RPiCrossCompiling/gcc-linaro-7.3-arm-rpi3-linux-gnueabihf.config -O .config
mkdir tarballs

Some minor changes have to be done

For GCC < 8 and kernel != 4.10.8 (check with 'uname -r' on the Raspberry Pi), the kernel version has to be updated:

export KERNEL_VERSION_MAJOR=4 # <- change this according to 'uname -r'
export KERNEL_VERSION_MINOR=10 # <- change this according to 'uname -r'
export KERNEL_VERSION_PATCH=8 # <- change this according to 'uname -r'
sed -i 's:^\(# \)\?CT_KERNEL_VERSION[= ].\+$:CT_KERNEL_VERSION="'${KERNEL_VERSION_MAJOR}'.'${KERNEL_VERSION_MINOR}'.'${KERNEL_VERSION_PATCH}'":g' .config
sed -i 's:^\(# \)\?CT_LIBC_GLIBC_MIN_KERNEL[= ].\+$:CT_LIBC_GLIBC_MIN_KERNEL="'${KERNEL_VERSION_MAJOR}'.'${KERNEL_VERSION_MINOR}'.'${KERNEL_VERSION_PATCH}'":g' .config
sed -i 's:^\(# \)\?CT_KERNEL_V_4_10[= ].\+$:CT_KERNEL_V_'${KERNEL_VERSION_MAJOR}'_'${KERNEL_VERSION_MINOR}'=y\n# CT_KERNEL_V_4_10 is not set:g' .config

For GCC >= 8 and kernel != 4.19.66 (check with 'uname -r' on the Raspberry Pi), the kernel version has to be updated:

export KERNEL_VERSION_MAJOR=4 # <- change this according to 'uname -r'
export KERNEL_VERSION_MINOR=19 # <- change this according to 'uname -r'
export KERNEL_VERSION_PATCH=66 # <- change this according to 'uname -r'
sed -i 's:^\(# \)\?CT_LINUX_VERSION[= ].\+$:CT_LINUX_VERSION="'${KERNEL_VERSION_MAJOR}'.'${KERNEL_VERSION_MINOR}'.'${KERNEL_VERSION_PATCH}'":g' .config
sed -i 's:^\(# \)\?CT_GLIBC_MIN_KERNEL[= ].\+$:CT_GLIBC_MIN_KERNEL="'${KERNEL_VERSION_MAJOR}'.'${KERNEL_VERSION_MINOR}'.'${KERNEL_VERSION_PATCH}'":g' .config
sed -i 's:^\(# \)\?CT_LINUX_V_4_19[= ].\+$:CT_LINUX_V_'${KERNEL_VERSION_MAJOR}'_'${KERNEL_VERSION_MINOR}'=y\n# CT_KERNEL_V_4_19 is not set:g' .config

For binutils != 2.28 (check with 'ld –version' on the Raspberry Pi), the binutils version has to be updated:

export BINUTILS_VERSION_MAJOR=2 # <- change this according to 'ld --version'
export BINUTILS_VERSION_MINOR=28 # <- change this according to 'ld --version'
sed -i 's:^\(# \)\?CT_BINUTILS_VERSION[= ].\+$:CT_BINUTILS_VERSION="'${BINUTILS_VERSION_MAJOR}'.'${BINUTILS_VERSION_MINOR}'":g' .config
sed -i 's:^\(# \)\?CT_BINUTILS_V_2_28[= ].\+$:CT_BINUTILS_V_'${BINUTILS_VERSION_MAJOR}'_'${BINUTILS_VERSION_MINOR}'=y\n# CT_BINUTILS_V_2_28 is not set:g' .config

For GCC < 8 and libc != 2.23 (check with 'ldd –version' on the Raspberry Pi), the libc version has to be updated:

export LIBC_VERSION_MAJOR=2 # <- change this according to 'ldd --version'
export LIBC_VERSION_MINOR=23 # <- change this according to 'ldd --version'
sed -i 's:^\(# \)\?CT_LIBC_VERSION[= ].\+$:CT_LIBC_VERSION="'${LIBC_VERSION_MAJOR}'.'${LIBC_VERSION_MINOR}'":g' .config
sed -i 's:^\(# \)\?CT_LIBC_V_2_23[= ].\+$:CT_LIBC_V_'${LIBC_VERSION_MAJOR}'_'${LIBC_VERSION_MINOR}'=y\n# CT_LIBC_V_2_23 is not set:g' .config

For GCC >= 8 and libc != 2.28 (check with 'ldd –version' on the Raspberry Pi), the libc version has to be updated:

export LIBC_VERSION_MAJOR=2 # <- change this according to 'ldd --version'
export LIBC_VERSION_MINOR=28 # <- change this according to 'ldd --version'
sed -i 's:^\(# \)\?CT_GLIBC_VERSION[= ].\+$:CT_GLIBC_VERSION="'${LIBC_VERSION_MAJOR}'.'${LIBC_VERSION_MINOR}'":g' .config
sed -i 's:^\(# \)\?CT_GLIBC_V_2_28[= ].\+$:CT_GLIBC_V_'${LIBC_VERSION_MAJOR}'_'${LIBC_VERSION_MINOR}'=y\n# CT_GLIBC_V_2_28 is not set:g' .config

To finally compile the toolchain, the following commands have to be executed:

Note
This takes a very long time!
LD_LIBRARY_PATH_BACKUP=${LD_LIBRARY_PATH}
unset LD_LIBRARY_PATH
ct-ng build
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH_BACKUP}
unset LD_LIBRARY_PATH_BACKUP

Copy libraries of target system to the build system

All target system libraries and header files needed to build MIRA have to be available on the build system while the build is in progress:

mkdir /home/remoteFS
Note
Some system libraries install linker scripts in /usr/lib that contain absolute paths. Since these paths are not resolved correctly on the build system the linker struggles to link libraries correctly. To solve this problem you may need to edit the linker scripts (for me this was the case for /usr/lib/arm-linux-gnueabihf/libpthread.so and /usr/lib/arm-linux-gnueabihf/libc.so). I recommend to edit the files so that the absolute paths are removed (I did not notice any negative effects, but please don't blame me if anything goes wrong). Please check that the linker can find the libraries. On some distributions you might have to use relative paths. If you are facing any linking problems you might want to add "-Wl,-verbose" to the linker flags defined in "make/Crosscompile.cmake" in order to get a verbose linker output. Therefore, the original libc.so:
/* GNU ld script
Use the shared library, but some functions are only in
the static library, so try that secondarily. */
OUTPUT_FORMAT(elf32-littlearm)
GROUP ( /lib/arm-linux-gnueabihf/libc.so.6 /lib/arm-linux-gnueabihf/libc_nonshared.a AS_NEEDED ( /lib/arm-linux-gnueabihf/ld-linux-armhf.so.3 ) )
needs to be changed to:
/* GNU ld script
Use the shared library, but some functions are only in
the static library, so try that secondarily. */
OUTPUT_FORMAT(elf32-littlearm)
GROUP ( libc.so.6 libc_nonshared.a AS_NEEDED ( ld-linux-armhf.so.3 ) )
Note
In newer versions of Raspbian, ldconfig is used to determine the pathes for run-time linking:
mkdir /home/remoteFS/etc
cat > /home/remoteFS/etc/ld.so.conf << EOF
/lib/arm-linux-gnueabihf
/usr/lib/arm-linux-gnueabihf
/opt/vc/lib
EOF

There are several different ways to make the files available on the build system. Two of them are explained in the following:

Mount files using SSHFS

The first method is really comfortable. It uses sshfs to mount the filesystem of the target system. Once sshfs is installed on the target and on the build system one can use the following commands to mount the whole file system:

sshfs user@192.168.0.1:/ /home/remoteFS -o transform_symlinks

After that you can access the complete file system of the target system in the /home/remoteFS folder.

Copy files using RSync

The first method is quite comfortable but slows down the building process of MIRA a lot (by factor 2-3 for fast connections). Therefore, a second method using RSync can be used to copy the required files to the build system. Since RSync cannot transform symlinks, we need to use the –copy-unsafe-links option of RSync to create real file copies if the links are pointing to absolute paths. This increases the required disk space a bit but should'nt be a big deal. To copy the required files, use the following commands:

mkdir /home/remoteFS/usr /home/remoteFS/opt
rsync -rlt --copy-unsafe-links user@192.168.0.1:/lib /home/remoteFS/
rsync -rlt --copy-unsafe-links user@192.168.0.1:/usr/include /home/remoteFS/usr/
rsync -rlt --copy-unsafe-links user@192.168.0.1:/usr/lib /home/remoteFS/usr/
rsync -rlt --copy-unsafe-links user@192.168.0.1:/opt/vc /home/remoteFS/opt/

Cross compile MIRA

Now that the compiler and libraries are available, we can start to build MIRA. Therefore, CMake needs to know where the compiler and the target system environment can be found. This configuration has to be stored in a CMake file e.g. in /home/Toolchain.cmake. Assuming that the Raspberry Pi toolchain is installed in /home/gcc-linaro-7.3-arm-rpi3-linux-gnueabihf/install and that the remote files are stored in /home/remoteFS, the file may look like this:

# this one is important
SET(CMAKE_SYSTEM_NAME Linux)
#this one not so much
SET(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_LIBRARY_ARCHITECTURE arm-rpi3-linux-gnueabihf)
set(CMAKE_SYSROOT /home/remoteFS)
# specify the cross compiler
SET(CMAKE_C_COMPILER
/home/gcc-linaro-7.3-arm-rpi3-linux-gnueabihf/install/bin/arm-linux-gnueabihf-gcc CACHE STRING "CMAKE_C_COMPILER")
SET(CMAKE_CXX_COMPILER
/home/gcc-linaro-7.3-arm-rpi3-linux-gnueabihf/install/bin/arm-linux-gnueabihf-g++ CACHE STRING "CMAKE_CXX_COMPILER")
SET(CMAKE_LINKER
/home/gcc-linaro-7.3-arm-rpi3-linux-gnueabihf/install/bin/arm-linux-gnueabihf-ld CACHE STRING "CMAKE_LINKER")
SET(CMAKE_AR
/home/gcc-linaro-7.3-arm-rpi3-linux-gnueabihf/install/bin/arm-linux-gnueabihf-ar CACHE STRING "CMAKE_AR")
# where is the target environment
SET(CMAKE_FIND_ROOT_PATH
"/home/remoteFS")
# search for programs in the build host directories
SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
# for libraries and headers in the target directories
SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
SET(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

More info on how to write a cmake cross compile configuration file can be found here: http://www.cmake.org/Wiki/CMake_Cross_Compiling

Once the file is configured, MIRA can be compiled in cross compile mode. Please go to the MIRA root dir and run:

make release TOOLCHAIN=/home/Toolchain.cmake
Note
If you are compiling for a Raspberry (ARMv6) you will see a lot of "swp{b} use is deprecated for this architecture" warnings. But you don't need to worry about this if the target architecture is ARMv6.
If you have added new search paths for libraries you need to remove the file "CMakeCache.txt" in the folder "build/BuildTarget" in order to force CMake to discover new paths or libraries.

The build system supresses calls to ManifestGen in cross compile mode since the binary and the libraries cannot be interpreted on the build system. The required commands are stored in a file instead.

In order to generate the final MIRA installation for the target system, the following command needs to be executed:

make install_release DESTDIR=/home/remoteFS/

All the files required will be stored in /home/remoteFS/opt/MIRA. Therefore, if you used rsync to copy the target system files, you need to copy this folder to the target system afterwards:

rsync -rlt --copy-unsafe-links /home/remoteFS/opt/MIRA user@192.168.0.1:/opt/

Generate manifest files on target system

Note
In newer versions of Raspbian, ldconfig is used to determine the pathes for run-time linking:
cat > /tmp/mira.conf << EOF
/opt/MIRA/lib
EOF
sudo mv /tmp/mira.conf /etc/ld.so.conf.d/
sudo ldconfig -v

Since the manifest files could not be generated on the build system, we need to call a special build target on the target system. Once you have configured your MIRA_PATH and LD_LIBRARY_PATH system variables (or ldconfig config) correctly, you need to go to the path you have copied MIRA to and execute the following command:

make manifest

Congratulations! You are done and can use MIRA on the target system.

Cross compile project

Update build system linker config

Note
In newer versions of Raspbian, ldconfig is used to determine the pathes for run-time linking: Add
/opt/MIRA/lib
to the file /home/remoteFS/etc/ld.so.conf

Cross compile project

Compile the project in cross compile mode. Please go to the project root dir and run:

make release TOOLCHAIN=/home/Toolchain.cmake

In order to generate the final project installation for the target system, the following command needs to be executed:

make install_release DESTDIR=/home/remoteFS/

All the files required will be stored in /home/remoteFS/usr/local/PROJECT. Therefore, if you used rsync to copy the target system files, you need to copy this folder to the target system afterwards:

rsync -rlt --copy-unsafe-links /home/remoteFS/usr/local/PROJECT user@192.168.0.1:/usr/local/

Generate manifest files on target system

Note
In newer versions of Raspbian, ldconfig is used to determine the pathes for run-time linking: Add
/usr/local/PROJECT
to the file /etc/ld.so.conf.d/mira.conf and run
sudo ldconfig -v

Once you have updated your MIRA_PATH and LD_LIBRARY_PATH system variables (or ldconfig config), you need to go to the path you have copied your project to and execute the following command:

make manifest

Congratulations! You are done and can use the project on the target system.

SVN-r10557 , Generated on Mon Dec 23 2024 03:36:07 for MIRA by   doxygen 1.8.14