blog:2017:0925_build_boost_for_android

Differences

This shows you the differences between two versions of the page.


Previous revision
blog:2017:0925_build_boost_for_android [2021/09/02 13:39] (current) – Discussion status changed manu
Line 1: Line 1:
 +====== Building Boost libraries for Android platform ======
 +
 +{{tag>devel cpp android boost}}
 +
 +For my personal project **NervSeed** I would like to be able to compile advanced c++ code for Android. So I was thinking: at some point I will need the boost libraries, so i might just as well try to figure out right now how to build boost for Android. Afterall, this is a good timing since I've just setup the required support in this project to support compilation for: linux64, win32, win64, msvc32 and msvc64 => we might just as well try to add android to the list now :-)
 +
 +====== ======
 +
 +====== Investigations ======
 +
 +  * https://stackoverflow.com/questions/37679587/how-to-compile-boost-1-61-for-android-ndk-11
 +  * https://developer.android.com/ndk/guides/standalone_toolchain.html
 +  * https://stackoverflow.com/questions/35839127/how-to-build-boost-for-android-as-shared-library-with-c11-support
 +  * https://github.com/dec1/Boost-for-Android
 +
 +====== Implementing support for compilation from windows ======
 +
 +It seems the easiest option is to start with creating a dedicated toolchain from the NDK tools.
 +
 +===== Creating Standalone NDK toolchain =====
 +
 +Trying to execute a simple script to build the standalone toolchain: <sxh bash>nv_install_ndk_toolchain()
 +{
 +  local ndkdir="`nv_get_android_ndk_dir`"
 +  echo "Using NDK folder: $ndkdir"
 +
 +  # Select destination folder:
 +  local destdir="`nv_get_project_dir`/tools/windows/ndk-api21"
 +
 +  # call the make toolchain script:
 +  $ndkdir/build/tools/make_standalone_toolchain.py --arch arm --api 21 --install-dir $destdir
 +
 +  echo "Done installing NDK toolchain for API 21"
 +}</sxh>
 +
 +But we get an error when running this from cygwin: <code>$ nv_install_ndk_toolchain
 +Using NDK folder: /cygdrive/d/Apps/CodeWorks-1R6/android-ndk-r12b
 +Unsupported platform: CYGWIN_NT-10.0
 +Done installing NDK toolchain for API 21
 +</code>
 +
 +=> Let's try using the make-standalone-toolchain.sh script instead: results are a little bit better... But now we have a message stating that this script will be removed in r13 anyway: <code>$ nv_install_ndk_toolchain
 +Using NDK folder: /cygdrive/d/Apps/CodeWorks-1R6/android-ndk-r12b
 +WARNING: make-standalone-toolchain.sh will be removed in r13. Please try make_standalone_toolchain.py now to make sure it works for your needs.
 +HOST_OS=cygwin
 +HOST_EXE=.exe
 +HOST_ARCH=x86_64
 +HOST_TAG=windows-x86_64
 +HOST_NUM_CPUS=8
 +BUILD_NUM_CPUS=16
 +ERROR: Option '--arch' requires an argument. See --help for usage.
 +Done installing NDK toolchain for API 21</code>
 +
 +=> So, not so much choice: we have to get ourself a decent portable python installation. Just retrieving a zip package of the python version available from chromium depot tools! :-)
 +
 +<note>It's weird that you can cannot easily find a minimal python portable zip package online: they all come with hundreds of MB of additional libraries, packaged as executable files...</note>
 +
 +**Note**: could check the following page to [[https://stackoverflow.com/questions/8097161/how-would-i-build-python-myself-from-source-code-on-ubuntu|build python from sources on linux]]
 +
 +=> **OK**: Can now build the toolchain without error with the portable python installation.
 +
 +===== Preparing script for boost compilation =====
 +
 +Tried the simple clang toolchain usage, getting this error: <code>/cygdrive/d/Projects/NervSeed/tools/windows/ndk-api21/bin/clang: line 8: syntax error: unexpected end of file</code> when running the script commands: <sxh bash>    echo "Running bootstrap..."
 +    local PREVPATH="$PATH"
 +    local chaindir="`nv_get_ndk_toolchain_dir`"
 +    echo "Using NDK toolchain path: $chaindir"
 +    export PATH="$chaindir/bin:$PATH"
 +
 +    ./bootstrap.sh --without-icu --with-toolset=clang</sxh>
 +
 +Tried a bit more complex script with: <sxh bash>  echo "Running bootstrap..."
 +  ./bootstrap.sh --without-icu
 +
 +    local chaindir="`nv_get_ndk_toolchain_dir`"
 +    echo "Using NDK toolchain path: $chaindir"
 +
 +    echo "using clang : android : $chaindir/bin/arm-linux-androideabi-clang++ : ;" > user-config.jam
 +
 +    # cxxflags="-stdlib=libc++" --layout=versioned
 +    ./b2 --prefix="$idir" --user-config=user-config.jam --reconfigure target-os=android toolset=clang-android \
 +      include=$chaindir/include/c++/4.9.x variant=release threading=multi threadapi=pthread link=static \
 +      runtime-link=static  install > build.log 2>&1
 +    </sxh> but this doesn't work either (not producing any library with errors in the build log). Checking the errors: <code>/cygdrive/d/Projects/NervSeed/tools/windows/ndk-api21/bin/arm-linux-androideabi-clang++: line 8: syntax error: unexpected end of file</code>
 +
 +Checking this script file it seems we are using CRLF lien endings, which might not be appropriate. So let's fix that by creating our own launcher:
 +
 +<sxh bash>   cat <<EOF >nvclang
 +#!/bin/bash
 +if [ "\$1" != "-cc1" ]; then
 +    $chaindir/bin/clang38++ -target armv7a-none-linux-androideabi --sysroot $chaindir/sysroot "\$@"
 +else
 +    # target/triple already spelled out.
 +    $chaindir/bin/clang38++ "\$@"
 +fi
 +EOF
 +    chmod +x nvclang
 +
 +
 +    echo "using clang : android : $PWD/nvclang : ;" > user-config.jam</sxh>
 +
 +=> With this change the compilation seems to be going smoothly :-) (crossing fingers...) => Nope... Still no static libraries built:, during compilation we now get errors such as: <code>In file included from libs/log/src/event.cpp:16:
 +In file included from .\boost/log/detail/config.hpp:34:
 +In file included from .\boost/config.hpp:48:
 +.\boost/config/stdlib/libstdcpp3.hpp:78:12: fatal error: 'unistd.h' file not found
 +#  include <unistd.h>
 +           ^
 +1 error generated.</code>
 +
 +Now trying the enhanced user-config version: **OK, this works!**
 +
 +====== Implementing support for compilation from linux ======
 +
 +Updated the build scripts to:
 +  - Create a standalone toolchain in **tools/linux/ndk-api21**
 +  - Build boost from this toolchain (based on **android-ndk-r15c**)
 +
 +=> This doesn't work because in version r15c we don't have clang38++ anymore (instead we have clang50++): <code>/mnt/array1/dev/projects/NervSeed/deps/build/boost_1_65_1/nvclang: line 3: /mnt/array1/dev/projects/NervSeed/tools/linux/ndk-api21/bin/clang38++: No such file or directory</code>
 +
 +One of our goal is to be able to build android libraries for use in Unreal Engine 4, and thus we should stick with the compilations capabilities available in Codeworks-1R6, so we should just use **android-ndk-r12b**.
 +
 +Downloading proper NDK version: <sxh bash>cd /mnt/array1/dev/compilers
 +wget https://dl.google.com/android/repository/android-ndk-r12b-linux-x86_64.zip</sxh>
 +
 +=> **OK** with this version of the android NDK we can build the boost libraries the same way as on windows. Now one thing I just noticed is that on linux we seem to get some support for C++11 features: <code>    - C++11 mutex              : no
 +    - lockfree boost::atomic_flag : yes
 +    - Boost.Config Feature Check: cxx11_auto_declarations : yes
 +    - Boost.Config Feature Check: cxx11_constexpr : yes
 +    - Boost.Config Feature Check: cxx11_defaulted_functions : yes
 +    - Boost.Config Feature Check: cxx11_final : yes
 +    - Boost.Config Feature Check: cxx11_hdr_mutex : no
 +    - Boost.Config Feature Check: cxx11_hdr_tuple : yes
 +    - Boost.Config Feature Check: cxx11_lambdas : yes
 +    - Boost.Config Feature Check: cxx11_noexcept : yes
 +    - Boost.Config Feature Check: cxx11_nullptr : yes
 +    - Boost.Config Feature Check: cxx11_rvalue_references : yes
 +    - Boost.Config Feature Check: cxx11_template_aliases : yes
 +    - Boost.Config Feature Check: cxx11_thread_local : yes
 +    - Boost.Config Feature Check: cxx11_variadic_templates : yes</code>
 +
 +Checking this again from windows: <code>    - C++11 mutex              : no
 +    - lockfree boost::atomic_flag : no
 +    - Boost.Config Feature Check: cxx11_auto_declarations : no
 +    - Boost.Config Feature Check: cxx11_constexpr : no
 +    - Boost.Config Feature Check: cxx11_defaulted_functions : no
 +    - Boost.Config Feature Check: cxx11_final : no
 +    - Boost.Config Feature Check: cxx11_hdr_mutex : no
 +    - Boost.Config Feature Check: cxx11_hdr_tuple : no
 +    - Boost.Config Feature Check: cxx11_lambdas : no
 +    - Boost.Config Feature Check: cxx11_noexcept : no
 +    - Boost.Config Feature Check: cxx11_nullptr : no
 +    - Boost.Config Feature Check: cxx11_rvalue_references : no
 +    - Boost.Config Feature Check: cxx11_template_aliases : no
 +    - Boost.Config Feature Check: cxx11_thread_local : no
 +    - Boost.Config Feature Check: cxx11_variadic_templates : no</code>
 +  
 +Hmmm... that's weird... Checking [[https://stackoverflow.com/questions/39826917/boost-build-fails-c11-feature-checks-when-using-custom-gcc-4-x-or-5-x|this page]] on this issue.
 +  * Added **cxxflags="--std=c++11"** on the b2 command line: **doesn't help**
 +  * Okay, the missing C++11 features above might not be that critical for now (waiting to see what we get when trying to build our projects on android ?) So for now, let's just move forward.
 +
 +====== Conclusion ======
 +
 +In the end, building boost for android was pretty easy, using mainly the following script commands: <sxh bash>  echo "Running bootstrap..."
 +  ./bootstrap.sh --without-icu
 +
 +  # cf. https://stackoverflow.com/questions/37679587/how-to-compile-boost-1-61-for-android-ndk-11
 +  local chaindir="`nv_get_ndk_toolchain_dir`"
 +  echo "Using NDK toolchain path: $chaindir"
 +  local ndkdir="`nv_get_android_ndk_dir`"
 +  echo "Using NDK root path: $ndkdir"
 +
 +  cat <<EOF >nvclang
 +#!/bin/bash
 +if [ "\$1" != "-cc1" ]; then
 +    $chaindir/bin/clang38++ -target armv7a-none-linux-androideabi --sysroot $chaindir/sysroot "\$@"
 +else
 +    # target/triple already spelled out.
 +    $chaindir/bin/clang38++ "\$@"
 +fi
 +EOF
 +  chmod +x nvclang
 +
 +  # echo "using clang : android : $PWD/nvclang : ;" > user-config.jam
 +  # echo "using clang : android : $chaindir/bin/arm-linux-androideabi-clang++ : ;" > user-config.jam
 +
 + cat <<EOF >user-config.jam
 +import os ;
 +local AndroidNDKRoot = `nv_to_win_path $ndkdir` ;
 +using clang : android
 +:
 +$PWD/nvclang
 +:
 +<compileflags>-fexceptions
 +<compileflags>-frtti
 +<compileflags>-fpic
 +<compileflags>-ffunction-sections
 +<compileflags>-funwind-tables
 +<compileflags>-Wno-psabi
 +<compileflags>-march=armv7-a
 +<compileflags>-mfloat-abi=softfp
 +<compileflags>-mfpu=vfpv3-d16
 +<compileflags>-fomit-frame-pointer
 +<compileflags>-fno-strict-aliasing
 +<compileflags>-finline-limit=64
 +<compileflags>-I\$(AndroidNDKRoot)/platforms/android-21/arch-arm/usr/include
 +<compileflags>-Wa,--noexecstack
 +<compileflags>-DANDROID
 +<compileflags>-D__ANDROID__
 +<compileflags>-DNDEBUG
 +<compileflags>-O2
 +<compileflags>-std=c++11
 +#<compileflags>-g
 +<compileflags>-I\$(AndroidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.9/include
 +<compileflags>-I\$(AndroidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi/include
 +<architecture>arm
 +<compileflags>-fvisibility=hidden
 +<compileflags>-fvisibility-inlines-hidden
 +<compileflags>-fdata-sections
 +<cxxflags>-D__arm__
 +<cxxflags>-D_REENTRANT
 +<cxxflags>-D_GLIBCXX__PTHREADS
 +;
 +EOF
 +
 +  ./b2 --prefix="$idir" --user-config=user-config.jam --reconfigure target-os=android toolset=clang-android \
 +      --without-mpi --without-python variant=release threading=multi threadapi=pthread link=static \
 +      runtime-link=static  install > build.log 2>&1</sxh>
 +
 +We might not need all the compileflags provided above in the user-config.jam file, but I didn't try to remove them anyway :-).
 +
 +~~DISCUSSION:off~~