blog:2017:0925_build_boost_for_android [NervTech's Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Building Boost libraries for Android platform ======

{{tag>devel cpp android boost}}

For my personal project **NervSeed** I would like to be able to compile advanced c++ code for Android. So I was thinking: at some point I will need the boost libraries, so i might just as well try to figure out right now how to build boost for Android. Afterall, this is a good timing since I've just setup the required support in this project to support compilation for: linux64, win32, win64, msvc32 and msvc64 => we might just as well try to add android to the list now :-)

====== ======

====== Investigations ======

  * https://stackoverflow.com/questions/37679587/how-to-compile-boost-1-61-for-android-ndk-11
  * https://developer.android.com/ndk/guides/standalone_toolchain.html
  * https://stackoverflow.com/questions/35839127/how-to-build-boost-for-android-as-shared-library-with-c11-support
  * https://github.com/dec1/Boost-for-Android

====== Implementing support for compilation from windows ======

It seems the easiest option is to start with creating a dedicated toolchain from the NDK tools.

===== Creating Standalone NDK toolchain =====

Trying to execute a simple script to build the standalone toolchain: <sxh bash>nv_install_ndk_toolchain()
{
  local ndkdir="`nv_get_android_ndk_dir`"
  echo "Using NDK folder: $ndkdir"

  # Select destination folder:
  local destdir="`nv_get_project_dir`/tools/windows/ndk-api21"

  # call the make toolchain script:
  $ndkdir/build/tools/make_standalone_toolchain.py --arch arm --api 21 --install-dir $destdir

  echo "Done installing NDK toolchain for API 21"
}</sxh>

But we get an error when running this from cygwin: <code>$ nv_install_ndk_toolchain
Using NDK folder: /cygdrive/d/Apps/CodeWorks-1R6/android-ndk-r12b
Unsupported platform: CYGWIN_NT-10.0
Done installing NDK toolchain for API 21
</code>

=> Let's try using the make-standalone-toolchain.sh script instead: results are a little bit better... But now we have a message stating that this script will be removed in r13 anyway: <code>$ nv_install_ndk_toolchain
Using NDK folder: /cygdrive/d/Apps/CodeWorks-1R6/android-ndk-r12b
WARNING: make-standalone-toolchain.sh will be removed in r13. Please try make_standalone_toolchain.py now to make sure it works for your needs.
HOST_OS=cygwin
HOST_EXE=.exe
HOST_ARCH=x86_64
HOST_TAG=windows-x86_64
HOST_NUM_CPUS=8
BUILD_NUM_CPUS=16
ERROR: Option '--arch' requires an argument. See --help for usage.
Done installing NDK toolchain for API 21</code>

=> So, not so much choice: we have to get ourself a decent portable python installation. Just retrieving a zip package of the python version available from chromium depot tools! :-)

<note>It's weird that you can cannot easily find a minimal python portable zip package online: they all come with hundreds of MB of additional libraries, packaged as executable files...</note>

**Note**: could check the following page to [[https://stackoverflow.com/questions/8097161/how-would-i-build-python-myself-from-source-code-on-ubuntu|build python from sources on linux]]

=> **OK**: Can now build the toolchain without error with the portable python installation.

===== Preparing script for boost compilation =====

Tried the simple clang toolchain usage, getting this error: <code>/cygdrive/d/Projects/NervSeed/tools/windows/ndk-api21/bin/clang: line 8: syntax error: unexpected end of file</code> when running the script commands: <sxh bash>    echo "Running bootstrap..."
    local PREVPATH="$PATH"
    local chaindir="`nv_get_ndk_toolchain_dir`"
    echo "Using NDK toolchain path: $chaindir"
    export PATH="$chaindir/bin:$PATH"

    ./bootstrap.sh --without-icu --with-toolset=clang</sxh>

Tried a bit more complex script with: <sxh bash>  echo "Running bootstrap..."
  ./bootstrap.sh --without-icu

    local chaindir="`nv_get_ndk_toolchain_dir`"
    echo "Using NDK toolchain path: $chaindir"

    echo "using clang : android : $chaindir/bin/arm-linux-androideabi-clang++ : ;" > user-config.jam

    # cxxflags="-stdlib=libc++" --layout=versioned
    ./b2 --prefix="$idir" --user-config=user-config.jam --reconfigure target-os=android toolset=clang-android \
      include=$chaindir/include/c++/4.9.x variant=release threading=multi threadapi=pthread link=static \
      runtime-link=static  install > build.log 2>&1
    </sxh> but this doesn't work either (not producing any library with errors in the build log). Checking the errors: <code>/cygdrive/d/Projects/NervSeed/tools/windows/ndk-api21/bin/arm-linux-androideabi-clang++: line 8: syntax error: unexpected end of file</code>

Checking this script file it seems we are using CRLF lien endings, which might not be appropriate. So let's fix that by creating our own launcher:

<sxh bash>	  cat <<EOF >nvclang
#!/bin/bash
if [ "\$1" != "-cc1" ]; then
    $chaindir/bin/clang38++ -target armv7a-none-linux-androideabi --sysroot $chaindir/sysroot "\$@"
else
    # target/triple already spelled out.
    $chaindir/bin/clang38++ "\$@"
fi
EOF
    chmod +x nvclang


    echo "using clang : android : $PWD/nvclang : ;" > user-config.jam</sxh>

=> With this change the compilation seems to be going smoothly :-) (crossing fingers...) => Nope... Still no static libraries built:, during compilation we now get errors such as: <code>In file included from libs/log/src/event.cpp:16:
In file included from .\boost/log/detail/config.hpp:34:
In file included from .\boost/config.hpp:48:
.\boost/config/stdlib/libstdcpp3.hpp:78:12: fatal error: 'unistd.h' file not found
#  include <unistd.h>
           ^
1 error generated.</code>

Now trying the enhanced user-config version: **OK, this works!**

====== Implementing support for compilation from linux ======

Updated the build scripts to:
  - Create a standalone toolchain in **tools/linux/ndk-api21**
  - Build boost from this toolchain (based on **android-ndk-r15c**)

=> This doesn't work because in version r15c we don't have clang38++ anymore (instead we have clang50++): <code>/mnt/array1/dev/projects/NervSeed/deps/build/boost_1_65_1/nvclang: line 3: /mnt/array1/dev/projects/NervSeed/tools/linux/ndk-api21/bin/clang38++: No such file or directory</code>

One of our goal is to be able to build android libraries for use in Unreal Engine 4, and thus we should stick with the compilations capabilities available in Codeworks-1R6, so we should just use **android-ndk-r12b**.

Downloading proper NDK version: <sxh bash>cd /mnt/array1/dev/compilers
wget https://dl.google.com/android/repository/android-ndk-r12b-linux-x86_64.zip</sxh>

=> **OK** with this version of the android NDK we can build the boost libraries the same way as on windows. Now one thing I just noticed is that on linux we seem to get some support for C++11 features: <code>    - C++11 mutex              : no
    - lockfree boost::atomic_flag : yes
    - Boost.Config Feature Check: cxx11_auto_declarations : yes
    - Boost.Config Feature Check: cxx11_constexpr : yes
    - Boost.Config Feature Check: cxx11_defaulted_functions : yes
    - Boost.Config Feature Check: cxx11_final : yes
    - Boost.Config Feature Check: cxx11_hdr_mutex : no
    - Boost.Config Feature Check: cxx11_hdr_tuple : yes
    - Boost.Config Feature Check: cxx11_lambdas : yes
    - Boost.Config Feature Check: cxx11_noexcept : yes
    - Boost.Config Feature Check: cxx11_nullptr : yes
    - Boost.Config Feature Check: cxx11_rvalue_references : yes
    - Boost.Config Feature Check: cxx11_template_aliases : yes
    - Boost.Config Feature Check: cxx11_thread_local : yes
    - Boost.Config Feature Check: cxx11_variadic_templates : yes</code>

Checking this again from windows: <code>    - C++11 mutex              : no
    - lockfree boost::atomic_flag : no
    - Boost.Config Feature Check: cxx11_auto_declarations : no
    - Boost.Config Feature Check: cxx11_constexpr : no
    - Boost.Config Feature Check: cxx11_defaulted_functions : no
    - Boost.Config Feature Check: cxx11_final : no
    - Boost.Config Feature Check: cxx11_hdr_mutex : no
    - Boost.Config Feature Check: cxx11_hdr_tuple : no
    - Boost.Config Feature Check: cxx11_lambdas : no
    - Boost.Config Feature Check: cxx11_noexcept : no
    - Boost.Config Feature Check: cxx11_nullptr : no
    - Boost.Config Feature Check: cxx11_rvalue_references : no
    - Boost.Config Feature Check: cxx11_template_aliases : no
    - Boost.Config Feature Check: cxx11_thread_local : no
    - Boost.Config Feature Check: cxx11_variadic_templates : no</code>
  
Hmmm... that's weird... Checking [[https://stackoverflow.com/questions/39826917/boost-build-fails-c11-feature-checks-when-using-custom-gcc-4-x-or-5-x|this page]] on this issue.
  * Added **cxxflags="--std=c++11"** on the b2 command line: **doesn't help**
  * Okay, the missing C++11 features above might not be that critical for now (waiting to see what we get when trying to build our projects on android ?) So for now, let's just move forward.

====== Conclusion ======

In the end, building boost for android was pretty easy, using mainly the following script commands: <sxh bash>  echo "Running bootstrap..."
  ./bootstrap.sh --without-icu

  # cf. https://stackoverflow.com/questions/37679587/how-to-compile-boost-1-61-for-android-ndk-11
  local chaindir="`nv_get_ndk_toolchain_dir`"
  echo "Using NDK toolchain path: $chaindir"
  local ndkdir="`nv_get_android_ndk_dir`"
  echo "Using NDK root path: $ndkdir"

  cat <<EOF >nvclang
#!/bin/bash
if [ "\$1" != "-cc1" ]; then
    $chaindir/bin/clang38++ -target armv7a-none-linux-androideabi --sysroot $chaindir/sysroot "\$@"
else
    # target/triple already spelled out.
    $chaindir/bin/clang38++ "\$@"
fi
EOF
  chmod +x nvclang

  # echo "using clang : android : $PWD/nvclang : ;" > user-config.jam
  # echo "using clang : android : $chaindir/bin/arm-linux-androideabi-clang++ : ;" > user-config.jam

	cat <<EOF >user-config.jam
import os ;
local AndroidNDKRoot = `nv_to_win_path $ndkdir` ;
using clang : android
:
$PWD/nvclang
:
<compileflags>-fexceptions
<compileflags>-frtti
<compileflags>-fpic
<compileflags>-ffunction-sections
<compileflags>-funwind-tables
<compileflags>-Wno-psabi
<compileflags>-march=armv7-a
<compileflags>-mfloat-abi=softfp
<compileflags>-mfpu=vfpv3-d16
<compileflags>-fomit-frame-pointer
<compileflags>-fno-strict-aliasing
<compileflags>-finline-limit=64
<compileflags>-I\$(AndroidNDKRoot)/platforms/android-21/arch-arm/usr/include
<compileflags>-Wa,--noexecstack
<compileflags>-DANDROID
<compileflags>-D__ANDROID__
<compileflags>-DNDEBUG
<compileflags>-O2
<compileflags>-std=c++11
#<compileflags>-g
<compileflags>-I\$(AndroidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.9/include
<compileflags>-I\$(AndroidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi/include
<architecture>arm
<compileflags>-fvisibility=hidden
<compileflags>-fvisibility-inlines-hidden
<compileflags>-fdata-sections
<cxxflags>-D__arm__
<cxxflags>-D_REENTRANT
<cxxflags>-D_GLIBCXX__PTHREADS
;
EOF

  ./b2 --prefix="$idir" --user-config=user-config.jam --reconfigure target-os=android toolset=clang-android \
      --without-mpi --without-python variant=release threading=multi threadapi=pthread link=static \
      runtime-link=static  install > build.log 2>&1</sxh>

We might not need all the compileflags provided above in the user-config.jam file, but I didn't try to remove them anyway :-).

~~DISCUSSION:off~~