blog:2017:0918_buillding_mingw64_compiler_on_linux [NervTech's Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Building Mingw64 compiler from sources on Linux ======

{{tag>devel linux}}

Yes I know you can install a package on your distro to get the mingw64 compilers ready in a few minutes. But there is a major issue with this solution: you never quite know which version you will get! Or well, you know which version you get, but it might not be the one you would expect!

Actually this is just happening to me right now: I need to install the mingw64/32 compilers on an ubuntu machine... and this would be a build a very modern project with all fancy C++11/14 stuff: so I would really like to get the latest (or close to the latest) version of gcc and all. So I went to check the [[https://mingw-w64.org/doku.php/download|official download page]] just to realize that for someone like me who doesn't want to mess my ubuntu 14.04 server completely I should stick with an official mingw64 package providing gcc 4.8 with mingw version 3.1... Seriously ?? :-). Time to fix this the hard way!

====== ======

====== Old build instructions ======

<note important>Those old build instructions cannot be used to build the compiler anymore, they are only left here for reference. Please use the **new build instructions** below instead.</note>

Let's start with the beginning and find some decent initial build instructions. [[https://sourceforge.net/p/mingw-w64/wiki2/mingw-w64-howto-build|This page]] seems to provide a good entry point. So let's try that!

First we download the require source files:
  * [[https://sourceforge.net/projects/mingw-w64/files/mingw-w64/mingw-w64-release/mingw-w64-v5.0.2.tar.bz2/download|mingw-w64-v5.0.2.tar.bz2]]: the actual mingw-w64 source files.
  * [[https://sourceforge.net/projects/mingw-w64/files/Toolchain%20sources/Personal%20Builds/mingw-builds/7.2.0/src-7.2.0-release-rt_v6-rev0.tar.7z/download|src-7.2.0-release-rt_v6-rev0.tar.7z]]: a complete toolchain build package ?

Trying to use the src-7.2.0-release-rt_v6-rev0.tar.7z package: <sxh bash>$ src-7.2.0-release-rt_v6-rev0.tar.7z
$ tar xvf src-7.2.0-release-rt_v6-rev0.tar
$ cd src
</sxh>

=> This source folder contains a lot of different libraries: <code>$ ls 
binutils-2.28  gcc-7.2.0  gmp-6.1.0     libiconv-1.15  mingw-libgnurx-2.5.1  mpfr-3.1.4      Python-2.7.9             tcl8.6.4       zlib-1.2.11
bzip2-1.0.6    gdb-8.0    isl-0.16.1    make-4.2.1     mingw-w64             ncurses-6.0     readline-6.2             termcap-1.3.1
expat-2.1.0    gdbm-1.10  libffi-3.2.1  MARKERS        mpc-1.0.3             openssl-1.0.2e  sqlite-autoconf-3081002  tk8.6.4</code>

===== Building the binutils =====

We create a build folder in the binutils folder, and then we build it: <sxh bash>cd binutils-2.28
mkdir build
cd build
../configure --target=x86_64-w64-mingw32 --prefix=/mnt/array1/dev/compilers/mingw64 --with-sysroot=/mnt/array1/dev/compilers/mingw64
make
make install

cd ../../mingw-w64/
cp -R mingw-w64-headers/include /mnt/array1/dev/compilers/mingw64/x86_64-w64-mingw32/
cd /mnt/array1/dev/compilers/mingw64
ln -s x86_64-w64-mingw32 mingw</sxh>

===== Building the gcc core =====

Building gcc core with: <sxh bash>cd /mnt/array1/dev/projects/mingw64_build/src/gcc-7.2.0/
mkdir build
cd build
../configure --target=x86_64-w64-mingw32 --prefix=/mnt/array1/dev/compilers/mingw64 --with-sysroot=/mnt/array1/dev/compilers/mingw64
</sxh>

We get an error on this point: <code>configure: error: Building GCC requires GMP 4.2+, MPFR 2.4.0+ and MPC 0.8.0+.</code>

So let's try to build these first: <sxh bash>cd /mnt/array1/dev/projects/mingw64_build/src/mpfr-3.1.4/
mkdir build
cd build
../configure --target=x86_64-w64-mingw32 --prefix=/mnt/array1/dev/compilers/mingw64 --with-sysroot=/mnt/array1/dev/compilers/mingw64
make
make install

cd ../../gmp-6.1.0/
mkdir build
cd build
export CFLAGS=-fPIC
export CXXFLAGS=-fPIC
../configure --host=x86_64-w64-mingw32 --prefix=/mnt/array1/dev/compilers/mingw64 --with-sysroot=/mnt/array1/dev/compilers/mingw64
make
make install

cd ../../mpc-1.0.3/
mkdir build
cd build
../configure --target=x86_64-w64-mingw32 --prefix=/mnt/array1/dev/compilers/mingw64 --with-sysroot=/mnt/array1/dev/compilers/mingw64 --with-mpfr=/mnt/array1/dev/compilers/mingw64 --with-gmp=/mnt/array1/dev/compilers/mingw64  --enable-shared=no
make
make install
</sxh>

Getting a compilation error: <code>/usr/bin/ld: /mnt/array1/dev/compilers/mingw64/lib/libgmp.a(realloc.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/mnt/array1/dev/compilers/mingw64/lib/libgmp.a: error adding symbols: Bad value</code>

=> So now building with PIC as indicated above. But this doesn't help! So we only build the static library for MPC: **OK** building now. 

With those dependencies we can try to build the gcc core again: <sxh bash>cd /mnt/array1/dev/projects/mingw64_build/src/gcc-7.2.0/
mkdir build
cd build
../configure --target=x86_64-w64-mingw32 --prefix=/mnt/array1/dev/compilers/mingw64 --with-sysroot=/mnt/array1/dev/compilers/mingw64 --with-mpc=/mnt/array1/dev/compilers/mingw64 --with-mpfr=/mnt/array1/dev/compilers/mingw64 --with-gmp=/mnt/array1/dev/compilers/mingw64
make all-gcc
make install-gcc
</sxh>

===== Building the CRT =====

Now building the C runtime library with: <sxh bash>cd /mnt/array1/dev/projects/mingw64_build/src/mingw-w64/mingw-w64-crt
../configure --host=x86_64-w64-mingw32 --prefix=/mnt/array1/dev/compilers/mingw64 --with-sysroot=/mnt/array1/dev/compilers/mingw64
</sxh>

Of course we get an error when doing this: <code>configure: error: source directory already configured; run "make distclean" there first</code>

And running make distclean doesn't help much: <sxh bash>$ make distclean
Makefile:13447: * missing separator.  Stop.
</sxh>

Instead trying to manually remove the **makefile** and **config.h** files: still **not working**

Time to check out additional mingw64 package ? <sxh bash>tar xvjf mingw-w64-v5.0.2.tar.bz2
cd mingw-w64-v5.0.2/mingw-w64-crt/
../configure --host=x86_64-w64-mingw32 --prefix=/mnt/array1/dev/compilers/mingw64 --with-sysroot=/mnt/array1/dev/compilers/mingw64
</sxh>

Still the same error !!Grrr...!! Time to try a new perspective.

====== New build instructions ======

<note important>Again, those instructions below are not good enough and we are stuck with a segmentation fault when trying to build the CRT. So please refer to the **Third Trial** section :-)</note>

Okay, so found updated instructions from this page: https://sourceforge.net/p/mingw-w64/wiki2/Cross%20Win32%20and%20Win64%20compiler/

So let's restart everything. This time we will build from **/mnt/dev/build/** folder.

===== Building the binutils =====

<sxh bash>cd /mnt/array1/dev/build/src/binutils-2.28
mkdir build
cd build
../configure --target=x86_64-w64-mingw32 --prefix=/mnt/array1/dev/compilers/linux-mingw64 --with-sysroot=/mnt/array1/dev/compilers/linux-mingw64 --disable-multilib
make
make install
cd ..
rm -Rf build</sxh>

Add our new bin folder to our path (from our master **profile.sh** file): <sxh bash>export PATH="/home/kenshin/dev/compilers/mingw64/bin:$PATH"</sxh>

===== Install the mingw-w64 headers =====

**Note**: In the command below it is very important to specify the **x86_64-w64-mingw32** ending of the prefix location:

<sxh bash>cd /mnt/array1/dev/build/src/mingw-w64/mingw-w64-headers/
mkdir build
cd build
../configure --host=x86_64-w64-mingw32 --prefix=/mnt/array1/dev/compilers/linux-mingw64/x86_64-w64-mingw32
make install
cd /mnt/array1/dev/compilers/linux-mingw64
ln -s x86_64-w64-mingw32 mingw
cd x86_64-w64-mingw32
ln -s lib lib64</sxh>

===== Building the gcc cross-compiler =====

<sxh bash>export MGWDIR=/mnt/array1/dev/compilers/linux-mingw64
cd /mnt/array1/dev/projects/mingw64_build/src/mpfr-3.1.4/
mkdir build
cd build
../configure --target=x86_64-w64-mingw32 --prefix=$MGWDIR --with-sysroot=$MGWDIR --enable-shared=no
make
make install
cd ..
rm -Rf build
cd ../gmp-6.1.0
mkdir build
cd build
../configure --host=x86_64-w64-mingw32 --prefix=$MGWDIR --with-sysroot=$MGWDIR
make
make install
cd ..
rm -Rf build
cd ../mpc-1.0.3
mkdir build
cd build
../configure --target=x86_64-w64-mingw32 --prefix=$MGWDIR --with-sysroot=$MGWDIR --with-mpfr=$MGWDIR --with-gmp=$MGWDIR --enable-shared=no
make
make install
cd ..
rm -Rf build
cd ../gcc-7.2.0/
mkdir build
cd build
../configure --target=x86_64-w64-mingw32 --prefix=$MGWDIR --with-sysroot=$MGWDIR --with-mpc=$MGWDIR --with-mpfr=$MGWDIR --with-gmp=$MGWDIR --disable-multilib
make all-gcc
make install-gcc
</sxh>

At this point we keep the gcc build folder as we will come back to it later.

===== Building the CRT =====

<sxh bash>cd /mnt/array1/dev/build/src/mingw-w64/mingw-w64-crt
mkdir build
cd build
../configure --host=x86_64-w64-mingw32 --prefix=$MGWDIR --with-sysroot=$MGWDIR
make
</sxh>

Great... Now we have a segmentation fault in our new compiler freshly installed: <code>make
echo "static int __attribute__((unused)) __mingw_libm_dummy;" > _libm_dummy.c
make  all-am
make[1]: Entering directory `/mnt/array1/dev/build/src/mingw-w64/mingw-w64-crt/build'
x86_64-w64-mingw32-gcc -DHAVE_CONFIG_H -I. -I..  -m32 -I../include -D_CRTBLD -I/mnt/array1/dev/compilers/linux-mingw64/include  -pipe -std=gnu99 -D_WIN32_WINNT=0x0f00 -Wall -Wextra -Wformat -Wstrict-aliasing -Wshadow -Wpacked -Winline -Wimplicit-function-declaration -Wmissing-noreturn -Wmissing-prototypes -g -O2 -MT intrincs/lib32_libkernel32_a-__movsb.o -MD -MP -MF intrincs/.deps/lib32_libkernel32_a-__movsb.Tpo -c -o intrincs/lib32_libkernel32_a-__movsb.o `test -f 'intrincs/__movsb.c' || echo '../'`intrincs/__movsb.c
In file included from /mnt/array1/dev/compilers/linux-mingw64/lib/gcc/x86_64-w64-mingw32/7.2.0/include/immintrin.h:45:0,
                 from /mnt/array1/dev/compilers/linux-mingw64/lib/gcc/x86_64-w64-mingw32/7.2.0/include/x86intrin.h:48,
                 from /mnt/array1/dev/compilers/linux-mingw64/x86_64-w64-mingw32/include/intrin.h:73,
                 from ../intrincs/__movsb.c:10:
/mnt/array1/dev/compilers/linux-mingw64/lib/gcc/x86_64-w64-mingw32/7.2.0/include/avx512fintrin.h: In function ‘_mm512_mask_reduce_mul_ps’:
/mnt/array1/dev/compilers/linux-mingw64/lib/gcc/x86_64-w64-mingw32/7.2.0/include/avx512fintrin.h:13471:3: internal compiler error: Segmentation fault
   __A = _mm512_mask_mov_ps (_mm512_set1_ps (1.0f), __U, __A);
   ^~~
0xa497ef crash_signal
        ../../gcc/toplev.c:337
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.
make[1]: *** [intrincs/lib32_libkernel32_a-__movsb.o] Error 1
make[1]: Leaving directory `/mnt/array1/dev/build/src/mingw-w64/mingw-w64-crt/build'</code>

=> Note: we did not disable multilib in GCC core build process above, trying again with that: **same error**. Seriously ?...

====== Third trial ======

Now also taking into account the info from: https://www.reactos.org/wiki/Building_MINGW-w64 and building a reusable script function to automate the complete build process: <sxh bash># Helper method used to build a library with configure/make/instal steps
# arg1: the name of the library to build
# arg2: the opts as a string to pass to the configure command
# arg3: an optional target specialization flag: if this is "gcc", then the make and install targets are customized and the build folder is not removed (as we need it to finish the gcc build on the second pass)
nv_std_build()
{
  local cpus=$(grep -c processor /proc/cpuinfo 2>/dev/null || sysctl -n hw.ncpu | tr -d "\n" 2>/dev/null)
  local lname="$1"
  local cfg="$2"
  local tgt="$3"
  local maketgt=""
  local insttgt="install"
  if [ "$tgt" == "gcc" ]; then
    maketgt="all-gcc"
    insttgt="install-gcc"
  fi

  cd "$lname"
  mkdir build
  cd build
  ../configure $cfg || { echo "Configuring $lname FAILED."; return 1; }
  make -j $cpus $maketgt || { echo "Building $lname FAILED."; return 1; }
  make $insttgt || { echo "Installing $lname FAILED."; return 1; }
  cd ../
  if [ "$tgt" != "gcc" ]; then  # Do not clean after GCC pass 1
    rm -Rf build
  fi
  cd ../
}

# Helper method used to build a compiler from sources:
nv_build_mingw_compiler()
{
  local arch="$1"

  local sdir="/mnt/array1/dev/compilers" # source file directory
  local bdir="/mnt/array1/dev/build/mingw$arch" # Build directory
  local idir="/mnt/array1/dev/compilers/linux-mingw$arch" # installation directory
  
  local tgt="x86_64-w64-mingw32"
  if [ "$arch" == "32" ]; then
    tgt="i686-w64-mingw32"
  fi

  local host="x86_64-linux-gnu"
  local build="$host"

  local srcfile="mingw64_sources" # archive file base name
  local __binutils="binutils-2.28"
  local __gmp="gmp-6.1.2"
  local __headers="mingw-w64/mingw-w64-headers"
  local __crt="mingw-w64/mingw-w64-crt"
  local __mpfr="mpfr-3.1.5"
  local __mpc="mpc-1.0.3"
  local __isl="isl-0.18"
  local __gcc="gcc-7.1.0"

  local cpus=$(grep -c processor /proc/cpuinfo 2>/dev/null || sysctl -n hw.ncpu | tr -d "\n" 2>/dev/null)
  echo "Using $cpus CPUs"

  # Remove previous stuff:
  echo "Removing previous build folder $bdir..."
  rm -Rf "$bdir"

  echo "Removing previous install folder $idir..."
  rm -Rf "$idir"

  echo "Extracting source archive $srcfile"
  mkdir -p "$bdir"
  cd "$bdir"
  7za x "${sdir}/${srcfile}.7z" 

  # Setup the directories:
  local srcdir="$bdir/src"
  
  cd "$srcdir"

  # Build the binutils:
  nv_std_build $__binutils "--target=$tgt --prefix=$idir --with-sysroot=$idir --disable-shared --enable-static --disable-multilib --disable-nls" || { echo "=> Error in binutils build"; return 1; }

  # Ensure our path contains our new compiler location:
  if [[ `echo $PATH | grep $idir/bin` == "" ]]; then
  	echo "Adding new compiler location to path: $idir/bin"
  	export PATH="$idir/bin:$PATH"
  fi

  # Install the headers:
  nv_std_build $__headers "--host=$tgt --build=$build --prefix=$idir/$tgt --enable-secure-api" || { echo "=> Error in mingw64 headers build"; return 1; }

  # Create the symlinks:
  cd $idir
  ln -s $tgt mingw

  cd "$srcdir"

  # Move dependencies into GCC folder:
  mv $__gmp $__gcc/gmp
  mv $__mpfr $__gcc/mpfr
  mv $__mpc $__gcc/mpc
  mv $__isl $__gcc/isl

  # Build GCC, pass 1
  nv_std_build $__gcc "--target=$tgt --prefix=$idir --enable-languages=c,c++ --disable-multilib --disable-shared --enable-static --disable-nls" "gcc" || { echo "=> Error in gcc build"; return 1; }

  # Build the crt
  nv_std_build $__crt "--host=$tgt --build=$build --prefix=$idir/$tgt --with-sysroot=$idir/$tgt" || { echo "=> Error in mingw64 CRT build"; return 1; }

  # Finish building GCC:
  cd "$srcdir/$__gcc/build"
  make || { echo "Building gcc pass 2 FAILED."; return 1; }
  make install || { echo "Installing gcc pass 2 FAILED."; return 1; }
  cd ..
  rm -Rf build
  
  echo "=> Compiler build SUCCESSFUL."
}

nv_build_mingw64_compiler()
{
  nv_build_mingw_compiler 64
}

nv_build_mingw32_compiler()
{
  nv_build_mingw_compiler 32
}</sxh>


**Note**: we can figure out our current machine triplet with: <sxh bash>$ gcc -dumpmachine
x86_64-linux-gnu</sxh>

Got a configuration error for MPFR 3.1.4: <code>configure: error: C preprocessor "/lib/cpp" fails sanity check</code>
  * According to [[https://askubuntu.com/questions/509663/c-preprocessor-lib-cpp-fails-sanity-check|this page]] this means the C++ compiler is missing, so installing with <sxh bash>sudo apt-get install g++</sxh>
  * => Nope, C++ compiler is already installed. So checking the config.log... From there we have: <code>configure:4936: /lib/cpp  -I/mnt/array1/dev/compilers/linux-mingw64/include conftest.c
In file included from /mnt/array1/dev/compilers/linux-mingw64/include/crtdefs.h:10:0,
                 from /mnt/array1/dev/compilers/linux-mingw64/include/limits.h:6,
                 from conftest.c:10:
/mnt/array1/dev/compilers/linux-mingw64/include/_mingw.h:264:2: error: #error Only Win32 target is supported!
 #error Only Win32 target is supported!
  ^
</code>
  * So we have to specify the proper target value when building MPFR ? => **No**: this will generate another error: "target" is not expected and we should use "host" instead.

Found additional indications on [[http://pete.akeo.ie/2010/07/compiling-mingw-w64-with-multilib-on.html|this page]] => so it seems we can put the GCC dependencies directly inside the gcc folder and expect it to use them ?

Got an error in GCC build pass 2: <code>checking for ld that supports -Wl,--gc-sections... configure: error: Link tests are not allowed after GCC_NO_EXECUTABLES</code>
  * Trying to build gcc without **--with-system-zlib**: **same error**
  * Checking the config.log file: **build/x86_64-w64-mingw32/libstdc++-v3/config.log**: **nothing**
  * Checking this build script: https://github.com/Zeranoe/mingw-w64-build/blob/master/mingw-w64-build and trying to update our own build script: **same error**
	* Using the previous script directly: **Working** !!
	* => So we need to figure out what we are doing wrong in our script...
  * Integrated the Zeranoe script in our library: **still working** but the outputs pipes need fixing: **fixed**.

=> So, on the whole, still no clear idea why our script didn't work... but maybe we should try to download the same sources as the one used in the zeranoe script! => **And this works!** so it's really the source package provided on the mingw64 sourforge download page which was to blame here. My script above is now working when using the same sources as Zeranoe!

Also tried to build the compiler for the i686 target: **OK! build successful** :-).

====== Conclusion ======

Well, this experiment was a lot more painful than I anticipated... I thought that building mingw compilers would be something at least somewhat appropriately documented. But **it is not**. Seriously, I think someone from the developer team should take the time to **clean** the various wiki pages and pseudo explanations on how to build from sources, to only keep an updated version that should be easy enough to find on the web...

But anyway, in the end the only working option was the one I found on the [[https://github.com/Zeranoe/mingw-w64-build/blob/master/mingw-w64-build|zeranoe github project]]: so I would advise anyone who is interested in building this compiler to check the script provided there directly.

====== Selecting threading model ======

Okay, this is all very good until you then try to build a project with those compilers where you try to use for instance the **std::mutex** class => Guess what ? It's not available! Why ? Most probably because the compilers we just built are using the so called "win32 threading model", and that support for all the new C++ threading stuff is only provided when using the "posix threading model". So, how do we change this threading model ?
  * cf. https://stackoverflow.com/questions/17242516/mingw-w64-threads-posix-vs-win32

<note>**Thought of the day**: It's just incredible how hard it can be to find the answer to what seems to be a simple and legitimate question on the web such as:

>"The download packages for mingw-w64 come in a 'posix' or 'win32' threading model, then how do you build mingw-w64 with either 'posix' or 'win32' threading model ?"
=> As most of the time, all you get is a collection of stupid ideas, useless or unrelated discussions and partial explanations going in all directions, and then of course you are the one who spend the complete day just trying to figure out where the world is going...</note>

Finally found this [[https://github.com/niXman/mingw-builds|github project]] where the build script mention the thread model to be used... investigating.
  * Found from that project that the threading model should be passed to gcc configuration command: <sxh bash>--enable-threads=(posix|win32)</sxh>
  * Also reading [[https://gcc.gnu.org/install/configure.html|GCC configure documentation]]

=> So rebuilding our compilers with **posix thread model** this time with threads and exception specifications: <sxh bash>--enable-threads=posix --enable-sjlj-exceptions</sxh>

But of course when trying just that you get an error: <code>In file included from ../../../libgcc/gthr.h:148:0,
                 from ../../../libgcc/unwind-sjlj.c:31:
./gthr-default.h:35:10: fatal error: pthread.h: No such file or directory
 #include <pthread.h>
          ^~~~~~~~~~~
compilation terminated.
make[2]: *** [unwind-sjlj.o] Error 1</code>

We need to provide this pthread library somehow obviously...
  * According to [[http://dlbeer.co.nz/articles/mingw64.html|this page]] it sounds like if we should build the winpthread library ourself just after building the CRT: **not working**: the winpthread build is successfull but we still have the same error in gcc pass 2, and the **pthread.h** is not available :-(
  * => **OK Now** :-)! We just needed to install the winpthreads library in the appropriate folder! Now the compilers are building again.
  * Testing with **i686** compiler: **OK**
  * Testing build a project library using std::mutex with the new win64 compiler: **Working!**
  * Testing build a project library using std::mutex with the new win32 compiler: **Working!**


~~DISCUSSION:off~~