Project

General

Profile

NDKBuild » History » Revision 4

Revision 3 (Beuc Beuc, 07/24/2015 09:56 PM) → Revision 4/20 (Beuc Beuc, 07/24/2015 10:05 PM)

h1. Build a Native Developement Kit (NDK) with Replicant 4.2 

 h2. Principle of operation 

 The [[ReplicantSourceCode|Android 4.2 source code]] ships a @ndk@ directory with NDK build scripts. 
 These tools download various upstream imports from https://android.googlesource.com/toolchain/XXX/ with optional commit date (no branches or tags), apply patches, and build them. 

 Note: as of 2015-07, the upcoming Android version moved to a different build system, and will populate $topdir/toolchain/ through @repo@ instead of invoking git in a temporary directory (i.e. ditched @download-toolchain-sources.sh@). 

 h2. Ubuntu 12.04 build environment with LXC 

 Replicant 4.2 is based on CyanogenMod 10.1 which is based on AOSP 4.2.2 (2013-02). 
 The build environment used by Google should be Ubuntu LTS 12.04. 
 We recreate it through LXC.    If you know how to use Triskel 6 instead, please contribute the instructions. 

 <pre> 
 lxc-create -n replicant -t download -- -d ubuntu -r precise -a amd64 
 # Note: or Triskel 6, if you know how to configure LXC/debootstrap for it 
 lxc-start -n replicant -d 
 lxc-attach -n replicant 

 # clean-up non-free sources 
 sed -i -e 's/ restricted//' -e 's/ multiverse//' /etc/apt/sources.list 
 apt-get update 

 # prepare build user 
 apt-get install openssh-server 
 useradd replicant --shell /bin/bash --create-home 
 mkdir -p -m 700 ~replicant/.ssh 
 cat <<EOF >> ~replicant/.ssh/authorized_keys 
 your public key 
 EOF 
 chown -R replicant: ~replicant 
 </pre> 

 Note: compiling from Debian 8 will yield errors from texinfo (doc in binutils) and Perl (POD in LLVM) due to more recent, stricter versions of these. 

 h2. Build dependencies 

 <pre> 
 # Base Android build dependencies (from https://source.android.com/source/initializing.html, section "Installing required packages (Ubuntu 12.04)") 
 # Dropping mingw to skip windows builds for now. 
 # Installing g++-multilib first otherwise apt-get complains. 
 apt-get install build-essential g++-multilib 
 apt-get install git gnupg flex bison gperf \ 
   zip curl libc6-dev libncurses5-dev:i386 x11proto-core-dev \ 
   libx11-dev:i386 libreadline6-dev:i386 libgl1-mesa-glx:i386 \ 
   libgl1-mesa-dev \ 
   python-markdown libxml2-utils xsltproc zlib1g-dev:i386 

 # Additional NDK build dependencies (completed from https://android.googlesource.com/platform/ndk/+/master/README.md) 
 apt-get install curl texinfo bison flex libtool pbzip2 groff 

 # Make 'bash' the default shell otherwise 'build-gabi++.sh' will fail 
 dpkg-reconfigure dash 
 </pre> 

 h2. Preparing the sources 

 Based on [[ReplicantSourceCode]] with additional configuration. 

 <pre> 
 mkdir ~/tools/ 
 cd ~/tools/ 
 wget http://commondatastorage.googleapis.com/git-repo-downloads/repo 
 chmod a+x repo 
 cd ~ 

 cat <<EOF >> ~/.bashrc 
 export USE_CCACHE=1 
 EOF 

 # Avoid prompts 
 git config --global user.email "you@example.com" 
 git config --global user.name "Your Name" 
 git config --global color.ui true 

 mkdir replicant-4.2/ 
 cd replicant-4.2/ 
 ../tools/repo init -u git://git.replicant.us/manifest.git -b replicant-4.2 
 ../tools/repo sync -j4  
 # > 1h, 27GB (inc. 17GB .repo) 
 </pre> 

 h2. Determining the checkout date 

 Since there's no NDK release tag for the AOSP @toolchain/@ checkouts, we need to find the right date. 

 @cd ndk/ && git log@ says the last commit is 2012-10-05, so it's around NDK r8b (2012-07) and NDK r8c (2012-11). 
 Checking @build/tools/toolchain-patches/@ in the NDKs shows r8c has matching patches. 

 Also NDKs from google contain several @SOURCES@ files listing Git repos and commit IDs. 
 With r8c's release date (@ls --full-time RELEASE.TXT@), @build/tools/download-toolchain-sources.sh@ checkouts the same Git commit IDs, and the patches apply. 
 Only, r8c references a LLVM Git repo, while our scripts download a tarball. 
 So our scripts match a NDK version between r8b and r8c, and can almost exactly reproduce r8c. Let's call it r8b2. 

 h2. Building the SDK 

 Based on @ndk/docs/DEVELOPMENT.html@, with much improvement :) 

 <pre> 
 export NDK=~/replicant-4.2/ndk 
 cd $NDK 

 # Allow checking build errors 
 export NDK_LOGFILE=$HOME/ndk.log 

 # Follow Python download redirection 
 sed -i -e 's/curl -S/curl -L -S/' build/tools/download-toolchain-sources.sh 

 # Get the sources from toolchain/ repos, with r8c release date: 
 bash $NDK/build/tools/download-toolchain-sources.sh --git-date='2012-10-22T07:59:40+0200' ~/ndk-dl 
 # ndk-dl: 2.5GB 

 # Define NDK_TMPDIR and and use --incremental so the build can be resumed on error 
 export NDK_TMPDIR=~/ndk-tmp 
 mkdir -p $NDK_TMPDIR/release-r8b2/ 

 # Build the release 
 bash $NDK/build/tools/make-release.sh --toolchain-src-dir=$HOME/ndk-dl --release=r8b2 --incremental 
 # 30mn with 4 cores, 28GB (inc. 17GB .repo), 2.5GB ndk-dl, 1GB /tmp 
 </pre> 

 The GNU/Linux NDK release is in @/tmp/ndk-replicant/release/android-ndk-r8b2-linux-x86.tar.bz2@ :) 


 h2. TODOs 

 h3. Windows build 

 Build deps: 
 <pre> 
 # windows build is triggered by the presence of mingw32 
 apt-get install mingw32 tofrodos 
 # and requires running maketab.exe 
 apt-get install wine 
 # I had troubles in LXC registering direct .exe execution: 
 # $ /tmp/ndk-replicant/tmp/build-3038/maketab.exe  
 # run-detectors: unable to find an interpreter for /tmp/ndk-replicant/tmp/build-3038/maketab.exe 
 # Work-around: 
 update-binfmts --disable 
 dpkg-reconfigure wine1.4 --pri=high 
 </pre> 

 Code fixes: 
 <pre> 
 # build-ndk-stack/elff/ produces a warning with mingw32, and uses -Werror 
 # cf. https://android.googlesource.com/platform/ndk/+/32e74f3f1b969ff65f037e1ee89e21a5cbc0ecf0 
 sed -i -e 's/-Werror//' sources/host-tools/ndk-stack/GNUMakefile 
 # TODO: maybe 'export EXTRA_CFLAGS=-Wall' would be enough? 
 # TODO: does this mean google was rebuilding from Ubuntu 10.04 instead of 12.04? 
 </pre> 

 ... 

 <pre> 
 bash $NDK/build/tools/make-release.sh --toolchain-src-dir=$HOME/ndk-dl --release=r8b2 --incremental 
 # /home/replicant/ndk-dl/build/../binutils/binutils-2.21/gold/arm.cc:2171: internal compiler error: in make_rtl_for_nonlocal_decl, at cp/decl.c:4971 
 # => let's disable the windows build for now.. maybe better with 10.04? or a later ndk.git revision? 
 </pre> 

 h3. Other build method 

 Attempt to rebuild r8e with the same build environment but getting .git repos directly from AOSP: 
 less patches + local LLVM repo + precise control over the ndk scripts revision 

 According to @ndk/docs/DEVELOPMENT.html@ one only needs: 
   git clone https://xxx/platform/ndk.git ndk 
   git clone https://xxx/platform/development.git development 

 This means we wouldn't rely on exactly the build scripts shipped in Replicant 4.2 - but we already grab all the @toolchain/@ sources from AOSP, don't we?