Replicant

h1. PathInterposer

{{toc}}

h2. Warnings

This page is a work in progress, so it might contains mistakes, inaccuracies, etc.

h2. Introduction

Sometimes we need to use host tools for building Replicant.

This has several consequences:

* When the tools that we inherit from AOSP or LineageOS lack certain features (like when the host xz provided for Android was compiled without the ARM BJC filter), we can easily use the host tools that have such features instead. The downside is that it makes the build process potentially less reproducible as the version of the tools provided by the host distributions can change with the distribution updates or when different people use different distributions.

* We typically trust the host distributions like Trisquel way better than binaries provided by Google. So even if both are free software, we would prefer to trust the distribution tools instead.

Ideally we could make Replicant build most of the tools we need in a reproducible way to not use the host tools without having to trust prebuilt binaries, but this is not always practical as doing that could take a lot of time. In addition we do need tools to build replacement tools in the first place.

In any case when using tools from the host distribution, it's a really good idea to not hardcode the path of the tool (like /usr/bin/python) and instead make sure that the host distribution @PATH@ is used instead.

This has several advantages:

* It helps a lot making Replicant compatible with more distributions as:

** not all the distributions have the tools in the same path (/usr/bin vs /usr/sbin)

** some distributions (like Guix) do respect the @PATH@ but don't use standard paths (repo could be in /gnu/store/4cw9j7c36cgv37g6bzs2dnygmfc5jxh1-git-repo-2.4.1/share/git-repo/repo or in ~/.guix-profile/bin/repo). And since Guix is reproducible and even bootstrapable we also want not to rule out this option, while also being compatible with other distributions (for simplicity and to not be stuck with Guix).

* users might need to override tools in some cases (that happened with proguard in Replicant 6)

This article shows how to use host tools in a way that makes use of the host @PATH@ and also how Android implements (with PathInterposer) the isolation between the tools provided by the host distribution and Android.

h2. How it works

During the build:

* The host path is saved to out/.path_interposer_origpath

* The host path is changed to a new one

* The path interposer is used to access host binaries. It uses the saved path for that.

With Trisquel 8, and Replicant 11, here's an example of how the path looks like for applications being launched through the path interposer[1]:

<pre>

/home/replicant/replicant-11/prebuilts/jdk/jdk11/linux-x86/bin

/home/replicant/replicant-11/prebuilts/jdk/jdk11/linux-x86/bin

/home/replicant/replicant-11/out/soong/host/linux-x86/bin

/home/replicant/replicant-11/out/host/linux-x86/bin

/home/replicant/replicant-11/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin

/home/replicant/replicant-11/development/scripts

/home/replicant/replicant-11/prebuilts/devtools/tools

/home/replicant/replicant-11/external/selinux/prebuilts/bin

/home/replicant/replicant-11/prebuilts/misc/linux-x86/dtc

/home/replicant/replicant-11/prebuilts/misc/linux-x86/libufdt

/home/replicant/replicant-11/prebuilts/clang/host/linux-x86/llvm-binutils-stable

/home/replicant/replicant-11/prebuilts/android-emulator/linux-x86_64

/home/replicant/replicant-11/prebuilts/asuite/acloud/linux-x86

/home/replicant/replicant-11/prebuilts/asuite/aidegen/linux-x86

/home/replicant/replicant-11/prebuilts/asuite/atest/linux-x86

/home/replicant/bin

/home/replicant/.local/bin

/usr/local/sbin

/usr/local/bin

/usr/sbin

/usr/bin

/sbin

/bin

/usr/games

</pre>

Here we can deduce that the paths set by the Android build system take precedence over the host paths.

fn1. This has been deduced by prints with the following patch:

<pre>

--- a/cmd/path_interposer/main.go

+++ b/cmd/path_interposer/main.go

@@ -116,6 +116,10 @@ func Main(stdout, stderr io.Writer, interposer string, args []string, opts mainO

                return 1, fmt.Errorf("Failed to set PATH env: %v", err)

        }

+       fmt.Fprintln(os.Stderr, "####################################")

+       fmt.Fprintln(os.Stderr, os.Getenv("PATH"))

+       fmt.Fprintln(os.Stderr, "####################################")

+

        if config := opts.config(base); config.Log || config.Error {

                var procs []paths.LogProcess

                if opts.lookupParents != nil {

</pre>

h2. Launching host binaries without touching the prebuilts

In Replicant 10, we could use out/.path/python3 as python3 interpreter assuming that we enabled it in build/soong/ui/build/paths/config.go

for instance in the Mesa (external/mesa3d) Android.mk we had:

<pre>

MESA_PYTHON2 := out/.path/python2

MESA_PYTHON3 := out/.path/python3

</pre>

This worked because 'python2' and 'python3' were only available in the host path:

<pre>

$ for path in $(echo /home/replicant/replicant-10/prebuilts/jdk/jdk9/linux-x86/bin:/home/replicant/replicant-10/prebuilts/jdk/jdk9/linux-x86/bin:/home/replicant/replicant-10/out/soong/host/linux-x86/bin:/home/replicant/replicant-10/out/host/linux-x86/bin:/home/replicant/replicant-10/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin:/home/replicant/replicant-10/development/scripts:/home/replicant/replicant-10/prebuilts/devtools/tools:/home/replicant/replicant-10/external/selinux/prebuilts/bin:/home/replicant/replicant-10/prebuilts/misc/linux-x86/dtc:/home/replicant/replicant-10/prebuilts/misc/linux-x86/libufdt:/home/replicant/replicant-10/prebuilts/android-emulator/linux-x86_64:/home/replicant/replicant-10/prebuilts/asuite/acloud/linux-x86:/home/replicant/replicant-10/prebuilts/asuite/aidegen/linux-x86:/home/replicant/replicant-10/prebuilts/asuite/atest/linux-x86:/home/replicant/bin:/home/replicant/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games | sed 's#:#\n#g') ; do ls "${path}/python3" ; done

ls: cannot access '/home/replicant/replicant-10/prebuilts/jdk/jdk9/linux-x86/bin/python3': No such file or directory

ls: cannot access '/home/replicant/replicant-10/prebuilts/jdk/jdk9/linux-x86/bin/python3': No such file or directory

ls: cannot access '/home/replicant/replicant-10/out/soong/host/linux-x86/bin/python3': No such file or directory

ls: cannot access '/home/replicant/replicant-10/out/host/linux-x86/bin/python3': No such file or directory

ls: cannot access '/home/replicant/replicant-10/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin/python3': No such file or directory

ls: cannot access '/home/replicant/replicant-10/development/scripts/python3': No such file or directory

ls: cannot access '/home/replicant/replicant-10/prebuilts/devtools/tools/python3': No such file or directory

ls: cannot access '/home/replicant/replicant-10/external/selinux/prebuilts/bin/python3': No such file or directory

ls: cannot access '/home/replicant/replicant-10/prebuilts/misc/linux-x86/dtc/python3': No such file or directory

ls: cannot access '/home/replicant/replicant-10/prebuilts/misc/linux-x86/libufdt/python3': No such file or directory

ls: cannot access '/home/replicant/replicant-10/prebuilts/android-emulator/linux-x86_64/python3': No such file or directory

ls: cannot access '/home/replicant/replicant-10/prebuilts/asuite/acloud/linux-x86/python3': No such file or directory

ls: cannot access '/home/replicant/replicant-10/prebuilts/asuite/aidegen/linux-x86/python3': No such file or directory

ls: cannot access '/home/replicant/replicant-10/prebuilts/asuite/atest/linux-x86/python3': No such file or directory

ls: cannot access '/home/replicant/bin/python3': No such file or directory

ls: cannot access '/home/replicant/.local/bin/python3': No such file or directory

ls: cannot access '/usr/local/sbin/python3': No such file or directory

ls: cannot access '/usr/local/bin/python3': No such file or directory

ls: cannot access '/usr/sbin/python3': No such file or directory

/usr/bin/python3

ls: cannot access '/sbin/python3': No such file or directory

ls: cannot access '/bin/python3': No such file or directory

ls: cannot access '/usr/games/python3': No such file or directory

</pre>

For instance @prebuilts/python/linux-x86/2.7.5/bin/python@ wasn't in the path.