Project

General

Profile

Actions

DeprecatedPortingToAndroid10 » History » Revision 192

« Previous | Revision 192/289 (diff) | Next »
Denis 'GNUtoo' Carikli, 07/24/2019 10:33 AM


Porting Replicant to Android 9

AOSP 9

Status with AOSP 9

  • The Galaxy SIII 4G (i9305) boots (with graphics hacks) under AOSP9, without having to import nonfree libraries:
    • The device boots until the lock screen
    • The graphics work but are really slow as they have not been optimized
    • Sometime the display content is corrupted
    • The touchscreen works
  • The kernel used is based on Linux 5.0-rc6 and has ~150 Android specific patches on top and some patches for the Galaxy SIII 4G (i9305) on top

Repositories and changes

  • git://git.putti.eu/aosp/manifest_i9305.git
  • git://git.putti.eu/aosp/kernel_i9305.git
  • git://git.putti.eu/aosp/device_i9305.git
  • git://git.putti.eu/aosp/external_mesa3d.git
  • git://git.putti.eu/aosp/build_make.git
  • git://git.putti.eu/aosp/core.git

Building AOSP version

Source code

First get the source code:

$ repo init -u git://git.putti.eu/aosp/manifest_i9305.git -b android-9.0.0

For Trisquel 8

sudo apt-get install git-core gnupg flex bison gperf build-essential zip curl zlib1g-dev gcc-multilib g++-multilib libc6-dev-i386 lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z-dev libgl1-mesa-dev libxml2-utils xsltproc unzip bc python-mako

Building it

Once this is done, see the device/putti/i9305/README.md for the build instructions.

Trisquel 8 LXC

If you use Parabola, you may be interested in running Trisquel 8 in LXC.

To do that first debootstrap a Trisquel 8 rootfs.

Parabola's debootstrap does support Trisquel 8 and its manual has an example on how to do that:

$ man debootstrap
[...]
# debootstrap flidas flidas-root http://archive.trisquel.info/trisquel

Then you can use virt-manager to setup the LXC instance.

The advantages of this solution are that:
  • The LXC guest and host shares their resources (CPU, RAM) with almost no penalty
  • Trisquel 8 is not a rolling release distribution
The disadvantage of this solution are that:
  • you need to configure again the Trisquel 8 LXC instance (vimrc, sshd_config, etc)
  • It's more complicated to setup
  • The Android build system output a warning message about not being able to use namespaces which may become mandatory in newer Android versions

LineageOS 16

Repositories and changes for LineageOS 16

Building LineageOS 16 version

Source code

$ repo init -u https://git.replicant.us/GNUtoo/manifest.git -b GNUtoo/lineage-16.0-base
$ repo sync

For Trisquel 8

sudo apt-get install bc bison build-essential ccache curl flex g++-multilib gcc-multilib gettext git gnupg gperf imagemagick lib32ncurses5-dev lib32readline-dev lib32z1-dev liblz4-tool libncurses5-dev libsdl1.2-dev libssl-dev libwxgtk3.0-dev libxml2 libxml2-utils lzop python-mako pngcrush rsync schedtool squashfs-tools xsltproc zip zlib1g-dev
sudo apt-get install gcc-5-arm-linux-gnueabi

Launching the build

Follow the i9300 manifest instructions:

$ subject='/C=US/ST=California/L=Mountain View/O=Android/OU=Android/CN=Android/emailAddress=android@android.com'
$ mkdir .android-certs
$ for x in releasekey platform shared media testkey; do \
  ./development/tools/make_key .android-certs/$x "$subject"; \
  done
$ parallel_tasks=$(echo "$(grep 'processor' /proc/cpuinfo | wc -l ) + 1" | bc)
$ source build/envsetup.sh
$ lunch lineage_i9305-eng
$ make -j${parallel_tasks} bacon

The images can then be flashed with heimdall. They are then in:

  out/target/product/i9305/obj/PACKAGING/target_files_intermediates/lineage_i9305-target_files-eng.replicant/IMAGES/

Getting lastest changes

  • The repositories are being constantly modified with git push --force as we are trying things out, and don't want to make the commits history look too much dirty, so be sure to backup your local changes.
  • Sometimes the manifest repository is also modified with git push --force. In that case the following commands will loose all the work you did locally but will make the repository consistent with upstream repositories again:
    $ rm -rf .repo/manifests/ # This enables me to force push and rebase the manifest repository as well
    $ repo init -u https://git.replicant.us/GNUtoo/manifest.git -b GNUtoo/lineage-16.0-base
    $ repo sync --force-sync
    
  • The following command might also be necessary to make the state consistent with upstream repositories again, when the manifest history wasn't rewritten, but it will also loose all the work you did locally:
    $ repo sync --force-sync
    

Install the images

$ sudo heimdall flash --BOOT out/target/product/i9305/obj/PACKAGING/target_files_intermediates/lineage_i9305-target_files-eng.replicant/IMAGES/boot.img --USERDATA out/target/product/i9305/obj/PACKAGING/target_files_intermediates/lineage_i9305-target_files-eng.replicant/IMAGES/userdata.img --SYSTEM out/target/product/i9305/obj/PACKAGING/target_files_intermediates/lineage_i9305-target_files-eng.replicant/IMAGES/system.img 

Get adb

As the device IDs are the ones given by the Linux kernel, they are not in the adb udev rules, so for now it requires to run adb as root:

$ sudo adb shell
* daemon not running; starting now at tcp:5037
* daemon started successfully
i9305:/ #                                 
$ sudo adb kill-server
$ adb shell
* daemon not running; starting now at tcp:5037
* daemon started successfully
error: no devices/emulators found

So make sure to kill the adb-server and run it as root:
$ adb kill-server
$ sudo adb shell
* daemon not running; starting now at tcp:5037
* daemon started successfully
i9305:/ # 

Booting

At some point during boot, the device goes into suspend, so you will need to press some buttons like the power or volume button to make the boot continue until the graphical interface.

You can also follow the boot progress with adb:

adb logcat
adb logcat -b main

Note that the device can go into suspend at any time, so adb might be interrupted. That looks like that:
First you get a shell

$ sudo adb shell
i9305:/ #

Then the connection is interrupted:

$ adb shell
i9305:/ # [randomdev@fullyfreelaptop ]$                                                                                                     

The effect with adb logcat is similar.

cleanups to be done

  • Make adb work as user by using the right USB IDs, and make userspace do the USB setup.
  • Make the kernel not use hardcoded CMDLINE_FORCE Not possible unless the bootloader is changed or Linux is very heavily patched.
  • Make the kernel not use hardcoded partitions if possible (though we use system as root)
  • Make a clean Gatekeeper HAL module implementation instead of using the same hack than goldfish
  • look at init.rc documentation to see if init.rc can be overriden clearly with the override statement to see if it's possible to keep the serial console patch for -eng

Using SwiftShader instead of Mesa3D llvmpipe for software rendering

To use SwifShader you need a kernel that supports UDIV/SDIV emulation, you can checkout the branch GNUtoo/udiv-emulation for kernel/replicant/linux. After doing that that run the following commands to use SwiftShader:

$ adb remount
$ adb shell
i9305:/ # rm vendor/lib/egl/libGLES_mesa.so # or move it somewhere safe meanwhile you test SwiftShader
i9305:/ # nano system/build.prop # set ro.hardware.hwcomposer=ranchu and ro.hardware.gralloc=default
$ adb reboot

Upstreaming status

TODO

First month of full time equivalent work:

Time estimation Task Comments
DONE boot a device under AOSP9 Only boots with graphics, not much more
7h DONE build it under a FSDG compliant distribution like Trisquel8 WIP for AOSP, It's difficult to do precise time estimations as it could work out of the box or require one full time month of work depending on how much issues are encountered
Builds under Trisquel8
21h * port the changes from AOSP9 to LineageOS 16
* cleanup the code
* build the kernel from the Android build system
* make sure it builds with an FSDG compliant distribution
* document the build procedure

Status:
* Boots with adb.
* Has ultra slow graphics
14h find, remove and document proprietary software in LineageOS 16
21h find, remove and document privacy issues in LineageOS 16
7h port the touch keys Linux driver dts for the galaxy-s3 boards Also requires a n710x for testing the patch
7h upstream the AAT1290 flash led Linux dts for the galaxy-s3 boards applied upstream
7h rebrand LineageOS as Replicant
70h port and cleanup the the Galaxy SIII (i9300) modem Linux driver from 4.16 to 5.0
Total: 147h (~1 month)
Second month of full time equivalent work:
* port libsamsung-ril and libsamsung-ipc to Android 9
* Make the modem driver and libsamsung-ipc work together
157h libsamsung-ipc should be trivial, libsamsung-ril probably needs to be heavily adapted to Trebble or the new RIL interfaces
libsamsung-ipc and samsung-ril were fixed to compile on Android 9.
The code was not yet tested at runtime.
It also probably still need to be adapted to trebble or the new RIL interfaces to be more substainable.
Status:
* The code has been updated to work without a wrapper on Android 9, but some cleanups are needed in all the modem related components.
* The modem doesn't work yet on Replicant 9 but some of the initialization already works.
On Replicant 6 we have:
 D use-Rlog/RLOG-RIL-IPC: Starting i9300 modem boot
 D use-Rlog/RLOG-RIL-IPC: Opened modem image device
 D use-Rlog/RLOG-RIL-IPC: Mapped modem image data to memory
 D use-Rlog/RLOG-RIL-IPC: Opened modem boot device
 D use-Rlog/RLOG-RIL-IPC: Opened modem link device
 D use-Rlog/RLOG-RIL-IPC: Turned the modem off
 D use-Rlog/RLOG-RIL-IPC: Turned the modem on
 D use-Rlog/RLOG-RIL-IPC: Waited for link connected
 D use-Rlog/RLOG-RIL-IPC: Wrote ATAT in ASCII
 D use-Rlog/RLOG-RIL-IPC: Reading boot ACK failed
 D use-Rlog/RLOG-RIL-IPC: Sending XMM626 HSIC PSI failed
 E use-Rlog/RLOG-RIL-IPC: Booting IPC FMT client failed
 E use-Rlog/RLOG-RIL: Opening IPC FMT client failed
 D use-Rlog/RLOG-RIL: Closed IPC FMT client
 D use-Rlog/RLOG-RIL-IPC: Starting i9300 modem boot
 D use-Rlog/RLOG-RIL-IPC: Opened modem image device
 D use-Rlog/RLOG-RIL-IPC: Mapped modem image data to memory
 D use-Rlog/RLOG-RIL-IPC: Opened modem boot device
 D use-Rlog/RLOG-RIL-IPC: Opened modem link device
 D use-Rlog/RLOG-RIL-IPC: Turned the modem off
 D use-Rlog/RLOG-RIL-IPC: Turned the modem on
 D use-Rlog/RLOG-RIL-IPC: Waited for link connected
 D use-Rlog/RLOG-RIL-IPC: Wrote ATAT in ASCII
 D use-Rlog/RLOG-RIL-IPC: Read chip id (0x16)
 D use-Rlog/RLOG-RIL-IPC: Wrote PSI header
 D use-Rlog/RLOG-RIL-IPC: Wrote PSI, CRC is 0xc8
 D use-Rlog/RLOG-RIL-IPC: Wrote PSI CRC (0xc8)
 D use-Rlog/RLOG-RIL-IPC: Read PSI CRC ACK
 D use-Rlog/RLOG-RIL-IPC: Read PSI ACK
 D use-Rlog/RLOG-RIL-IPC: Sent XMM626 HSIC PSI
 D use-Rlog/RLOG-RIL-IPC: Wrote EBL size
 D use-Rlog/RLOG-RIL-IPC: Wrote EBL, CRC is 0xb5
 D use-Rlog/RLOG-RIL-IPC: Wrote EBL CRC (0xb5)
 D use-Rlog/RLOG-RIL-IPC: Sent XMM626 HSIC EBL
 D use-Rlog/RLOG-RIL-IPC: Read port config
 D use-Rlog/RLOG-RIL-IPC: Sent XMM626 HSIC port config
 D use-Rlog/RLOG-RIL-IPC: Sent XMM626 HSIC SEC start
 D use-Rlog/RLOG-RIL-IPC: Sent XMM626 HSIC firmware
 D use-Rlog/RLOG-RIL-IPC: Checked nv_data path
 D use-Rlog/RLOG-RIL-IPC: Checked nv_data md5 path
 D use-Rlog/RLOG-RIL-IPC: Calculated nv_data md5: 50e04bd1ae13445977f3aec356ada08d
 D use-Rlog/RLOG-RIL-IPC: Read nv_data md5: 50e04bd1ae13445977f3aec356ada08d
 D use-Rlog/RLOG-RIL-IPC: Checked nv_data backup path
 D use-Rlog/RLOG-RIL-IPC: Loaded nv_data
 D use-Rlog/RLOG-RIL-IPC: Sent XMM626 HSIC nv_data
 D use-Rlog/RLOG-RIL-IPC: Sent XMM626 HSIC HW reset
 D use-Rlog/RLOG-RIL-IPC: Waiting for host wake failed
 D use-Rlog/RLOG-RIL-IPC: Waited for host wake
 D use-Rlog/RLOG-RIL-IPC: Waited for link connected
 D use-Rlog/RLOG-RIL: Opened IPC FMT client
 D use-Rlog/RLOG-RIL-IPC: 
 D use-Rlog/RLOG-RIL-IPC: xmm626_sec_modem_fmt_recv: Received FMT message
 D use-Rlog/RLOG-RIL-IPC: xmm626_sec_modem_fmt_recv: Message: aseq=0xff, command=IPC_PWR_PHONE_PWR_UP, type=IPC_TYPE_NOTI, size=0
 D use-Rlog/RLOG-RIL: Updating RIL radio state to 0
 [...]

On Replicant 9 we have:
 
 # adb shell
 # /system/bin/xmm6262_boot0.sh 
 # exit
 # adb logcat -b radio
 [...] D use-Rlog/RLOG-RIL-IPC: Starting generic modem boot
 D use-Rlog/RLOG-RIL-IPC: Opened modem image device
 D use-Rlog/RLOG-RIL-IPC: Mapped modem image data to memory
 D use-Rlog/RLOG-RIL-IPC: Turned the modem off
 D use-Rlog/RLOG-RIL-IPC: Turned the modem on
 D use-Rlog/RLOG-RIL-IPC: Opened modem boot device
 D use-Rlog/RLOG-RIL-IPC: Wrote ATAT in ASCII
 D use-Rlog/RLOG-RIL-IPC: Read chip id (0x16)
 D use-Rlog/RLOG-RIL-IPC: Wrote PSI header
 D use-Rlog/RLOG-RIL-IPC: Wrote PSI, CRC is 0xc8
 D use-Rlog/RLOG-RIL-IPC: Wrote PSI CRC (0xc8)
 D use-Rlog/RLOG-RIL-IPC: Read PSI CRC ACK
 D use-Rlog/RLOG-RIL-IPC: Read PSI ACK
 D use-Rlog/RLOG-RIL-IPC: Sent XMM626 HSIC PSI
 D use-Rlog/RLOG-RIL-IPC: Wrote EBL size
 D use-Rlog/RLOG-RIL-IPC: Wrote EBL, CRC is 0xb5
 D use-Rlog/RLOG-RIL-IPC: Wrote EBL CRC (0xb5)
 D use-Rlog/RLOG-RIL-IPC: Sent XMM626 HSIC EBL
 D use-Rlog/RLOG-RIL-IPC: Read port config
 D use-Rlog/RLOG-RIL-IPC: Sent XMM626 HSIC port config
 D use-Rlog/RLOG-RIL-IPC: Sent XMM626 HSIC SEC start
 D use-Rlog/RLOG-RIL-IPC: Sent XMM626 HSIC firmware
 D use-Rlog/RLOG-RIL-IPC: ipc_nv_data_path_check: nv_data path: /efs/nv_data.bin
 D use-Rlog/RLOG-RIL-IPC: ipc_nv_data_path_check: nv_data size: 2097152
 D use-Rlog/RLOG-RIL-IPC: Checked nv_data path
 D use-Rlog/RLOG-RIL-IPC: Checked nv_data md5 path
 D use-Rlog/RLOG-RIL-IPC: Calculated nv_data md5: 50e04bd1ae13445977f3aec356ada08d
 D use-Rlog/RLOG-RIL-IPC: Read nv_data md5: 50e04bd1ae13445977f3aec356ada08d
 D use-Rlog/RLOG-RIL-IPC: Checked nv_data backup path
 D use-Rlog/RLOG-RIL-IPC: Loaded nv_data
 D use-Rlog/RLOG-RIL-IPC: Sent XMM626 HSIC nv_data
 D use-Rlog/RLOG-RIL-IPC: Sent XMM626 HSIC SEC end
 D use-Rlog/RLOG-RIL-IPC: Sent XMM626 HSIC HW reset
 D use-Rlog/RLOG-RIL-IPC: Waited for host wake
 D use-Rlog/RLOG-RIL-IPC: Waited for link connected
 E use-Rlog/RLOG-RIL-IPC: Opening IPC FMT client failed
 E use-Rlog/RLOG-RIL: Opening IPC FMT client failed
 [...]
Total 157h ~1 month
Third month of full time equivalent work:
Task Time estimation Comments
port the sensors libraries and other device specific libraries 70h Already done by the unofficial LineageOS port of the Galaxy SIII (i9300), needs testing
add support for Audio with the upstream kernel driver 70h Might be way faster, depending on what Android 9 uses
add partial (no modem) support for the Galaxy SIII (i9300) Galaxy SIII 4G (i9305) and factorize the code with i9305 14h The work for the Galaxy SIII (i9300) has already been done by the unofficial LineageOS port.
It needs to be adapted to the upstream kernel and not to use proprietary libraries.
It also needs to be adapted to the Galaxy SIII 4G (i9305)
Total 154h ~1 month
Task Time estimation Comments
create a recovery 21h
add internal WiFi support and validate the functionality 6h
add external WiFi dongles support 20h External dongles support might be tricky
create new update the install and upgrade instructions 35h Our current install instructions don't scale as we have one copy for each device.
We also created generic instructions but they tend to be harder to follow1 than the device specific ones.
This will be made in a modular format (for instance in LaTeX) that enables to generate per device install instructions without requiring to have multiples copy of the same text.
The instructions will need to be able to be modified and compiled on an FSDG compliant distribution.
Task Time estimation Comments
Estimate the amount of work to Reduce the attack surface ?
Estimate the amount of work to add in-system upgrades ?

1 The generic instructions were tested at Install parties in Paris

Devices support:

Easy, because it's similar enough to the Galaxy SIII (I9300)

Galaxy Note II (N7100)
Task Time estimation Comments
port the EA8061 LCD Linux driver 35h
port the S6EVR02 LCD Linux driver 35h
port the MAX77693 flash led Linux driver 7h
android: add support for the Note II (N7100) and factorize the code with Galaxy SIII (i9300) and Galaxy SIII 4G (i9305) 14h Should be similar to the Galaxy SIII
port the sensors libraries and other device specific libraries 70h It's difficult to evaluate how much time it could take
add support for Audio with the upstream kernel driver 14h Should be similar to the Galaxy SIII
Galaxy Note 8.0 (N5100) and 8.0 WiFi (N5110)
Task Time estimation Comments
Evaluate the time required to do the port 14h TODO

Needs more work and unknown upstream Linux status

Device Time estimation Comments
Galaxy S II (i9100) Linux: devboard dts upstream? unknown status
Galaxy Note (N7000) unknown Linux upstream status
Galaxy Nexus (I9250) OMAP4, no dts upsrteam
Galaxy Tab 2 7.0 (P3100), 7.0 WiFi (P3110), 10.1 (P5100), 10.1 WiFi (P5110)
GTA04 >= A4 TODO: a RIL needs to be written, userspace GPS support is missing, audio scenarios, etc

Documentation

Replicant 6.0 changes

Repository path rebase worked Changes pushed rebase in Replicant 6.0 Tag for the old history
build No
manifest No N/A
bootable/recovery Yes * UI changes to improve usability
* Branding
* Small fix(es)
external/boringssl Yes * Small fix(es)
external/heimdall No
frameworks/av Yes * Software rendering fix(es) Yes replicant-6.0-aug-2018
frameworks/base Yes * WiFi fixe(s) for external dongles
* Software rendering fix(es)
* Privacy fix(es)
* Security improvements
* Usability improvements
Yes replicant-6.0-sept-2018
frameworks/native No
hardware/broadcom/wlan Yes * Remove nonfree software
hardware/libhardware Yes * Software rendering fix(es) Yes replicant-6.0-0003
hardware/libhardware_legacy Yes * Don't depend on nonfree software
hardware/ti/omap4 Yes * Remove nonfree software
* Don't depend on nonfree software
* Small fix(es)
libcore Yes * Small fix(es)
packages/apps/Browser Yes * Privacy fix(es)
* Branding
* Small fix(es)
packages/apps/LegacyCamera Yes * Software rendering fix(es)
* Small fix(es)
packages/apps/RepWifi Not upstreamed, obsolete N/A
packages/apps/ReplicantWallpapers Not upstreamed * Branding
packages/apps/Settings No
packages/apps/SetupWizard Yes * Privacy fix(es)
* Don't depend on nonfree software
* Branding
packages/apps/ThemeChooser Yes * Privacy fix(es) ?
* Don't depend on nonfree software ?
packages/providers/ContactsProvider Yes * Small fix(es)
system/core Yes * Branding
* various fixes
vendor/replicant No
vendor/cmsdk Yes * Branding
external/chromium-webview ? ?
external/drm_gralloc Yes * Software rendering fix(es) Yes replicant-6.0-aug-2018
external/llvm Yes * Software rendering fix(es) Already rebased replicant-6.0-0003
external/mesa3d Yes * Software rendering fix(es)
* Performance debugging for software rendering
external/parameter-framework Yes * Small fix(es)
external/protobuf N/A No changes
prebuilts/ndk ? ?
prebuilts/sdk ? ?
sdk Yes * Small fix(es)
toolchain/src/binutils Yes * Small fix(es)
toolchain/src/gcc No
toolchain/src/jack Yes * Small fix(es)
toolchain/src/jill Yes * Small fix(es)
kernel/samsung/smdk4412 Yes * Security fixes
* Privacy Fixes (TrustZone)
* Don't depend on nonfree software
* Remove nonfree software
* Small fix(es)
Yes replicant-6.0-0003
kernel/samsung/tuna Yes replicant-6.0-0003
kernel/samsung/espresso10
device/samsung/i9100 No * Replace nonfree software by free software
* free software sensors implementation(s)
* free software modem audio implementation(s)
* Software rendering fix(es)
* Privacy Fixes (TrustZone)
* Don't depend on nonfree software
* Remove nonfree software
* Branding
* Small fix(es)
device/samsung/n7000 Yes
device/samsung/i9300 No
device/samsung/i9305 No
device/samsung/n7100 Yes Already rebased replicant-6.0-0003
device/samsung/n5100 Yes
device/samsung/n5110 Yes
device/samsung/maguro Yes
device/samsung/tuna Yes
device/samsung/espressowifi Yes
device/samsung/espresso3g Yes
device/samsung/galaxys2-common Yes
device/samsung/smdk4412-common No
device/samsung/kona-common Yes
hardware/ril/samsung-ril Not upstreamed
external/libsamsung-ipc Not upstreamed
hardware/tinyalsa-audio Not upstreamed?

CustomROMs i9300 components

Repository Tree path Dependencies Function Comments
https://github.com/CustomROMs/android_hardware_samsung
lineage-16.0 branch
hardware/samsung/macloader Loads the MAC Address of the WiFi network interface Might be useful
hardware/samsung/wifiloader Loads the wifi kernel module (like modprobe) and setup firmware filesystems permissions May be useful
hardware/samsung/audio seems to contains ril related stuff as well Look if the ril stuff is required, go for standard audio
hardware/samsung/lineagehw/hidl/livedisplay livedisplay is a feature similar to what redshift does on GNU/Linux Not sure if it works with mainline
hardware/samsung/exynos/multimedia/utils/ seem meant for audio/video decoding offload assembly obtimized color conversion and resize code check assembly code license, not sure if useful
all other directories in hardware/samsung/exynos/ nonfree firmwares, nonfree software?, smdk kernel? audio/video decoding offload Avoid using that
hardware/samsung/exynos3 nonfree firmwares?, nonfree software?, smdk kernel? some light libraries, display stuff (gralloc, etc), 2D acceleration (FIMG), camera (FIMC), 3D acceleration, etc Avoid using that for now
hardware/samsung/exynos4

Links

Updated by Denis 'GNUtoo' Carikli about 5 years ago · 192 revisions

Also available in: PDF HTML TXT