DeprecatedPortingToAndroid10 » History » Revision 224
Revision 223 (Denis 'GNUtoo' Carikli, 09/18/2019 03:36 PM) → Revision 224/289 (Denis 'GNUtoo' Carikli, 09/18/2019 03:36 PM)
h1. Porting Replicant to Android 9 {{>toc}} h2. LineageOS 16 h3. Repositories and changes for LineageOS 16 * https://git.replicant.us/GNUtoo/kernel_replicant_linux.git * git://git.putti.eu/aosp/external_mesa3d.git * https://git.replicant.us/GNUtoo/android_build.git * https://git.replicant.us/GNUtoo/device_i9305.git * https://git.replicant.us/GNUtoo/manifest.git * https://git.replicant.us/GNUtoo/vendor_replicant.git h3. Building LineageOS 16 version h4. Source code <pre> $ repo init -u https://git.replicant.us/replicant-9/manifest.git -b replicant-9 $ repo sync </pre> Alternatively a shallow copy of the source tree can be fetched in order to save on disk space: <pre> $ repo init -u https://git.replicant.us/replicant-9/manifest.git -b replicant-9 --depth=1 $ repo sync -c </pre> To `unshallow` a specific module: <pre> $ cd path/to/module $ git fetch --unshallow <remote> </pre> h4. For Trisquel 8 <pre> sudo apt-get install bc bison build-essential bsdmainutils ccache curl flex g++-multilib gcc-multilib gettext git gnupg gperf imagemagick lib32ncurses5-dev lib32readline-dev lib32z1-dev liblz4-tool libncurses5-dev libsdl1.2-dev libssl-dev libwxgtk3.0-dev libxml2 libxml2-utils lzop python-mako pngcrush rsync schedtool squashfs-tools xsltproc zip zlib1g-dev sudo apt-get install gcc-5-arm-linux-gnueabi </pre> h4. Launching the build Follow the "i9300 manifest instructions:":https://github.com/CustomRoms/android_local_manifests_i9300 <pre> $ subject='/C=US/ST=California/L=Mountain View/O=Android/OU=Android/CN=Android/emailAddress=android@android.com' $ mkdir .android-certs $ for x in releasekey platform shared media testkey; do \ ./development/tools/make_key .android-certs/$x "$subject"; \ done $ parallel_tasks=$(echo "$(grep 'processor' /proc/cpuinfo | wc -l ) + 1" | bc) $ source build/envsetup.sh $ lunch lineage_i9305-eng $ make -j${parallel_tasks} bacon </pre> The images can then be flashed with heimdall. They are then in: <pre> out/target/product/i9305/obj/PACKAGING/target_files_intermediates/lineage_i9305-target_files-eng.replicant/IMAGES/ </pre> h4. Getting lastest changes * The repositories are being constantly modified with git push --force as we are trying things out, and don't want to make the commits history look too dirty, so be sure to backup your local changes. * Sometimes the manifest repository is also modified with git push --force. In that case the following commands will loose all the work you did locally but will make the repository consistent with upstream repositories again: <pre> $ rm -rf .repo/manifests/ # This enables me to force push and rebase the manifest repository as well $ repo init -u https://git.replicant.us/replicant-9/manifest.git -b replicant-9 $ repo sync --force-sync </pre> * The following command might also be necessary to make the state consistent with upstream repositories again, when the manifest history wasn't rewritten, but it will also loose all the work you did locally: <pre> $ repo sync --force-sync </pre> h4. Install the images <pre> $ sudo heimdall flash --BOOT out/target/product/i9305/obj/PACKAGING/target_files_intermediates/lineage_i9305-target_files-eng.replicant/IMAGES/boot.img --USERDATA out/target/product/i9305/obj/PACKAGING/target_files_intermediates/lineage_i9305-target_files-eng.replicant/IMAGES/userdata.img --SYSTEM out/target/product/i9305/obj/PACKAGING/target_files_intermediates/lineage_i9305-target_files-eng.replicant/IMAGES/system.img </pre> h4. Get adb As the device IDs are the ones given by the Linux kernel, they are not in the adb udev rules, so for now it requires to run adb as root: <pre> $ sudo adb shell * daemon not running; starting now at tcp:5037 * daemon started successfully i9305:/ # $ sudo adb kill-server $ adb shell * daemon not running; starting now at tcp:5037 * daemon started successfully error: no devices/emulators found </pre> So make sure to kill the adb-server and run it as root: <pre> $ adb kill-server $ sudo adb shell * daemon not running; starting now at tcp:5037 * daemon started successfully i9305:/ # </pre> h4. Booting At some point during boot, the device goes into suspend, so you will need to press some buttons like the power or volume button to make the boot continue until the graphical interface. You can also follow the boot progress with adb: <pre> adb logcat adb logcat -b main </pre> Note that the device can go into suspend at any time, so adb might be interrupted. That looks like that: First you get a shell <pre> $ sudo adb shell i9305:/ # </pre> Then the connection is interrupted: <pre> $ adb shell i9305:/ # [randomdev@fullyfreelaptop ]$ </pre> The effect with adb logcat is similar. h3. cleanups to be done * Make adb work as user by using the right USB IDs, and make userspace do the USB setup. * -Make the kernel not use hardcoded CMDLINE_FORCE- Not possible unless the bootloader is changed or Linux is very heavily patched. * Make the kernel not use hardcoded partitions if possible (though we use system as root) * Make a clean Gatekeeper HAL module implementation instead of using the same hack than goldfish * look at "init.rc":https://android.googlesource.com/platform/system/core/+/master/init/README.md documentation to see if init.rc can be overriden clearly with the override statement to see if it's possible to keep the serial console patch for -eng h3. Using SwiftShader instead of Mesa3D llvmpipe for software rendering To use SwifShader you need a kernel that supports UDIV/SDIV emulation, you can checkout the branch GNUtoo/udiv-emulation for kernel/replicant/linux. After doing that that run the following commands to use SwiftShader: <pre> $ adb remount $ adb shell i9305:/ # rm vendor/lib/egl/libGLES_mesa.so # or move it somewhere safe meanwhile you test SwiftShader i9305:/ # nano system/build.prop # set ro.hardware.hwcomposer=ranchu and ro.hardware.gralloc=default $ adb reboot </pre> h2. AOSP 9 h3. Status with AOSP 9 * The Galaxy SIII 4G (i9305) boots (with graphics hacks) under AOSP9, without having to import nonfree libraries: ** The device boots until the lock screen ** The graphics work but are really slow as they have not been optimized ** Sometime the display content is corrupted ** The touchscreen works * The kernel used is based on Linux 5.0-rc6 and has ~150 Android specific patches on top and some patches for the Galaxy SIII 4G (i9305) on top h3. Repositories and changes * git://git.putti.eu/aosp/manifest_i9305.git * git://git.putti.eu/aosp/kernel_i9305.git * git://git.putti.eu/aosp/device_i9305.git * git://git.putti.eu/aosp/external_mesa3d.git * git://git.putti.eu/aosp/build_make.git * git://git.putti.eu/aosp/core.git h3. Building AOSP version h4. Source code First get the source code: <pre> $ repo init -u git://git.putti.eu/aosp/manifest_i9305.git -b android-9.0.0 </pre> h4. For Trisquel 8 <pre> sudo apt-get install git-core gnupg flex bison gperf build-essential zip curl zlib1g-dev gcc-multilib g++-multilib libc6-dev-i386 lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z-dev libgl1-mesa-dev libxml2-utils xsltproc unzip bc python-mako gcc-5-arm-linux-gnueabi </pre> h4. Building it Once this is done, see the device/putti/i9305/README.md for the build instructions. h4. Trisquel 8 LXC If you use Parabola, you may be interested in running Trisquel 8 in LXC. To do that first debootstrap a Trisquel 8 rootfs. Parabola's debootstrap does support Trisquel 8 and its manual has an example on how to do that: <pre> $ man debootstrap [...] # debootstrap flidas flidas-root http://archive.trisquel.info/trisquel </pre> Then you can use virt-manager to setup the LXC instance. The advantages of this solution are that: * The LXC guest and host shares their resources (CPU, RAM) with almost no penalty * Trisquel 8 is not a rolling release distribution The disadvantage of this solution are that: * you need to configure again the Trisquel 8 LXC instance (vimrc, sshd_config, etc) * It's more complicated to setup * The Android build system output a warning message about not being able to use namespaces which may become mandatory in newer Android versions h2. Upstreaming status * "Unbreak 5.2 booting:":https://www.openwall.com/lists/kernel-hardening/2019/06/14/4 This is really tricky as to work with the stock bootloader we'd have to disable the caches and the MMU. We need to look if upstream u-boot would accept something like that. Else we'd need to maintain a stub it. * -"add flash support to galaxy-s3 boards":https://lists.infradead.org/pipermail/linux-arm-kernel/2019-June/660361.html- "Made it in 5.3":https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v5.3&id=4e4dfcb2a425cccc6dd1fb7d46e060cd57999afc "applied":https://lists.infradead.org/pipermail/linux-arm-kernel/2019-June/661471.html * Battery charging: ** "mfd: max77693: Add defines for charger current control":http://lkml.iu.edu/hypermail/linux/kernel/1909.1/02216.html : No response ** "power_supply: max77693: Listen for cable events and enable charging":http://lkml.iu.edu/hypermail/linux/kernel/1909.1/02217.html : No response * "ARM: dts: add touchkey nodes for midas":https://lists.infradead.org/pipermail/linux-arm-kernel/2019-August/671598.html ** Testing required on N7100, N7105: Waiting for devices to test on before resuming to work on that patch ** Needs some fixes h2. TODO First month of full time equivalent work: |_. Time estimation |_.Task |_. Comments | | DONE | -boot a device under AOSP9- | Only boots with graphics, not much more | | -7h- DONE | -build it under a "FSDG compliant distribution":https://www.gnu.org/distros/free-distros.html like Trisquel8- | -WIP for AOSP, It's difficult to do precise time estimations as it could work out of the box or require one full time month of work depending on how much issues are encountered- Builds under Trisquel8| | -21h- DONE | * -port the changes from AOSP9 to LineageOS 16- * -cleanup the code- * -build the kernel from the Android build system- * -make sure it builds with an FSDG compliant distribution- * -document the build procedure- | Status: * Boots with adb. * Has ultra slow graphics | | 14h | find, remove and document proprietary software in LineageOS 16 | | | 21h | find, remove and document privacy issues in LineageOS 16 | | | 7h | port the touch keys Linux -driver- dts for the galaxy-s3 boards | Also requires a n710x for testing the patch | | 7h | -upstream the AAT1290 flash led Linux dts for the galaxy-s3 boards- | "In 5.3":https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v5.3&id=4e4dfcb2a425cccc6dd1fb7d46e060cd57999afc | | 7h | rebrand LineageOS as Replicant | | | 70h | -port and- cleanup the the Galaxy SIII (i9300) "modem":https://github.com/fourkbomb/linux/tree/modem Linux driver from 4.16 to 5.0| * The porting is done * The code still needs to be cleaned up * The code has not been validated completely yet, see below on the section with libsamung-ipc | |_\3. Total: 147h (~1 month) | Second month of full time equivalent work: | * port libsamsung-ril and libsamsung-ipc to Android 9 * Make the modem driver and libsamsung-ipc work together | 157h | -libsamsung-ipc should be trivial, libsamsung-ril probably needs to be heavily adapted to Trebble or the new RIL interfaces- -libsamsung-ipc and samsung-ril were fixed to compile on Android 9.- -The code was not yet tested at runtime.- -It also probably still need to be adapted to trebble or the new RIL interfaces to be more substainable.- Status: * The code has been updated to work without a wrapper on Android 9, but some cleanups are needed in all the modem related components. * The modem initialization is now complete. It has been validated with ipc-test of libsamsung-ipc under GNU/Linux. * TODO: ** Validate that it can send-receive IPC messages ** Test and/or fix libsamsung-ril with Replicant 9 See #1954 for more details on the progress | |_. Total |_. 157h | ~1 month | Third month of full time equivalent work: |_. Task |_. Time estimation |_. Comments | | -port the sensors libraries and other device specific libraries- Look which sensor libraries can be used | 70h | Already done by the unofficial LineageOS port of the Galaxy SIII (i9300), needs testing | | add support for Audio with the upstream kernel driver | 70h | Might be way faster, depending on what Android 9 uses | | -add partial (no modem) support for the Galaxy SIII 4G (i9305) and- factorize the code with i9300 | 14h | * The source code on which the work was based changed from AOSP to an unofficial LineageOS port to a port of i9305 support for AOSP by Joonas to the official LineageOS so it's now supported by default * The work to factorize the code between the i9300 and i9305 still need to be done | | |_. Total |_. 154h | ~1 month | |_. Task |_. Time estimation |_. Comments | | create a recovery | 21h | | | add internal WiFi support and validate the functionality | 6h | | | add external WiFi dongles support | 20h | External dongles support might be tricky | | -create new- update the install and upgrade instructions | -35h- | -Our current install instructions don't scale as we have one copy for each device.- -We also created generic instructions but they tend to be harder to follow[1] than the device specific ones.- -This will be made in a modular format (for instance in LaTeX) that enables to generate per device install instructions without requiring to have multiples copy of the same text.- -The instructions will need to be able to be modified and compiled on an FSDG compliant distribution.- Mostly done: * The installation instructions are now generic enough. * Some long standing TODO were also done along the way like adding backup instructions for the EFS. * The current instructions are still for Replicant 6.0 and will need to be updated for Replicant 9.0 | |_. Task |_. Time estimation |_. Comments | | Estimate the amount of work to Reduce the attack surface | ? | | | Estimate the amount of work to add in-system upgrades | ? | | fn1. The generic instructions were tested at Install parties in Paris h3. Devices support: h4. Easy, because it's similar enough to the Galaxy SIII (I9300) |_\3. Galaxy Note II (N7100) | |_. Task |_. Time estimation |_. Comments | | port the EA8061 LCD Linux driver | 35h | | | port the S6EVR02 LCD Linux driver | 35h | | | port the MAX77693 flash led Linux driver | 7h | | | android: add support for the Note II (N7100) and factorize the code with Galaxy SIII (i9300) and Galaxy SIII 4G (i9305) | 14h | Should be similar to the Galaxy SIII | | port the sensors libraries and other device specific libraries | 70h | It's difficult to evaluate how much time it could take | | add support for Audio with the upstream kernel driver | 14h | Should be similar to the Galaxy SIII | |_\3. Galaxy Note 8.0 (N5100) and 8.0 WiFi (N5110) | |_. Task |_. Time estimation |_. Comments | | Evaluate the time required to do the port | 14h | TODO | h4. Needs more work and unknown upstream Linux status |_. Device |_. Time estimation |_. Comments | | Galaxy S II (i9100) | | Linux: devboard dts upstream? unknown status | | Galaxy Note (N7000) | | unknown Linux upstream status | | Galaxy Nexus (I9250) | |/2. OMAP4, no dts upsrteam | | Galaxy Tab 2 7.0 (P3100), 7.0 WiFi (P3110), 10.1 (P5100), 10.1 WiFi (P5110) | | | GTA04 >= A4 | | TODO: a RIL needs to be written, userspace GPS support is missing, audio scenarios, etc | h2. Documentation h3. Replicant 6.0 changes |_. Repository path |_. rebase worked |_. Changes |_. pushed rebase in Replicant 6.0 | Tag for the old history | | build | No | | | | manifest | No | N/A | | bootable/recovery | Yes | * UI changes to improve usability * Branding * Small fix(es) | | | external/boringssl | Yes | * Small fix(es) | | external/heimdall | No | | | | frameworks/av | Yes | * Software rendering fix(es) | Yes | replicant-6.0-aug-2018 | | frameworks/base | Yes | * WiFi fixe(s) for external dongles * Software rendering fix(es) * Privacy fix(es) * Security improvements * Usability improvements | Yes | replicant-6.0-sept-2018 | | frameworks/native | No | | | hardware/broadcom/wlan | Yes | * Remove nonfree software | | | hardware/libhardware | Yes | * Software rendering fix(es) | Yes | replicant-6.0-0003 | | hardware/libhardware_legacy | Yes | * Don't depend on nonfree software | | | hardware/ti/omap4 | Yes | * Remove nonfree software * Don't depend on nonfree software * Small fix(es) | | | libcore | Yes | * Small fix(es) | | | packages/apps/Browser | Yes | * Privacy fix(es) * Branding * Small fix(es) | | | packages/apps/LegacyCamera | Yes | * Software rendering fix(es) * Small fix(es) | | | packages/apps/RepWifi | Not upstreamed, obsolete | N/A | | | packages/apps/ReplicantWallpapers | Not upstreamed | * Branding | | | packages/apps/Settings | No | | | | packages/apps/SetupWizard | Yes | * Privacy fix(es) * Don't depend on nonfree software * Branding | | | packages/apps/ThemeChooser | Yes | * Privacy fix(es) ? * Don't depend on nonfree software ? | | | packages/providers/ContactsProvider | Yes | * Small fix(es) | | | system/core | Yes | * Branding * various fixes | | | vendor/replicant | No | | | vendor/cmsdk | Yes | * Branding | | | external/chromium-webview | ? | ? | | | external/drm_gralloc | Yes | * Software rendering fix(es) | Yes | replicant-6.0-aug-2018 | | external/llvm | Yes | * Software rendering fix(es) | Already rebased| replicant-6.0-0003 | | external/mesa3d | Yes | * Software rendering fix(es) * Performance debugging for software rendering | | external/parameter-framework | Yes | * Small fix(es) | | | external/protobuf | N/A | No changes | | | prebuilts/ndk | ? | ? | | | prebuilts/sdk | ? | ? | | | sdk | Yes | * Small fix(es) | | | toolchain/src/binutils | Yes | * Small fix(es) | | | toolchain/src/gcc | No | | | | toolchain/src/jack | Yes | * Small fix(es) | | | toolchain/src/jill | Yes | * Small fix(es) | | | kernel/samsung/smdk4412 |/3. Yes |/3. * Security fixes * Privacy Fixes (TrustZone) * Don't depend on nonfree software * Remove nonfree software * Small fix(es) | Yes | replicant-6.0-0003 | | kernel/samsung/tuna | Yes | replicant-6.0-0003 | | kernel/samsung/espresso10 | | | device/samsung/i9100 | No |/14. * Replace nonfree software by free software * free software sensors implementation(s) * free software modem audio implementation(s) * Software rendering fix(es) * Privacy Fixes (TrustZone) * Don't depend on nonfree software * Remove nonfree software * Branding * Small fix(es) | | | device/samsung/n7000 | Yes | | | device/samsung/i9300 | No | | | device/samsung/i9305 | No | | | device/samsung/n7100 | Yes | Already rebased| replicant-6.0-0003 | | device/samsung/n5100 | Yes | | | device/samsung/n5110 | Yes | | | device/samsung/maguro | Yes | | | device/samsung/tuna | Yes | | device/samsung/espressowifi | Yes | | | device/samsung/espresso3g | Yes | | | device/samsung/galaxys2-common | Yes | | | device/samsung/smdk4412-common | No | | | device/samsung/kona-common | Yes | | | hardware/ril/samsung-ril |\4. Not upstreamed | | external/libsamsung-ipc |\4. Not upstreamed | | hardware/tinyalsa-audio |\4. Not upstreamed? | h3. Other rebases TODO: move this in its own wiki page. |_. Repository path |_. date | Tag for the old history |_. Changes | Rationale | | hardware/ril/samsung-ril | 24 July 2019 | replicant-6.0-0003 | Pointed replicant-6.0 to master | * See the "libsamsung-ril/libsamsung-ipc: delete or sync master branch? Was: [PATCH 6/6] Fix undefined references to MD5_{Init, Update, Final} ":https://lists.osuosl.org/pipermail/replicant/2019-June/001950.html thread * This also has been made to prevent similar mistakes from hapening again | | external/libsamsung-ipc | 24 July 2019 | replicant-6.0-0003 | Pointed replicant-6.0 to master | * See the "libsamsung-ril/libsamsung-ipc: delete or sync master branch? Was: [PATCH 6/6] Fix undefined references to MD5_{Init, Update, Final} ":https://lists.osuosl.org/pipermail/replicant/2019-June/001950.html thread * This also has been made to prevent similar mistakes from hapening again | h2. Other attempts * It might be interesting to contact the people doing ports once we have something working well enough. * It is also interesting to look at other attempts to understand if a given device is powerful enough to run Android 9 and what configuration was used to achieve it. |_. Device(s) |_. Repository |_. status |_. Comments | | i9300 | "CustomROMs":https://github.com/CustomROMs/android_local_manifests_i9300 | * "March 3 2019 release":https://forum.xda-developers.com/galaxy-s3/development/rom-lineageos-16-0-t3875899 * "July 12, 2019 Go port":https://forum.xda-developers.com/galaxy-s3/development/rom-optimized-lineageos-16-0-team-t3940142 | Issues: * Use a Samsung kernel * Use too many nonfree libraries => Probably nothing we could reuse from its code | | I9100 | | "XDA":https://forum.xda-developers.com/galaxy-s2/development-derivatives/pie-i9100-t3850588 | | h3. Links for other attempts * https://www.xda-developers.com/android-pie-android-9-port-custom-roms/ h3. CustomROMs i9300 components |_. Repository |_. Tree path |_. Dependencies |_. Function |_. Comments | |/8. https://github.com/CustomROMs/android_hardware_samsung lineage-16.0 branch | hardware/samsung/macloader | | Loads the MAC Address of the WiFi network interface | Might be useful | | hardware/samsung/wifiloader | | Loads the wifi kernel module (like modprobe) and setup firmware filesystems permissions | May be useful | | hardware/samsung/audio | | seems to contains ril related stuff as well | Look if the ril stuff is required, go for standard audio | | hardware/samsung/lineagehw/hidl/livedisplay | | livedisplay is a feature similar to what redshift does on GNU/Linux | Not sure if it works with mainline | | hardware/samsung/exynos/multimedia/utils/ | seem meant for audio/video decoding offload | assembly obtimized color conversion and resize code | check assembly code license, not sure if useful | | all other directories in hardware/samsung/exynos/ | nonfree firmwares, nonfree software?, smdk kernel? | audio/video decoding offload | Avoid using that | | hardware/samsung/exynos3 |/2. nonfree firmwares?, nonfree software?, smdk kernel? |/2. some light libraries, display stuff (gralloc, etc), 2D acceleration (FIMG), camera (FIMC), 3D acceleration, etc |/2. Avoid using that for now | | hardware/samsung/exynos4 | h2. Links * "Android build requirements - hardware and software":https://source.android.com/setup/build/requirements * "Why LineageOS Developers are building Android Go-optimized custom ROMs":https://www.xda-developers.com/android-go-old-android-8-1-oreo/ * "Android Go recommended default values for propreties for optimization":https://android.googlesource.com/platform/build/+/master/target/product/go_defaults_common.mk * "Team InFusion i9300 optimized system.prop":https://github.com/team-infusion-developers/android_device_samsung_i9300/blob/lineage-16.0/system.prop#L23 * "Hack to fix high CPU usage caused by logd":https://github.com/team-infusion-developers/android_device_samsung_smdk4412-common/commit/c66015514fc9779edfed2665f67e841f3620c71e * "Use low-end video codecs":https://github.com/team-infusion-developers/android_device_samsung_smdk4412-common/commit/adc919f04b981a4fe005fab2cf443596a22992b2 * "Optimize ActivityManager cached apps":https://github.com/team-infusion-developers/android_device_samsung_smdk4412-common/commit/24bc62970d35eb5a7a257463e7dbfe873ceb5779 * "Use 1Gb Dalvik config":https://github.com/team-infusion-developers/android_device_samsung_smdk4412-common/commit/8a7cebc5af772a2949420ac4a383f5f75236c65c