Project

General

Profile

Actions

Upstream » History » Revision 281

« Previous | Revision 281/295 (diff) | Next »
Kurtis Hanna, 08/25/2019 09:10 PM
added link to oFono video presentation


Upstream Linux

Existing work

The issue #1882 has some documentation on where the work is being done

Benefits of using Upstream Linux

Currently, Replicant uses device specific Hardware Abstraction Layers, because device manufacturers implemented non-standard kernel interfaces. However, Android works with upstream kernels and supports plug-n-play hardware nowadays, so it makes sense to have generic Hardware Abstraction Layers for the standard interfaces of the Linux kernel (ALSA, V4L2, etc).

Benefits:
  • It would allow supporting external WiFi dongles such as the ones supported by the ath9k_htc driver and free firmwares without the need for a specific application or configuration.
  • It would make devices last longer by alleviating the device specific maintenance burden: If LineageOS stops supporting a Replicant supported device, Replicant would need to maintain it by its own. This would require a lot of work, unless the device is already supported the upstream Linux kernel and generic hardware abstractions layers. This would also enable Replicant to support devices that are not currently supported by LineageOS with a lot less work.
  • It would enable the support for devices that are or will be added to upstream Linux.
As GNU/Linux expects standard kernel interfaces, this would also enable to run GNU/Linux out of the box on such devices.
This has some interesting outcomes:
  • The device specific work could be shared between GNU/Linux communities and Replicant communities. This could result in less work to do to support individual devices. Since Android libraries depends on Android's libc, non-standard proprietary libraries might be harder to reuse than the free software implementations, so we might get even more collaboration thanks to that.
  • It would enable GNU/Linux distributions to more easily support smartphones and tablets, which would hopefully enable FSDG distributions to be able to focus on usability instead of hardware support. This way, if one day Android devices stop using the Linux kernel, stops being free software, or if the code takes directions that are too much problematic, already having GNU/Linux based Android alternatives would reduce the amount of work needed to be able to get again a fully free software distribution for smartphones and tablets.
  • Older devices with less amount of RAM than Replicant current minimum requirements could be used with GNU/Linux and possibly repurposed for other usages, reducing the amount of electronic devices waste.

Requirements

  • For the other standard interfaces (like ASLA, etc) a device running a upstream Linux Kenrel with as few patches as possible is required.

Devices

It is best to use a device that requires the least amount of work to be functional under Replicant.
More precisely we want to minimize:
  • The work needed to have the device usable with upstream Linux.
  • The work porting or writing Android hardware abstractions layers.
To achieve that we can choose a device that:
  • requires no or very minimal work to be fully supported by Linux.
  • have less hardware features (so we don't need to support them in Linux and in the HALs).
  • is easy to buy, so the work can be shared among multiple people.
  • doesn't have more freedom flaws than the devices currently supported by Replicant

It is also a good idea to keep one image per device at first, as trying to make a single image that
would work on all ARM device supported by upstream Linux is complicated: Even GNU/Linux
distributions have a hard time doing that for ARM devices.

Linux upstream status

Smartphones and tablets with a free software bootloader and work in progress upstream Linux support

Formfactor Vendor Product Linux dts Linux status Verdict
Smartphone LG Optimus Black omap3-sniper.dts no display(no driver), very few peripherals Too much work required
Tablet Amazon Kindle Fire (first generation) omap4-kc1.dts no display(no driver), very few peripherals Too much work required
Smartphone GTA04 A3 omap3-gta04a3.dts http://projects.goldelico.com/p/gta04-kernel/page/UpstreamStatus/ Good fit: Free software bootloader, very few parts not upstream
Smartphone GTA04 A4 omap3-gta04a4.dts
Smartphone GTA04 A5 omap3-gta04a5.dts

Replicant supported Samsung Exynos devices

Formfactor Vendor Product Linux dts Linux status page Issues Verdict
Smartphone Samsung Galaxy S (i9000) s5pv210-galaxys.dts https://github.com/PabloPL/linux/wiki (needs verified, but looks right) * Probably has a signed bootloader. See the bootloader status below for more information on it.
* Has shared memory between the modem and the processor running Replicant
* Is supported by Replicant 4.2 but not Replicant 6.0
* Barely meets Android 9 requirements
Smartphone Samsung Galaxy S II (i9100) exynos4210-trats.dts? (devboard or i9100?) We don't know of anyone working to mainline the kernel, but an older kernel was updated that can run Android Pie https://github.com/rINanDO/android_kernel_samsung_smdk4412 Probably has a signed bootloader
Smartphone Samsung Galaxy S III (i9300) exynos4412-i9300.dts https://blog.forkwhiletrue.me/pages/midas-mainline/ + patch required for Samsung bootloader (reference) The first stage is signed. See the bootloader status below for more information on it. Good fit: Only the modem, touch keys, and small parts are missing upstream but are available as patches on top of it
Smartphone Samsung Galaxy Note 2 3G (N7100) exynos4412-n710x.dts (needs to be modified to exynos4412-n7100.dts via this patch.) Good fit: the LCD, the modem, touch keys, and small parts are missing but are available as patches on top of it
Tablet Samsung Galaxy Note 8.0 (N51XX) The platform is kona, not midas, but it might be fairly easy to port to mainline since it uses the same SoC, same fuelgauge, same audio, same radio chip, and maybe other parts apart from LCD and touchscreen as the i9300 and n7100 * The bootloader is most probably signed. See the bootloader status below for more information on it.
* Soldered battery => Shops might be able to replace it
* Need to find a way to get access to the serial port => The special connector might have serial on it

[PATCH] ARM: decompressor: Flush tlb before swiching domain 0 to client mode

Background on that patch:
  • v3.4 worked without that patch. This means that even with the patch, newer kernel could break at any time, which would force us to (1) understand from where problem comes from (2) look where it broke (3) try to find a way to fix it.
  • It's needed because the proprietary bootloader turns on the MMU [wiki page on what the MMU is needs to be added].
  • It's possible to replace part of it by u-boot, avoiding the issue, but it's a tricky operation as the first part of the bootloader is signed, and needs to be replaced by a version that doesn't check the signatures.
  • The boot order probably makes it difficult to recover the device if something went wrong replacing the bootloader.
  • There is no procedure yet to get back to the original bootloader

A response to the patch states:

Yes. The MMU is suppose to be turned off by boot-loader before jumping
to the kernel. That explains the problem. No need to patch
the kernel here.

However it might still be possible to solve that as:
  • Some arm CPUs require the Linux kernel to turn off the MMU like in head-xscale.S
  • Support for some specific bootloader do exist, for instance in head-sharpsl.S
  • There are some comments related to cleaning and disabling the MMU in this Exynos4 lowlevel_init.S u-boot code that might be valuable
  • We could ask HardKernel to get a BL1 signed that turns off the MMU and also doesn't check a signature for BL2.

Replicant supported Texas Instruments OMAP4 devices

Formfactor Vendor Product Linux dts Linux status page Issues Verdict
Smartphone Samsung Galaxy Nexus (I9250) None * Status on PostmarketOS wiki
* mainline fork
? Need to write a DTS
Tablet Samsung Galaxy Tab 2 7.0 (P31xx) None ? * Soldered battery => Shops might be able to replace it
* Need to find a way to get access to the serial port => The special connector might have serial on it
?
Tablet Samsung Galaxy Tab 2 10.1 (P51xx)

Other devices with some upstream support.

Formfactor Vendor Product Linux dts Linux status page Issues Verdict
Tablet Difrnce DIT4350 sun5i-a13-difrnce-dit4350.dts
Tablet Empire Electronix D709 tablet sun5i-a13-empire-electronix-d709.dts
Tablet Empire Electronix M712 tablet sun5i-a13-empire-electronix-m712.dts
Tablet Gemei G9 Tablet sun4i-a10-gemei-g9.dts TODO in the dts Missing touchscreen
Smartphone Samsung Galaxy SIII 4G (i9305) exynos4412-i9305.dts https://blog.forkwhiletrue.me/pages/midas-mainline/ + patch required for Samsung bootloader The bootloader is signed. See the bootloader status below for more information on it. Good fit:
* Not fully supported by Replicant because it lacks modem support but the work can be reused by the Galaxy SIII, Galaxy SII, Note 2, Note 8.0, and Note 10.1 (2012 edition).
* touch keys, and small parts are missing upstream but are available as patches on top of it.
Smartphone Samsung Galaxy Note 2 4G (N7105) exynos4412-n710x.dts needs to be modified to exynos4412-n7105.dts via this patch since it has a different modem than the n7100. Good fit: the LCD, touch keys, and small parts are missing but are available as patches on top of it. The modem might be in mainline, but probably lacks firmware loading.
Smartphone Nokia N900 omap3-n900.dts https://elinux.org/N900 * Has a signed bootloader
* Has only 256M of RAM
Bad fit: not enough RAM
Smartphone Nokia N9 omap3-n9.dts https://elinux.org/N9 * Probably have a signed bootloader
* Needs investigation to see if the touchscreen firmware is mandatory
Would need investigation to see if the touchscreen firmware is mandatory
Smartphone Nokia N950 omap3-n950.dts https://elinux.org/N950
Smartphone Motorolla Droid 4 (XT894) omap4-droid4-xt894.dts http://elektranox.org/droid4/ Probably has a signed bootloader, may have a signed kernel requiring kexec Bad fit: requires a signed Linux kernel in the boot chain
Smartphone Nexus 7 (2012) qcom-apq8064-asus-nexus7-flo.dts Qualcomm SOC (signed bootloader, unknown modem isolation) Bad fit:
* Would need more guarantees requarding the modem isolation on recent qualcomm SOCs
* Would need more research to on the state of free software for more recent qualcom SOCs
* Not enough support in the Linux kernel for devices with Qualcomm SOCs
qcom-apq8064-sony-xperia-yuga.dts
qcom-msm8974-sony-xperia-amami.dts
Tablet Sony Xperia Z2 Tablet qcom-msm8974-sony-xperia-castor.dts
qcom-msm8974-sony-xperia-honami.dts
Smartphone Nexus 5 qcom-msm8974-lge-nexus5-hammerhead.dts
Smartphone Samsung Galaxy S5 qcom-msm8974-samsung-klte.dts
Fairphone 2 qcom-msm8974-fairphone-fp2.dts
Tablet MSI Primo81 sun6i-a31s-primo81.dts
Tablet Yones TopTech BS1078 v2 Tablet sun6i-a31s-yones-toptech-bs1078-v2.dts
Tablet Allwinner? Q8 A13 Tablet sun5i-a13-q8-tablet.dts
Tablet Utoo P66 sun5i-a13-utoo-p66.dts
Tablet Primux INet-98V Rev 02 sun5i-a13-inet-98v-rev2.dts
Tablet Primux INet-86DZ Rev 01 sun8i-a23-inet86dz.dts
Tablet Wondermedia? WM8650-MID Tablet wm8650-mid.dts
Tablet Wondermedia? WM8850-W70v2 Tablet wm8850-w70v2.dts
Tablet Colorfly E708 Q1 tablet sun6i-a31s-colorfly-e708-q1.dts
Tablet Polaroid MID2407PXE03 tablet sun8i-a23-polaroid-mid2407pxe03.dts
Tablet Polaroid MID2809PXE04 tablet sun8i-a23-polaroid-mid2809pxe04.dts
Tablet Allwinner? Q8 A23 Tablet sun8i-a23-q8-tablet.dts
Tablet Allwinner? Q8 A33 Tablet sun8i-a33-q8-tablet.dts
Tablet TBS Biometrics A711 Tablet sun8i-a83t-tbs-a711.dts
Tablet iNet Tek iNet Q972 tablet sun6i-a31s-inet-q972.dts
Tablet Allwinner? GT90H Dual Core Tablet (v4) sun8i-a23-gt90h-v4.dts
Tablet Allwinner? GA10H Quad Core Tablet (v1.1) sun8i-a33-ga10h-v1.1.dts
Tablet Allwinner? INet-D978 Rev 02 sun8i-a33-inet-d978-rev2.dts

More upcoming Exynos related mainlining work could potentially be discovered at this Tizen wiki page

Allwinner devices

Devices with Allwinner SOCs are an interesting targets because:
  • Many of them do not use signed bootloaders.
  • Many of the SOCs and various devices using them have good Linux and u-boot mainline support

For instance the Lime 2 from Olimex is pretty well supported and is easy to find.
However this device is a single board computer and, as such it doesn't have the have the usual peripherals that are commonly found in tablets and smartphones. This makes a port on this device less relevant and less useful.

Some research is needed to identify which devices are easiest to work with. Tablets that don't have a modem seem to be better than smartphones, as supporting the modem would require to have it supported in Linux and the userspace libraries. This might even require to write and upstream a Linux driver for the modem.

A good tablet for this task should have:
  • A SOC that has good mainline support, see the Linux mainlining effort page on linux-sunxi for more details.
  • A Free software bootloader, or the ability to easily add support for the tablet to a free software bootloader.
  • The ability to power and use an USB WiFi card or chip that is compatible with the ath9k_htc driver.

It would also be better if the chosen tablet doesn't use an AllWinner SOC with a PowerVR GPU, as MALI GPU have more probability to be usable with free software in the future.

Rockchip devices

Formfactor Vendor Product Linux dts Linux status page Issues Verdict
Tablet Acer Chromebook Tab 10 D651N-K9WT - codenamed Scarlet rk3399-gru-scarlet.dtsi The Asus Chromebook Flip C101PA (codenamed Bob) has the same RK3399 SoC and was recently added to mainline u-boot . RK3399 in upstream u-boot doesn't include these tablets yet. These tablets, like the C101PA, ship with Coreboot bootloaders. Google used the Depthcharge payload in Coreboot to boot Android on the Pixel C, which has a difference SoC, but also shipped with Coreboot. Upstream Coreboot repo Android specific code in Depthcharge payload u-boot README.rockchip u-boot README.rockusb rkdevelopmenttool First (and only?) XDA thread about porting AOSP/CyanogenMod to a Chromebook
Tablet Asus Chromebook Tablet CT100 - codenamed Dumo rk3399-gru-scarlet.dtsi (needs to be verified, but both device's Board Names and Base Boards are codenamed Scarlet)
Tablet CTL Chromebook Tab Tx1 - codenamed Druwl

Bootloaders

Bootloader status

Formfactor Vendor Product signatures Source code u-boot status Verdict
Smartphone Samsung Galaxy S - i9000 The bootloader is probably signed downstream u-boot * u-boot mainline was placed in BL2, copying much of the bootloader work done on the i9300 and n7100
* That u-boot also supports being installed in place of the Linux kernel, leaving the stock bootloader in place
* Probably signed
* The port to run instead of the Linux kernel seem interesting as it could be used on "midas" to disable the MMU and caches
Galaxy S 2 - I9100 Signed, see emulating-exynos-4210-bootrom-in-qemu.html ? ? ?
Galaxy SIII - I9300 * The first stage is signed.
* The stock first stage checks the second stage signatures.
* A signed first stage that doesn't enforce subsequent signatures exists but it's nonfree and non-redistributable
downstream u-boot for the second stage this downstream code was likely forked from the trats2 upstream u-boot code which is a Samsung eval board which has the same Exynos4412 SoC. The only other upstream u-boot code for the Exynos4412 seems to be for an O-Droid device * That u-boot version is meant to be used in combination with a nonfree and non-redistributable first stage bootloader.
* This combination doesn't load the nonfree TrustZone OS
* Replicant 9 can run on devices with this bootloader, but the current Replicant 6 kernel expects a TrustZone OS to work.
* We need to find a way to completely replace the first bootloader stage (BL1) as the one that is used to run u-boot is nonfree and non-redistributable.
* More details on that issue can be found in the Exynos4_Bootrom wiki page.
Galaxy SIII 4G - I9305
Galaxy Note 2 - N7100
Galaxy Note 2 4G - N7105
Tablet Samsung Galaxy Note 8.0 - N51XX ? Has the same System On a Chip (SOC) as the Galaxy SIII and Note 2, the Exynos4412, but slightly modified u-boot might need to be written. Testing needed.
Galaxy Note 10.1 - N8000
Camera Samsung Galaxy Camera 2 - EK-GC200 ? First Exynos4412 Samsung device to get u-boot support in 2012
Smartphone Google and Samsung Galaxy Nexus - I9250 downstream u-boot for the second stage
Smartphone LG Optimus Black unsigned upstream u-boot no display(no driver), very few peripherals but enough to be usable
Tablet Amazon Kindle Fire (first generation) upstream u-boot
Smartphone GTA04 A3 downstream u-boot and xloader
GTA04 A4
GTA04 A5

See also

Upstream GNU/Linux

Modem support

Protocols Implmentation comments
QMI Android <-> RIL <-> libqmi-ril to be completed <-> libqmi
* QMI
* AT
* Other
Android <-> RIL <-> libraries to be written
android_frameworks_opt_telephony_ril_ofono + ofono + ofono backend (AT, QMI, etc) * Using ofono would enable us to share more effort with upstream GNU/Linux and support many other protocol like AT for the GTA04 or qmi-ril for the Galaxy SIII 4G (I9305) or the Galaxy Note II 4G (N7105)
* According to the README, it has already been tested with most of Replicant 6 code but on a smartphones not yet supported by Replicant. Calls, Audio, SMS, etc are knwon to work.
* BuildRilWrapper.java seems to use introspection to automatically generate the API between the Framework Java RIL and itself (which replaces rild) (See the official documentation for background information on the Android architecture)
* Replicant and oFono based Java RIL video presentation https://redmine.replicant.us/projects/replicant/wiki/ContributorsMeetingJuly2019#Presentations
samsung-ipc Ofono (rilmodem backend/driver) <-> rild <-> libsamsung-ril <-> libsamsung-ipc * Might be usable for GNU/Linux distributions with libhybris
* Could be usable for testing Replicant as ofono could run on the host computer and the rild socket could be exported with adb
* Some forks exist: check if they still have interesting patches
* https://github.com/rilmodem/ofono
Android <-> rild <-> libsamsung-ril <-> libsamsung-ipc * Currently in use in Replicant
Android <-> Ofono <-> libsamsung-ipc * An ofono fork with libsamsung-ipc support is available
Patches to add that upstream were refused because upstream didn't want to make the project become GPLv3 (libsamsung-ipc was GPLv3 at the time) but now libsamsung-ipc has been relicensed to GPLv2+
FSO <-> libsamsung-ipc * Probably not easily usable in Replicant
* Is FSO still actively maintained?
* Was used by SHR and potentially other GNU/Linux distributions supporting the Openmoko GTA04 smartphones

Upstream userspace hardware support libraries

Usage Replicant GNU/Linux comments
Bluetooth stack BlueDroid Bluez
GPS hardware support ? gpsd

Upstream non-hardware specific userspace

Usage Replicant GNU/Linux comments
Unix command line tools ? * Busybox
* Coreutils
* Busybox already has Android specific code in it but no Android.mk
* Busybox build is very similar to Linux, and Linux can be built by Android

libc

Feature Advantages Disadvantages sustainability
glibc + libhybris * You just need an Android.mk to compile GNU/Linux software * You need to link the Android part that need bionic functions to libhybris TODO: Evaluate how close to bionic is libhybris
bionic Android default * You spend lot of time trying to run GNU/Linux debug tools like evtest on Android:
* Parabola cannot be used on old kenrels (FATAL: kernel too old)
* GuiX and libreCMC might not have the package and might need tweaking to be recompiled with an old glibc and kernel headers
* TODO: try crosstool-ng
* Most of the other GNU/Linux distributions are not FSDG compliant or do not support ARM
* The software might not work on Android due to missing bionic functions like versionsort(...)

Other projects interested in using upstream Linux and/or contributing to it

Upstream Android

LineageOS

The devices initially supported by LineageOS 16 have either:

It may be because they rely on nonfree software to support devices, which has not been ported to Android 9, or may be because they need more time to add devices with other SOCs like exynos.

AOSP and LineageOS

Feature AOSP LineageOS
Code quality Better than LineageOS
Documentation Better than LineageOS
root Not sure if supported or not Supported
Minor release versioning supported with Git tags Not supported.
For instance frameworks/native has no LineageOS tags (only cm-1x tags) and has no other branch for lineage OS 16.0 than the lineage-16.0 branch, as the other branches name are for other things. This was confirmed after running 'git fetch github' in frameworks/native to fetch the other branches that are not fetched by repo by default.
Despite that, some code was even merged in after the lineage-16.0 release:
commit 22abc3cf4077644463a2dc1c59a5a74e9518ea16
Merge: 9e96d54 8a6b6c3
Date:   Sat Jul 13 18:57:45 2019 +0200

    Merge remote-tracking branch 'aosp/pie-gsi' into lineage-16.0-pie-gsi
GUI features for developers
* Advanced reboot
? Supported

Design decisions

Some decisions have been taken by upstream projects, for instance the the Android Open Source Project (AOSP) is pushing device manufacturers to use signed bootloaders and not give the users the ability to replace those bootloaders. Therefor it is best to revisit such decisions and decide whether or not to implement a given feature.

Various

Feature Advantages Disadvantages
adb and root at boot Easier to debug:
* We get the logs at boot
* May be able to diagnose non-booting devices (partition not mountable, etc)
Way less secure:
* Vulnerable to Juice_Jacking
* Vulnerable to an attacker that just connect an usb cable to a running phone
* Breaks the user's expectation of security (lock screen etc)

root filesystem related

Feature Advantages Disadvantages sustainability
System as root * The kernel size can be bigger
* You have to hardcode the root partition in the cmdline or use PARTLABEL which might be a security issue: if a microSD has a partition named SYSTEM, the kernel may boot on it instead
having an initramfs adds some flexibility:
* Selecting partitions can potentially be more flexible
System as root + dm-verity + dm-init * The kernel size can be bigger
* The partition selection is flexible and secure
read-only /system * more secure and more easily understandable by users and developers * need to ship in replicant user-scripts or add them to vendor/

Gatekeeper HAL backend

Background information: Gatekeeper is a daemon that is used to store passwords and other secrets.

Backend used Advantages Disadvantages
simple userspace implementation * Fast to do
* Simple to understand
* Good enough for most use cases
kernel keyring (man 7 keyrings) * Secure
* The Linux kernel is well known and updated regularily
* Some users are already used to the userspace/kernel security model
* Probably fun to implement, can learn how t implement Android daemons and how to use the keyring along the way
Free software Trusted Execution Environment (TEE) * Android does it * Require access to TrustZone, which doesn't work for all SOCs
* Unfamiliar to users and developers (it's supposed to tick in suspend, knowledge aobut TrustZone is less spread than Linux, etc)
* Probably requires to port a TrustZone OS to every SOC or phone
* The Linux kernel is well known and updated regularily
Proprietary software Trusted Execution Environment (TEE) None: we really want to get rid of it if possible
Cannot be trusted:
* Not free software
* Not under the user control

Handling dynamic major/minor /dev/ nodes

Background information: We can manage to avoid this use case for now.

Backend used Advantages Disadvantages sustainability
Android default + Upstream Linux * does not work by default, probably impossible to make it work as-is
Android default + Upstream Linux + very dirty userspace scripts * Minimal changes * Not robust
* Might eat up resources
* Already implemented cleanly in mdev
Android default + hacked drivers to use fixed major/minor * Minimal changes in Linux
* No changes required on Android side
* Require to change userspace software (libsamsung-ipc) * Not upstreamable in Linux
devtmpfs + hacked Andrdroid init * Minimal changes if upstreamed
* Init is hard to debug
* Complex to do as debugging at this stage is complicated * Need to be usptreamed in Android
Stock Andrdroid init + mdev (busybox) * Bad integration in the Android build system * Changes are minimal * Android build system integration need to be upstreamed or maintained

Firmware loading

Backend used Advantages Disadvantages sustainability
system/core/init/firmware_handler.cpp * Stock Android implementation None Already upstream in Android
Something else None The stock Android implementation is good enough and firmware loading is trivial anyway Not upstream in Android

Tools

Tool Upstream Comments
Heimdall Heimdall Packaged in most GNU/Linux distributions
Fastboot, adb, etc Android Requires the Android build system
Archlinux
Debian
GuiX

Updated by Kurtis Hanna about 2 months ago · 281 revisions