[Archived] Replicant 6.0, at least for the Samsung Galaxy S3

Added by Wolfgang Wiedmeyer 4 months ago

This is an archived version of this post
I rewrote the original post and wanted to save the previous version as documentation of my initial work.

EDIT: added toolchain
EDIT 2: added more needed dependencies and change my intent to run this version on my main device to the current status where I am actually running this version on my main device.
EDIT 3: jack/jill toolchain is now also included
EDIT 4: updated encryption section
EDIT 5 updated build instructions and image section
EDIT 6 updated SELinux and Wifi section
EDIT 7 more hints for the build instructions, thanks to Loic Dachary!

The last weeks, I was working on getting Replicant updated to the CyanogenMod 13.0 branch, respectively to the AOSP 6.0.x release. This post should give an overview of what work has been done, what functionality is working on the device and what is left to be done. I only own a Galaxy S3 for testing, so I could only test my changes on this device and I did not work on the specific device sources of other phones. However, my work should also ease the migration for other devices, as a lot of the work is device-independent and many device specific changes also apply to other Samsung devices.

First of all: Why did I choose the latest CyanogenMod branch and not an older one that is already marked stable, like the 12.1 branch? I started with the 12.1 branch, but I quickly noticed that going this route results in more work than working with a branch that is still under development. The main issue was that the Galaxy S3 was not supported on the 12.x releases, so I had to backport changes from the 13.0 branch where support was added again and I had to look at forks that had maintained the device. When I hit a very strange compile error with the bionic lib, I decided to try the 13.0 branch. Working with a branch where changes are added daily is actually only very little additional work. Merging the changes from CyanogenMod is quickly done and there were no big changes anymore since quite some time. And on the upside: the latest security fixes are already included.

The following sections should give an overview of the most important areas where changes have to be made for getting Replicant working on a newer Android release.

Graphics

Android does not honor egl.cfg anymore. More information can be found here: http://www.2net.co.uk/tutorial/android-egl-cgf-is-dead
Android only falls back to the software renderer if it detects that it is running inside the emulator. I reused the ro.softwaregl property to add fallbacks. The overall performance is faster than on Replicant 4.2 in my experience and no crashes occurred in my latest tests.

Camera

CyanogenMod integrated the free software implementation from Replicant, but they adapted it for the proprietary graphics acceleration. I stayed with the version from replicant-4.2 as it works so far, but I noticed a few errors in the log. CyanogenMod's camera code may include some enhancements that could also be useful for Replicant.

I added the LegacyCamera app and taking pictures and recording video works. The preview is (annoyingly) slow, however. The gray background in the lower area gets rendered differently, so I would assume that some changes in libraries on which the app depends led to a worse software rendering performance. The camera app code itself contains no changes in this regard compared to Replicant 4.2. Further investigation has to be done to get the preview back to the old speed.

The old Gallery app still works and even the thumbnails work now. I still don't know what exactly made them work, though.

Webview

A lot changed here. The Android webview is now fully maintained by the Chromium project. The AOSP code only includes binaries now. Building the webview with the AOSP code is not supported anymore. It has to be built by setting up a Chromium build environment.

The downside of this is that another huge and complex build environment has to be set up in order to build the Webview apk. The apk has to be copied and committed to the corresponding repo in the Replicant source tree. Then it gets automatically included in the build. The build environment for the Chromium code has to be reviewed, just like the one for Replicant itself. All dependencies must be free software and the prebuilt binaries should be verifiable and the aim should be to build them from source, too.

The upside is that it is very easy now to build the Webview based on the latest Chromium release and to install it in Replicant. This makes it finally possible to have a Webview in Android that is on the same level as the desktop browsers when it comes to latest security fixes and adherence to current security standards. Please note that the Webview includes its own versions of various libraries. So if e.g. a security issue gets fixed in the AOSP Boringssl code, this will have no effect on the Webview as it includes its own Boringssl version.

Unfortunately, reality looks not that bright. All Webviews versions > 43 crashed because of missing hardware acceleration. I tried to fix a recent version, but I did not succeed. The latest versions have to be tested if they include fixes for this problem. There are open bugreports on the Chromium issue tracker for this as also everyone is affected who uses the emulator without hardware acceleration. So let's hope that the Chromium devs care for this. Otherwise the changes between version 43 and 44 have to be closer investigated. If that also doesn't help, then we're stuck for now with v43 and this version and its dependencies have to be patched. This would still be a much better situation than having the unmaintained Webview as in Replicant 4.2. The version I included for now (43.0.2357.134) is still fairly new.

Sensors

CyanogenMod also includes now a free software implementation, based on the code from Replicant. I stayed with the code from CyanogenMod as it seems to work. Screen rotation works and I also tested the SatStat app and noticed no missing functionality. Please let me know if there is additional testing necessary.

Wifi and Bluetooth

I could make Wifi and Bluetooth work in the case that the proprietary firmware files are installed.

If you are also using an external GPS receiver over Bluetooth, then you might be interested in my fix for the BlueGps app in order to make it work with Marshmallow: https://code.fossencdi.org/BlueGPS.git/

Device Encryption

The encryption setup does not ask you to set up a password first. So I recommend to set a lock screen password first. Then do the encryption setup.

If you want to use a passphrase for the encryption and something shorter for the lock screen (like a pin):
run this command in a root shell after you have set the short lock screen password and after you rebooted:

vdc cryptfs changepw password "short lockscreen password" "new very long and secure password"

The return code should be 200 0 0

SELinux

SELinux is running in enforcing mode (also in recovery mode).

RIL

I was able to get the Samsung-RIL and libsamsung-ipc code updated with the help from Paul on irc. Telephony, SMS and mobile data should work.

Toolchain and build instructions

There are even more binaries in the source code than with Replicant 4.2. I don't think that this is an acceptable situation. Most of these binaries are not well documented and very hard to rebuild. Thus they should not be trusted and removed. I was able to use more build tools that come with the GNU/Linux Distribution (in my case Debian Jessie) and get a big part of the toolchain build from source. For the GCC toolchain that builds the kernel the gcc-arm-none-eabi toolchain from Debian is used. The gcc-arm-linux-androideabi and host toolchains for LLVM and GCC are build from source. The host toolchains are built natively and don't rely on Google's modified Ubuntu sysroot, while the gcc-arm-linux-androideabi toolchain still relies on the sysroot from the NDK. I did not yet figure out how to completely bootstrap the gcc-arm-linux-androideabi toolchain with proper 1st and 2nd stage compilers and get rid of the dependency on prebuilt libs from the NDK.
The Jack/Jill toolchain is also built from source.
The whole build system is now more complex and likely breaks on other Distributions than Debian Jessie. Additionally to the current Replicant 4.2 dependencies, you need to install these:

sudo apt-get build-dep gcc-4.9 binutils llvm-defaults
sudo apt-get install gcc-arm-none-eabi cmake python-dev swig ant bc proguard maven-debian-helper libemma-java libasm4-java libguava-java libnb-platform18-java libnb-org-openide-util-java libandroidsdk-ddmlib-java libmaven-source-plugin-java libfreemarker-java libmaven-javadoc-plugin-java

Before you can build the ROM in the regular way and before you run any of the commands like . build/envsetup.sh, lunch, make, you need to run a build script that takes care of building the GCC/LLVM toolchains: ./vendor/replicant/build-toolchain
In order to prevent strange errors, I recommend running the script in a newly opened shell, where you didn't already run one of the commands like . build/envsetup.sh, lunch, make that change the environment.
The build script is very rudimentary for now. So if something breaks, you will need to take a look at it and figure out what exactly went wrong.
All my code is hosted at https://code.fossencdi.org. If you want to get it: Just do a repo init -u https://code.fossencdi.org/replicant_manifest.git -b replicant-6.0 and then run repo sync.
I also added a script (vendor/replicant/sign-build) that signs your build, takes care of generating the necessary keys and puts everything in the out/dist directory. Using this script, it is possible to rely on your own keys and not on the test keys which are not recommended to use because the private keys are publicly available. Building from source and flashing the default image that is signed with the test keys basically disables all the security measures in Android that are based on signing keys. The script also makes it possible to use password-encrypted private keys. The images below are signed with my keys using this script.
In summary, this are the steps you have to take in order to build an image and get it onto the device:
repo init -u https://code.fossencdi.org/replicant_manifest.git -b replicant-6.0
repo sync
./vendor/replicant/get-prebuilts # also checks the signature of the downloaded apk (see https://f-droid.org/wiki/page/Release_Channels_and_Signing_Keys)
./vendor/replicant/build-toolchain
. build/envsetup.sh
lunch replicant_i9300-userdebug
make -j $(nproc) bacon
./vendor/replicant/sign-build
heimdall flash --BOOT out/dist/recovery.img --RECOVERY out/dist/recovery.img # while in Downloading mode
adb sideload out/dist/replicant-6.0.zip # sideloading in recovery mode

The current F-Droid release key can be retrieved with the command gpg --recv-key 7A029E54DD5DCE7A.
If you have it previously already built and now you are getting build errors: Run make clobber and retry in a newly opened shell.
If make fails, it may be necessary that you need to run mka org.cyanogenmod.platform-res.

General

There are still a lot of graphics with the CyanogenMod logo that need to be replaced with the Replicant logo. Additionally, I did not care for the repos that include code which is not relevant for building the ROM or that did not add any improvements to functionality. The sdk repo comes to mind, for example. These repos need to be merged with the Replicant branch or the relevant commits have to be cherry-picked.

In the current state the ROM should be usable as a daily driver, at least for my use cases it is. I replaced the Replicant 4.2 version with this version on my main device and it works great so far.

Before you can flash your build to your device, you need to flash the new recovery first. A full wipe is also necessary. If you start adb from the terminal you already built the ROM from, you will actually use an adb version from the out directory that was added to your $PATH. If you use adb from a different terminal, you might notice that it won't work. You will need to update your android-tools-adb installation. I backported the package from Debian Testing.

If you want to add support for a different device: Add the missing device repositories and try to merge the changes from CyanogenMod. If the device is not supported on the CyanogenMod 13 branch, you will have to look at forks that took up the development. Merging with an older branch could also be enough to get basic functionality working.

Compiled images if you can't or don't want to build it yourself (build date: 25.3.16):

  1. Download the following files:
    https://fossencdi.org/replicant/replicant-6.0.zip
    https://fossencdi.org/replicant/replicant-6.0.zip.asc
    https://fossencdi.org/replicant/recovery.img
    https://fossencdi.org/replicant/recovery.img.asc
    https://fossencdi.org/replicant/images.sha256
  2. Add my key to your GPG keyring
    You can retrieve it from the keyserver of your choice (gpg --recv-key 5816A24C10757FC4).
    Alternatively, you can download it from here: https://fossencdi.org/replicant/5816A24C10757FC4.asc
    and import it with gpg --armor --import path/to/5816A24C10757FC4.asc
    The key should have the following fingerprint: 0F30 D1A0 2F73 F70A 6FEE 048E 5816 A24C 1075 7FC4
  3. Verify the signatures of the files:
    gpg --armor --verify path/to/replicant-6.0.zip.asc path/to/replicant-6.0.zip
    gpg --armor --verify path/to/recovery.img.asc path/to/recovery.img
    
  4. Check the checksums (in the folder with all the downloaded files):
     sha256sum -c images.sha256
    

Only install the images if all of these checks succeed!

I hope that my work benefits the Replicant project in some way and that we can have an official Replicant 6.0 release soon!