Project

General

Profile

PortingToAndroid11 » History » Version 2

Joonas Kylmälä, 11/17/2020 02:09 PM

1 1 Joonas Kylmälä
h1. Porting Replicant to Android 11
2
3
{{>toc}}
4
5
Active development has moved towards AOSP 11.
6
7 2 Joonas Kylmälä
The basic features working on Replicant 10 have been confirmed to work on Replicant 11, but not everything has been tested yet on either version. Replicant 10 source code and build instruction have been kept to do regression tracking: [[PortingToAndroid10|Porting Replicant to Android 10]].
8 1 Joonas Kylmälä
9
h2. Building Replicant 11
10
11
h3. Source code
12
13
<pre>
14
$ repo init -u https://git.replicant.us/replicant-next/manifest.git -b replicant-11-dev
15
$ repo sync
16
</pre>
17
18
Alternatively a shallow copy of the source tree can be fetched in order to save on disk space:
19
20
<pre>
21
$ repo init -u https://git.replicant.us/replicant-next/manifest.git -b replicant-11-dev --depth=1
22
$ repo sync -c
23
</pre>
24
25
To unshallow a specific module:
26
27
<pre>
28
$ cd path/to/module
29
$ git fetch --unshallow <remote>
30
</pre>
31
32
h3. Build dependencies
33
34
h4. For Trisquel 8
35
36
<pre>
37
sudo apt-get install bc bison build-essential bsdmainutils ccache curl flex g++-multilib gcc-multilib gettext git gnupg gperf imagemagick lib32ncurses5-dev lib32readline-dev lib32z1-dev liblz4-tool libncurses5-dev libsdl1.2-dev libssl-dev libwxgtk3.0-dev libxml2 libxml2-utils lzop python-mako pngcrush rsync schedtool squashfs-tools xsltproc zip zlib1g-dev
38
sudo apt-get install gcc-5-arm-linux-gnueabi
39
</pre>
40
41
h3. Fixing the build environment
42
43
h4. Allow system binaries for building
44
45
By default, the Android 11 build system can only use the prebuilt binaries it ships.
46
47
While having binary toolchains is better for reproducible builds, and that the binaries are free software, this creates a number of issues:
48
* We need to be able to rebuild the binaries, and so far no one did it yet.
49
* Binaries are way harder to trust than source code and not everyone trust Google.
50
51
As GNU/Linux distribution's tools can be rebuilt and are easier to trust, we are using that for now.
52
53
h4. Java heap space
54
55
The Java heap size is automatically set according to the available system memory. On machines with 8 GB or less RAM, it is set to a value which is too low, and will result in the following error during the build:
56
57
<pre>
58
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
59
</pre>
60
61
The heap size can be "increased with an envirnoment variable":https://stackoverflow.com/a/60474592/1313087:
62
63
<pre>
64
$ export _JAVA_OPTIONS="-Xmx3g"
65
</pre>
66
67
h4. Reduce parallel jobs to avoid killed processes
68
69
Increasing the Java heap space is not enough to get a successful build on machines with 8 GB or less RAM. It is also necessary to reduce the number of parallel jobs, to avoid processes from being killed due to lack of memory. This typically happens during the build of @frameworks/base@ components.
70
71
For greater speed, you may let your build run with the defaults, wait for it to fail due to killed processes, and then relunch the build with:
72
73
<pre>
74
$ make -j1
75
</pre>
76
77
By default, "Ninja":https://ninja-build.org/manual.html, the underlaying build system for Android, used when you run @make bacon@, computes the number of parallel jobs according to the number of CPUs on your machine (typically #CPUs + 2 parallel jobs).
78
79
80
h3. Launching the build
81
82
<pre>
83
$ cd kernel/replicant/linux && ARCH=arm CROSS_COMPILE=arm-none-eabi- make replicant_defconfig && ARCH=arm CROSS_COMPILE=arm-none-eabi- make -j9 && cat arch/arm/boot/zImage arch/arm/boot/dts/exynos4412-i9305.dtb > zImage-dtb
84
$ source build/envsetup.sh
85
$ lunch lineage_i9305-eng
86
$ make
87
</pre>
88
89
h3. Install the images
90
91
h4. From scratch
92
93
<pre>
94
$ cd out/target/product/i9305
95
$ sudo heimdall flash --BOOT boot.img --USERDATA userdata.img --SYSTEM system.img 
96
</pre>
97
98
h4. Update previous installation
99
100
<pre>
101
adb remount
102
adb sync
103
</pre>
104
105
h3. Get adb
106
107
As the device IDs are the ones given by the Linux kernel, they are not in the adb udev rules, so for now it requires to run adb as root:
108
<pre>
109
$ sudo adb shell
110
* daemon not running; starting now at tcp:5037
111
* daemon started successfully
112
i9305:/ #                                 
113
$ sudo adb kill-server
114
$ adb shell
115
* daemon not running; starting now at tcp:5037
116
* daemon started successfully
117
error: no devices/emulators found
118
</pre>
119
So make sure to kill the adb-server and run it as root:
120
<pre>
121
$ adb kill-server
122
$ sudo adb shell
123
* daemon not running; starting now at tcp:5037
124
* daemon started successfully
125
i9305:/ # 
126
</pre>
127
128
h3. Boot progress
129
130
You can also follow the boot progress with adb:
131
<pre>
132
adb logcat
133
adb logcat -b main
134
</pre>
135
136
Note that the device can go into suspend at any time, so adb might be interrupted. That looks like that:
137
First you get a shell
138
<pre>
139
$ sudo adb shell
140
i9305:/ #
141
</pre>
142
143
Then the connection is interrupted:
144
<pre>
145
$ adb shell
146
i9305:/ # [randomdev@fullyfreelaptop ]$                                                                                                     
147
</pre>
148
149
The effect with adb logcat is similar.
150
151
h3. Getting the latest changes
152
153
* The repositories are being constantly modified with @git push --force@ as we are trying things out, and don't want to make the commits history look too dirty, so be sure to backup your local changes.
154
* Sometimes the manifest repository is also modified with @git push --force@. In that case the following commands will loose all the work you did locally but will make the repository consistent with upstream repositories again:
155
<pre>
156
$ rm -rf .repo/manifests .repo/manifests.git .repo/manifest.xml
157
$ repo init -u https://git.replicant.us/replicant-next/manifest.git -b replicant-11-dev
158
$ repo sync --force-sync
159
</pre>
160
* The following command might also be necessary to make the state consistent with upstream repositories again, when the manifest history wasn't rewritten, but it will also loose all the work you did locally: 
161
<pre>
162
$ repo sync --force-sync
163
</pre>
164
165
h3. Build VM
166
167
If you use Parabola, you may be interested in running Trisquel 8 in LXC.
168
169
To do that first debootstrap a Trisquel 8 rootfs. 
170
171
Parabola's debootstrap does support Trisquel 8 and its manual has an example on how to do that:
172
<pre>
173
$ man debootstrap
174
[...]
175
# debootstrap flidas flidas-root http://archive.trisquel.info/trisquel
176
</pre>
177
178
Then you can use virt-manager to setup the LXC instance.
179
180
The advantages of this solution are that:
181
* The LXC guest and host shares their resources (CPU, RAM) with almost no penalty
182
* Trisquel 8 is not a rolling release distribution
183
184
The disadvantage of this solution are that:
185
* you need to configure the Trisquel 8 LXC instance (vimrc, sshd_config, etc)
186
* It's more complicated to setup
187
* The Android build system outputs a warning message about not being able to use namespaces which may become mandatory in newer Android versions
188
189
h2. Cleanups to be done
190
191
* Make adb work as user by using the right USB IDs, and make userspace do the USB setup.
192
* -Make the kernel not use hardcoded CMDLINE_FORCE- Not possible unless the bootloader is changed or Linux is very heavily patched.
193
* Make the kernel not use hardcoded partitions if possible (though we use system as root)
194
* Make a clean Gatekeeper HAL module implementation instead of using the same hack than goldfish
195
* Look at "init.rc":https://android.googlesource.com/platform/system/core/+/master/init/README.md documentation to see if init.rc can be overriden clearly with the override statement to see if it's possible to keep the serial console patch for -eng
196
197
h2. Upstreaming status
198
199
* The stock bootloader is incompatible with Linux, see [[BootloadersIncompatibleWithLinux#Devices-with-the-Exynos-4412]] for more information. So we maintain patches to enable the Galaxy SIII, and Galaxy Note II to boot with the stock bootloader. In the long run we need to look into using u-boot in the kernel partition as using u-boot instead of the stock bootloader currently require nonfree and non-redistributable software (BL1).
200
* For the patches that are not merged yet, see the "issues of the redmine upstreaming sub-project":https://redmine.replicant.us/projects/upstreaming/issues
201
202
h2. Graphics status
203
204
Progress of the graphics related tasks is tracked at [[GraphicsReplicant10]].
205
206
h2. Modem status
207
208
*libsamsung-ipc:* 
209
* Fully tested under GNU/Linux only
210
* Can initialize completely the modem and receive messages (see the #1954 bug report for the logs)
211
* Needs more cleanup but there is now a better abstraction
212
213
*libsamsung-ril:* 
214
* Ported to Replicant 9 using a wrapper for the API >=12 in libsamsung-ril source code that needs to be removed
215
* Tested under Replicant 9 without up to date libsamsung-ipc (no modem init)
216
* Tested and validated under Replicant 6 (doesn't break telephony)
217
218
h3. Modem status TODO
219
220
* Implement the missing part to shut down the modem, close the interfaces and such, in order to need to reboot after each test.
221
* Continue to clean up the libsamsung-ipc and libsamsung-ril patches, test the patches in Replicant 6 when applicable, and merge them in the upstream repositories.
222
* Convert the firmware loading driver to the upstream API and then adapt libsamsung-ipc for that. This should also benefit other devices like the Galaxy SIII 4G, and the Galaxy Note II 4G which probably don't need much more to get their modem supported by upstream Linux.
223
* Cleanup and convert the rest of the drivers to look like the ones for the Nokia N900 and adapt the userspace in libsamsung-ipc, and merge libsamsung-ipc support for that once the kernel API is stable.
224
* Test the code under Replicant 10 when the graphics status will enable to have decent enough speed to do some testing through human interaction.
225
226
h2. TODO
227
228
First month of full time equivalent work:
229
230
|_. Time estimation  |_.Task |_. Comments |
231
| DONE | -boot a device under AOSP9- | Only boots with graphics, not much more |
232
| -7h- DONE | -build it under a "FSDG compliant distribution":https://www.gnu.org/distros/free-distros.html like Trisquel8- | -WIP for AOSP, It's difficult to do precise time estimations as it could work out of the box or require one full time month of work depending on how much issues are encountered- 
233
Builds under Trisquel8|
234
| -21h- DONE | * -port the changes from AOSP9 to LineageOS 16-
235
* -cleanup the code-
236
* -build the kernel from the Android build system-
237
* -make sure it builds with an FSDG compliant distribution-
238
* -document the build procedure- |
239
Status:
240
* Boots with adb.
241
* Has ultra slow graphics |
242
| 14h | find, remove and document proprietary software in LineageOS 16 | |
243
| 21h | find, remove and document privacy issues in LineageOS 16 | |
244
| 7h | -Add support for the touch keys driver in the  galaxy-s3 dts- | "applied":https://lists.infradead.org/pipermail/linux-arm-kernel/2019-November/694100.html |
245
| 7h | -upstream the AAT1290 flash led Linux dts for the galaxy-s3 boards- | "Now in 5.3":https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v5.3&id=4e4dfcb2a425cccc6dd1fb7d46e060cd57999afc |
246
| 7h | rebrand LineageOS as Replicant | |
247
| 70h | -port and- cleanup the the Galaxy SIII (i9300) "modem":https://github.com/fourkbomb/linux/tree/modem Linux driver from 4.16 to 5.0| See the [[PortingToAndroid9/#Modem-status|modem status]] for more details |
248
|_\3. Total: 147h (~1 month) |
249
250
Second month of full time equivalent work:
251
| * port  libsamsung-ril and libsamsung-ipc to Android 9
252
* Make the modem driver and libsamsung-ipc work together | 157h | See the [[PortingToAndroid9/#Modem-status|modem status]] for more details |
253
|_. Total |_. 157h | ~1 month |
254
255
Third month of full time equivalent work:
256
|_. Task |_. Time estimation |_. Comments |
257
| -port the sensors libraries and other device specific libraries-
258
Look which sensor libraries can be used | 70h | Already done by the unofficial LineageOS port of the Galaxy SIII (i9300), needs testing |
259
| add support for Audio with the upstream kernel driver | 70h | Might be way faster, depending on what Android 9 uses
260
                                                               See also "this bugreport":https://github.com/CustomROMs/android_local_manifests_i9300/issues/1 |
261
| -add partial (no modem) support for the Galaxy SIII 4G (i9305) and- factorize the code with i9300 | 14h | * The source code on which the work was based changed from AOSP to an unofficial LineageOS port to a port of i9305 support for AOSP by Joonas to the official LineageOS so it's now supported by default
262
* The work to factorize the code between the i9300 and i9305 still need to be done |
263
 |
264
|_. Total |_. 154h | ~1 month |
265
266
|_. Task |_. Time estimation |_. Comments |
267
| create a recovery | 21h | |
268
| add internal WiFi support and validate the functionality | 6h | |
269
| add external WiFi dongles support | 20h | External dongles support might be tricky |
270
| -create new- update the install and upgrade instructions | -35h- | -Our current install instructions don't scale as we have one copy for each device.-
271
-We also created generic instructions but they tend to be harder to follow[1] than the device specific ones.-
272
-This will be made in a modular format (for instance in LaTeX) that enables to generate per device install instructions without requiring to have multiples copy of the same text.-
273
-The instructions will need to be able to be modified and compiled on an FSDG compliant distribution.-
274
Mostly done: 
275
* The installation instructions are now generic enough. 
276
* Some long standing TODO were also done along the way like adding backup instructions for the EFS.
277
* The current instructions are still for Replicant 6.0 and will need to be updated for Replicant 9.0 |
278
279
|_. Task |_. Time estimation |_. Comments |
280
| Estimate the amount of work to Reduce the attack surface | ? | |
281
| Estimate the amount of work to add in-system upgrades | ? | |
282
283
fn1. The generic instructions were tested at Install parties in Paris 
284
285
h3. Devices support:
286
287
h4. Easy, because it's similar enough to the Galaxy SIII (I9300)
288
289
|_\3. Galaxy Note II (N7100) |
290
|_. Task |_. Time estimation |_. Comments |
291
| port the EA8061 LCD Linux driver | 35h | |
292
| port the S6EVR02 LCD Linux driver | 35h | |
293
| port the MAX77693 flash led Linux driver | 7h | |
294
| android: add support for the Note II (N7100) and factorize the code with Galaxy SIII (i9300) and Galaxy SIII 4G (i9305) | 14h | Should be similar to the Galaxy SIII |
295
| port the sensors libraries and other device specific libraries | 70h | It's difficult to evaluate how much time it could take |
296
| add support for Audio with the upstream kernel driver | 14h | Should be similar to the Galaxy SIII |
297
298
|_\3. Galaxy Note 8.0 (N5100) and 8.0 WiFi (N5110) |
299
|_. Task |_. Time estimation |_. Comments |
300
| Evaluate the time required to do the port | 14h | TODO |
301
302
h4. Needs more work and unknown upstream Linux status
303
304
|_. Device |_. Time estimation |_. Comments |
305
| Galaxy S II (i9100) | | Linux: devboard dts upstream? unknown status |
306
| Galaxy Note (N7000) | | unknown Linux upstream status |
307
| Galaxy Nexus (I9250) | |/2. OMAP4, no dts upstream |
308
| Galaxy Tab 2 7.0 (P3100), 7.0 WiFi (P3110), 10.1 (P5100), 10.1 WiFi (P5110) | |
309
| GTA04 >= A4 | | TODO: a RIL needs to be written, userspace GPS support is missing, audio scenarios, etc |
310
311
h2. Documentation
312
313
h3. Replicant 6.0 changes
314
315
See [[Replican6Changes]].
316
317
h3. Other rebases
318
319
See the [[Samsung-ipc]] page.
320
321
h2. Other attempts
322
323
* It might be interesting to contact the people doing ports once we have something working well enough.
324
* It is also interesting to look at other attempts to understand if a given device is powerful enough to run Android 9 and what configuration was used to achieve it.
325
326
|_. Device(s) |_. Repository |_. status |_. Comments |
327
| i9300 | "CustomROMs":https://github.com/CustomROMs/android_local_manifests_i9300 | * "February 8 2020 Pie release":https://forum.xda-developers.com/galaxy-s3/development/rom-lineageos-16-0-t3875899 | |
328
| i9300 | "Team InFusion":https://github.com/team-infusion-developers/android_local_manifests_i9300 | * "August 20 2019 Pie release":https://forum.xda-developers.com/galaxy-s3/development/rom-optimized-lineageos-16-0-team-t3940142 | Issues: * Uses a Samsung kernel
329
* Uses too many nonfree libraries
330
=> Probably nothing we could reuse from its code |
331
| n7100 | "ComicoTeam":https://github.com/ComicoTeam/android_device_samsung_n7100 | * "January 4 2020 Pie release":https://forum.xda-developers.com/galaxy-note-2/development/rom-lineageos-16-0-20190112-comico-t3889281 | |
332
| i9100 | "rINanDO":https://github.com/rINanDO/android_device_samsung_galaxys2-common | * "March 20 2020 Pie release":https://forum.xda-developers.com/galaxy-s2/development-derivatives/pie-i9100-t3850588 
333
* "July 19 2020 Android 10 release":https://forum.xda-developers.com/galaxy-s2/development-derivatives/rom-lineageos-17-0-t4022733 | |
334
335
h3. Links for other attempts
336
337
* https://www.xda-developers.com/android-pie-android-9-port-custom-roms/
338
339
h3. CustomROMs i9300 components
340
341
|_. Repository |_. Tree path |_. Dependencies |_. Function |_. Comments |
342
|/8. https://github.com/CustomROMs/android_hardware_samsung
343
lineage-16.0 branch | hardware/samsung/macloader | | Loads the MAC Address of the WiFi network interface | Might be useful |
344
| hardware/samsung/wifiloader | | Loads the wifi kernel module (like modprobe) and setup firmware filesystems permissions | May be useful |
345
| hardware/samsung/audio | | seems to contains ril related stuff as well | Look if the ril stuff is required, go for standard audio |
346
| hardware/samsung/lineagehw/hidl/livedisplay | | livedisplay is a feature similar to what redshift does on GNU/Linux | Not sure if it works with mainline |
347
| hardware/samsung/exynos/multimedia/utils/ | seem meant for audio/video decoding offload | assembly obtimized color conversion and resize code | check assembly code license, not sure if useful |
348
| all other directories in hardware/samsung/exynos/ | nonfree firmwares, nonfree software?, smdk kernel? | audio/video decoding offload | Avoid using that |
349
| hardware/samsung/exynos3 |/2. nonfree firmwares?, nonfree software?, smdk kernel? |/2. some light libraries, display stuff (gralloc, etc), 2D acceleration (FIMG), camera (FIMC), 3D acceleration, etc |/2. Avoid using that for now |
350
| hardware/samsung/exynos4 |
351
352
h2. Known error messages that are safe to ignore
353
354
* TestHarnessModeService: Failed to start Test Harness Mode; no implementation of PersistentDataBlockManagerInternal was bound
355
* JniUtils: Could not load native library jni_latinimegoogle
356
357
h2. Links 
358
359
* "Android build requirements - hardware and software":https://source.android.com/setup/build/requirements
360
* "Why LineageOS Developers are building Android Go-optimized custom ROMs":https://www.xda-developers.com/android-go-old-android-8-1-oreo/
361
* "Android Go recommended default values for propreties for optimization":https://android.googlesource.com/platform/build/+/master/target/product/go_defaults_common.mk
362
* "Team InFusion i9300 optimized system.prop":https://github.com/team-infusion-developers/android_device_samsung_i9300/blob/lineage-16.0/system.prop#L23
363
* "Hack to fix high CPU usage caused by logd":https://github.com/team-infusion-developers/android_device_samsung_smdk4412-common/commit/c66015514fc9779edfed2665f67e841f3620c71e
364
* "Use low-end video codecs":https://github.com/team-infusion-developers/android_device_samsung_smdk4412-common/commit/adc919f04b981a4fe005fab2cf443596a22992b2
365
* "Optimize ActivityManager cached apps":https://github.com/team-infusion-developers/android_device_samsung_smdk4412-common/commit/24bc62970d35eb5a7a257463e7dbfe873ceb5779
366
* "Use 1Gb Dalvik config":https://github.com/team-infusion-developers/android_device_samsung_smdk4412-common/commit/8a7cebc5af772a2949420ac4a383f5f75236c65c