Graphics on Replicant 10¶

This page documents the current progress and future plans for the graphics acceleration on Replicant 10. The original plan can be found at TasksFunding. Since then, through more in depth research and hands-on experience, several things have diverged.

The full effort of Porting Replicant to Android 10 can be tracked at: PortingToAndroid10.

Background information, as well as details on the software components and acronyms used on this document, can be found at GraphicsResearch.

Graphics stack tasks¶

status	origin	short description	notes	estimated man-hours	actual man-hours
done	original plan	Set up the development environment.	Required: i9305 phones, LXC Trisquel container (systemd nspawn fails due to old systemd on Trisquel 8), larger SSDs, 1.8V serial-USB adapters (BS101P FT232RL) plus makeshift resistors' banks.	24	24
ongoing	new	Update development environment for Replicant 10.	Android 10 has higher resource consumption during builds. Hacks and workarounds had to be found to be able to build in our machines. Build servers will be set up to get faster builds.	0	8
ongoing	original plan	Read graphics related AOSP documentation.	Never-ending task that, besides actual documentation, involves scouring through source-code, bug trackers, mailing lists and IRC logs.	16	40
ongoing	new	Ask for help.	Bothering free-software developers1 that have experience with or contribute to graphics sub-systems has been the most fruitful way to clear most roadblocks.	0	0
done	original plan	Use Mesa's llvmpipe backend instead of softpipe.	Merge requests on Mesa: !1402 and !1403. There was no need to update LLVM version.	40	28
todo	new	Find out why we are getting away without using the libEGL patch.	Android 10 no longer needs EGL? Take a look at frameworks/native/opengl/libs/EGL/Loader.cpp	0	0
ongoing	new	Proper dri node for Mesa's kms_swrast driver.	Currently using a simple hack that allows it to use drm/exynos. Should rather use vGEM.	0	4
tentative	new	Combine kmsro with kms_swrast on vGEM render node?	Is kms_swrast working on top of the vGEM render node able to share PRIME buffers with the display node (Exynos)? If not, would adding kmsro to the mix help? Architectural ideia: kmsro + kms_swrast on vGEM render node -> PRIME -> drm_hwcomposer on display node (Exynos) Where should gbm_gralloc go to? vGEM render node? Advantages: - no need to copy buffers between kms_swrast and Exynos (PRIME takes care of that); - can take advantage of HW planes.	0	0
todo	original plan	Implement the missing pixel formats in drm/exynos.		72	0
todo	new	Get entire stack to use RGB555 pixel format.	Had a huge performance boost on Replicant 6.	0	0
ongoing	original plan	Proper way to use DRM-Master and DRM-Auth with gbm_gralloc and drm_hwcomposer.	Tried: 1. Auth hack (both on /dev/dri/card0) 2. vGEM (gbm gralloc on /dev/dri/card1) - Requires memory sync between vGEM and exynos. Use PRIME? gbm gralloc cannot take advantage of exynos hardware planes? 3. Allow dumb buffers on render node (gbm gralloc on /dev/dri/renderD128) - Dumb buffers are used for scanout. Should not be created on a render node. This hack is not needed with gbm_gralloc on Lima render node. Why? gbm_gralloc no longer allocates dumb buffers?   Tentative solutions: * use DRM Magic and have both on /dev/dri/card0 * link gbm_gralloc to vGEM's render node (/dev/dri/renderD129)	40	4
ongoing	new	Use hardware planes for better composer performance.	Enabling HW planes with drm_hwcomposer was straightforward but led to severe graphics corruption. Disabling devfreq fixed the corruption. Tentative explanation: display controller frequency gets too low for timely DMA transfer of overlays. Reported upstream. TODO: lock display controller frequencies through sysfs and re-enable devfreq, or try to remove low freq OPP steps.	0	16
ongoing	new	Use Skia instead of HWUI to render the Canvas.	Unlike Replicant 6, none of the usual system props (e.g. ro.kernel.qemu=1, ro.config.avoid gfx accel=1) would yield the expected performance. Got there by forcing hardwareAccelerated=false on all apps. TODO: turn this dirty hack into a system property that can be toggled on the device tree.	0	22
ongoing	original plan	Create test scenarios and check if the graphics stack works as expected.	Stock apps work. Check Tested apps bellow for the current status with apps that require advanced graphics features. TODO: compliance tests.	40	14
abandoned	original plan	Make the graphics stack work with vGEM driver besides drm/exynos.	No need. All Replicant targets have a display and, likely, a companion display controller with a fitting drm driver too. Most drm drivers expose a render node that Mesa can use. Better use that instead of vGEM.	40	0
ongoing	original plan	Document the design decisions.	Done at this wiki page plus the presentation at ContributorsMeetingJuly2019.	16	59
ongoing	new	Try out the Android Go low RAM switches and check their impact on graphics rendering performance and overall system usability.		0	1
total sum:				288	208

¹ A big thanks to Joonas Kylmälä, Paul Kocialkowski, Andrés Domínguez, Denis Carikli, Mauro Rossi, Emil Velikov, Andrzej Hajda, Marek Szyprowski and LiquidAcid.

SwiftShader tasks¶

status	origin	short description	notes	estimated man-hours	actual man-hours
done	original plan	Find a way to use SwiftShader instead of Mesa.	Joonas got there with ranchu composer (from Android Emulator) and the default gralloc, plus a patch to support UDIV/SDIV emulation in the kernel.	40	0
done	new	Use LLVM as backend instead of SubZero.	Found a SwiftShader revision that uses LLVM and is still compatible with Android 9 frameworks/native. No noticeable performance difference.	0	6
done	new	Do UDIV/SDIV emulation on JIT compiled shader code instead of kernel patch. Avoids performance penalty of interrupt handling.	It seems that SwiftShader does not send the processor model (microarchitecture) to LLVM, leaving it without a way to decide whether the processor has hardware division. Fixed upon update to Replicant 10. SwiftShader now uses LLVM 7 instead of 3, which fixed this.	0	30
todo	new	Use drm_hwcomposer instead of ranchu. Advantages: uses hardware planes and DRM nodes instead of direct framebuffer.	Joonas was close.	0	0
todo	new	Use mainline SwiftShader. Brings in a Vulkan software renderer for Replicant.	Difficult to due incompatibilities with frameworks/native. Check if is fixed with Replicant 10.	0	0
total sum:				40	36

llvmpipe optimization tasks¶

status	origin	short description	notes	estimated man-hours	actual man-hours
todo	original plan	Setup a testing and benchmarking environment.	Profiling: turn on profiling switch on Mesa + simpleperf? Benchmarks: android-fps-count, 0xBenchmark, GearsES2 Conformance: dEQP, Android CTS, piglit, freedreno/tests-*, glmark2	40	1
todo	original plan	Disable expensive OpenGL operations.		24	0
todo	original plan	Recap matrix operations and study ARM NEON.		48	0
todo	original plan	Profile apps to find the most used GLES operations.		32	0
todo	original plan	Use Ne10 library or Neon Intrinsics for the most used GLES operations.	Optimizations have to be done on LLVM and not on llvmpipe. llvmpipe only outputs LLVM IR. LLVM already has autovectorization for ARM NEON, try it.	80	0
todo	original plan	Fix bugs, re-write the code where needed, get it stable.		80	0
total sum:				304	1

Lima driver tasks¶

status	origin	short description	notes	estimated man-hours	actual man-hours
done	original plan	Rebase Lima's Linux kernel DRM driver on top of forkbomb's Midas on Mainline kernel.	Done by others. Lima DRM driver was accepted into mainline Linux, which also has forkbomb's patches and is now used on Replicant 10.	80	0
done	original plan	Replace mainline Mesa for Lima's Mesa (with their driver).	Done by others. Lima is now on mainline Mesa. Lima wiki	16	0
done	new	Lima DRM driver bringup on Exynos.	Lima development is done on AllWinner devices. We expected some issues to get it working on Exynos. Although there were encouraging reports by ChronoMonochrome, hexdump0815 and Viciouss (manifest, xda). Joonas added Lima to Replicant 10 and faced no major bringup issues.	0	1
todo	new	Use kmsro with Lima render node.	Architecture: kmsro (Mesa) + gbm_gralloc on renderD129 (Lima) -> PRIME -> drm_hwcomposer on card0 (Exynos) Advantages: - no need to copy buffers between Lima and Exynos (PRIME takes care of that); - can take advantage of HW planes.	0	0
ongoing	new	Fix graphics corruption with hardware planes.	Buffer sharing is broken when hardware planes are turned on at drm_hwcomposer. Corruption happens when compositing GL planes with non-GL planes. E.g. try Shader Editor, run a shader and open a menu. Does the hardware support GPU and hardware planes at the same time? Stock kernel used the GPU with a single plane. Disabling devfreq didn't solve it (as it did with llvmpipe).	0	1
todo	new	Advertise GLES 2. Shader Editor can only detect GLES 1.		0	0
todo	original plan	Build and test thoroughly with synthetic and real applications.	Use conformance tests to figure out the current GLES implementation status.	40	0
abandoned	original plan	Create a fallback mechanism that uses the software renderer for GLES functions not yet implemented in Lima.	There is no sane way to switch between different GLES drivers at the function level. Abandoned in favour of the tasks bellow.	100	1
todo	new	Lima as SurfaceFlinger backend.	Even if not GLES 1 or 2 feature complete, Lima should already be usable as backend renderer for SurfaceFlinger.	0	0
todo	new	Lima as HWUI (SkiaGL) backend.	Even if not GLES 1 or 2 feature complete, Lima should already be usable as backend renderer for HWUI.	0	0
todo	new	Lima on a per-app basis.	Current state of Lima may allow it to work with certain apps depending on their GLES usage. We can re-work the per process libagl/llvmpipe patch into a patch that switches between Lima and a software renderer (llvmpipe or SwiftShader).	0	0
total sum:				236	3

2D optimization tasks¶

status	origin	short description	notes	estimated man-hours	actual man-hours
todo	new	Investigate the possibility of using Pixman or Exynos G2D as RenderEngine for SurfaceFlinger.	There are interesting reports of people using G2D to hardware-accelerate X11 EXA	0	1
todo	new	Accelerate Skia with G2D.	Rework old patches (Hillenbrand 2013 and raymanfx 2016 ) to make them work on current Skia.	0	1
total sum:				0	2

Tested apps¶

app	Replicant 6		Replicant 10			notes
	libagl	LLVMpipe	Lima	LLVMpipe	SwiftShader
Fennec F-Droid	crashes	slow	fast			Needs GLES 2.0
LibreOffice Viewer	crashes	slow	cannot test (missing storage)
Red Reader	crashes	usable	cannot test (no network)
Shader Editor	crashes	7 fps	30 fps (freezes when changing resolution)			FPS measured on default shader with 1/1 resolution.
Marine Compass	bad render	bad render	crashes			Only uses GLES 1.0
Gears	crashes	no render	crashes
GL TRON	4 fps	2 fps	23 fps			Has a nice FPS counter.