Project

General

Profile

GraphicsResearch » History » Version 46

dl lud, 07/02/2020 04:33 PM
Update Multiple backens with latest info from Replicant 6 0004. Add Vulkan software renderers. Move GLES software renderers to their own category.

1 1 Wolfgang Wiedmeyer
h1. Research on free graphics-related software
2
3 12 Wolfgang Wiedmeyer
{{>toc}}
4
5 5 Wolfgang Wiedmeyer
On this page, information is collected that could help solving graphics issues in Replicant (see #1539). Besides evaluating free implementations that are relevant for currently supported devices, other implementations should also be listed if they are useful for potential future target devices.
6 1 Wolfgang Wiedmeyer
7 46 dl lud
External resources:
8
"Free and open-source graphics device drivers - Wikipedia":https://en.wikipedia.org/wiki/Free_and_open-source_graphics_device_driver
9
"Mobile drivers - Debian Wiki":https://wiki.debian.org/Mobile#Drivers
10 8 Wolfgang Wiedmeyer
11 21 Denis 'GNUtoo' Carikli
h2. Multiple backends
12 35 Jeremy Rand
13 46 dl lud
"Replicant 6.0 0004 RC1":https://lists.osuosl.org/pipermail/replicant/2020-January/002452.html includes a [[Graphics#Choosing-the-software-renderer-for-a-specific-app|mechanism]] that allows choosing between different OpenGL ES implementations for each app or process. This came into place as means to achieve a balance between fully-compliant but slower implementations (e.g. llvmpipe) and non-compliant but fast implementations (e.g. libagl).
14 1 Wolfgang Wiedmeyer
15 46 dl lud
A similar mechanism may be used in future Replicant versions, to take advantage of GPU backed implementations, even if they do not achieve full OpenGL ES compliance.
16 1 Wolfgang Wiedmeyer
17
h2. Software rendering
18 21 Denis 'GNUtoo' Carikli
19 46 dl lud
Software rendering uses the CPU and not a dedicated graphics processor for graphics rendering. It is slower than per-GPU implementations and is mostly used as a fallback, when GPU acceleration is not available. An advantage is that the same software renderer can work across many different types of hardware. Therefore, improving a software renderer benefits many different devices, regardless of the SoC and graphics unit that they have. Furthermore, a software renderer doesn't require a kernel driver. This makes it easier to work on mainline Linux kernel support for a device, until the graphics driver is in mainline.
20 1 Wolfgang Wiedmeyer
21 46 dl lud
h3. OpenGL ES software renderers
22 2 Wolfgang Wiedmeyer
23 46 dl lud
h4. libagl
24 1 Wolfgang Wiedmeyer
25 46 dl lud
* Source code: https://git.replicant.us/replicant/frameworks_native/tree/opengl/libagl
26
27 45 dl lud
libagl is the fastest software renderer available for Replicant devices. It is used by default on all Replicant-supported devices up until Replicant 6.0 0003 (0004 switched to llvmpipe).
28 1 Wolfgang Wiedmeyer
29 45 dl lud
libagl was developed specifically for Android and it is part of the AOSP source code. The renderer includes optimizations for ARM via "libpixelflinger":https://android.googlesource.com/platform/system/core/+/master/libpixelflinger and "codeflinger":https://android.googlesource.com/platform/system/core/+/master/libpixelflinger/codeflinger/ that do JIT compilation into platform optimized code. Development ceased in 2013 and no work was done to support newer OpenGL ES (GLES) versions (which causes #705).
30
31 26 dl lud
Until ca. 2011 (Android 4.0), another library with the name @libagl2@ existed and @surfaceflinger2@ was developed based on Mesa. The source code was later removed and no further development is known. At the time, the work was done to support a newer GLES version for the software renderer. It was abandoned later, probably when Google made it mandatory for Android 4.0 and later devices to pack their own hardware GPU with OpenGL ES 2.0 support. It is questionable if it is worth it to port the old @libagl2@ library to a recent Replicant version, also given that we would be the only ones using and maintaining the code.
32 1 Wolfgang Wiedmeyer
33 46 dl lud
h4. Mesa's llvmpipe
34 1 Wolfgang Wiedmeyer
35 46 dl lud
* Replicant llvmpipe support source code: https://git.replicant.us/replicant/external_mesa3d
36
* Replicant documentation: [[Graphics]]
37
* Upstream documentation: https://www.mesa3d.org/llvmpipe.html
38
* Upstream performance improvement documentation: https://www.mesa3d.org/perf.html
39 20 Denis 'GNUtoo' Carikli
40 1 Wolfgang Wiedmeyer
Mesa's llvmpipe is the default software renderer for Replicant since version 6.0 0004. It has a GLES implementation that is more complete than libagl, although slower when used by some system components (e.g. SurfaceFlinger).
41
42
Besides llvmpipe, Mesa has two other software rasterizers: swrast/swr and softpipe, but both are of no interest. swrast's GLES implementation is incomplete and this driver is mostly deprecated in favor of those built upon Gallium (softpipe and llvmpipe). softpipe on the other hand, "is as complete as llvmpipe":https://mesamatrix.net/ but is slower than it.
43 45 dl lud
44 1 Wolfgang Wiedmeyer
The "Android-x86 project":http://www.android-x86.org/ is using llvmpipe. A few of their Android-specific framework patches are applied in Replicant 6.0. Their Mesa source code fork is also used in Replicant 6.0. A lot of porting work of llvmpipe to Android was done by Jide while Intel is contributing as well. So there is an interest from different parties to have llvmpipe working on Android. Android patches are upstreamed to mainline Mesa.
45
46
llvmpipe is still not ported to ARM which makes it slow. Also for Android, it is mostly used on the x86 platform in other projects. See #705 for more information. Optimizing llvmpipe for ARM seems currently the most promising approach to fix graphics-related issues with Replicant.
47
48 46 dl lud
h5. Tuning llvmpipe
49
50 36 Jeremy Rand
The following environment variables might speed up llvmpipe somewhat:
51
52 46 dl lud
<pre>
53 1 Wolfgang Wiedmeyer
LP_PERF=no_mipmap,no_linear,no_mip_linear,no_tex,no_blend,no_depth,no_alphatest MESA_NO_DITHER=1
54 46 dl lud
</pre>
55 1 Wolfgang Wiedmeyer
56 46 dl lud
@LP_PERF@ seems to be undocumented; @MESA_NO_DITHER@ is documented in the Mesa performance tips. According to adjtm, @MESA_NO_DITHER=1@ improved glxgears performance by 3% on a GNU/Linux PC without breaking any apps that they tested. @LP_PERF=no_mipmap,no_linear,no_mip_linear@ also didn't break any apps on GNU/Linux in tests but there was no noticeable increase in performance. @no_tex,no_blend,no_depth,no_alphatest@ broke rendering of all tested apps in GNU/Linux.
57 1 Wolfgang Wiedmeyer
58 46 dl lud
To test with those environment variables on a per-app basis, you can disable SELinux, and then run the following, substituting the name of the app (here @info.guardianproject.orfox@) for the one you wish to test; then run the app from the launcher:
59 1 Wolfgang Wiedmeyer
60 46 dl lud
<pre>
61 1 Wolfgang Wiedmeyer
setprop wrap.info.guardianproject.orfox "LP_PERF=no_mipmap,no_linear,no_mip_linear,no_tex,no_blend,no_depth,no_alphatest MESA_NO_DITHER=1"
62 46 dl lud
</pre>
63 1 Wolfgang Wiedmeyer
64 46 dl lud
Unfortunately, wrap seems to crash Replicant sometimes. There was allegedly a fix at https://android-review.googlesource.com/c/platform/frameworks/base/+/318859 .
65 1 Wolfgang Wiedmeyer
66
Setting those environment variables globally might be possible by editing https://git.replicant.us/replicant/system_core/tree/rootdir/init.environ.rc.in .
67
68
Benchmarking performance in real-world apps might be feasible via this script: https://github.com/romannurik/env/blob/master/bin/android-fps-count
69
70 46 dl lud
h4. SwiftShader
71 1 Wolfgang Wiedmeyer
72 46 dl lud
* Source code: https://swiftshader.googlesource.com/SwiftShader
73 1 Wolfgang Wiedmeyer
74 46 dl lud
"Google released SwiftShader as free software in mid 2016":https://blog.chromium.org/2016/06/universal-rendering-with-swiftshader.html. It supports x86 and ARM architectures with SDIV/UDIV support. It is used in the Chromium project but also with Android, for example in the Android-x86 project. Swiftshader doesn't seem to depend on any external libraries besides what are provided in its Git repository, therefore it is very easy to compile it and use it as a software renderer for Replicant. All the Android build files are provided so you just need to the add the following packages to PRODUCT_PACKAGES in order to be able to use it:
75 1 Wolfgang Wiedmeyer
76 46 dl lud
 * libGLESv2_swiftshader
77
 * libEGL_swiftshader
78 1 Wolfgang Wiedmeyer
79 46 dl lud
SwiftShader features a broadly compliant GLES implementation like llvmpipe.
80 36 Jeremy Rand
81 46 dl lud
h3. Vulkan software renderers
82 36 Jeremy Rand
83 46 dl lud
h4. SwiftShader
84 36 Jeremy Rand
85 46 dl lud
Apart from GLES, SwiftShader also supports Vulkan. Actually, Vulkan is SwiftShader's main focus now.
86
87
h4. Vallium
88
89
* Source code: https://cgit.freedesktop.org/~airlied/mesa/log/?h=not-a-vulkan-swrast-4
90
91
Vallium is a Vulkan software renderer based on Mesa's llvmpipe and Gallium. It is still a work-in-progress, led single-handedly by David Airlie from Red Hat. Hopefully it will get merged into Mesa once deemed mature enough.
92
93
h4. Kazan
94
95
* Source code: https://salsa.debian.org/Kazan-team/kazan
96
97
Kazan is another work-in-progress Vulkan software renderer. It is an independent stack, not relying on any underlying graphics library. However, much like Vallium and SwiftShader, it uses LLVM for its shader compiler. Kazan is programmed in Rust.
98
99
h3. kms_swrast
100
101
"kms_swrast":https://memcpy.io/kms_swrast-a-hardware-backed-graphics-driver.html is a Mesa driver, built upon Gallium, that uses DRM nodes for memory allocation and to present images on the display, but does the actual OpenGL rendering through a Mesa software renderer such as llvmpipe or softpipe.
102
It may allow noticeable performance improvements by avoiding expensive memory copy operations between the software renderer and DRM.
103
104
Exynos based devices have a working free-software DRM driver that can be used with kms_swrast. Devices that do not have DRM driver can still benefit from kms_swrast by usage of the "VGEM (Virtual GEM) driver":https://www.phoronix.com/scan.php?page=news_item&px=MTA0MTQ.
105
106 34 dl lud
h3. Pixman
107
108 46 dl lud
* Source code: https://cgit.freedesktop.org/pixman/
109 34 dl lud
110
> "Pixman":http://pixman.org/ is a low-level software library for pixel manipulation, providing features such as image compositing and trapezoid rasterization.
111
112
It is highly optimized for ARM processors and has a "fast path for ARM NEON":https://www.phoronix.com/scan.php?page=news_item&px=OTAyOQ. It may be worthwhile to write an OpenGL ES backend for Replicant that detects 2D operations and translates them to Pixman. This could provide a considerable performance improvement over llvmpipe or SwiftShader and, if complete enough, could even replace them.
113
114 1 Wolfgang Wiedmeyer
On the other hand, writing such translation layer may prove to be an enormous task, as the OpenGL ES 2.0 API is quite extensive. In this scenario we can still benefit from Pixman by reproducing it's ARM NEON fast paths on llvmpipe.
115
116 9 Wolfgang Wiedmeyer
h2. Per-GPU implementations
117 1 Wolfgang Wiedmeyer
118 23 dl lud
In the following, free software implementations are listed that should make it possible to use the respective GPU with free software.
119 7 Wolfgang Wiedmeyer
120 42 Kurtis Hanna
h3. ARM Mali-4xx (Utgard) with Lima
121 7 Wolfgang Wiedmeyer
122 22 dl lud
Supported devices that use Mali 400: Galaxy S 2, Galaxy S 3, Galaxy S 3 4G, Galaxy Note, Galaxy Note 2, Galaxy Note 8.0
123 1 Wolfgang Wiedmeyer
124 22 dl lud
Project page: https://gitlab.freedesktop.org/lima/web/wikis/home
125 7 Wolfgang Wiedmeyer
126 23 dl lud
Lima is still unfinished as of January 2019 but development is going on at a fast pace. The detailed implementation status can be checked "with the piglit test suite":https://gitlab.freedesktop.org/lima/mesa/issues/39#note_79193.
127 22 dl lud
128
Lima saw first light with the reverse-engineering efforts by Luc Verhaegen, that produced an experimental driver with it's original page at  
129 16 Jeremy Rand
https://limadriver.org . Luc's development stopped around 2014, but in 2017 Qiang Yu took on the task and started development on top of Mesa's Gallium3D driver as "reported by Phoronix":https://www.phoronix.com/scan.php?page=news_item&px=Mali-400-New-Open-Source. 
130 43 Kurtis Hanna
The code is now hosted at "freedesktop.org's GitLab":https://gitlab.freedesktop.org/lima and is getting contributions by many developers besides Qiang Yu.
131
132 1 Wolfgang Wiedmeyer
The lima kernel driver is now loading with hacks on Replicant 9 using the 5.3.0-rc5+ kernel: https://github.com/CustomROMs/android_local_manifests_i9300/issues/1#issuecomment-524375690
133
134
h3. Imagination PowerVR
135 5 Wolfgang Wiedmeyer
136 8 Wolfgang Wiedmeyer
supported devices that use PowerVR SGX540: Nexus S, Galaxy S, Galaxy Nexus, Galaxy Tab 2 7.0, Galaxy Tab 2 10.1
137
supported devices that use PowerVR SGX530: GTA04
138 10 Wolfgang Wiedmeyer
139
A reverse-engineering project exists:
140 8 Wolfgang Wiedmeyer
"website":http://powervr.gnu.org.ve/doku.php
141 1 Wolfgang Wiedmeyer
"mailing list":http://listas.gnu.org.ve/listinfo/powervr-devel
142
"Libreplanet group":https://libreplanet.org/wiki/Group:PowerVR_drivers
143 23 dl lud
144
Despite initial reverse-engineering progress until 2013, no further development updates seem to be available. The project website provides documentation.
145
146
h3. ARM Mali-T6xx/T7xx/T8xx (Midgard) and G7x (Bifrost) with Panfrost
147
148
not used by a supported device
149
Used on several Samsung Galaxy S series phones that are now supported by LineageOs and may be potential targets for Replicant ports: S6
150
(Mali T760MP8), S7 (Mali-T880 MP12) and S9 (Mali-G72 MP18).
151
152
Project page: https://panfrost.freedesktop.org/
153 33 dl lud
Source code: https://gitlab.freedesktop.org/panfrost
154
155
Panfrost is another reverse-engineered driver for ARM Mali GPUs, but aimed at the latest architectures (Midgard and Bifrost). It is still experimental but under active development with a strong community (as of April 2019). "Its implementation of OpenGL ES 2.0 is approaching feature-completeness":https://rosenzweig.io/blog/kodi-supertuxkart-panfrost.html.
156 23 dl lud
Panfrost's main focus are the ARM SoCs present on some laptops and single-board computers (SBC), but it is "Intended to work on as many SoCs as possible to make everyone's lives easier":https://gitlab.freedesktop.org/panfrost/mali_kbase.
157 1 Wolfgang Wiedmeyer
Panfrost is developed by Alyssa Rosenzweig and Lyude Paul, and recently attracted other contributors such as "Tomeu Vizoso":https://blog.tomeuvizoso.net/ and "Rob Herring":https://gitlab.freedesktop.org/robh.
158 24 dl lud
Development can be followed on "Alyssas's blog":https://rosenzweig.io/blog/.
159 1 Wolfgang Wiedmeyer
160
h3. Qualcomm Adreno with freedreno
161 9 Wolfgang Wiedmeyer
162
not used by a supported device
163
164
wiki: https://github.com/freedreno/freedreno/wiki
165
166
freedreno is actively developed. Linux kernel component is available in mainline since version 3.12. It needs to be investigated how well freedreno could work on potential target devices. However, even when using freedreno, non-free firmware is very likely still needed.
167 15 Wolfgang Wiedmeyer
168 24 dl lud
Generally speaking, Qualcomm devices have a lot of blobs and no modem isolation which is the reason why no Qualcomm-based device is yet supported by Replicant. See [[TargetsEvaluation]] for some analysis. We have not yet identified a Qualcomm-based device that would be a promising target for Replicant and where freedreno could be used on.
169 15 Wolfgang Wiedmeyer
170
h3. Vivante GCxxxx with Etnaviv
171
172
not used by a supported device
173 29 dl lud
174
For some Vivante GCxxxx GPU variants, a free driver, Etnaviv, exists. See "wikipedia":https://en.wikipedia.org/wiki/Free_and_open-source_graphics_device_driver#Vivante for some details. It would be interesting to investigate devices using such a GPU if they would be good Replicant targets. It also needs to be checked if non-free firmware needs to be loaded to the GPU.
175
176
h2. Gralloc
177
178
In order to have a working software rendering on Replicant 9 we will need a gralloc (graphics memory allocator) library that:
179
* Implements the "Android Gralloc HAL":https://source.android.com/devices/graphics/implement#gralloc_hal API version 1.
180
* Is compatible with "drm-hwcomposer":https://gitlab.freedesktop.org/drm-hwcomposer/drm-hwcomposer (a libre implementation, based on Linux's DRM, of Android's Hardware Composer HAL).
181
* Is compatible with Mesa and particularly its llvmpipe software renderer.
182 30 dl lud
* (optional) Is compatible with SwiftShader.
183 29 dl lud
184
There are currently 3 free-software grallocs under evaluation:
185 1 Wolfgang Wiedmeyer
* "drm_gralloc":https://osdn.net/projects/android-x86/scm/git/external-drm_gralloc/summary - Directly uses Linux DRM for buffer allocation. Built by the Android-x86 team and now getting phased out in favor of gbm_gralloc.
186 30 dl lud
* "gbm_gralloc":https://osdn.net/projects/android-x86/scm/git/external-gbm_gralloc/summary - Uses Mesa's GBM (Generic Buffer Management) for buffer allocation through libgbm. GBM itself will then call DRM. Originally "by Rob Herring":https://github.com/robherring/gbm_gralloc, but now maintained by Android-x86.
187
* "minigbm":https://github.com/intel/minigbm - Uses its own embedded GBM implementation. Does not depend on Mesa. Built by Intel for their Android-IA project now dubbed "Project Celadon":https://github.com/projectceladon. Also used by Google "in ChromiumOS":https://chromium.googlesource.com/chromiumos/platform/minigbm/.
188
189
The table bellow summarizes the capabilities of these 3 gralloc implementations (sourced from the "slides":https://xdc2018.x.org/slides/XDC2018_Android-x86_Tech_Talk.pdf of "Mauro's Rossi talk at XDC 2018":https://www.youtube.com/watch?v=O1D_XYaxs-c):
190
191
|_. project |_. API version |_. GEM/flink names |_. PRIME fd |_. binderization |
192
|=. drm_gralloc |=. 0 |=. Yes |=. incomplete |=. No |
193
|=. gbm_gralloc |=. 0 |=. N/A |=. Yes |=. Yes |
194
|=. minigbm |=. 0, 1 |=. N/A |=. Yes |=. Yes |
195
196
Key:
197
* *project*: the implementation name.
198
* *API version*: lists the supported versions of Android's gralloc HAL.
199
* *GEM/flink names*: whether there is support for GEM (Graphics Execution Manager) object sharing through GEM names. GEM names are unique 32 bit integers, created via the flink operation, that point to a unique GEM object inside a DRM device. GEM names are mostly deprecated due to security reasons.
200 38 Denis 'GNUtoo' Carikli
* *PRIME fd*: whether there is support for buffer sharing between different DRM drivers through PRIME, the successor of GEM names. PRIME allows the conversion of local GEM handles to DMA-BUF file descriptors and vice-versa, via the DMA Buffer Sharing API.
201 44 dl lud
* *binderization*: whether there is support for using a binder, a structure that allows memory sharing between different processes on the same machine (for instance a Queue).
202
203
h2. Composer
204
205
h3. Non Android Hardware Composer HAL compatible
206 38 Denis 'GNUtoo' Carikli
* "libliftoff":https://github.com/emersion/libliftoff - Eases the use of KMS planes.
207 39 Denis 'GNUtoo' Carikli
208
h2. Wayland
209 40 Denis 'GNUtoo' Carikli
210
| Implementation | Architecture | Advantages | Disadvantages | Sustainability |
211 1 Wolfgang Wiedmeyer
| "SPURV":https://www.collabora.com/news-and-blog/blog/2019/04/01/running-android-next-to-wayland/ | Android<->Hwcomposer<->Wayland | | Probably too low in the stack, maybe should replace surfaceflinger ideally?
212
=> Do some benchmarks |