Project

General

Profile

GraphicsReplicant11 » History » Version 76

dl lud, 07/21/2020 11:01 AM
Add note about drm_hwcomposer only using 2 HW planes.

1 42 dl lud
h1. Graphics on Replicant 10
2 2 dl lud
3 41 dl lud
This page documents the current progress and future plans for the graphics acceleration on Replicant 10. The original plan can be found at [[TasksFunding#Graphics-acceleration]]. Since then, through more in depth research and hands-on experience, several things have diverged.
4 2 dl lud
5 54 dl lud
The full effort of Porting Replicant to Android 10 can be tracked at: [[PortingToAndroid10]].
6
7 2 dl lud
Background information, as well as details on the software components and acronyms used on this document, can be found at [[GraphicsResearch]].
8 3 dl lud
9
h2. Graphics stack tasks
10
11
|_. status |_. origin |_. short description |_. notes |_. estimated man-hours |_. actual man-hours |
12 6 dl lud
| done | original plan | Set up the development environment. | Required: i9305 phones, LXC Trisquel container (systemd nspawn fails due to old systemd on Trisquel 8), larger SSDs, 1.8V serial-USB adapters (BS101P FT232RL) plus makeshift resistors' banks. |>. 24 |>. 24 |
13 57 dl lud
| ongoing | new | Update development environment for Replicant 10. | Android 10 has higher resource consumption during builds.
14
"Hacks and workarounds":https://redmine.replicant.us/projects/replicant/wiki/PortingToAndroid10#Java-heap-space had to be found to be able to build in our machines.
15
Build servers will be set up to get faster builds. |>. 0 |>. 8 |
16 72 dl lud
| ongoing | original plan | Read graphics related AOSP documentation. | Never-ending task that, besides actual documentation, involves scouring through source-code, bug trackers, mailing lists and IRC logs. |>. 16 |>. 48 |
17 8 dl lud
| ongoing | new | Ask for help. | Bothering free-software developers[1] that have experience with or contribute to graphics sub-systems has been the most fruitful way to clear most roadblocks. |>. 0 |>. 0 |
18 1 dl lud
| done | original plan | Use Mesa's llvmpipe backend instead of softpipe. | Merge requests on Mesa: "!1402":https://gitlab.freedesktop.org/mesa/mesa/merge_requests/1402 and "!1403":https://gitlab.freedesktop.org/mesa/mesa/merge_requests/1403. There was no need to update LLVM version. |>. 40 |>. 28 |
19 41 dl lud
| todo | new | Find out why we are getting away without using the "libEGL patch.":https://patchwork.freedesktop.org/patch/131741/?series=17611&rev=1 | Android 10 no longer needs EGL?
20 56 dl lud
Take a look at @frameworks/native/opengl/libs/EGL/Loader.cpp@ |>. 0 |>. 0 |
21 1 dl lud
| ongoing | original plan | Implement the missing pixel formats in drm/exynos. | Joonas found that Exynos4412 allows switching between RGB and BGR formats with the VIDCON0 register. There is VIDCON0_PNRMODE_RGB and VIDCON0_PNRMODE_BGR. Both formats cannot co-exist.
22 49 dl lud
GNUtoo points out that upstreaming requires a solution that does a check like @.atomic_check(...){ if (using RGB && asked for BGR) => return NOT_POSSIBLE;@
23 1 dl lud
If application A starts using RGB, and application B asks for BGR, the kernel refuses. As if the BGR format is removed from "the list":https://git.replicant.us/contrib/replicant-next/kernel_replicant_linux/commit/?h=replicant-10&id=6f59aa92223f0f2df2ded4e60acb8c2365b0e76f at runtime. Once no more applications use the display engine, then it's like if it was re-added to the list.|>. 72 |>. 1 |
24
| todo | new | Get entire stack to use RGB555 pixel format. | Had a huge performance boost on Replicant 6. |>. 0 |>. 0 |
25 72 dl lud
| abandonned | original plan | Proper way to use DRM-Master and DRM-Auth with gbm_gralloc and drm_hwcomposer. | DRM-Auth is no longer needed for gbm_gralloc because, on December 2019, "DRM_AUTH was dropped from PRIME_TO/FROM_HANDLE ioctls":https://git.replicant.us/contrib/replicant-next/kernel_replicant_linux/commit/?h=replicant-10&id=30a958526d2cc6df38347336a602479d048d92e7. drm_hwcomposer and gbm_gralloc can now share the display/kms node with no need for DRM Auth. drm_hwcomposer, which uses KMS ioctls, must attach to the node first, in order to become DRM Master. gbm_gralloc should attach after it.
26
Before DRM_AUTH had been dropped, we had tried:
27 68 dl lud
1. "Auth hack":https://git.replicant.us/contrib/replicant-next/kernel_replicant_linux/commit/?h=history/lineage-16.0_i9300&id=e0f327c553befec2d367ac9a5ebef29b4291187a (both on @/dev/dri/card0@)
28 72 dl lud
2. vGEM (gbm gralloc on @/dev/dri/card1@) - gbm gralloc cannot take advantage of exynos hardware planes; memory may not be properly allocated.
29
3. "Allow dumb buffers on render node":https://git.replicant.us/contrib/replicant-next/kernel_replicant_linux/commit/?id=805cc511ee2adc62ae40c4253fb8d378c3710a81 (gbm gralloc on @/dev/dri/renderD128@) - Dumb buffers are used for scanout. Should not be created on a render node. |>. 40 |>. 8 |
30
| ongoing | new | Ensure initialization order for drm_hwcomposer and gbm_gralloc services. | gbm_gralloc must be started after drm_hwcomposer as explained above.
31
"Android init":https://android.googlesource.com/platform/system/core/+/master/init/ is quite primitive, but should be possible to accomplish this with its "triggers":https://android.googlesource.com/platform/system/core/+/master/init/#triggers. |>. 0 |>. 2 |
32 67 dl lud
| ongoing | new | Use hardware planes for better composer performance. | Enabling HW planes with drm_hwcomposer was straightforward but led to severe graphics corruption.
33 9 dl lud
"Disabling devfreq":https://git.replicant.us/contrib/replicant-next/kernel_replicant_linux/commit/?id=90ceab148872e9515522c5150df16722822cf9d5 fixed the corruption. Tentative explanation: display controller frequency gets too low for timely DMA transfer of overlays. "Reported upstream.":https://www.spinics.net/lists/linux-samsung-soc/msg66752.html
34 76 dl lud
TODO: lock display controller frequencies through sysfs and re-enable devfreq, or try to "remove low freq OPP steps":https://github.com/ReplicantOS-midas/android_kernel_samsung_smdk4412/commit/ce08237e39e2d74c1a25b3eadde4ac0f1043f270.
35
Joonas found out that drm_hwcomposer is just using 2 HW planes, when exynos has at least 4 available (1 primary, 3 overlay). Double-check and try to fix. |>. 0 |>. 16 |
36 40 dl lud
| ongoing | new | Use Skia instead of HWUI to render the Canvas. | Unlike Replicant 6, none of the usual system props (e.g. @ro.kernel.qemu=1@, @ro.config.avoid gfx accel=1@) would yield the expected performance.
37 10 dl lud
Got there by forcing "@hardwareAccelerated=false@":https://git.replicant.us/contrib/replicant-next/frameworks_base/commit/?h=replicant-9&id=da9bab529acdb0bf5a0cbf3ac1962ca57fb6b5d7 on all apps.
38 56 dl lud
TODO: turn this dirty hack into a system property that can be toggled on the device tree. |>. 0 |>. 22 |
39 1 dl lud
| ongoing | original plan | Create test scenarios and check if the graphics stack works as expected. | Stock apps work.
40
Check [[GraphicsReplicant10#Tested-apps|Tested apps]] bellow for the current status with apps that require advanced graphics features.
41
TODO: compliance tests. |>. 40 |>. 14 |
42
| ongoing | original plan | Make the graphics stack work with vGEM driver besides drm/exynos. | vGEM seem to be the proper dri node for Mesa's "kms_swrast driver":https://memcpy.io/kms_swrast-a-hardware-backed-graphics-driver.html
43
We are currently using a "simple hack":https://git.replicant.us/contrib/replicant-next/external_mesa3d/commit/?id=e0a5e6c056c8281712bf6366acfe876215992613 that kms_swrast to use drm/exynos instead. Should rather use "vGEM":https://git.replicant.us/contrib/replicant-next/kernel_replicant_linux/commit/?id=29fab049bea6a17e5f85ccb147df0e0046bbd487. |>. 40 |>. 4 |
44 73 dl lud
| tentative | new | Combine "kmsro":https://gitlab.freedesktop.org/mesa/mesa/-/blob/master/src/gallium/winsys/kmsro/drm/kmsro_drm_winsys.c with kms_swrast on vGEM render node? | Is kms_swrast working on top of the vGEM render node able to share PRIME buffers with the display node (Exynos)? If not, would adding kmsro to the mix help? 
45
Architectural ideia: @kmsro + kms_swrast on vGEM render node -> PRIME -> drm_hwcomposer and gbm_gralloc on display node (Exynos)@
46
Advantages:
47
- no need to copy buffers between kms_swrast and Exynos (PRIME takes care of that);
48
- can take advantage of HW planes. |>. 0 |>. 0 |
49 72 dl lud
| ongoing | original plan | Document the design decisions. | Done at this wiki page plus [[GraphicsResearch]] and the presentation at [[ContributorsMeetingJuly2019#Presentations]]. |>. 16 |>. 64 |
50 29 dl lud
| ongoing | new | Try out the "Android Go low RAM switches":https://redmine.replicant.us/projects/replicant/wiki/PortingToAndroid9#Links and check their impact on graphics rendering performance and overall system usability. | |>. 0 |>. 1 |
51 75 dl lud
|\4>. total sum: |>. 288 |>. 240 |
52 1 dl lud
53 4 dl lud
fn1. A big thanks to Joonas Kylmälä, Paul Kocialkowski, Andrés Domínguez, Denis Carikli, Mauro Rossi, Emil Velikov, Andrzej Hajda, Marek Szyprowski and LiquidAcid.
54 15 dl lud
55
h2. SwiftShader tasks
56
57
|_. status |_. origin |_. short description |_. notes |_. estimated man-hours |_. actual man-hours |
58 56 dl lud
| done | original plan | Find a way to use SwiftShader instead of Mesa. | Joonas got there [[PortingToAndroid9#Using-SwiftShader-instead-of-Mesa3D-llvmpipe-for-software-rendering|with ranchu composer (from Android Emulator) and the default gralloc]], plus a "patch to support UDIV/SDIV emulation in the kernel":https://git.replicant.us/contrib/replicant-next/kernel_replicant_linux/commit/?h=udiv-emulation&id=c5954294d3935774ef25375c2783bd3afa60e421. |>. 40 |>. 0 |
59 17 dl lud
| done | new | Use LLVM as backend instead of SubZero. | Found a "SwiftShader revision":https://swiftshader.googlesource.com/SwiftShader/+/fde88d96a58b92beab76035393b3acd849445160 that uses LLVM and is still compatible with Android 9 frameworks/native. No noticeable performance difference. |>. 0 |>. 6 |
60 55 dl lud
| done | new | Do UDIV/SDIV emulation on JIT compiled shader code instead of kernel patch. Avoids performance penalty of interrupt handling. | It seems that "SwiftShader does not send the processor model (microarchitecture) to LLVM":https://swiftshader.googlesource.com/SwiftShader/+/fde88d96a58b92beab76035393b3acd849445160/src/Reactor/LLVMReactor.cpp#596, leaving it without a way to decide "whether the processor has hardware division":https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/divide-and-conquer.
61
Fixed upon update to Replicant 10. SwiftShader now uses LLVM 7 instead of 3, which fixed this.  |>. 0 |>. 30 |
62 1 dl lud
| todo | new | Use drm_hwcomposer instead of ranchu. Advantages: uses hardware planes and DRM nodes instead of direct framebuffer. | "Joonas was close":https://people.freedesktop.org/~cbrill/dri-log/?channel=dri-devel&date=2018-08-03. |>. 0 |>. 0 |
63 55 dl lud
| todo | new | Use mainline SwiftShader. Brings in a Vulkan software renderer for Replicant. | Difficult to due incompatibilities with "frameworks/native":https://android.googlesource.com/platform/frameworks/native/+/408eda0002ed37a2d30a3dddea6466dfc5c288e7.
64
Check if is fixed with Replicant 10. |>. 0 |>. 0 |
65 35 dl lud
|\4>. total sum: |>. 40 |>. 36 |
66 20 dl lud
67
h2. llvmpipe optimization tasks
68
69
|_. status |_. origin |_. short description |_. notes |_. estimated man-hours |_. actual man-hours |
70
| todo | original plan | Setup a testing and benchmarking environment. | Profiling: turn on profiling switch on Mesa + simpleperf?
71 27 dl lud
Benchmarks: "android-fps-count":https://github.com/romannurik/env/blob/master/bin/android-fps-count, "0xBenchmark":https://f-droid.org/wiki/page/org.zeroxlab.zeroxbenchmark, "GearsES2":https://f-droid.org/wiki/page/com.jeffboody.GearsES2eclair
72 45 dl lud
Conformance: "dEQP":https://android.googlesource.com/platform/external/deqp, "Android CTS":https://source.android.com/compatibility/cts/, "piglit":https://gitlab.freedesktop.org/mesa/piglit/tree/master, "freedreno/tests-*":https://github.com/freedreno/freedreno, "glmark2":https://github.com/glmark2/glmark2 |>. 40 |>. 1 |
73 21 dl lud
| todo | original plan | Disable expensive OpenGL operations. | |>. 24 |>. 0 |
74
| todo | original plan | Recap matrix operations and study ARM NEON. | |>. 48 |>. 0 |
75
| todo | original plan | Profile apps to find the most used GLES operations. | |>. 32 |>. 0 |
76
| todo | original plan | Use "Ne10 library":https://github.com/projectNe10/Ne10 or "Neon Intrinsics":https://developer.arm.com/architectures/instruction-sets/simd-isas/neon/intrinsics for the most used GLES operations. | Optimizations have to be done on LLVM and not on llvmpipe. llvmpipe only outputs LLVM IR. LLVM already has autovectorization for ARM NEON, try it. |>. 80 |>. 0 |
77
| todo | original plan | Fix bugs, re-write the code where needed, get it stable. | |>. 80 |>. 0 |
78 35 dl lud
|\4>. total sum: |>. 304 |>. 1 |
79 22 dl lud
80
h2. Lima driver tasks
81 23 dl lud
82 22 dl lud
|_. status |_. origin |_. short description |_. notes |_. estimated man-hours |_. actual man-hours |
83 41 dl lud
| done | original plan | Rebase "Lima's Linux kernel DRM driver":https://gitlab.freedesktop.org/lima/linux on top of "forkbomb's Midas on Mainline kernel":https://blog.forkwhiletrue.me/pages/midas-mainline/. | Done by others. Lima DRM driver was accepted into mainline Linux, which also has forkbomb's patches and is now used on Replicant 10. |>. 80 |>. 0 |
84 39 Kurtis Hanna
| done | original plan | Replace mainline Mesa for "Lima's Mesa":https://gitlab.freedesktop.org/lima/mesa (with their driver). | Done by others. Lima is now on mainline Mesa. "Lima wiki":https://gitlab.freedesktop.org/lima/web/-/wikis/home |>. 16 |>. 0 |
85 43 dl lud
| done | new | Lima DRM driver bringup on Exynos. | Lima development is done on AllWinner devices.
86 44 dl lud
We expected "some issues to get it working on Exynos":https://forum.odroid.com/viewtopic.php?p=267259#p264856.
87
Although there were encouraging reports by "ChronoMonochrome":https://github.com/CustomROMs/android_local_manifests_i9300/issues/1#issuecomment-524375690, "hexdump0815":https://github.com/hexdump0815/linux-mainline-and-mali-on-odroid-u3 and Viciouss ("manifest":https://github.com/Viciouss/manifest_aosp_n80xx, "xda":https://forum.xda-developers.com/galaxy-note-10-1/general/mainline-n8000-progress-t3964980).
88
Joonas added Lima to Replicant 10 and faced no major bringup issues. |>. 0 |>. 1 |
89 74 dl lud
| todo | new | Fully test proper architecture. | @drm_hwcomposer and gbm_gralloc on card0 (Exynos) -> PRIME -> Mesa on renderD129 (Lima)@
90 1 dl lud
Advantages:
91
- no need to copy buffers between Lima and Exynos (PRIME takes care of that);
92 50 dl lud
- can take advantage of HW planes. |>. 0 |>. 0 |
93 65 dl lud
| ongoing | new | Fix graphics corruption with hardware planes. | Buffer sharing is broken when hardware planes are turned on at drm_hwcomposer.
94 69 dl lud
Corruption happens when compositing GL planes with non-GL planes. E.g. try "Shader Editor":https://f-droid.org/en/packages/de.markusfisch.android.shadereditor/, run a shader and open a menu.
95 1 dl lud
Does the hardware support GPU and hardware planes at the same time? Stock kernel used the GPU with a single plane.
96 74 dl lud
Disabling devfreq didn't solve it (as it did with llvmpipe).
97
Probably was due to having gbm_gralloc working on Lima's render node, which cannot do CMA. |>. 0 |>. 1 |
98 1 dl lud
| todo | new | Advertise GLES 2. "Shader Editor":https://f-droid.org/en/packages/de.markusfisch.android.shadereditor/ can only detect GLES 1. | |>. 0 |>. 0 |
99
| todo | original plan | Build and test thoroughly with "synthetic":https://source.android.com/devices/graphics/testing and real applications. | Use conformance tests to figure out the "current GLES implementation status":https://gitlab.freedesktop.org/lima/mesa/issues/102. |>. 40 |>. 0 |
100 24 dl lud
| abandoned | original plan | Create a fallback mechanism that uses the software renderer for GLES functions not yet implemented in Lima. | There is no sane way to switch between different GLES drivers at the function level. Abandoned in favour of the tasks bellow. |>. 100 |>. 1 |
101 74 dl lud
| done | new | Lima as SurfaceFlinger backend. | This is the default (SurfaceFlinger using the default GLES implementation). No problems found. |>. 0 |>. 0 |
102
| done | new | Lima as HWUI (SkiaGL) backend. | This is the default (SkiaGL using the default GLES implementation). No problems found. |>. 0 |>. 0 |
103
| todo | new | Lima on a per-app basis. | Lima will at most support GLES2. Therefore it may not work with certain apps depending on their GLES usage. We can re-work the "per process libagl/llvmpipe patch":https://lists.osuosl.org/pipermail/replicant/2019-August/002054.html into a patch that switches between Lima and a software renderer (llvmpipe or SwiftShader). |>. 0 |>. 0 |
104 53 dl lud
|\4>. total sum: |>. 236 |>. 3 |
105 30 dl lud
106
h2. 2D optimization tasks
107
108
|_. status |_. origin |_. short description |_. notes |_. estimated man-hours |_. actual man-hours |
109
| todo | new | Investigate the possibility of using "Pixman":http://pixman.org or "Exynos G2D":http://linux-exynos.org/wiki/G2D as RenderEngine for SurfaceFlinger. | There are interesting reports of people using "G2D to hardware-accelerate X11 EXA":https://forum.odroid.com/viewtopic.php?f=81&t=21035 |>. 0 |>. 1 |
110 32 dl lud
| todo | new | Accelerate Skia with G2D. | Rework old patches ("Hillenbrand 2013":https://github.com/CyanogenMod/android_external_skia/commit/647876b665f2cf011e75adc6ff2238d467c47635 and "raymanfx 2016":https://review.lineageos.org/c/LineageOS/android_external_skia/+/61162 ) to make them work on current Skia. |>. 0 |>. 1 |
111 35 dl lud
|\4>. total sum: |>. 0 |>. 2 |
112 58 dl lud
113
h2. Tested apps
114
115
|_/2. app |_\2. Replicant 6 |_\3. Replicant 10 |_/2. notes |
116
|_. libagl |_. LLVMpipe |_. Lima |_. LLVMpipe |_. SwiftShader  |
117 62 dl lud
| "Fennec F-Droid":https://f-droid.org/en/packages/org.mozilla.fennec_fdroid | crashes | slow | fast | | | Needs GLES 2.0 |
118 60 dl lud
| "LibreOffice Viewer":https://f-droid.org/en/packages/org.documentfoundation.libreoffice | crashes | slow |\3. cannot test (missing storage) | |
119
| "Red Reader":https://f-droid.org/en/packages/org.quantumbadger.redreader | "crashes":https://github.com/QuantumBadger/RedReader/issues/279 | usable |\3. cannot test (no network) | |
120 64 dl lud
| "Shader Editor":https://f-droid.org/en/packages/de.markusfisch.android.shadereditor | crashes | 7 fps | 30 fps (freezes when changing resolution) | | | FPS measured on default shader with 1/1 resolution. |
121 58 dl lud
| "Marine Compass":https://f-droid.org/en/packages/net.pierrox.mcompass | bad render | bad render | crashes | | | Only uses GLES 1.0 |
122
| "Gears":https://f-droid.org/en/packages/com.jeffboody.GearsES2eclair | crashes | no render | crashes | | | |
123 59 dl lud
| "GL TRON":https://f-droid.org/en/packages/com.glTron | 4 fps | 2 fps | 23 fps | | | Has a nice FPS counter. |