Project

General

Profile

GraphicsReplicant9 » History » Revision 29

Revision 28 (dl lud, 09/05/2019 09:01 PM) → Revision 29/35 (dl lud, 09/05/2019 09:32 PM)

h1. Graphics on Replicant 9

This page documents the current progress and future plans for the graphics acceleration on Replicant 9. The original plan can be found at [[TasksFunding#Graphics-acceleration]]. Since then, through more in depth research and hands-on experience, several things have diverged.

Background information, as well as details on the software components and acronyms used on this document, can be found at [[GraphicsResearch]].



h2. Graphics stack tasks

|_. status |_. origin |_. short description |_. notes |_. estimated man-hours |_. actual man-hours |
| done | original plan | Set up the development environment. | Required: i9305 phones, LXC Trisquel container (systemd nspawn fails due to old systemd on Trisquel 8), larger SSDs, 1.8V serial-USB adapters (BS101P FT232RL) plus makeshift resistors' banks. |>. 24 |>. 24 |
| ongoing | original plan | Read graphics related AOSP documentation. | Never-ending task that, besides actual documentation, involves scouring through source-code, bug trackers, mailing lists and IRC logs. |>. 16 |>. 40 |
| ongoing | new | Ask for help. | Bothering free-software developers[1] that have experience with or contribute to graphics sub-systems has been the most fruitful way to clear most roadblocks. |>. 0 |>. 0 |
| done | original plan | Use Mesa's llvmpipe backend instead of softpipe. | Merge requests on Mesa: "!1402":https://gitlab.freedesktop.org/mesa/mesa/merge_requests/1402 and "!1403":https://gitlab.freedesktop.org/mesa/mesa/merge_requests/1403. There was no need to update LLVM version. |>. 40 |>. 28 |
| ongoing | new | Use Mesa's kms_swrast driver with drm/exynos. | Solved with a "simple hack":https://git.replicant.us/contrib/replicant-9/external_mesa3d/commit/?id=e0a5e6c056c8281712bf6366acfe876215992613.
TODO: make it more generic and send upstream.
Perhaps have the if clause read a list of additional drivers that would be supplied at compile time. |>. 0 |>. 4 |
| todo | original plan | Implement the missing pixel formats in drm/exynos. | |>. 72 |>. 0 |
| todo | new | Get entire stack to use RGB555 pixel format. | Had a huge performance boost on Replicant 6. |>. 0 |>. 0 |
| ongoing | original plan | Proper way to use DRM-Master and DRM-Auth with gbm_gralloc and drm_hwcomposer. | Tried:
1. "Auth hack":https://git.replicant.us/contrib/GNUtoo/kernel_replicant_linux/commit/?h=history/lineage-16.0_i9300&id=e0f327c553befec2d367ac9a5ebef29b4291187a (both on @/dev/dri/card0@)
2. "vGEM":https://git.replicant.us/contrib/replicant-9/kernel_replicant_linux/commit/?id=29fab049bea6a17e5f85ccb147df0e0046bbd487 (gbm gralloc on @/dev/dri/card1@) - Requires memory sync between vGEM and exynos. gbm gralloc cannot take advantage of exynos hardware planes.
3. "Allow dumb buffers on render node":https://git.replicant.us/contrib/replicant-9/kernel_replicant_linux/commit/?id=805cc511ee2adc62ae40c4253fb8d378c3710a81 (gbm gralloc on @/dev/dri/renderD128@) - Dumb buffers are used for scanout. Should not be created on a render node.
 
Correct solution is using DRM Magic and have both on @/dev/dri/card0@. |>. 40 |>. 4 |
| ongoing | new | Use hardware planes for better composer performance. | Enabling HW planes with drm_hwcomposer was straightforward but led to severe graphics corruption.
"Disabling devfreq":https://git.replicant.us/contrib/replicant-9/kernel_replicant_linux/commit/?id=90ceab148872e9515522c5150df16722822cf9d5 fixed the corruption. Tentative explanation: display controller frequency gets too low for timely DMA transfer of overlays.
"Reported upstream.":https://www.spinics.net/lists/linux-samsung-soc/msg66752.html TODO: lock display controller frequencies through sysfs and re-enable devfreq. |>. 0 |>. 16 |
| ongoing | new | Use Skia instead of HWUI to render the Canvas. | Unlike Replicant 6, none of the usual system props (e.g. @ro.kernel.qemu=1@, @ro.config.avoid gfx accel=1@) would yield the expected performance.
Got there by forcing "@hardwareAccelerated=false@":https://git.replicant.us/contrib/replicant-9/frameworks_base/commit/?h=replicant-9&id=da9bab529acdb0bf5a0cbf3ac1962ca57fb6b5d7 on all apps.
TODO: turn this dirty hack into a system property that can be toggled on the device tree. |>. 0 |>. 22 |
| ongoing | original plan | Create test scenarios and check if the graphics stack works as expected. | Testing has been done manually with the default apps. TODO: use apps that specifically require GLES 2.0 (e.g. Firefox-based browsers). |>. 40 |>. 10 |
| abandoned | original plan | Make the graphics stack work with vGEM driver besides drm/exynos. | No need. All Replicant targets have a display and, likely, a companion display controller with a fitting drm driver too. Most drm drivers expose a render node that Mesa can use. Better use that instead of vGEM. |>. 40 |>. 0 |
| ongoing | original plan | Document the design decisions. | Done at this wiki page plus the presentation at [[ContributorsMeetingJuly2019#Presentations]]. |>. 16 |>. 58 |
| ongoing | new | Try out the "Android Go low RAM switches":https://redmine.replicant.us/projects/replicant/wiki/PortingToAndroid9#Links and check their impact on graphics rendering performance and overall system usability. | |>. 0 |>. 1
|

fn1. A big thanks to Joonas Kylmälä, Paul Kocialkowski, Andrés Domínguez, Denis Carikli, Mauro Rossi, Emil Velikov, Andrzej Hajda, Marek Szyprowski and LiquidAcid.

h2. SwiftShader tasks

|_. status |_. origin |_. short description |_. notes |_. estimated man-hours |_. actual man-hours |
| done | original plan | Find a way to use SwiftShader instead of Mesa. | Joonas got there [[PortingToAndroid9#Using-SwiftShader-instead-of-Mesa3D-llvmpipe-for-software-rendering|with ranchu composer (from Android Emulator) and the default gralloc]], plus a "patch to support UDIV/SDIV emulation in the kernel":https://git.replicant.us/contrib/replicant-9/kernel_replicant_linux/commit/?h=udiv-emulation&id=c5954294d3935774ef25375c2783bd3afa60e421. |>. 40 |>. 0 |
| done | new | Use LLVM as backend instead of SubZero. | Found a "SwiftShader revision":https://swiftshader.googlesource.com/SwiftShader/+/fde88d96a58b92beab76035393b3acd849445160 that uses LLVM and is still compatible with Android 9 frameworks/native. No noticeable performance difference. |>. 0 |>. 6 |
| ongoing | new | Do UDIV/SDIV emulation on JIT compiled shader code instead of kernel patch. Avoids performance penalty of interrupt handling. | It seems that "SwiftShader does not send the processor model (microarchitecture) to LLVM":https://swiftshader.googlesource.com/SwiftShader/+/fde88d96a58b92beab76035393b3acd849445160/src/Reactor/LLVMReactor.cpp#596, leaving it without a way to decide "whether the processor has hardware division":https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/divide-and-conquer. |>. 0 |>. 30 |
| todo | new | Use drm_hwcomposer instead of ranchu. Advantages: uses hardware planes and DRM nodes instead of direct framebuffer. | |>. 0 |>. 0 |
| todo | new | Use mainline SwiftShader. Brings in a Vulkan software renderer for Replicant. | Difficult to due incompatibilities with "frameworks/native":https://android.googlesource.com/platform/frameworks/native/+/408eda0002ed37a2d30a3dddea6466dfc5c288e7. |>. 0 |>. 0 |

h2. llvmpipe optimization tasks

|_. status |_. origin |_. short description |_. notes |_. estimated man-hours |_. actual man-hours |
| todo | original plan | Setup a testing and benchmarking environment. | Profiling: turn on profiling switch on Mesa + simpleperf?
Benchmarks: "android-fps-count":https://github.com/romannurik/env/blob/master/bin/android-fps-count, "0xBenchmark":https://f-droid.org/wiki/page/org.zeroxlab.zeroxbenchmark, "GearsES2":https://f-droid.org/wiki/page/com.jeffboody.GearsES2eclair
Conformance: "dEQP":https://android.googlesource.com/platform/external/deqp, "Android CTS":https://source.android.com/compatibility/cts/, "piglit":https://gitlab.freedesktop.org/mesa/piglit/tree/master, "freedreno/tests-*":https://github.com/freedreno/freedreno |>. 40 |>. 1 |
| todo | original plan | Disable expensive OpenGL operations. | |>. 24 |>. 0 |
| todo | original plan | Recap matrix operations and study ARM NEON. | |>. 48 |>. 0 |
| todo | original plan | Profile apps to find the most used GLES operations. | |>. 32 |>. 0 |
| todo | original plan | Use "Ne10 library":https://github.com/projectNe10/Ne10 or "Neon Intrinsics":https://developer.arm.com/architectures/instruction-sets/simd-isas/neon/intrinsics for the most used GLES operations. | Optimizations have to be done on LLVM and not on llvmpipe. llvmpipe only outputs LLVM IR. LLVM already has autovectorization for ARM NEON, try it. |>. 80 |>. 0 |
| todo | original plan | Fix bugs, re-write the code where needed, get it stable. | |>. 80 |>. 0 |

h2. Lima driver tasks

|_. status |_. origin |_. short description |_. notes |_. estimated man-hours |_. actual man-hours |
| done | original plan | Rebase "Lima's Linux kernel DRM driver":https://gitlab.freedesktop.org/lima/linux on top of "forkbomb's Midas on Mainline kernel":https://blog.forkwhiletrue.me/pages/midas-mainline/. | Done by others. Lima DRM driver was accepted into mainline Linux, which also has forkbomb's patches and is now used on Replicant 9. |>. 80 |>. 0 |
| done | original plan | Replace mainline Mesa for "Lima's Mesa":https://gitlab.freedesktop.org/lima/mesa (with their driver). | Done by others. Lima is now on mainline Mesa. |>. 16 |>. 0 |
| todo | new | Lima DRM driver bringup on Exynos. | Lima development is done on AllWinner devices.
We will "face some issues to get it working on Exynos":https://forum.odroid.com/viewtopic.php?p=267259#p264856.
There are some encouraging reports by "ChronoMonochrome":https://github.com/CustomROMs/android_local_manifests_i9300/issues/1#issuecomment-524375690. |>. 0 |>. 1 |
| todo | original plan | Build and test thoroughly with "synthetic":https://source.android.com/devices/graphics/testing and real applications. | Use conformance tests to figure out the "current GLES implementation status":https://gitlab.freedesktop.org/lima/mesa/issues/102. |>. 40 |>. 0 |
| abandoned | original plan | Create a fallback mechanism that uses the software renderer for GLES functions not yet implemented in Lima. | There is no sane way to switch between different GLES drivers at the function level. Abandoned in favour of the tasks bellow. |>. 100 |>. 1 |
| todo | new | Lima as SurfaceFlinger backend. | Even if not GLES 1 or 2 feature complete, Lima should already be usable as backend renderer for SurfaceFlinger. |>. 0 |>. 0 |
| todo | new | Lima as HWUI (SkiaGL) backend. | Even if not GLES 1 or 2 feature complete, Lima should already be usable as backend renderer for HWUI. |>. 0 |>. 0 |
| todo | new | Lima on a per-app basis. | Current state of Lima may allow it to work with certain apps depending on their GLES usage. We can re-work the "per process libagl/llvmpipe patch":https://lists.osuosl.org/pipermail/replicant/2019-August/002054.html into a patch that switches between Lima and a software renderer (llvmpipe or SwiftShader). |>. 0 |>. 0 |