Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard
4 likes7,773 views
Every modern multimedia-oriented ARM SoC usually has a number of display controllers, to drive a screen or an LCD panel, and a GPU, to provide 3D acceleration. The Linux kernel framework of choice to support these controllers is the DRM subsystem. This talk will walk through the DRM stack, the architecture of a DRM/KMS driver and the interaction between the display and GPU drivers. The presentation is based on the work we have done to develop a DRM driver for the Allwinner SoCs display controller with multiple outputs, such as parallel display interfaces, HDMI or MIPI-DSI. The work done to make the ARM Mali OpenGL driver work on top of a mainline DRM/KMS driver will also be detailed, as well as the more traditional, Mesa-based, solution used in a variety of other platforms. Maxime Ripard, Free Electrons
Ad
More Related Content
What's hot (20)
Similar to Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard (20)
Ad
More from Anne Nicolas (20)
Ad
Recently uploaded (20)
Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard
- 1. Kernel Recipes 2017 An introduction to the Linux DRM subsystem Maxime Ripard Free Electrons [email protected] c Copyright 2004-2017, Free Electrons. Creative Commons BY-SA 3.0 license. Corrections, suggestions, contributions and translations are welcome! Embedded Linux Developers Free Electrons Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 1/1
- 2. Maxime Ripard Embedded Linux engineer and trainer at Free Electrons Embedded Linux development: kernel and driver development, system integration, boot time and power consumption optimization, consulting, etc. Embedded Linux training, Linux driver development training and Android system development training, with materials freely available under a Creative Commons license. http://free-electrons.com Contributions Co-maintainer for the sunXi SoCs from Allwinner Contributor to a couple of other open-source projects, Buildroot, U-Boot, Barebox Living in Toulouse, south west of France Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 2/1
- 3. An introduction to the Linux DRM subsystem Introduction Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 3/1
- 4. A long long time ago, in a galaxy (not so) far, far away Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 4/1
- 5. Framebuffers Display hardware was dead simple... .. and so was the API to drive it. Introducing... fbdev! Allows for three things: Mode-Setting Accessing the (only) buffer Optional 2d acceleration: draw, copy, etc. And access to the device registers... Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 5/1
- 6. That 90’s show Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 6/1
- 7. Back to the future Two different trends Embedded devices starting to show up, with their low power needs ⇒ Display engines need to accelerate more things Desktop displays getting more and more fancy in order to play Quake in 4k VR ⇒ Bigger and bigger GPUs Led to two different outcomes: Interface to drive GPU devices through the kernel: DRM Hacks piling on in order to fit embedded use-cases: omapdss, pxafb Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 7/1
- 8. Composition Evolved Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 8/1
- 9. The droid you’re looking at Sean Paul and Zach Reizner - Google - XDC2016 Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 9/1
- 10. Can I talk to the manager? DRM was initially introduced to deal with the GPU’s need Initialize the card, load its firmware, etc. Share the GPU command queue between multiple applications Manage the memory (allocation, access) But not modesetting! All the modesetting was in the userspace, especially in X Race between rendering and modesetting Only one graphical application that needed to remain there all the time (Lack of) Abstraction! Introduction of the Kernel Mode-Setting (KMS) to move the modesetting back into the kernel Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 10/1
- 11. Kill it with fire! Now, fbdev could be implemented on top of KMS... ... Or removed entirely Call for deprecation in 2012 (Hi Laurent!) Last fbdev driver merged in 2014 First ARM DRM driver: exynos in 2011 Followed: arm, armada, atmel-hclcdc, fsl-dcu, hisilicon, imx, mediatek, meson, msm, mxsfb, omapdrm, pl111, rcar-du, rockchip, shmobile, sti, stm, sun4i, tegra, tve200, etc... Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 11/1
- 12. Two nodes to go please Initially, DRM was created for devices that were both displaying and rendering (your traditionnal PC graphics card). On embedded devices, it’s never really been like that the GPU is discrete and comes from a third party the display engine is usually designed by the SoC vendor DRM and KMS APIs requiring the same level of privilege, with one master, and were both exposed on the same device file Creation of render nodes Also useful for things like GPGPU, off-screen rendering, more flexible access control Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 12/1
- 13. An introduction to the Linux DRM subsystem DRM/KMS Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 13/1
- 14. The pixels must flow Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 14/1
- 15. KMS components Planes Image source Associated with one (or more!) framebuffers Holds a resized / cropped version of that framebuffer Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 15/1
- 16. Page flipping Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 16/1
- 17. KMS components Planes Image source Associated with one (or more!) framebuffers Holds a resized / cropped version of that framebuffer CRTCs Take the planes, and does the composition Contains the display mode and parameters Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 17/1
- 18. KMS components Planes Image source Associated with one (or more!) framebuffers Holds a resized / cropped version of that framebuffer CRTCs Take the planes, and does the composition Contains the display mode and parameters Encoders Take the raw data from the CRTC and convert it to a particular format Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 17/1
- 19. KMS components Planes Image source Associated with one (or more!) framebuffers Holds a resized / cropped version of that framebuffer CRTCs Take the planes, and does the composition Contains the display mode and parameters Encoders Take the raw data from the CRTC and convert it to a particular format Connectors Outputs the encoded data to an external display Handles hotplug events Reads EDIDs Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 17/1
- 20. Allwinner display pipeline Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 18/1
- 21. DRM vs SoC pipeline Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 19/1
- 22. DRM Stack: KMS Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 20/1
- 23. DRM Stack: GEM Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 21/1
- 24. DRM Stack: CMA Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 22/1
- 25. DRM Stack: PRIME Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 23/1
- 26. DRM Stack: GPUs Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 24/1
- 27. An introduction to the Linux DRM subsystem Vendor solutions... Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 25/1
- 28. Solutions The GPU found in most Allwinner SoCs is the Mali-400 from ARM (with a variable number of cores) There are two options to support that GPU: Lima Reversed engineered proof-of-concept Triggered the reverse engineering effort of the GPUs (freedreno, etnaviv, etc.) Development (close to?) stopped three years ago, and then resumed a couple of monthes ago ARM-Provided support Featureful Two parts: GPL kernel driver and proprietary OpenGL ES implementation Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 26/1
- 29. DRM Stack: GPU Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 27/1
- 30. Development Everything is provided by ARM on their website (if you’re lucky) On the userspace side, you just need to put the library they provided on your system On the driver side, you need to create a platform glue that will deal with: Memory mapping Interrupts Clocks Reset lines Power Domains Basically everything needed for the GPU to operate properly on your SoC Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 28/1
- 31. X11 integration We need a DDX (Device Dependent X) driver xf86-video-modesetting is working on top of KMS and GBM (MESA-defined user-space API to allocate buffers) ARM developped xf86-video-armsoc for SoC using a 3rd party GPU (Mali, PowerVR, Vivante, etc.) Relies on KMS for the display configuration, driver-specific ioctl for buffer allocations and vendor-provided OpenGL ES implementation Just have to write a small glue to use your driver allocator, and give some hints to X about what your hardware support (hw cursor, vblank, etc.) Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 29/1
- 32. Questions? Maxime Ripard [email protected] Slides under CC-BY-SA 3.0 http://free-electrons.com/pub/conferences/2017/kr/ripard-drm Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 30/1