# XDC 2020 Allocation Constraints Workshop Talk slides: https://xdc2020.x.org/event/9/contributions/615/attachments/704/1301/XDC_2020__Allocation_Constraints.pdf Attendees (Add yourself): * James Jones (NVIDIA) jajones 'at' nvidia.com * Simon Ser contact@emersion.fr * Daniel Stone (Collabora) daniels@collabora.com * John Reitan (Arm) john.reitan 'at' arm.com * Liviu Dudau (Arm) * Laurent Pinchart (Ideas on Board) * Bas Nieuwenhuizen (Google) Ideas: * Feedback on high level design? * Usage described via existing API mechanisms * Query constraints from existing APIs for a given usage * One constraint set for each modifier/layout * Merge constraint sets via common logic/library/shared code * Provide constraint set to allocation API of choice * For wayland use mmap FD to share constraint lists. To that end, add serialize/unserialize. To that end, add serialize/unserialize. * Already have serialize/deserialize. There is no "non-serialized" view ATM in fact. * For EGL query, how is texture/render specified? * external_only flag a rough proxy for this. Corresponds to OES external textures * Do we need constraint per modifier? Too much? * Seems like this should be fine. * Common code to merge constraints good? * No new objections. * Allocators? * GBM + Vulkan OK? * What if we want camera buffers, for example? GBM not a good fit * Design can work with any allocator. A new unified allocator could take constraint sets as input. * GBM often lowest common denominator for shared buffers. Graphics often involved * However, consider encode. GBM not a good fit here. * Would that proposal work fine with your hw? * Luma/Chroma planes in two different banks * Vivante needs shadow buffers with blits between them to handle layout transitions * This is not an attempt to get an optimal, global setup -- it's local * Have used extra pixels in image to include meta-data. Did not see good way to express in modifiers * Could express that in a constraint, but would not notify users of their presence. * Rely on dma-buf/buffer size to find extra bytes? Or pad stride/pitch? Not fully solved by constraints themselves. * Where does the constraint merging library go? * Current proposal: Put it in a uapi header as header-only/inlined library * No way to add "private" functions/structs. Everything is public. * Various versions get inlined * Other ideas: * Add a helper syscall in DRM. Might be heavyweight. * Could put it in the vdso. Benefit: Keeps in sync across driver components on a given kernel * Is a general mechanism in kernel, needs discussion. Code, but not much info * Still need uapi header for interface and constraint def, but lib would be shared code. * Need to justify inclusion in the kernel * Kernel needs to generate constraint sets (KMS) * Kernel needs to parse at allocation step * XXX Depends on whether kernel takes constraint sets as parameter, or derived parameters * Would mean extending/changing many existing driver ioctls * Tradeoff of duplicating all parameters into each allocation ioctl, Vs a unified ioctl or at least unified type of parameters * May be difficult to answer this question (constraint merging code home) without stepping into that debate * Historically hasn't been much love for a unified allocation mechanism/parameter namespace. May be stuck with driver-specific or "the unified one" and picking between the two as needed. * (some agreement here) * If it remains in userspace-only, should we use a shared library instead? * Yes, definitely. However, may want to keep option to share with kernel open * Would be easy to [start with shared library], transition to kernel later * If a shared lib, at least headers defining constraints need to be kept in sync with kernel releases. * Only an issue if this changes a lot. Hopefully won't long-term * Should it be a whole new library, or is libdrm acceptable? OK with camera people? * Might be problematic if required by default, but probably OK for interaction with graphics (had trouble hearing this) * Problematic if code is copied rather than shared? Two versions resulting in different buffers? * Versioning addresses this to some extent * Considering case of shipping fixed BSP/system thoguh, then updating kernel. Would be nice if new constraints got picked up automatically. * Does this require userspace updates anyway to use them though? * Only if userspace is generating constraints itself, userspace may just be serving as a conduit and merger for constraints * Consider licensing as well. DRM is MIT, making it usable by BSD as well. * Feedback on EGL/Vulkan/KMS/GBM integration? * Question from romangg: `Can you outline which components (compositor, client, driver, etc.) have to do what for typical usage of constraints?` * Compositors collect constraints from various APIs, merge them, and provide them to GBM * Need clients to provide constraints? * Buffers are client-allocated. No good solution now for clients to allocate buffers for HW planes, for example. * First step here, give modifier hints * Generalize to give constraints for each modifier as well * Already working on modifier solution * In general, handle similar to modifiers. Whenever you pass around a modifier, include its constraints. Add yours, if you have local ones. Likely hidden from apps by "drivers" like EGL/Vulkan on client, but compositors need to get into details * Example: Camera displaying on compositor as texture or HW plane? * Start from steady-state (which is texturing to composite?) * Compositor observes it would like to promote. * Queries modifiers for this usage * Sends them back to client * Client, if it's aware, uses them to reallocate * When new buffer is presented, compositor promotes to plane. * With constraints, need to add those in, but following same workflow/pipeline. * Vendor-specific constraints? Fully-opaque constraints? * Any need for non-public/non-upstream constraints * No known demand, but design leaves the door open. * Constraint "CPU access", "cached/uncached" * Heaps maybe not enough? * Do the case study and figure out if that works * Expand usage to include CPU mapping usage? (RO, RW, sequential or random-access, etc?) * Heap IDs? * "Regular memory" heap? Generic/well-known heaps? * Heap flags for cacheable/etc? * SoC: who's responsible for exposing the heaps, how do drivers discover other heaps? Multipe drivers need to agree. * Good time to work with dma-buf heap designers * Start with Local/unshareable, non-local shareable * What kind of API to expose those heaps * What kind of API to discover those heaps * John Stultz, Ezequiel Garcia, Sumit Semwal, Laura Abbott * Good to propose sensible path towards implementing it, get a MVP * James to take next step of starting email chain * Try to make some more concrete proposals * Post gitlab issues here: https://gitlab.freedesktop.org/emersion/drm-constraints