#XDC 2020 Allocation Constraints Workshop
Talk slides: https://xdc2020.x.org/event/9/contributions/615/attachments/704/1301/XDC_2020__Allocation_Constraints.pdf
Attendees (Add yourself):
- James Jones (NVIDIA) jajones 'at' nvidia.com
- Simon Ser firstname.lastname@example.org
- Daniel Stone (Collabora) email@example.com
- John Reitan (Arm) john.reitan 'at' arm.com
- Liviu Dudau (Arm)
- Laurent Pinchart (Ideas on Board)
- Bas Nieuwenhuizen (Google)
Feedback on high level design?
- Usage described via existing API mechanisms
- Query constraints from existing APIs for a given usage
- One constraint set for each modifier/layout
- Merge constraint sets via common logic/library/shared code
- Provide constraint set to allocation API of choice
For wayland use mmap FD to share constraint lists. To that end, add serialize/unserialize. To that end, add serialize/unserialize.
- Already have serialize/deserialize. There is no "non-serialized" view ATM in fact.
For EGL query, how is texture/render specified?
- external_only flag a rough proxy for this. Corresponds to OES external textures
Do we need constraint per modifier? Too much?
- Seems like this should be fine.
Common code to merge constraints good?
- GBM + Vulkan OK?
- What if we want camera buffers, for example? GBM not a good fit
- Design can work with any allocator. A new unified allocator could take constraint sets as input.
- GBM often lowest common denominator for shared buffers. Graphics often involved
- However, consider encode. GBM not a good fit here.
Would that proposal work fine with your hw?
- Luma/Chroma planes in two different banks
- Vivante needs shadow buffers with blits between them to handle layout transitions
- This is not an attempt to get an optimal, global setup -- it's local
- Have used extra pixels in image to include meta-data. Did not see good way to express in modifiers
- Could express that in a constraint, but would not notify users of their presence.
- Rely on dma-buf/buffer size to find extra bytes? Or pad stride/pitch? Not fully solved by constraints themselves.
Where does the constraint merging library go?
- Current proposal: Put it in a uapi header as header-only/inlined library
- No way to add "private" functions/structs. Everything is public.
- Various versions get inlined
- Other ideas:
- Add a helper syscall in DRM. Might be heavyweight.
- Could put it in the vdso. Benefit: Keeps in sync across driver components on a given kernel
- Is a general mechanism in kernel, needs discussion. Code, but not much info
- Still need uapi header for interface and constraint def, but lib would be shared code.
- Need to justify inclusion in the kernel
- Kernel needs to generate constraint sets (KMS)
- Kernel needs to parse at allocation step
- XXX Depends on whether kernel takes constraint sets as parameter, or derived parameters
- Would mean extending/changing many existing driver ioctls
- Tradeoff of duplicating all parameters into each allocation ioctl, Vs a unified ioctl or at least unified type of parameters
- May be difficult to answer this question (constraint merging code home) without stepping into that debate
- Historically hasn't been much love for a unified allocation mechanism/parameter namespace. May be stuck with driver-specific or "the unified one" and picking between the two as needed.
- If it remains in userspace-only, should we use a shared library instead?
- Yes, definitely. However, may want to keep option to share with kernel open
- Would be easy to [start with shared library], transition to kernel later
- If a shared lib, at least headers defining constraints need to be kept in sync with kernel releases.
- Only an issue if this changes a lot. Hopefully won't long-term
- Should it be a whole new library, or is libdrm acceptable? OK with camera people?
- Might be problematic if required by default, but probably OK for interaction with graphics (had trouble hearing this)
- Problematic if code is copied rather than shared? Two versions resulting in different buffers?
- Versioning addresses this to some extent
- Considering case of shipping fixed BSP/system thoguh, then updating kernel. Would be nice if new constraints got picked up automatically.
- Does this require userspace updates anyway to use them though?
- Only if userspace is generating constraints itself, userspace may just be serving as a conduit and merger for constraints
- Consider licensing as well. DRM is MIT, making it usable by BSD as well.
Feedback on EGL/Vulkan/KMS/GBM integration?
- Question from romangg:
Can you outline which components (compositor, client, driver, etc.) have to do what for typical usage of constraints?
- Compositors collect constraints from various APIs, merge them, and provide them to GBM
- Need clients to provide constraints?
- Buffers are client-allocated. No good solution now for clients to allocate buffers for HW planes, for example.
- First step here, give modifier hints
- Generalize to give constraints for each modifier as well
- Already working on modifier solution
- In general, handle similar to modifiers. Whenever you pass around a modifier, include its constraints. Add yours, if you have local ones. Likely hidden from apps by "drivers" like EGL/Vulkan on client, but compositors need to get into details
- Example: Camera displaying on compositor as texture or HW plane?
- Start from steady-state (which is texturing to composite?)
- Compositor observes it would like to promote.
- Queries modifiers for this usage
- Sends them back to client
- Client, if it's aware, uses them to reallocate
- When new buffer is presented, compositor promotes to plane.
- With constraints, need to add those in, but following same workflow/pipeline.
Vendor-specific constraints? Fully-opaque constraints?
- Any need for non-public/non-upstream constraints
- No known demand, but design leaves the door open.
Constraint "CPU access", "cached/uncached"
- Heaps maybe not enough?
- Do the case study and figure out if that works
- Expand usage to include CPU mapping usage? (RO, RW, sequential or random-access, etc?)
- "Regular memory" heap? Generic/well-known heaps?
- Heap flags for cacheable/etc?
- SoC: who's responsible for exposing the heaps, how do drivers discover other heaps? Multipe drivers need to agree.
- Good time to work with dma-buf heap designers
- Start with Local/unshareable, non-local shareable
- What kind of API to expose those heaps
- What kind of API to discover those heaps
- John Stultz, Ezequiel Garcia, Sumit Semwal, Laura Abbott
Good to propose sensible path towards implementing it, get a MVP