#XDC 2020 Allocation Constraints Workshop

Talk slides: https://xdc2020.x.org/event/9/contributions/615/attachments/704/1301/XDC_2020__Allocation_Constraints.pdf

Attendees (Add yourself):

• James Jones (NVIDIA) jajones 'at' nvidia.com
• Simon Ser contact@emersion.fr
• Daniel Stone (Collabora) daniels@collabora.com
• John Reitan (Arm) john.reitan 'at' arm.com
• Liviu Dudau (Arm)
• Laurent Pinchart (Ideas on Board)
• Bas Nieuwenhuizen (Google)

Ideas:

• Feedback on high level design?

  • Usage described via existing API mechanisms
  • Query constraints from existing APIs for a given usage
  • One constraint set for each modifier/layout
  • Merge constraint sets via common logic/library/shared code
  • Provide constraint set to allocation API of choice

• For wayland use mmap FD to share constraint lists. To that end, add serialize/unserialize. To that end, add serialize/unserialize.

  • Already have serialize/deserialize. There is no "non-serialized" view ATM in fact.

• For EGL query, how is texture/render specified?

  • external_only flag a rough proxy for this. Corresponds to OES external textures

• Do we need constraint per modifier? Too much?

  • Seems like this should be fine.

• Common code to merge constraints good?

  • No new objections.

• Allocators?

  • GBM + Vulkan OK?
  • What if we want camera buffers, for example? GBM not a good fit
  • Design can work with any allocator. A new unified allocator could take constraint sets as input.
  • GBM often lowest common denominator for shared buffers. Graphics often involved
  • However, consider encode. GBM not a good fit here.

• Would that proposal work fine with your hw?

  • Luma/Chroma planes in two different banks
  • Vivante needs shadow buffers with blits between them to handle layout transitions
  • This is not an attempt to get an optimal, global setup -- it's local
  • Have used extra pixels in image to include meta-data. Did not see good way to express in modifiers
    • Could express that in a constraint, but would not notify users of their presence.
    • Rely on dma-buf/buffer size to find extra bytes? Or pad stride/pitch? Not fully solved by constraints themselves.

• Where does the constraint merging library go?

  • Current proposal: Put it in a uapi header as header-only/inlined library
    • No way to add "private" functions/structs. Everything is public.
    • Various versions get inlined
  • Other ideas:
    • Add a helper syscall in DRM. Might be heavyweight.
    • Could put it in the vdso. Benefit: Keeps in sync across driver components on a given kernel
      • Is a general mechanism in kernel, needs discussion. Code, but not much info
      • Still need uapi header for interface and constraint def, but lib would be shared code.
  • Need to justify inclusion in the kernel
    • Kernel needs to generate constraint sets (KMS)
    • Kernel needs to parse at allocation step
      • XXX Depends on whether kernel takes constraint sets as parameter, or derived parameters
        • Would mean extending/changing many existing driver ioctls
    • Tradeoff of duplicating all parameters into each allocation ioctl, Vs a unified ioctl or at least unified type of parameters
      • May be difficult to answer this question (constraint merging code home) without stepping into that debate
    • Historically hasn't been much love for a unified allocation mechanism/parameter namespace. May be stuck with driver-specific or "the unified one" and picking between the two as needed.
      • (some agreement here)
  • If it remains in userspace-only, should we use a shared library instead?
    • Yes, definitely. However, may want to keep option to share with kernel open
    • Would be easy to [start with shared library], transition to kernel later
    • If a shared lib, at least headers defining constraints need to be kept in sync with kernel releases.
    • Only an issue if this changes a lot. Hopefully won't long-term
    • Should it be a whole new library, or is libdrm acceptable? OK with camera people?
      • Might be problematic if required by default, but probably OK for interaction with graphics (had trouble hearing this)
      • Problematic if code is copied rather than shared? Two versions resulting in different buffers?
      • Versioning addresses this to some extent
      • Considering case of shipping fixed BSP/system thoguh, then updating kernel. Would be nice if new constraints got picked up automatically.
      • Does this require userspace updates anyway to use them though?
      • Only if userspace is generating constraints itself, userspace may just be serving as a conduit and merger for constraints
      • Consider licensing as well. DRM is MIT, making it usable by BSD as well.

• Feedback on EGL/Vulkan/KMS/GBM integration?

  • Question from romangg: Can you outline which components (compositor, client, driver, etc.) have to do what for typical usage of constraints?
    • Compositors collect constraints from various APIs, merge them, and provide them to GBM
    • Need clients to provide constraints?
    • Buffers are client-allocated. No good solution now for clients to allocate buffers for HW planes, for example.
      • First step here, give modifier hints
      • Generalize to give constraints for each modifier as well
      • Already working on modifier solution
    • In general, handle similar to modifiers. Whenever you pass around a modifier, include its constraints. Add yours, if you have local ones. Likely hidden from apps by "drivers" like EGL/Vulkan on client, but compositors need to get into details
    • Example: Camera displaying on compositor as texture or HW plane?
      • Start from steady-state (which is texturing to composite?)
      • Compositor observes it would like to promote.
        • Queries modifiers for this usage
        • Sends them back to client
        • Client, if it's aware, uses them to reallocate
        • When new buffer is presented, compositor promotes to plane.
      • With constraints, need to add those in, but following same workflow/pipeline.

• Vendor-specific constraints? Fully-opaque constraints?

  • Any need for non-public/non-upstream constraints
  • No known demand, but design leaves the door open.

• Constraint "CPU access", "cached/uncached"

  • Heaps maybe not enough?
  • Do the case study and figure out if that works
  • Expand usage to include CPU mapping usage? (RO, RW, sequential or random-access, etc?)

• Heap IDs?

  • "Regular memory" heap? Generic/well-known heaps?
  • Heap flags for cacheable/etc?
  • SoC: who's responsible for exposing the heaps, how do drivers discover other heaps? Multipe drivers need to agree.
  • Good time to work with dma-buf heap designers
  • Start with Local/unshareable, non-local shareable
    • What kind of API to expose those heaps
    • What kind of API to discover those heaps
  • John Stultz, Ezequiel Garcia, Sumit Semwal, Laura Abbott

• Good to propose sensible path towards implementing it, get a MVP

  • James to take next step of starting email chain
  • Try to make some more concrete proposals
  • Post gitlab issues here: https://gitlab.freedesktop.org/emersion/drm-constraints