This talk was presented at Vulkanised 2023 (Feb 7-9, Munich Germany). Vulkanised is organised by the Khronos Group and is the largest event dedicated to 3D developers using the Vulkan API. The event provides a unique opportunity to bring the Vulkan developer community together to exchange ideas, solve problems and help steer the future development of the Vulkan API and ecosystem. (c) Vulkanised 2023 Feb 7-9 Munich (Germany) https://www.youtube.com/watch?v=5PV_yTculaM https://vulkan.org/events/vulkanised-2023

1. Implementing a Vulkan decoder
Víctor Jáquez
Igalia
The 5th
Vulkan Developer Conference
Munich, Germany / February 7–9
Platinum Sponsors:

2. Implementing a Vulkan Video decoder
Víctor Jáquez, February 2023
ººº

3. Implementing a Vulkan Video decoder
Víctor Jáquez, February 2023
API Overview
 Semi-stateless
 Low level
 Fine grained
● Each driver implementation can operate differently
● Verbose
 Big specification

4.  Create a session parameters object
 Add new parameters
 Destroy session parameters object
Implementing a Vulkan Video decoder
Víctor Jáquez, February 2023
Stream Parameters Sets
This approach is very different from current parsers.

5. Implementing a Vulkan Video decoder
Víctor Jáquez, February 2023
Picture references (DPB)
 References and output coincide
 Distinct references and output sets
● references are a set of images
● references are in a single multi-
layered image
General use implementation should manage each case.

6. Implementing a Vulkan Video decoder
Víctor Jáquez, February 2023
Bitstream buffers
 Upload all slices
● Extra memory copies
● VK_EXT_external_memory_host
 Prepend codec start-code

7. Implementing a Vulkan Video decoder
Víctor Jáquez, February 2023
Multiplanar Images
 VK_FORMAT_G8_B8R8_2PLANE_420_UNORM → NV12
● YUV420 2 planes in single image
 Currently, multiplane images are emulated with
separate images per-plane
GStreamer format mapping issue:
2 Vulkan image layouts = 1 GStreamer format

8. Implementing a Vulkan Video decoder
Víctor Jáquez, February 2023
GStreamer Capabilities
template
 At element registering time
● Query all possible decoding devices
● Query all profiles each device is
capable
● Query all color formats each
device/profile can output