Slides for the "Getting AV1/SVC to work in the Janus WebRTC Server" presentation I made at the Real-Time Communications devroom of FOSDEM 2024 in Brussels. It describes in detail how AV1 is used in real-time communications (e.g., RTP packetization rules) and how the Dependency Descriptor extensions allows for SVC to be used in a server, by sharing my experience integrating it in the Janus WebRTC Server.
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Getting AV1/SVC to work in the Janus WebRTC Server
1. Getting AV1/SVC to work in the
Janus WebRTC Server
Lorenzo Miniero
@lminiero@fosstodon.org
FOSDEM 2024 Real Time Communications
3rd February 2024, Brussels
2. Who am I?
Lorenzo Miniero
• Ph.D @ UniNA
• Chairman @ Meetecho
• Main author of Janus
Contacts and info
• lorenzo@meetecho.com
• https://fosstodon.org/@lminiero
• https://www.meetecho.com
• https://lminiero.it
3. AOMedia Video 1 (AV1)
• Open, royalty free, video codec
• Developed by Alliance for Open Media (AOM)
• https://aomedia.org/av1-features/get-started/
• Specifically designed for real-time applications
• Support for higher resolutions
• Natively conceived to support SVC as well
• Scalable Video Coding
4. AOMedia Video 1 (AV1)
• Open, royalty free, video codec
• Developed by Alliance for Open Media (AOM)
• https://aomedia.org/av1-features/get-started/
• Specifically designed for real-time applications
• Support for higher resolutions
• Natively conceived to support SVC as well
• Scalable Video Coding
5. AOMedia Video 1 (AV1)
• Open, royalty free, video codec
• Developed by Alliance for Open Media (AOM)
• https://aomedia.org/av1-features/get-started/
• Specifically designed for real-time applications
• Support for higher resolutions
• Natively conceived to support SVC as well
• Scalable Video Coding
6. AOMedia Video 1 (AV1)
• Open, royalty free, video codec
• Developed by Alliance for Open Media (AOM)
• https://aomedia.org/av1-features/get-started/
• Specifically designed for real-time applications
• Support for higher resolutions
• Natively conceived to support SVC as well
• Scalable Video Coding
7. Why is SVC important?
• Simulcast
• Same source, same m-line
• Streams of different “quality” are separate tracks
• Each stream uses a different SSRC
• Each stream can be decoded indepedently from others
• SVC
• Same source, same m-line
• Streams of different “quality” are layers of the same “thing”
• All streams share the same SSRC (since they’re layers)
• Each stream depends on the previous to be decoded
• Less bandwidth, but more CPU intensive
Fun fact – Simulcast in browsers also enables temporal scalability
Allows to drop to lower framerate without sacrificing quality
8. Why is SVC important?
• Simulcast
• Same source, same m-line
• Streams of different “quality” are separate tracks
• Each stream uses a different SSRC
• Each stream can be decoded indepedently from others
• SVC
• Same source, same m-line
• Streams of different “quality” are layers of the same “thing”
• All streams share the same SSRC (since they’re layers)
• Each stream depends on the previous to be decoded
• Less bandwidth, but more CPU intensive
Fun fact – Simulcast in browsers also enables temporal scalability
Allows to drop to lower framerate without sacrificing quality
9. Why is SVC important?
• Simulcast
• Same source, same m-line
• Streams of different “quality” are separate tracks
• Each stream uses a different SSRC
• Each stream can be decoded indepedently from others
• SVC
• Same source, same m-line
• Streams of different “quality” are layers of the same “thing”
• All streams share the same SSRC (since they’re layers)
• Each stream depends on the previous to be decoded
• Less bandwidth, but more CPU intensive
Fun fact – Simulcast in browsers also enables temporal scalability
Allows to drop to lower framerate without sacrificing quality
10. Simulcast in a nutshell
https://webrtchacks.com/sfu-simulcast/
11. SVC as a different way to encode multiple tracks
https://webrtchacks.com/chrome-vp9-svc/
12. Enter the Janus WebRTC Server
Janus
General purpose, open source WebRTC server
• https://github.com/meetecho/janus-gateway
• Demos and documentation: https://janus.conf.meetecho.com
• Community: https://janus.discourse.group/
13. Adding AV1 support to Janus
• A few different (incremental) requirements
1 Negotiate AV1 in the SDP
2 Detect keyframes when receiving packets (useful for different reasons)
3 Copy AV1 frames from Janus recordings to playable format
4 Negotiate Dependency Descriptor extension in the SDP (for SVC)
5 Parse Dependency Descriptor extension format
6 Use Dependency Descriptor for SVC in different plugins
• Negotiating AV1 is easy
• a=rtpmap:XX AV1/90000 (note: was AV1X/90000 in first Chrome integration)
• Detecting keyframes and supporting AV1 in recordings is trickier
• Requires understanding of RTP packetization rules for AV1
• Let’s start there, before diving in SVC!
14. Adding AV1 support to Janus
• A few different (incremental) requirements
1 Negotiate AV1 in the SDP
2 Detect keyframes when receiving packets (useful for different reasons)
3 Copy AV1 frames from Janus recordings to playable format
4 Negotiate Dependency Descriptor extension in the SDP (for SVC)
5 Parse Dependency Descriptor extension format
6 Use Dependency Descriptor for SVC in different plugins
• Negotiating AV1 is easy
• a=rtpmap:XX AV1/90000 (note: was AV1X/90000 in first Chrome integration)
• Detecting keyframes and supporting AV1 in recordings is trickier
• Requires understanding of RTP packetization rules for AV1
• Let’s start there, before diving in SVC!
15. Adding AV1 support to Janus
• A few different (incremental) requirements
1 Negotiate AV1 in the SDP
2 Detect keyframes when receiving packets (useful for different reasons)
3 Copy AV1 frames from Janus recordings to playable format
4 Negotiate Dependency Descriptor extension in the SDP (for SVC)
5 Parse Dependency Descriptor extension format
6 Use Dependency Descriptor for SVC in different plugins
• Negotiating AV1 is easy
• a=rtpmap:XX AV1/90000 (note: was AV1X/90000 in first Chrome integration)
• Detecting keyframes and supporting AV1 in recordings is trickier
• Requires understanding of RTP packetization rules for AV1
• Let’s start there, before diving in SVC!
16. Adding AV1 support to Janus
• A few different (incremental) requirements
1 Negotiate AV1 in the SDP
2 Detect keyframes when receiving packets (useful for different reasons)
3 Copy AV1 frames from Janus recordings to playable format
4 Negotiate Dependency Descriptor extension in the SDP (for SVC)
5 Parse Dependency Descriptor extension format
6 Use Dependency Descriptor for SVC in different plugins
• Negotiating AV1 is easy
• a=rtpmap:XX AV1/90000 (note: was AV1X/90000 in first Chrome integration)
• Detecting keyframes and supporting AV1 in recordings is trickier
• Requires understanding of RTP packetization rules for AV1
• Let’s start there, before diving in SVC!
17. Adding AV1 support to Janus
• A few different (incremental) requirements
1 Negotiate AV1 in the SDP
2 Detect keyframes when receiving packets (useful for different reasons)
3 Copy AV1 frames from Janus recordings to playable format
4 Negotiate Dependency Descriptor extension in the SDP (for SVC)
5 Parse Dependency Descriptor extension format
6 Use Dependency Descriptor for SVC in different plugins
• Negotiating AV1 is easy
• a=rtpmap:XX AV1/90000 (note: was AV1X/90000 in first Chrome integration)
• Detecting keyframes and supporting AV1 in recordings is trickier
• Requires understanding of RTP packetization rules for AV1
• Let’s start there, before diving in SVC!
18. Adding AV1 support to Janus
• A few different (incremental) requirements
1 Negotiate AV1 in the SDP
2 Detect keyframes when receiving packets (useful for different reasons)
3 Copy AV1 frames from Janus recordings to playable format
4 Negotiate Dependency Descriptor extension in the SDP (for SVC)
5 Parse Dependency Descriptor extension format
6 Use Dependency Descriptor for SVC in different plugins
• Negotiating AV1 is easy
• a=rtpmap:XX AV1/90000 (note: was AV1X/90000 in first Chrome integration)
• Detecting keyframes and supporting AV1 in recordings is trickier
• Requires understanding of RTP packetization rules for AV1
• Let’s start there, before diving in SVC!
19. Adding AV1 support to Janus
• A few different (incremental) requirements
1 Negotiate AV1 in the SDP
2 Detect keyframes when receiving packets (useful for different reasons)
3 Copy AV1 frames from Janus recordings to playable format
4 Negotiate Dependency Descriptor extension in the SDP (for SVC)
5 Parse Dependency Descriptor extension format
6 Use Dependency Descriptor for SVC in different plugins
• Negotiating AV1 is easy
• a=rtpmap:XX AV1/90000 (note: was AV1X/90000 in first Chrome integration)
• Detecting keyframes and supporting AV1 in recordings is trickier
• Requires understanding of RTP packetization rules for AV1
• Let’s start there, before diving in SVC!
20. Adding AV1 support to Janus
• A few different (incremental) requirements
1 Negotiate AV1 in the SDP
2 Detect keyframes when receiving packets (useful for different reasons)
3 Copy AV1 frames from Janus recordings to playable format
4 Negotiate Dependency Descriptor extension in the SDP (for SVC)
5 Parse Dependency Descriptor extension format
6 Use Dependency Descriptor for SVC in different plugins
• Negotiating AV1 is easy
• a=rtpmap:XX AV1/90000 (note: was AV1X/90000 in first Chrome integration)
• Detecting keyframes and supporting AV1 in recordings is trickier
• Requires understanding of RTP packetization rules for AV1
• Let’s start there, before diving in SVC!
21. Adding AV1 support to Janus
• A few different (incremental) requirements
1 Negotiate AV1 in the SDP
2 Detect keyframes when receiving packets (useful for different reasons)
3 Copy AV1 frames from Janus recordings to playable format
4 Negotiate Dependency Descriptor extension in the SDP (for SVC)
5 Parse Dependency Descriptor extension format
6 Use Dependency Descriptor for SVC in different plugins
• Negotiating AV1 is easy
• a=rtpmap:XX AV1/90000 (note: was AV1X/90000 in first Chrome integration)
• Detecting keyframes and supporting AV1 in recordings is trickier
• Requires understanding of RTP packetization rules for AV1
• Let’s start there, before diving in SVC!
22. Starting from the basics: AV1 and RTP
• All codecs need packetization rules, to be used in RTP
• Especially true for video, since data may me split in multiple packets
• Need to know what’s what, how to split, and how to stitch back together
• Activity usually carried within the IETF (Internet Engineering Task Force)
• AVTCORE Working Group (Audio/Video Transport Core Maintenance)
• https://datatracker.ietf.org/wg/avtcore/about/
• RTP payload format for AV1 developed by AOM, though
• https://aomediacodec.github.io/av1-rtp-spec/
• Two main concepts introduced in the document
1 AV1 aggregation header (for the RTP payload)
2 Dependency descriptor (for SVC)
23. Starting from the basics: AV1 and RTP
• All codecs need packetization rules, to be used in RTP
• Especially true for video, since data may me split in multiple packets
• Need to know what’s what, how to split, and how to stitch back together
• Activity usually carried within the IETF (Internet Engineering Task Force)
• AVTCORE Working Group (Audio/Video Transport Core Maintenance)
• https://datatracker.ietf.org/wg/avtcore/about/
• RTP payload format for AV1 developed by AOM, though
• https://aomediacodec.github.io/av1-rtp-spec/
• Two main concepts introduced in the document
1 AV1 aggregation header (for the RTP payload)
2 Dependency descriptor (for SVC)
24. Starting from the basics: AV1 and RTP
• All codecs need packetization rules, to be used in RTP
• Especially true for video, since data may me split in multiple packets
• Need to know what’s what, how to split, and how to stitch back together
• Activity usually carried within the IETF (Internet Engineering Task Force)
• AVTCORE Working Group (Audio/Video Transport Core Maintenance)
• https://datatracker.ietf.org/wg/avtcore/about/
• RTP payload format for AV1 developed by AOM, though
• https://aomediacodec.github.io/av1-rtp-spec/
• Two main concepts introduced in the document
1 AV1 aggregation header (for the RTP payload)
2 Dependency descriptor (for SVC)
25. Starting from the basics: AV1 and RTP
• All codecs need packetization rules, to be used in RTP
• Especially true for video, since data may me split in multiple packets
• Need to know what’s what, how to split, and how to stitch back together
• Activity usually carried within the IETF (Internet Engineering Task Force)
• AVTCORE Working Group (Audio/Video Transport Core Maintenance)
• https://datatracker.ietf.org/wg/avtcore/about/
• RTP payload format for AV1 developed by AOM, though
• https://aomediacodec.github.io/av1-rtp-spec/
• Two main concepts introduced in the document
1 AV1 aggregation header (for the RTP payload)
2 Dependency descriptor (for SVC)
27. AV1 aggregation header
• Fundamental concept to map OBUs (Open Bitstream Unit) to RTP packets
• How to split OBU across multiple packets
• How to aggregate different OBUs in the same packet
• How to regenerate original OBUs from RTP packets
0 1 2 3 4 5 6 7
Z Y W N – – –
• Z: 1 if OBU continues from previous packet
• Y: 1 if OBU will continue in next packet
• W: number of OBUs in packet (2 bits)
• N: 1 if first packet of a sequence
• Other bits are currently reserved (and unused)
28. AV1 aggregation header example (W=0)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Z Y 0 0 N – – – OBU element 1 size (leb128)
OBU element 1 data
OBU element 2 size (leb128)
OBU element 2 data
· · ·
OBU el. N size (leb128)
OBU element N data
29. AV1 aggregation header example (W!=0)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Z Y 1 0 N – – – OBU element 1 size (leb128)
OBU element 1 data
OBU element 2 data
30. Using the Aggregation Header
• Detecting a keyframe can be approximated
• Z=0 and N=1 (not a continuation + first packet) −→ keyframe
• Simple and quick enough to use while relaying traffic
• Smarter process when post-processing recordings, instead
• Reconstructing OBUs performed following rules of Aggregation Header
• Parse RTP payloads and OBUs until we have a full one
• When one is ready, write it to file
• Also need to know width/height of video (for MP4 metadata)
• Resolution information is in a specific OBU (Sequence Header, type=1)
• All this is enough for “normal” AV1 usage (no SVC)
31. Using the Aggregation Header
• Detecting a keyframe can be approximated
• Z=0 and N=1 (not a continuation + first packet) −→ keyframe
• Simple and quick enough to use while relaying traffic
• Smarter process when post-processing recordings, instead
• Reconstructing OBUs performed following rules of Aggregation Header
• Parse RTP payloads and OBUs until we have a full one
• When one is ready, write it to file
• Also need to know width/height of video (for MP4 metadata)
• Resolution information is in a specific OBU (Sequence Header, type=1)
• All this is enough for “normal” AV1 usage (no SVC)
32. Using the Aggregation Header
• Detecting a keyframe can be approximated
• Z=0 and N=1 (not a continuation + first packet) −→ keyframe
• Simple and quick enough to use while relaying traffic
• Smarter process when post-processing recordings, instead
• Reconstructing OBUs performed following rules of Aggregation Header
• Parse RTP payloads and OBUs until we have a full one
• When one is ready, write it to file
• Also need to know width/height of video (for MP4 metadata)
• Resolution information is in a specific OBU (Sequence Header, type=1)
• All this is enough for “normal” AV1 usage (no SVC)
33. SVC and the Dependency Descriptors (DD)
• With SVC, the video stream can have multiple layers
• One or more spatial layers (different resolutions/bitrates)
• One or more temporal layers (different framerats/bitrates)
• Layers may have dependencies
• e.g., can’t decode spatial layer X without layer Y
• Has an impact on which packets can be dropped
• Need to figure out these dependencies at runtime
• SFU needs to know this info to decide when/what to drop/relay
• Can’t parse the payload (too heavy; impossible when E2EE is used)
Dependency Descriptor
Custom RTP extension with dynamic info on SVC/layers
34. SVC and the Dependency Descriptors (DD)
• With SVC, the video stream can have multiple layers
• One or more spatial layers (different resolutions/bitrates)
• One or more temporal layers (different framerats/bitrates)
• Layers may have dependencies
• e.g., can’t decode spatial layer X without layer Y
• Has an impact on which packets can be dropped
• Need to figure out these dependencies at runtime
• SFU needs to know this info to decide when/what to drop/relay
• Can’t parse the payload (too heavy; impossible when E2EE is used)
Dependency Descriptor
Custom RTP extension with dynamic info on SVC/layers
35. SVC and the Dependency Descriptors (DD)
• With SVC, the video stream can have multiple layers
• One or more spatial layers (different resolutions/bitrates)
• One or more temporal layers (different framerats/bitrates)
• Layers may have dependencies
• e.g., can’t decode spatial layer X without layer Y
• Has an impact on which packets can be dropped
• Need to figure out these dependencies at runtime
• SFU needs to know this info to decide when/what to drop/relay
• Can’t parse the payload (too heavy; impossible when E2EE is used)
Dependency Descriptor
Custom RTP extension with dynamic info on SVC/layers
36. SVC and the Dependency Descriptors (DD)
• With SVC, the video stream can have multiple layers
• One or more spatial layers (different resolutions/bitrates)
• One or more temporal layers (different framerats/bitrates)
• Layers may have dependencies
• e.g., can’t decode spatial layer X without layer Y
• Has an impact on which packets can be dropped
• Need to figure out these dependencies at runtime
• SFU needs to know this info to decide when/what to drop/relay
• Can’t parse the payload (too heavy; impossible when E2EE is used)
Dependency Descriptor
Custom RTP extension with dynamic info on SVC/layers
37. Example of decode targets/chains (L2T3)
https://aomediacodec.github.io/av1-rtp-spec/#a10-examples
38. Adding support for DD in Janus
• Continuing the requirements from before
4 Negotiate Dependency Descriptor extension in the SDP (for SVC)
5 Parse Dependency Descriptor extension format
6 Use Dependency Descriptor for SVC in different plugins
• Negotiating was supposed to be the easy part
• a=extmap:X https://aomediacodec.github.io/av1-rtp-spec/[..]
• ... but we also needed to add support for two-byte headers (DD can be BIG!)
• a=extmap-allow-mixed (RFC8285)
• Parsing the format proved to be a painful task, though...
• https://www.meetecho.com/blog/av1-svc/
• https://www.meetecho.com/blog/vp9-av1-simulcast-svc/
39. Adding support for DD in Janus
• Continuing the requirements from before
4 Negotiate Dependency Descriptor extension in the SDP (for SVC)
5 Parse Dependency Descriptor extension format
6 Use Dependency Descriptor for SVC in different plugins
• Negotiating was supposed to be the easy part
• a=extmap:X https://aomediacodec.github.io/av1-rtp-spec/[..]
• ... but we also needed to add support for two-byte headers (DD can be BIG!)
• a=extmap-allow-mixed (RFC8285)
• Parsing the format proved to be a painful task, though...
• https://www.meetecho.com/blog/av1-svc/
• https://www.meetecho.com/blog/vp9-av1-simulcast-svc/
40. Adding support for DD in Janus
• Continuing the requirements from before
4 Negotiate Dependency Descriptor extension in the SDP (for SVC)
5 Parse Dependency Descriptor extension format
6 Use Dependency Descriptor for SVC in different plugins
• Negotiating was supposed to be the easy part
• a=extmap:X https://aomediacodec.github.io/av1-rtp-spec/[..]
• ... but we also needed to add support for two-byte headers (DD can be BIG!)
• a=extmap-allow-mixed (RFC8285)
• Parsing the format proved to be a painful task, though...
• https://www.meetecho.com/blog/av1-svc/
• https://www.meetecho.com/blog/vp9-av1-simulcast-svc/
41. Adding support for DD in Janus
• Continuing the requirements from before
4 Negotiate Dependency Descriptor extension in the SDP (for SVC)
5 Parse Dependency Descriptor extension format
6 Use Dependency Descriptor for SVC in different plugins
• Negotiating was supposed to be the easy part
• a=extmap:X https://aomediacodec.github.io/av1-rtp-spec/[..]
• ... but we also needed to add support for two-byte headers (DD can be BIG!)
• a=extmap-allow-mixed (RFC8285)
• Parsing the format proved to be a painful task, though...
• https://www.meetecho.com/blog/av1-svc/
• https://www.meetecho.com/blog/vp9-av1-simulcast-svc/
42. Adding support for DD in Janus
• Continuing the requirements from before
4 Negotiate Dependency Descriptor extension in the SDP (for SVC)
5 Parse Dependency Descriptor extension format
6 Use Dependency Descriptor for SVC in different plugins
• Negotiating was supposed to be the easy part
• a=extmap:X https://aomediacodec.github.io/av1-rtp-spec/[..]
• ... but we also needed to add support for two-byte headers (DD can be BIG!)
• a=extmap-allow-mixed (RFC8285)
• Parsing the format proved to be a painful task, though...
• https://www.meetecho.com/blog/av1-svc/
• https://www.meetecho.com/blog/vp9-av1-simulcast-svc/
43. Adding support for DD in Janus
• Continuing the requirements from before
4 Negotiate Dependency Descriptor extension in the SDP (for SVC)
5 Parse Dependency Descriptor extension format
6 Use Dependency Descriptor for SVC in different plugins
• Negotiating was supposed to be the easy part
• a=extmap:X https://aomediacodec.github.io/av1-rtp-spec/[..]
• ... but we also needed to add support for two-byte headers (DD can be BIG!)
• a=extmap-allow-mixed (RFC8285)
• Parsing the format proved to be a painful task, though...
• https://www.meetecho.com/blog/av1-svc/
• https://www.meetecho.com/blog/vp9-av1-simulcast-svc/
44. Adding support for DD in Janus
• Continuing the requirements from before
4 Negotiate Dependency Descriptor extension in the SDP (for SVC)
5 Parse Dependency Descriptor extension format
6 Use Dependency Descriptor for SVC in different plugins
• Negotiating was supposed to be the easy part
• a=extmap:X https://aomediacodec.github.io/av1-rtp-spec/[..]
• ... but we also needed to add support for two-byte headers (DD can be BIG!)
• a=extmap-allow-mixed (RFC8285)
• Parsing the format proved to be a painful task, though...
• https://www.meetecho.com/blog/av1-svc/
• https://www.meetecho.com/blog/vp9-av1-simulcast-svc/
45. Spoiler alert... DD is a f***ing mess!!
• Closer to a codec bitstream, than a protocol format
• Almost all fields are variable length (often at the bit level!)
• Very hard to parse (even harder to validate parsing, when debugging)
• Has a few mandatory fields
• start_of_frame
• end_of_frame
• frame_number
• frame_dependency_template_id
• Everything else is optional
• Allows DD to describe full chain, or just indicate scope of current frame
46. Spoiler alert... DD is a f***ing mess!!
• Closer to a codec bitstream, than a protocol format
• Almost all fields are variable length (often at the bit level!)
• Very hard to parse (even harder to validate parsing, when debugging)
• Has a few mandatory fields
• start_of_frame
• end_of_frame
• frame_number
• frame_dependency_template_id
• Everything else is optional
• Allows DD to describe full chain, or just indicate scope of current frame
47. Spoiler alert... DD is a f***ing mess!!
• Closer to a codec bitstream, than a protocol format
• Almost all fields are variable length (often at the bit level!)
• Very hard to parse (even harder to validate parsing, when debugging)
• Has a few mandatory fields
• start_of_frame
• end_of_frame
• frame_number
• frame_dependency_template_id
• Everything else is optional
• Allows DD to describe full chain, or just indicate scope of current frame
60. Using DD for SVC in Janus
• Dependency nature of DD means keeping a state
• We need to store the full state of DD templates (from RTP packet X)
• DD in other RTP packets will reference existing DD templates
• Once we know which spatial (SL) and temporal (TL) layer a packet belongs to...
• ... we can decide whether to relay or drop it
• Depends on what subscriber wants/needs
• Don’t forget to relay the DD along with the video data!
• We may be done with it, but the receiver will need it too
• Of course, outgoing RTP headers need to be updated accordingly
• We’ll drop packets, but the subscriber must see no gaps in sequence numbers
• Last packet of sequence must have marker bit set to 1 (super-important!!)
61. Using DD for SVC in Janus
• Dependency nature of DD means keeping a state
• We need to store the full state of DD templates (from RTP packet X)
• DD in other RTP packets will reference existing DD templates
• Once we know which spatial (SL) and temporal (TL) layer a packet belongs to...
• ... we can decide whether to relay or drop it
• Depends on what subscriber wants/needs
• Don’t forget to relay the DD along with the video data!
• We may be done with it, but the receiver will need it too
• Of course, outgoing RTP headers need to be updated accordingly
• We’ll drop packets, but the subscriber must see no gaps in sequence numbers
• Last packet of sequence must have marker bit set to 1 (super-important!!)
62. Using DD for SVC in Janus
• Dependency nature of DD means keeping a state
• We need to store the full state of DD templates (from RTP packet X)
• DD in other RTP packets will reference existing DD templates
• Once we know which spatial (SL) and temporal (TL) layer a packet belongs to...
• ... we can decide whether to relay or drop it
• Depends on what subscriber wants/needs
• Don’t forget to relay the DD along with the video data!
• We may be done with it, but the receiver will need it too
• Of course, outgoing RTP headers need to be updated accordingly
• We’ll drop packets, but the subscriber must see no gaps in sequence numbers
• Last packet of sequence must have marker bit set to 1 (super-important!!)
63. Using DD for SVC in Janus
• Dependency nature of DD means keeping a state
• We need to store the full state of DD templates (from RTP packet X)
• DD in other RTP packets will reference existing DD templates
• Once we know which spatial (SL) and temporal (TL) layer a packet belongs to...
• ... we can decide whether to relay or drop it
• Depends on what subscriber wants/needs
• Don’t forget to relay the DD along with the video data!
• We may be done with it, but the receiver will need it too
• Of course, outgoing RTP headers need to be updated accordingly
• We’ll drop packets, but the subscriber must see no gaps in sequence numbers
• Last packet of sequence must have marker bit set to 1 (super-important!!)