Slides for the presentation on how to bridge WebRTC and NDI thanks to Janus, made at the virtual edition of ClueCon TGI2021.

1. Bringing WebRTC to producers using NDI
Lorenzo Miniero
ClueCon TGI2021
February 17th 2021

2. Who am I?
Lorenzo Miniero
• Ph.D @ UniNA
• Chairman @ Meetecho
• Main author of Janus®
Contacts and info
• lorenzo@meetecho.com
• https://twitter.com/elminiero
• https://www.slideshare.net/LorenzoMiniero
• https://soundcloud.com/lminiero

3. Just a few words on Meetecho
• Co-founded in 2009 as an academic spin-off
• University research efforts brought to the market
• Completely independent from the University
• Focus on real-time multimedia applications
• Strong perspective on standardization and open source
• Several activities
• Consulting services
• Commercial support and Janus licenses
• Streaming of live events (IETF, ACM, etc.)
• Proudly brewed in sunny Napoli, Italy

4. Home Sweet Home!

5. You may have heard of NDI already
https://youtu.be/qazlTnSjVzA?t=3703 (ClueCon 2020 Dangerous Demos)

6. Broadcasters love NDI
https://www.meetecho.com/blog/webrtc-ndi/
https://www.meetecho.com/blog/webrtc-ndi-part-2/

7. What is NDI?
• Network Device Interface (NDI)
• Royalty-free software standard developed by NewTek
• https://www.ndi.tv/
• Live exchange of multimedia streams within the same LAN
• Multichannel and uncompressed media streams (high quality)
• mDNS used for service discovery
• Easy to use (and integrate) native SDK
• Available on Windows, Linux, MacOS, Android, etc.
• https://www.ndi.tv/sdk/
• VLC team working on an alternative implementation
• https://code.videolan.org/jbk/libndi
• Plenty of applications for different use cases

8. What is NDI?
• Network Device Interface (NDI)
• Royalty-free software standard developed by NewTek
• https://www.ndi.tv/
• Live exchange of multimedia streams within the same LAN
• Multichannel and uncompressed media streams (high quality)
• mDNS used for service discovery
• Easy to use (and integrate) native SDK
• Available on Windows, Linux, MacOS, Android, etc.
• https://www.ndi.tv/sdk/
• VLC team working on an alternative implementation
• https://code.videolan.org/jbk/libndi
• Plenty of applications for different use cases

9. What is NDI?
• Network Device Interface (NDI)
• Royalty-free software standard developed by NewTek
• https://www.ndi.tv/
• Live exchange of multimedia streams within the same LAN
• Multichannel and uncompressed media streams (high quality)
• mDNS used for service discovery
• Easy to use (and integrate) native SDK
• Available on Windows, Linux, MacOS, Android, etc.
• https://www.ndi.tv/sdk/
• VLC team working on an alternative implementation
• https://code.videolan.org/jbk/libndi
• Plenty of applications for different use cases

10. What is NDI?
• Network Device Interface (NDI)
• Royalty-free software standard developed by NewTek
• https://www.ndi.tv/
• Live exchange of multimedia streams within the same LAN
• Multichannel and uncompressed media streams (high quality)
• mDNS used for service discovery
• Easy to use (and integrate) native SDK
• Available on Windows, Linux, MacOS, Android, etc.
• https://www.ndi.tv/sdk/
• VLC team working on an alternative implementation
• https://code.videolan.org/jbk/libndi
• Plenty of applications for different use cases

11. No easy way to do WebRTC, though..?

12. No easy way to do WebRTC, though..?

13. No easy way to do WebRTC, though..?

14. Ouch, that hurts...

15. Ouch, that hurts...

16. That hurts even MORE!

17. That hurts even MORE!

18. A real use case: CommCon Virtual 2020
• Due to the pandemic, CommCon was done virtually in 2020
• https://2020.commcon.xyz/
• Dan Jenkins, the organizer, wanted pre-recorded talks for more quality
• Speakers would present via WebRTC
• Media production team would record and process
• Per Dan’s words, a bit of a painful process, though
• Mix of different applications for different things
• Jitsi and vMix both used at the same time
• NDI, required by media team, hard to capture
• Manual capture of the Jitsi window for live content
• vMix only supports mixed streams, apparently?
• “Life would have been WAY easier with direct WebRTC-to-NDI...”

19. A real use case: CommCon Virtual 2020
• Due to the pandemic, CommCon was done virtually in 2020
• https://2020.commcon.xyz/
• Dan Jenkins, the organizer, wanted pre-recorded talks for more quality
• Speakers would present via WebRTC
• Media production team would record and process
• Per Dan’s words, a bit of a painful process, though
• Mix of different applications for different things
• Jitsi and vMix both used at the same time
• NDI, required by media team, hard to capture
• Manual capture of the Jitsi window for live content
• vMix only supports mixed streams, apparently?
• “Life would have been WAY easier with direct WebRTC-to-NDI...”

20. A real use case: CommCon Virtual 2020
• Due to the pandemic, CommCon was done virtually in 2020
• https://2020.commcon.xyz/
• Dan Jenkins, the organizer, wanted pre-recorded talks for more quality
• Speakers would present via WebRTC
• Media production team would record and process
• Per Dan’s words, a bit of a painful process, though
• Mix of different applications for different things
• Jitsi and vMix both used at the same time
• NDI, required by media team, hard to capture
• Manual capture of the Jitsi window for live content
• vMix only supports mixed streams, apparently?
• “Life would have been WAY easier with direct WebRTC-to-NDI...”

21. A real use case: CommCon Virtual 2020
• Due to the pandemic, CommCon was done virtually in 2020
• https://2020.commcon.xyz/
• Dan Jenkins, the organizer, wanted pre-recorded talks for more quality
• Speakers would present via WebRTC
• Media production team would record and process
• Per Dan’s words, a bit of a painful process, though
• Mix of different applications for different things
• Jitsi and vMix both used at the same time
• NDI, required by media team, hard to capture
• Manual capture of the Jitsi window for live content
• vMix only supports mixed streams, apparently?
• “Life would have been WAY easier with direct WebRTC-to-NDI...”

22. A real use case: CommCon Virtual 2020
• Due to the pandemic, CommCon was done virtually in 2020
• https://2020.commcon.xyz/
• Dan Jenkins, the organizer, wanted pre-recorded talks for more quality
• Speakers would present via WebRTC
• Media production team would record and process
• Per Dan’s words, a bit of a painful process, though
• Mix of different applications for different things
• Jitsi and vMix both used at the same time
• NDI, required by media team, hard to capture
• Manual capture of the Jitsi window for live content
• vMix only supports mixed streams, apparently?
• “Life would have been WAY easier with direct WebRTC-to-NDI...”

23. A real use case: CommCon Virtual 2020
• Due to the pandemic, CommCon was done virtually in 2020
• https://2020.commcon.xyz/
• Dan Jenkins, the organizer, wanted pre-recorded talks for more quality
• Speakers would present via WebRTC
• Media production team would record and process
• Per Dan’s words, a bit of a painful process, though
• Mix of different applications for different things
• Jitsi and vMix both used at the same time
• NDI, required by media team, hard to capture
• Manual capture of the Jitsi window for live content
• vMix only supports mixed streams, apparently?
• “Life would have been WAY easier with direct WebRTC-to-NDI...”

24. GStreamer NDI plugin can only receive
https://github.com/teltek/gst-plugin-ndi

25. Challenge accepted!

26. First experiments (RTP-to-NDI)

27. First experiments (RTP-to-NDI)

28. First experiments (RTP-to-NDI)

29. First experiments (RTP-to-NDI)

30. First experiments (RTP-to-NDI)

31. First experiments (RTP-to-NDI)

32. First experiments (RTP-to-NDI)

33. How to make that WebRTC compliant?
• First very simple step may be RTP forwarders1
• This prototype supports RTP ingestion already
• VideoRoom + RTP forwarders could easily fill the gap
• Why not something more integrated, though?
• RTP forwarders do require a bit of orchestration (e.g., ports exchange)
• Custom plugin to receive WebRTC streams, maybe?
• Audio/video decoding + NDI translation in plugin itself (lightweight enough)
• Quite a straightforward process
• Janus has its own RTP stack, no need for the one in rtp2ndi
• Everything else can work pretty much the same way
1
Don’t miss my workshop, in a couple of hours

34. How to make that WebRTC compliant?
• First very simple step may be RTP forwarders1
• This prototype supports RTP ingestion already
• VideoRoom + RTP forwarders could easily fill the gap
• Why not something more integrated, though?
• RTP forwarders do require a bit of orchestration (e.g., ports exchange)
• Custom plugin to receive WebRTC streams, maybe?
• Audio/video decoding + NDI translation in plugin itself (lightweight enough)
• Quite a straightforward process
• Janus has its own RTP stack, no need for the one in rtp2ndi
• Everything else can work pretty much the same way
1
Don’t miss my workshop, in a couple of hours

35. How to make that WebRTC compliant?
• First very simple step may be RTP forwarders1
• This prototype supports RTP ingestion already
• VideoRoom + RTP forwarders could easily fill the gap
• Why not something more integrated, though?
• RTP forwarders do require a bit of orchestration (e.g., ports exchange)
• Custom plugin to receive WebRTC streams, maybe?
• Audio/video decoding + NDI translation in plugin itself (lightweight enough)
• Quite a straightforward process
• Janus has its own RTP stack, no need for the one in rtp2ndi
• Everything else can work pretty much the same way
1
Don’t miss my workshop, in a couple of hours

36. Bringing Janus in the picture

37. Bringing Janus in the picture

38. Bringing Janus in the picture

39. Bringing Janus in the picture

40. Bringing Janus in the picture

41. Users will not be in your own LAN, though...
• As it is, the plugin works great for local users
• Browsers can connect to the Janus + NDI plugin directly
• The WebRTC-to-NDI feature works seamlessly
• What if you need to bring in someone remote, though?
• Janus + NDI plugin needs to be in the LAN (or NDI won’t work)
• Hard to make Janus server reachable, if it’s behind NAT/firewall
• (and you’ll definitely want the server to be publicly reachable anyway)
• A couple of options, here
1 Transform the code (again) to a “client” that can talk to the public instance
2 Leave Janus as it is, and “use” it as a programmable client instead!

42. Users will not be in your own LAN, though...
• As it is, the plugin works great for local users
• Browsers can connect to the Janus + NDI plugin directly
• The WebRTC-to-NDI feature works seamlessly
• What if you need to bring in someone remote, though?
• Janus + NDI plugin needs to be in the LAN (or NDI won’t work)
• Hard to make Janus server reachable, if it’s behind NAT/firewall
• (and you’ll definitely want the server to be publicly reachable anyway)
• A couple of options, here
1 Transform the code (again) to a “client” that can talk to the public instance
2 Leave Janus as it is, and “use” it as a programmable client instead!

43. Users will not be in your own LAN, though...
• As it is, the plugin works great for local users
• Browsers can connect to the Janus + NDI plugin directly
• The WebRTC-to-NDI feature works seamlessly
• What if you need to bring in someone remote, though?
• Janus + NDI plugin needs to be in the LAN (or NDI won’t work)
• Hard to make Janus server reachable, if it’s behind NAT/firewall
• (and you’ll definitely want the server to be publicly reachable anyway)
• A couple of options, here
1 Transform the code (again) to a “client” that can talk to the public instance
2 Leave Janus as it is, and “use” it as a programmable client instead!

44. Users will not be in your own LAN, though...
• As it is, the plugin works great for local users
• Browsers can connect to the Janus + NDI plugin directly
• The WebRTC-to-NDI feature works seamlessly
• What if you need to bring in someone remote, though?
• Janus + NDI plugin needs to be in the LAN (or NDI won’t work)
• Hard to make Janus server reachable, if it’s behind NAT/firewall
• (and you’ll definitely want the server to be publicly reachable anyway)
• A couple of options, here
1 Transform the code (again) to a “client” that can talk to the public instance
2 Leave Janus as it is, and “use” it as a programmable client instead!

45. Users will not be in your own LAN, though...
• As it is, the plugin works great for local users
• Browsers can connect to the Janus + NDI plugin directly
• The WebRTC-to-NDI feature works seamlessly
• What if you need to bring in someone remote, though?
• Janus + NDI plugin needs to be in the LAN (or NDI won’t work)
• Hard to make Janus server reachable, if it’s behind NAT/firewall
• (and you’ll definitely want the server to be publicly reachable anyway)
• A couple of options, here
1 Transform the code (again) to a “client” that can talk to the public instance
2 Leave Janus as it is, and “use” it as a programmable client instead!

46. Bringing Janus (and the Internet) in the picture

47. Bringing Janus (and the Internet) in the picture

48. Bringing Janus (and the Internet) in the picture

49. Bringing Janus (and the Internet) in the picture

50. Now that it works, why not turn it into a “service”?

51. Now that it works, why not turn it into a “service”?
• Collaboration between Dan Jenkins (Nimble Ape) and Meetecho
• https://broadcaster.vc/
• Approach pretty much the one we’ve seen in the previous slides
• Public Janus instances for conversations (SaaS, hosted by Nimble Ape)
• Docker images with { Janus + Meetecho NDI plugin } to launch in your LAN
• Orchestration between the two done “automagically”
• Local Janus instance automatically subscribes (WebRTC) to active participants
• NDI feeds automatically generated for who joins your room
Give it a try!
We had to improve a few things first (see next slides), but in Beta now

52. Now that it works, why not turn it into a “service”?
• Collaboration between Dan Jenkins (Nimble Ape) and Meetecho
• https://broadcaster.vc/
• Approach pretty much the one we’ve seen in the previous slides
• Public Janus instances for conversations (SaaS, hosted by Nimble Ape)
• Docker images with { Janus + Meetecho NDI plugin } to launch in your LAN
• Orchestration between the two done “automagically”
• Local Janus instance automatically subscribes (WebRTC) to active participants
• NDI feeds automatically generated for who joins your room
Give it a try!
We had to improve a few things first (see next slides), but in Beta now

53. Now that it works, why not turn it into a “service”?
• Collaboration between Dan Jenkins (Nimble Ape) and Meetecho
• https://broadcaster.vc/
• Approach pretty much the one we’ve seen in the previous slides
• Public Janus instances for conversations (SaaS, hosted by Nimble Ape)
• Docker images with { Janus + Meetecho NDI plugin } to launch in your LAN
• Orchestration between the two done “automagically”
• Local Janus instance automatically subscribes (WebRTC) to active participants
• NDI feeds automatically generated for who joins your room
Give it a try!
We had to improve a few things first (see next slides), but in Beta now

54. Now that it works, why not turn it into a “service”?
• Collaboration between Dan Jenkins (Nimble Ape) and Meetecho
• https://broadcaster.vc/
• Approach pretty much the one we’ve seen in the previous slides
• Public Janus instances for conversations (SaaS, hosted by Nimble Ape)
• Docker images with { Janus + Meetecho NDI plugin } to launch in your LAN
• Orchestration between the two done “automagically”
• Local Janus instance automatically subscribes (WebRTC) to active participants
• NDI feeds automatically generated for who joins your room
Give it a try!
We had to improve a few things first (see next slides), but in Beta now

55. Two channels are better than one!

56. Fixing artifacts with a jitter buffer

57. Fixing wrong image management

58. Test patterns (“Is this working?”)

59. Test patterns (“Is this working?”)

60. Water(marks), water(marks), everywhere

61. Now we have AV1 in there too!
https://www.meetecho.com/blog/av1-h265-janus/

62. Action!
https://www.youtube.com/watch?v=6MtH9XV4DfQ

63. Action!
https://www.youtube.com/watch?v=6MtH9XV4DfQ

64. Thanks! Questions? Comments?
Get in touch!
• https://twitter.com/elminiero
• https://twitter.com/meetecho
• http://www.meetecho.com