The document discusses the challenges of hybrid meetings that involve both in-person and remote attendees. It describes how a company called Meetecho has adapted their virtual event platform to address these challenges. Some of the key points discussed are integrating local and remote audio through a Janus AudioBridge, adding multiple cameras and dynamic video feeds, sharing all presentation content through the platform regardless of source, putting all attendees in a single queue, and using an on-site Meetecho tool to give local attendees remote-like functions.

1. The challenges of hybrid meetings
Lorenzo Miniero
@lminiero@fosstodon.org
CommCon 2023
June 20th 2023, Mercure Blackburn Dunkenhalgh Hotel & Spa (UK)

2. I’ve been doing VoIP/WebRTC for way too long!
Lorenzo Miniero
• Ph.D @ UniNA
• Chairman @ Meetecho
• Main author of Janus
Contacts and info
• lorenzo@meetecho.com
• https://fosstodon.org/@lminiero
• https://www.slideshare.net/LorenzoMiniero
• https://lminiero.bandcamp.com

3. Just a few words on Meetecho
• Co-founded in 2009 as an academic spin-off
• University research efforts brought to the market
• Completely independent from the University
• Focus on real-time multimedia applications
• Strong perspective on standardization and open source
• Several activities
• Consulting services
• Commercial support and Janus licenses
• Streaming of live events (IETF, RIPE, etc.)
• Proudly brewed in sunny Napoli(*), Italy

4. Just a few words on Meetecho
• Co-founded in 2009 as an academic spin-off
• University research efforts brought to the market
• Completely independent from the University
• Focus on real-time multimedia applications
• Strong perspective on standardization and open source
• Several activities
• Consulting services
• Commercial support and Janus licenses
• Streaming of live events (IETF, RIPE, etc.)
• Proudly brewed in sunny Napoli(*), Italy

5. ((*)
A different picture, for once!)

6. Streaming live events via WebRTC pre-pandemic
• Service we provided for several different events
• Mainly IETF (official remote participation tool) and some ACM conferences
• A few other events too (a growing business)
• Typically involved some specific activities
• Fly a couple people + equipment there, of course
• Setup cameras, split/capture projector signals, hooks to room mixer
• Embed some event-specific channel (e.g., Twitter feed, or Sli.do channel)
• ... and then stream everything via WebRTC (sometimes allowing remote speakers)
Until the world changed...
No more live events in person for a while!

7. Streaming live events via WebRTC pre-pandemic
• Service we provided for several different events
• Mainly IETF (official remote participation tool) and some ACM conferences
• A few other events too (a growing business)
• Typically involved some specific activities
• Fly a couple people + equipment there, of course
• Setup cameras, split/capture projector signals, hooks to room mixer
• Embed some event-specific channel (e.g., Twitter feed, or Sli.do channel)
• ... and then stream everything via WebRTC (sometimes allowing remote speakers)
Until the world changed...
No more live events in person for a while!

8. Streaming live events via WebRTC pre-pandemic
• Service we provided for several different events
• Mainly IETF (official remote participation tool) and some ACM conferences
• A few other events too (a growing business)
• Typically involved some specific activities
• Fly a couple people + equipment there, of course
• Setup cameras, split/capture projector signals, hooks to room mixer
• Embed some event-specific channel (e.g., Twitter feed, or Sli.do channel)
• ... and then stream everything via WebRTC (sometimes allowing remote speakers)
Until the world changed...
No more live events in person for a while!

9. A look at how that worked for IETF meetings

10. A look at how that worked for IETF meetings

11. A look at how that worked for IETF meetings

12. A look at how that worked for IETF meetings

13. A look at how that worked for IETF meetings

14. A look at how that worked for IETF meetings

15. A look at how that worked for IETF meetings

16. A look at how that worked for IETF meetings

17. A look at how that worked for IETF meetings

18. A simpler approach for other conferences (e.g., ACM)

19. Streaming JanusCon (no remote pizza, though)

20. Then a global pandemic hit...

21. Then a global pandemic hit...

22. Reinventing the platform to go fully virtual
https://commcon.xyz/session/turning-live-events-to-virtual-with-janus

23. Eating our own dog food
Janus
General purpose, open source WebRTC server
• https://github.com/meetecho/janus-gateway
• Demos and documentation: https://janus.conf.meetecho.com
• Community: https://janus.discourse.group/

24. Eating our own dog food
https://www.slideshare.net/LorenzoMiniero/scaling-webrtc-applications-with-janus
https://www.youtube.com/watch?v=zxRwELmyWU0

25. Mixed Audio + Streaming

26. Mixed Audio + Streaming

27. Mixed Audio + Streaming

28. SFU Video + Streaming

29. SFU Video + Streaming

30. Bringing it all together for Virtual Events

31. Bringing it all together for Virtual Events

32. Bringing it all together for Virtual Events

33. Bringing it all together for Virtual Events

34. Then, eventually, the world started opening up again

35. New challenges to overcome
• Before the pandemic, remote participants were few and far between
• Limited number of remote presentations
• Occasional remote question from the queue
• Focus was more on the venue and the local attendees
• In virtual platform, remote participants were not 2nd class citizens anymore...
• ... they were the only ones there!
• Every functionality shared with all other attendees
• After the pandemic, what to expect from a “hybrid” meeting instead?
• At the very least, a much more even distribution
• ~50% local attendees, ~50% remote participants
• How to get them to seamlessly interact with each other?

36. New challenges to overcome
• Before the pandemic, remote participants were few and far between
• Limited number of remote presentations
• Occasional remote question from the queue
• Focus was more on the venue and the local attendees
• In virtual platform, remote participants were not 2nd class citizens anymore...
• ... they were the only ones there!
• Every functionality shared with all other attendees
• After the pandemic, what to expect from a “hybrid” meeting instead?
• At the very least, a much more even distribution
• ~50% local attendees, ~50% remote participants
• How to get them to seamlessly interact with each other?

37. New challenges to overcome
• Before the pandemic, remote participants were few and far between
• Limited number of remote presentations
• Occasional remote question from the queue
• Focus was more on the venue and the local attendees
• In virtual platform, remote participants were not 2nd class citizens anymore...
• ... they were the only ones there!
• Every functionality shared with all other attendees
• After the pandemic, what to expect from a “hybrid” meeting instead?
• At the very least, a much more even distribution
• ~50% local attendees, ~50% remote participants
• How to get them to seamlessly interact with each other?

38. What about the infrastructure?
• Pretty much the same as the virtual one!
• Needed to accomodate potentially large remote attendance
• Feeds from the venue can be scaled automatically as “virtual participants”
• Streams from the room similar to before, with a few tweaks
• Audio mixer is now the Janus AudioBridge, not SIP-based
• Static video feeds are not static anymore, but dynamic
• Support for multiple cameras per room
• Local and remote streams can be consumed dynamically in the venue
• e.g., slides (more on that later)
• Remote presenters to display in the room
• Virtual queues, polls, transcriptions, etc.

39. What about the infrastructure?
• Pretty much the same as the virtual one!
• Needed to accomodate potentially large remote attendance
• Feeds from the venue can be scaled automatically as “virtual participants”
• Streams from the room similar to before, with a few tweaks
• Audio mixer is now the Janus AudioBridge, not SIP-based
• Static video feeds are not static anymore, but dynamic
• Support for multiple cameras per room
• Local and remote streams can be consumed dynamically in the venue
• e.g., slides (more on that later)
• Remote presenters to display in the room
• Virtual queues, polls, transcriptions, etc.

40. What about the infrastructure?
• Pretty much the same as the virtual one!
• Needed to accomodate potentially large remote attendance
• Feeds from the venue can be scaled automatically as “virtual participants”
• Streams from the room similar to before, with a few tweaks
• Audio mixer is now the Janus AudioBridge, not SIP-based
• Static video feeds are not static anymore, but dynamic
• Support for multiple cameras per room
• Local and remote streams can be consumed dynamically in the venue
• e.g., slides (more on that later)
• Remote presenters to display in the room
• Virtual queues, polls, transcriptions, etc.

41. Handling local and remote audio (before)

42. Handling local and remote audio (now)

43. Static camera and projector (before)

44. Adding cameras and making video more dynamic

45. Adding cameras and making video more dynamic

46. Adding cameras and making video more dynamic

47. What about the duplicated feed from the projector?
• Before the pandemic, we’d use a splitter
• Local attendee would plug the laptop to the splitter
• Splitter would feed the content to projector and our streamer
• Streamer would send the content as video remotely
• Got rid of that for fully hybrid events
• All presentation content is always shared via Meetecho, even when local
• Presentation can be light (preloaded slides) or WebRTC-based
• In the venue, agent shows slides/screen feed on main screen
• This allows to never worry about local vs. remote content
• No matter where it’s from, it’s published via WebRTC (or preloaded slides)
• No matter where it’s from, local and remote attendees consume it the same way

48. What about the duplicated feed from the projector?
• Before the pandemic, we’d use a splitter
• Local attendee would plug the laptop to the splitter
• Splitter would feed the content to projector and our streamer
• Streamer would send the content as video remotely
• Got rid of that for fully hybrid events
• All presentation content is always shared via Meetecho, even when local
• Presentation can be light (preloaded slides) or WebRTC-based
• In the venue, agent shows slides/screen feed on main screen
• This allows to never worry about local vs. remote content
• No matter where it’s from, it’s published via WebRTC (or preloaded slides)
• No matter where it’s from, local and remote attendees consume it the same way

49. What about the duplicated feed from the projector?
• Before the pandemic, we’d use a splitter
• Local attendee would plug the laptop to the splitter
• Splitter would feed the content to projector and our streamer
• Streamer would send the content as video remotely
• Got rid of that for fully hybrid events
• All presentation content is always shared via Meetecho, even when local
• Presentation can be light (preloaded slides) or WebRTC-based
• In the venue, agent shows slides/screen feed on main screen
• This allows to never worry about local vs. remote content
• No matter where it’s from, it’s published via WebRTC (or preloaded slides)
• No matter where it’s from, local and remote attendees consume it the same way

50. Displaying content in the venue (before)

51. Displaying content in the venue (now)

52. Displaying content in the venue (now)

53. One queue to rule them all!
• Historically, the IETF relied on mic lines for queues
• You’d stand up in line and wait for your turn
• Caused issues with remote speakers (when was their turn?)
• Partially handled at the time with “Pacman ghosts” + Big Red Button
• In fully virtual platform, not an issue
• Everyone remote, so everyone in the same queue!
• More of a problem in hybrid scenario
• Many remote and local attendees alike
• Solved by putting them all in the same queue
• Local attendees use “companion” web app to get in line
• No need to stand in line, they can do that when it’s their turn

54. One queue to rule them all!
• Historically, the IETF relied on mic lines for queues
• You’d stand up in line and wait for your turn
• Caused issues with remote speakers (when was their turn?)
• Partially handled at the time with “Pacman ghosts” + Big Red Button
• In fully virtual platform, not an issue
• Everyone remote, so everyone in the same queue!
• More of a problem in hybrid scenario
• Many remote and local attendees alike
• Solved by putting them all in the same queue
• Local attendees use “companion” web app to get in line
• No need to stand in line, they can do that when it’s their turn

55. One queue to rule them all!
• Historically, the IETF relied on mic lines for queues
• You’d stand up in line and wait for your turn
• Caused issues with remote speakers (when was their turn?)
• Partially handled at the time with “Pacman ghosts” + Big Red Button
• In fully virtual platform, not an issue
• Everyone remote, so everyone in the same queue!
• More of a problem in hybrid scenario
• Many remote and local attendees alike
• Solved by putting them all in the same queue
• Local attendees use “companion” web app to get in line
• No need to stand in line, they can do that when it’s their turn

56. One queue to rule them all!
• Historically, the IETF relied on mic lines for queues
• You’d stand up in line and wait for your turn
• Caused issues with remote speakers (when was their turn?)
• Partially handled at the time with “Pacman ghosts” + Big Red Button
• In fully virtual platform, not an issue
• Everyone remote, so everyone in the same queue!
• More of a problem in hybrid scenario
• Many remote and local attendees alike
• Solved by putting them all in the same queue
• Local attendees use “companion” web app to get in line
• No need to stand in line, they can do that when it’s their turn

57. Using the full Meetecho tool

58. Using the on-site Meetecho tool

59. Using the on-site Meetecho tool

60. Displaying the queue in the room

61. The on-site Meetecho tool
• Via QR code, but also easily accessible from the main agenda
• Uses same credentials to authenticate
• Presents limited set of Meetecho features
• Allows local attendees to use some “remote” features seamlessly
• Getting in line to talk (queue)
• Advancing slides, if preloaded (remote clicker)
• Raising/not raising hands when needed (polls)
• Local attendees recognized from remote ones
• Queue displays different icons depending on device/location
• “Bluesheets” automatically filled in

62. The on-site Meetecho tool
• Via QR code, but also easily accessible from the main agenda
• Uses same credentials to authenticate
• Presents limited set of Meetecho features
• Allows local attendees to use some “remote” features seamlessly
• Getting in line to talk (queue)
• Advancing slides, if preloaded (remote clicker)
• Raising/not raising hands when needed (polls)
• Local attendees recognized from remote ones
• Queue displays different icons depending on device/location
• “Bluesheets” automatically filled in

63. The on-site Meetecho tool
• Via QR code, but also easily accessible from the main agenda
• Uses same credentials to authenticate
• Presents limited set of Meetecho features
• Allows local attendees to use some “remote” features seamlessly
• Getting in line to talk (queue)
• Advancing slides, if preloaded (remote clicker)
• Raising/not raising hands when needed (polls)
• Local attendees recognized from remote ones
• Queue displays different icons depending on device/location
• “Bluesheets” automatically filled in

64. Show of hands (“hum” replacement)

65. New from IETF 116: live transcriptions

66. New from IETF 116: live transcriptions

67. New from IETF 116: live transcriptions

68. New from IETF 116: live transcriptions

69. Shared bus to control all the local agents
• Every device in the venue rooms has an agent to control it remotely
• Audio bridge device
• Raspberry Pis controlling cameras and screen duplication
• Raspberry Pis to display content on main and second screen
• All devices register at a controller and wait for commands
• Dynamic behaviour can be orchestrated remotely (e.g., from the NOC)
• Multiple actions that can be triggered
• e.g., start/stop audio, change gains, etc.
• e.g., start/stop cameras, pan/tilt/zoom, etc.
• e.g., start/stop content displays, reset/restart, etc.

70. Shared bus to control all the local agents
• Every device in the venue rooms has an agent to control it remotely
• Audio bridge device
• Raspberry Pis controlling cameras and screen duplication
• Raspberry Pis to display content on main and second screen
• All devices register at a controller and wait for commands
• Dynamic behaviour can be orchestrated remotely (e.g., from the NOC)
• Multiple actions that can be triggered
• e.g., start/stop audio, change gains, etc.
• e.g., start/stop cameras, pan/tilt/zoom, etc.
• e.g., start/stop content displays, reset/restart, etc.

71. Shared bus to control all the local agents
• Every device in the venue rooms has an agent to control it remotely
• Audio bridge device
• Raspberry Pis controlling cameras and screen duplication
• Raspberry Pis to display content on main and second screen
• All devices register at a controller and wait for commands
• Dynamic behaviour can be orchestrated remotely (e.g., from the NOC)
• Multiple actions that can be triggered
• e.g., start/stop audio, change gains, etc.
• e.g., start/stop cameras, pan/tilt/zoom, etc.
• e.g., start/stop content displays, reset/restart, etc.

72. Monitoring and controlling sessions

73. Monitoring and controlling sessions

74. Monitoring and controlling sessions

75. Monitoring and controlling sessions

76. Monitoring and controlling sessions

77. What about NDI?
• Wrote about this a few times
• https://www.meetecho.com/blog/webrtc-ndi/
• https://www.meetecho.com/blog/webrtc-ndi-part-2/
• Spoke about this a few times too
• ClueCon: https://youtu.be/RZ58vapWfUw?t=1755
• CommCon:
https://2021.commcon.xyz/talks/whip-ndi-and-janus-genesis-of-a-broadcasting-demo
• Used in practice @ CommCon 2021
• Janus NDI plugin part of early Broadcaster.vc stack
• Could be very useful for patching streams in/out of the venue
• Better interaction with AV team workflows
• Not currently used in our own workflows (yet?), though

78. What about NDI?
• Wrote about this a few times
• https://www.meetecho.com/blog/webrtc-ndi/
• https://www.meetecho.com/blog/webrtc-ndi-part-2/
• Spoke about this a few times too
• ClueCon: https://youtu.be/RZ58vapWfUw?t=1755
• CommCon:
https://2021.commcon.xyz/talks/whip-ndi-and-janus-genesis-of-a-broadcasting-demo
• Used in practice @ CommCon 2021
• Janus NDI plugin part of early Broadcaster.vc stack
• Could be very useful for patching streams in/out of the venue
• Better interaction with AV team workflows
• Not currently used in our own workflows (yet?), though

79. What about NDI?
• Wrote about this a few times
• https://www.meetecho.com/blog/webrtc-ndi/
• https://www.meetecho.com/blog/webrtc-ndi-part-2/
• Spoke about this a few times too
• ClueCon: https://youtu.be/RZ58vapWfUw?t=1755
• CommCon:
https://2021.commcon.xyz/talks/whip-ndi-and-janus-genesis-of-a-broadcasting-demo
• Used in practice @ CommCon 2021
• Janus NDI plugin part of early Broadcaster.vc stack
• Could be very useful for patching streams in/out of the venue
• Better interaction with AV team workflows
• Not currently used in our own workflows (yet?), though

80. What about NDI?
• Wrote about this a few times
• https://www.meetecho.com/blog/webrtc-ndi/
• https://www.meetecho.com/blog/webrtc-ndi-part-2/
• Spoke about this a few times too
• ClueCon: https://youtu.be/RZ58vapWfUw?t=1755
• CommCon:
https://2021.commcon.xyz/talks/whip-ndi-and-janus-genesis-of-a-broadcasting-demo
• Used in practice @ CommCon 2021
• Janus NDI plugin part of early Broadcaster.vc stack
• Could be very useful for patching streams in/out of the venue
• Better interaction with AV team workflows
• Not currently used in our own workflows (yet?), though

81. What’s next?
• A refactoring of the web UI
• Almost finished!
• Completely redesigned the UI, to make it mobile-ready too
• Better usage of the ton of features we added (and add!) over time
• Improvements on transcription
• Works fine already, but needs tailoring to specific context(s)
• Expanding the events base
• Bespoke solutions for big customers (e.g., IETF, RIPE, etc.)
• Doing the same for other events, possibly with modular feature set

82. What’s next?
• A refactoring of the web UI
• Almost finished!
• Completely redesigned the UI, to make it mobile-ready too
• Better usage of the ton of features we added (and add!) over time
• Improvements on transcription
• Works fine already, but needs tailoring to specific context(s)
• Expanding the events base
• Bespoke solutions for big customers (e.g., IETF, RIPE, etc.)
• Doing the same for other events, possibly with modular feature set

83. What’s next?
• A refactoring of the web UI
• Almost finished!
• Completely redesigned the UI, to make it mobile-ready too
• Better usage of the ton of features we added (and add!) over time
• Improvements on transcription
• Works fine already, but needs tailoring to specific context(s)
• Expanding the events base
• Bespoke solutions for big customers (e.g., IETF, RIPE, etc.)
• Doing the same for other events, possibly with modular feature set

84. Thanks! Questions? Comments?
Get in touch!
• https://fosstodon.org/@lminiero
• https://twitter.com/elminiero
• https://twitter.com/meetecho
• https://twitter.com/januswebrtc
• https://www.meetecho.com/blog/