Recommended
Recommended
More Related Content
Similar to IMA/Thales SceneGate Viewer for OpenSimulator Presentation at OSCC19
Similar to IMA/Thales SceneGate Viewer for OpenSimulator Presentation at OSCC19 (20)
Recently uploaded
Recently uploaded (20)
IMA/Thales SceneGate Viewer for OpenSimulator Presentation at OSCC19
- 1. OSCC 2019 SCENEGATE VIEWER FOR OPENSIMULATOR LISA LAXTON NATACHA BRU
- 2. 2 Do we need a new viewer for OpenSimulator? ▌ Development Gaps Use Cases: Accessibility by design is needed to combat digital marginalization - Hearing Impaired - Mobility Impaired - Vision Impaired - Cognition Impaired Use Cases: Onboarding remains a challenge for new users - Classes and Training Sessions - Meetings and Conferences - Interactive Environments
- 3. 3 Analysis ▌ Development Challenges Original project priorities Different stakeholders Third-party project priorities Standards compliance Documentation
- 4. 4 Analysis ▌ Development Solution New development team Stakeholder participation Future viewer research
- 5. 5 Design Scope ▌ Requirements Open Source Platform agnostic OpenSimulator agnostic Adaptable Customizable Standards compliant ▌ Primary Stakeholders New users Disabled users Grid owners Creators Educators Collaborators
- 6. 6 Solution ▌ Alchemy Viewer codebase selected for Project SceneGate ▌ Current Development Focus Accessibility Onboarding Performance Usability ▌ Future Development Focus EchoVoice Integration Renderer decoupling Standards Security Documentation Advanced Mode The SceneGate Viewer provides the user with a gate to a 3D scene in a virtual world.
- 7. 7 Project Roadmap ▌ 2018 Research Surveys Analyses and feedback Extensive testing Proof of concept Codebase selection ▌ 2019 Development Alpha Development Closed Alpha Testing Beta Development Closed Beta Testing Open Beta Testing
- 8. 8 Who’s Involved ▌ Participation Hypergrid users Development team members Industry interns (students) ▌ Design Collaborative approach HCI / Usability reviews Industry and usability standards
- 9. 9 What’s New: Accessibility Improvements ▌ New feature related to Hearing, Cognition, and Mobility Issues Listen from All Positions
- 10. 10 What’s New: Accessibility Improvements ▌ New feature related to Vision and Cognition Issues Default Custom Colors and Contrast Options
- 11. 11 What’s New: Onboarding Improvements ▌ New modes take a bulldozer to the learning curve Simplified mode presents fewer menu options
- 12. 12 What’s New: Onboarding Improvements ▌ New user interface layout per mode Minimal toolbar buttons in simplified mode
- 13. 13 What’s New: Onboarding Improvements ▌ New preferences option to change modes Extended mode for experienced users
- 14. 14 What’s New: Onboarding Improvements ▌ New standard avatar camera views Navigation is eased within virtual structures
- 15. 15 What’s New: Usability Improvements ▌ New code for grid manager error checking and management Users get immediate feedback
- 16. 16 What’s New: Usability Improvements ▌ New button group layout related to human computer interaction design Toolbar button groups for both modes
- 17. 17 What’s New: Performance Improvements ▌ Improved graphics handling Bug fix Resolves issue caused by Win 10 1903 update ▌ Dynamic texture loading time reduced Beneficial for presentation screens in-world Faster rendering of a full textured scene
- 18. 18 Future Viewer Viewer Render Application ObjectStore RenderInterface Viewer Data Handling OpenSimInterface OpenSim Server VoIPPlugin(s) Viewer Data Handling Application Future version of the SceneGate Viewer UserInterface Plugin UserInterface Plugin
- 19. 19 Future Viewer ▌ Modern Look and Feel Use gaming engines & modern UI packages, split rendering from data handling part viewer (mainly the loop over the server) - Viewer Render Application -> render engine that’s desired for a specific use - Object store <- input to the renderer - Render Interface -> to integrate the renderer with the other parts of the viewer ▌ Customizable & simplified UI (HMI) Design a new UI that allows users to customize for their needs in a simple way. Components include: - VoIP plugins(s) different VoIP services to be connected to the viewer - User Interface Plugins customize the User Interface for the viewer - Other plugins that can be defined in the future
- 20. 20 Future Viewer ▌ Maintainability Create a clear and structured documented design Refactor the SceneGate code if needed Create a package (OpenSim Interface) to handle the interface with OpenSim - Separate the internals from the OpenSim interface Design using state of the art libraries and packages to optimize security ▌ Current development State Splitting the renderer & data handling parts of the viewer 2 Student interns are working on this
- 21. 21 Directly Related Projects ▌ Project Echo Future EchoVoice Integration Future Text to Speech and Speech to Text R&D ▌ Project DreamGate (Firestorm fork) Future accessibility improvements Future usability improvements ▌ Project Helios Metaverse Depot R&D Grid (ADA or 508 compliance) OpenSimulator R&D
- 22. 22 Where IMA does R&D ▌ IMA Metaverse Depot R&D Grid grid.metaversedepot.com:8002 IMA Projects in Decentralized Virtual Worlds
- 23. 23 Why is security important ? ▌ Security is a matter of TRUST > Preserve users trust by granting the safety of their data > Security is important for every user Military Civilians ▌ Currently the viewer is not secure ▌ How to improve security ?
- 24. 24 Login ▌ Credentials are sensible information > The viewer sends username and MD5 hash of the password to the server ▌ MD5 should not be used - Cryptographic collisions - Brute force attacks - Rainbow tables attacks ▌ How to improve security ? - SHA256 or SHA512 - Salt hashes - implement TLS
- 25. 25 Exchanges with the server ▌ Sensible data can be exchanged > The viewer exchanges different information with the server ▌ Currently there is no data encryption - User information - Avatar information - Video/media links - Code - Chat messages (base 64 encoded zip file) - etc. ▌ How to improve security ? - Use Transport Layer Security (TLS) - Encrypt sensible data ⚠ Server-side modifications
- 26. 26 Third-party libraries ▌ Usage of third-party libraries is needed > Gain of time > Better tools for development ▌ But it increases the attack surface - A lot of lines of code - libraries calling other libraries - known vulnerabilities - outdated libraries - no longer supported libraries ▌How to improve security ?
- 27. 27 Third-party libraries 79% 15% 6% libraries version overview old up to date unknown 1% 40% 38% 21% Risk severity repartition in third-party libraries Low Medium High Critical
- 28. 28 Third-party libraries ▌ Usage of third-party libraries is needed > Gain of time > Better tools for development ▌ But it increases the attack surface - A lot of lines of code - libraries calling other libraries - known vulnerabilities - outdated libraries - no longer supported libraries ▌ How to improve security ? - Update libraries - Watch for new vulnerabilities - Stay up to date
- 29. 29 Conclusion ▌ Changes must be done on server side and client side > No implementation of TLS = no security > Be careful of how data are sent ▌ Security is an everyday challenge > Need to watch for new vulnerabilities > Need regular updates ▌Thank you for your attention
- 30. 30 Expanding Development ▌ We need your help with Wiki, Testing, Contributing, and Developing Join Project SceneGate! Code repository opening soon
- 31. 31 Panel Q&A Thank you for your attention Panel Discussion Lisa Laxton – lisa@infinitemetaverse.com Natacha Bru – natacha.bru12@gmail.com Frank Rulof – frank.rulof@hotmail.com
- 32. 32 Project SceneGate Links ▌ Downloads http://downloads.infinitemetaverse.org/index.php/downloads ▌ Release Notes https://bitbucket.org/imadevelopmentteam/scenegate- viewer/wiki/Release%20Notes:/Latest%20Beta%20Test%20Version ▌ Issue Tracker https://bitbucket.org/imadevelopmentteam/scenegate- viewer/wiki/Policies%20and%20Procedures:/Report%20a%20Viewer%20Bug ▌ Roadmap https://bitbucket.org/imadevelopmentteam/scenegate- viewer/wiki/Development%20Resources:/Project%20Roadmap ▌ Wiki Home https://bitbucket.org/imadevelopmentteam/scenegate-viewer/wiki/Home
Editor's Notes
- Good Morning! Thank you for allowing us to present our work on the new SceneGate Viewer for OpenSimulator. Since the open beta launch, there have been quite a few questions. We hope to have time for Q&A with our panel after the slide presentation. Infinite Metaverse Alliance (IMA) and its strategic partner Thales share a common mission to focus on inclusive "design thinking" to advance virtual worlds, virtual reality, and synthetic environments. Project SceneGate is one of several integrated projects supporting this mission. The SceneGate Viewer provides the user with a gate to a 3D scene in a virtual world.
- We asked the research question from a use case perspective: Do we need a new viewer for OpenSimulator? The answer is yes. As a result of the research, development gaps were revealed and use cases defined. Accessibility by design is needed to combat digital marginalization. This occurs when the needs of users with disabilities are not met. Hearing impaired users may be unable to communicate effectively using the listen from avatar position option. Mobility impaired users may be unable to communicate effectively when frequently changing the avatar camera position. Visually impaired users may have an immediate need for variations in color and contrast. Cognition impaired users may face immediate challenges related to hearing, mobility, or vision stressors. Onboarding remains a challenge due to steep learning curves and orientation time. Specific virtual world user cases include: Classes with users who are students and educators; Meetings with users who are attendees and speakers; and Interactive Environments with users who are new users or trainees. Usability improvements are needed to put the power into the hands of the users and to accommodate these needs.
- During our analysis, non-trivial challenges in development were found. The original Linden Lab source code project developers have priorities related to its business focus. OpenSimulator stakeholders are not a consideration for original source developers. Existing third-party viewer project developer priorities are divergent. The lack of standards compliance impacts interoperability with other applications. Documentation is non-existent or not up-to-date. It’s time to stop “doing it this way”.
- It became clear open source software forks were needed after extensive discussion and investigation into existing project roadmaps and priorities. So we began forming a new development team to meet immediate needs identified and consider “users as developers”. Surveys were conducted and software analyzed to identify and involve stakeholders. Through strategic collaboration, IMA and Thales are implementing systems engineering approaches to support future viewer development.
- The viewer design scope defines requirements and stakeholders. Requirements defined include: Open Source and distributable; Reliably run on Windows, Mac, and Linux systems; Support all active versions or derivatives of OpenSimulator; Based on an adaptable modern codebase; Be customizable by users and organizations; and Be standards compliant where possible. The primary stakeholders are: New users or those attending classes and meetings; Disabled users of all skill levels; Grid owners focused on user needs; Creators of interactive environments and objects; Educators using virtual worlds; and Collaborators using virtual worlds.
- The Alchemy Viewer codebase was selected for the SceneGate Viewer. Our current development focus is on immediate improvements related to accessibility, onboarding, performance, and usability. Our future development focus includes EchoVoice Integration, renderer decoupling, standards compliance, improved security, better documentation, and development of an advanced mode.
- Viewer efforts for IMA began with research in 2018. This involved: previous community and viewer user surveys; survey analyses and community feedback; extensive testing of multiple viewers by multiple stakeholders; proof of concept called Project EduGate; and available codebase feature analyses. Systems engineering milestones were established for the development timeline and project roadmap. Specific milestones included: Alpha Development, Closed Alpha Testing, Beta Development, Closed Beta Testing, and Open Beta Testing which is active now.
- Project SceneGate has various areas of participation in its design approach. Users from multiple grids with different OpenSimulator versions are involved in testing. Experienced development team members are active in OpenSimulator communities. Industry interns (students) are also part of the development team. IMA meeting attendees voted on most default preference changes. HCI / Usability design reviews drove initial User Interface changes. Industry standards and usability design drove default avatar camera changes. The most frequently asked question has been: What’s New? So let’s see some screeshots!
- The ability to hear voice equally or listen from all positions was added and established as a default. This feature helps users who may have hearing, cognitive, or mobility issues communicate in voice immediately. Without this capability, users only had the option to hear from avatar position or camera position creating a hostile environment for people with disabilities - also known as digital marginalization. While this is equally important for new users in simplified mode, it exists for users in extended mode as well.
- The default user interface skin for SceneGate provides users the ability to customize the color and contrast with far greater choices than other skins to meet their needs immediately. The improvement provides greater accessibility by design to specifically improve the user experience for those with visual or cognitive issues. This user interface skin is the default for both simplified and extended modes.
- To reduce cognitive overload caused by too many menu items and buttons on a new user's screen, a simplified mode was developed and made the default mode. In this mode, a limited number of user menu options effectively taking a bulldozer to the steep learning curve.
- Toolbar button defaults presented to new users are also limited to only those needed to learn to use virtual worlds. This further reduces onboarding time for new users.
- When new users are ready to learn more, they do not need to download and learn how to use another viewer. Extended mode is available in preferences with a few clicks. In this mode, the user is presented with more menu items and toolbar buttons. Their user experience is "extended" rather than changed.
- The default avatar camera settings were adjusted to provide a more natural, contemporary or modern sense of virtual presence in third person view. This change improves ease of navigation inside virtual structures and applies to all modes. Advanced users can change their camera settings temporarily or permanently.
- Grid Manager error checking was added to prevent a user adding a grid that is not available or that has a typo in the entry. Users get immediate feedback when this occurs aiding grid list management. Users can also load the short default grid list when needed - the usability mantra is "Don't Make Me Think". This feature is available in both simplified and extended modes.
- Default toolbar button locations were grouped by function categories based on human computer interaction design review. For simplified mode, the left-side button group presents two tools associated with the user's account. The right-side button group presents two tools associated with places the avatar can go and how to get there. The center-bottom button group presents commonly used buttons to communicate, navigate, and find people. Extended mode can maintain the grouping but present more toolbar buttons in each group. Users with specific disability needs or preferences can customize the toolbar button layout if needed.
- With any software project, we will always need to fix bugs and improve performance. Shortly after our fork, Windows released an update that caused the viewer to crash on exit. This was our first bug to fix. Dynamic texture loading times was our first performance improvement reducing loading times and was noticed by new users.
- Research is underway for a future version of the SceneGate Viewer for existing and new use case applications. The initial activity involves splitting the renderer & data handling parts of the viewer.
- Development goals include a modern look and feel by using gaming and modern UI packages as the code is developed. If the R&D is successful, gaming, VR, and web render engines can be used with the viewer. We also want to provide users with new user UI options to support new features designed as plugins.
- One of the primary considerations involves maintainability. This includes establishing a clear, structured, and documented design. This will likely involve some refactoring of the existing SceneGate codebase. A new package seen in the diagram as OpenSim Interface is needed to begin decoupling the renderer. Design goals also include the use of modern libraries and packages to optimize security. Two student interns at Thales are actively working on this R&D.
- IMA has seven primary project areas with most having some relation or integration to at least one other. Three of these are directly related to Project SceneGate. Project Echo includes EchoVoice source code which will integrate with the SceneGate Viewer to provide seamless configuration of different voice solutions. A future roadmap item common to viewer and voice projects, includes text to speech and speech to text capabilities to address accessibility and provide live transcription by design. Project DreamGate is a Firestorm Viewer fork for OpenSimulator. Improvements are needed so those with disabilities who prefer the Firestorm user interface do not experience digital marginalization. Project Helios encompasses OpenSimulator R&D and support for all IMA projects on the Metaverse Depot Grid. Grid work directly related to Project SceneGate includes bringing the DIVA WiFi user pages into standards compliance with the Americans with Disabilities Act.
- The IMA Metaverse Depot Grid extends Research and Development championed by the former MOSES team. MOSES In A Box used a single virtual machine image of a grid to deliver a platform agnostic “plug and play” solution. The research question we asked was: Can multiple virtual machines be used to decentralize virtual worlds on an OpenSimulator grid and what will the benefits be? The research answer is yes with tradeoffs. OpenSimulator related development has evolved from this research. Some benefits include segregation, ease of installation, ease of administration, and ease of remote machine connections. Tradeoffs include increased vCPU and RAM resources, but the approach will deliver a platform agnostic “plug and play” solution called IMABox. Work continues with a focus on further easing grid administration, security of remote machine connections, and security of SceneGate Viewer connections. On that note, please welcome Natacha who will discuss her research around security issues.
- Security is important because it is a matter of TRUST. We should preserve users trust by granting the safety of their data. Security is important for every user: both military and civilian. Unfortunately, currently the viewer is not secure.
- Credentials are sensible information. The viewer sends username and MD5 hash of the password to the server. MD5 should not be used due to risk of cryptographic collisions, brute force attacks, and rainbow tables attacks. Security can be improved with SHA256 or SHA512, salt hashes, and the implementation of TLS.
- Sensible data can be exchanged with the server. Currently there is no data encryption of user information, avatar information, links, code, or chat. Server-side modifications are needed to implement TLS and encrypt sensible data.
- Usage of third-party libraries is needed for a gain in time and better tools for development. But this can increase the attack surface with more lines of code, libraries calling others, known vulnerabilities, and outdated or unsupported libraries.
- 79% of libraries are old while 15% are up-to-date. This leaves 6% with unknown status. There are risks that can be classified by severity: 1% low, 40% medium, 38% high, and 21% critical.
- There are steps that can be taken to minimize risks associated with third-party libraries. Update the libraries, watch for new vulnerabilities, and stay up-to-date.
- In conclusion, changes must be done on the server side and the client side to improve security. No implementation of TLS equals no security. We must be careful how data is sent. Security is an everyday challenge because we need to watch for new vulnerabilities and do regular updates.
- The repository wiki structure is built but we need help to finish and maintain it. Items to finish include tutorials and frequently asked questions. Build instructions for Mac are pending. Anyone can participate in testing and bug reporting or even contribute bug fixes. We plan to expand the development team as work progresses. If you are interested, contact me!
- Thank you for joining our session today! We have some time for discussion. Are there any questions for the panel?