Understanding ocean dynamics is a challenging and costly endeavor that requires advanced technologies and methods to collect, process and represent data. An essential aspect of a digital twin for the ocean is the capability to visualize multi-source heterogeneous data in a digital representation. We propose an agnostic technology solution that supports collaborative, interactive and multiplatform visualization in the context of digital twins using virtual choreographies

2. A 3D Immersive novel tool for
digital twins using virtual
choreographies: the oil spill use
case
International Forum on Database Management
and Digital Transformation for Marine Conservation
Fernando Cassola (INESC TEC)
Alexandre Carvalho (INESC TEC), Demetrius Lacet (INESC TEC) and
Marco Amaro Oliveira (INESC TEC)
06.03.2024

3. Ocean dynamics requires advanced technologies and methods to collect,
process and represent data
https://rfablet.github.io/images/im-datawave.jpg

4. The Digital Twin for the ocean must visualize multi-source heterogeneous data in a digital representation
https://ditto-oceandecade.org/wp-content/uploads/2023/04/Group-25-min.png

5. What is our proposal?

6. Conceptual pipeline

7. Sets of interconnected
behaviors, interactions
and events that take place
in a given time and space,
with well-defined
objectives and rules
Virtual Choreographies
Cassola et al. (2022). Design and Evaluation of a Choreography-Based Virtual Reality Authoring Tool for Experiential Learning in Industrial Training. IEEE Transactions on Learning Technologies, 15(5), 526-539.

8. Let’s get into details…

9. • Run simulations (OceanDrift)
• Represent simulations as time-
changing 3D particle systems
• Represent 3D particle systems as
temporal snapshots of density
contours
• Based on contours, encode change
and alert user depending on user-
defined types of events of interest
[ESTIMATED].
Filtering and analysis
(WP5 execution model) WP5 WP6
WP4

10. Filtering and analysis WP5 WP6
WP4
• Inputs
• Oilpill Simulation (date/time and location)
NetCDF resource
• Bathimetry NetCDF resource
• Coastal areas NetCDF resource
• Points of interest resource
• Requirements:
• Hability to consult registry and load existing
resources
• Hability to access selected resources
• For points of interest, a characterization is
required, for instance type and corresponding
3D representation.
• Outputs
• Processed data
• Extracted 2D boundaries (oil spill density
contours, coastal área delimitation)
• Extracted 3D shapes (terrain elevation)
• Extracted 3D particle systems
• Extracted location for points of interest
• Some of the previous features are spatial
while others are spatiotemporal (change
location, shape, both)
• Requirements:
• Hability to store there results and call them for distinct
commands.
• Standalone WP5-compatible commands that perform
the transformations

11. • Visual metaphors that transform
processed data into a visually
compatible representation
• A representational model that is
rendering technology neutral, but that
encompasses the knowledge to allow
the render the visual result.
Visual mapping model
* Virtual choreography encoding
• Inputs
Processed data
• Outputs
Virtual choreography
xAPI Json
• Requirements:
A virtual choreography encoder
that fetches processed data,
other user inputs and writes the
xAPI json virtual choreographies
WP5 WP6
WP4

12. Rendering
* Virtual choreography decoding and rendering
2 prototypes
Density contour representation
Particle representation (in progress)
WP5 WP6
WP4

13. Density contour representation
https://drive.inesctec.pt/s/TXgaEsj3PWQat93
https://drive.inesctec.pt/s/xE89x9qM2YyCCfg

14. Information flow (ongoing)
* forecast
forecast
extract contour extract particle
build virtual choreography build virtual choreography
build STAC
create new
scenario
repo
netcdf netcdf
spartiotemporal
contours
spartiotemporal
particles
T
T T
T T
render
url: https://thredds.met.no/thredds/dodsC/sea/norkyst800m/1h/aggregate_be
engine: OCEANDRIFT vX.Y
arguments: –lat:4.85 –long: 60.0 – samples:5000 – hours:72 –seed:1
Oil spill use case
Create new scenario….
Go

15. Information flow (ongoing)
* real data
url: folder/folder/scenario A.B
repo: Repo Z
arguments: –lat:4.85 –long: 60.0 – samples:5000 – hours:72 –seed:1
Oil spill use case
Load scenario…
+
Go
+
fetch data
extract contour extract particle
build virtual choreography build virtual choreography
build STAC repo
netcdf
spartiotemporal
contours
spartiotemporal
particles
T
T T
T T
netcdf
create new
scenario
render

16. Rendering flow (ongoing)
fetch STAC
render overview
render detail
repo
virtual
choreography
T
T
T
render <

17. Rendering flow (ongoing)
particle-based rendering
fetch STAC
render overview
render detail
repo
virtual
choreography
T
T
T
render
<
<
contour-based rendering

18. Demonstration
https://drive.inesctec.pt/s/nwYEZMtYa9RdZ8t

19. Thank you!
Questions?

20. A 3D Immersive novel tool for
digital twins using virtual
choreographies: the oil spill use
case
International Forum on Database Management
and Digital Transformation for Marine Conservation
Fernando Cassola (INESC TEC)
fernando.c.marques@inesctec.pt
06.03.2024