Serving earth observation data with GeoServer: addressing real world requirements with Open Source

Serving earth observation data
with GeoServer: addressing real
world requirements with Open
Source
Ing. Simone Giannecchini
Ing. Andrea Aime

Outline
 About Us
 GeoServer Intro
 Current Support for EO/MetOc data
 The new integrated model with OpenSearch
Support
EO OpenScience Conference 2017

Quick Facts
 Founded in late 2006
 Expertise
• Image Processing, GeoSpatial Data Fusion
• Java, Java Enterprise, C++, Python
• JPEG2000, JPIP, Advanced visualization
• Web Based Mashups, Mobile Solutions
• OGC, ISO, INSPIRE Standards
 Supporting/Developing FOSS4G projects
 MapStore, GeoServer, GeoNetwork
 CKAN, GeoNode
 Clients
 UN FAO (CIOK, FIGIS, NRL, FORESTRY, ESTG), UN WFP, World Bank, DLR, EUMETSAT, JRC,
ARPAT, NATO CMRE, UNESCO, IGAD, UNEP, etc..
 Private Companies all over the world

Our Offer
 Enterprise Support Services
 Bug Fixing, Support, Customizations & New Features
 5 packages  different types of needs
 Deployment Warranty
 Deploy with confidence, we monitor and fix when needed
 Professional Training
 End-To-End Projects (Integration)
 Tell me what you need, I’ll put it together for you
 We take our core products and
 Bend them, twist them, embed them
 Hammer them to make clients happy
 FOSS4G Software is core for us
 Not simple use but develop and support

GeoServer
 GeoSpatial enterprise gateway
Java Enterprise
Management and Dissemination of raster
and vector data
 Standards compliant
OGC WCS 1.0, 1.1.1 (RI), 2.0.1
OGC WFS 1.0, 1.1 (RI), 2.0.0
OGC WMS 1.1.1, 1.3.0
OGC WPS 1.0.0
OGC CSW 2.0.2
OGC EO WMS, EO WCS
 Google Earth/Maps support
KML, GeoSearch, etc..

This is GeoServer!

Enough fuss, here’s the real stuff

Reference scenario

ImageMosaic – the EO/MetOc
workhorse

Terminology
 Granule/Tile  individual raster element composing the mosaic
 (Granule) Index  collection of metadata records describing the
location, spatial coverage and other attributes of each single granule
 Dimensions/Domains  dimensions besides the spatial ones used to
distinguish individual granules

Dimensions
 Map to alphanumeric attributes in the index
 TIME and ELEVATION receive special treatment for WMS and WCS
 Custom/Additional dimensions
 Everything besides TIME & ELEVATION
 They can be dynamically discovered from WMS GetCapabilities and WCS
DescribeCoverage
TIME Dimension

REST configuration
 (Granule) Index CRUD Operations via REST
 Create mosaic
 Read its index schema
 Read its index contents
 Update/Delete/Insert granules
GS
Pixels Index
Client
 Support for
programmatic access
 Workflow ETL
Integration

NetCDF format support
 NetCDF support
 Support COARDS/CF* conventions loosely
 Expose NetCDF internal data as a set of 2D slices
 Fast 2D (time, elevation) slice extraction
 Supports NCML

One or more variables
 Polyphemus Sample Dataset
 1 File  Multiple Coverages!

ImageMosaic – NetCDF integration

NetCDF/Mosaic integration
 ImageMosaic NetCDF integration
 Allow the ImageMosaic to handle multiple NetCDF files
 Expose NetCDF internal structure (times, elevations)
 Make ImageMosaic handle slices of the NetCDF file as granules
 5-dim mosaic (space/time/runtime other dimensions)
Mosaic
NetCDF 1 NetCDF 2 NetCDF … N
…

WMS-TIME support
TIME, ELEVATION & Custom dimensions
http://localhost:8080/geoserver/geosolutions/wms?...
&time=2013-03-1T00:00:00.000Z
&elevation=35.0
&DIM_FILEDATE=2013-03-01T00:00:00.000Z
&DIM_UPDATED=2013-04-08T08:18:41.597Z

&CQL_FILTER=platform=SENTINEL2  filter WMS on the fly!
WMS – custom filtering and sorting
&sortBy=time A&sortBy=cloudCoverage D

Rendering transformations
 SLD Based transformations
 On-the-fly contouring
 On-the-fly poligonalization
 Current arrows
 Wind Barbs
 Pluggable: add your own
windbarbs
contouring
currents

WMTS with Multidim
Domain Discovery

Multidim Domain Discovery
 Spatiotemporal data has complex domains
 Multiple dimensions
 Spatial, elevation, acquisition time, forecast time, reference_time, etc…
 Data scattered along space and time
 Think about different products within a mission
 Values vary in near real-time
 New acquisition (EO)
 New model runs (MetOc, EMR..)
 Difficult to convey info about complex domains
 WMS and WMTS very clunky, (EO) WCS much better
 We miss (at least) two things in the standards above:
 Ability to convey domain coverage quickly
 Ability to drill down in dimension for refined queries
ESA EOP-GTS Meeting
14th December 2016

 Extending OGC services (proposal here from OWS12)
 A few Domain Discovery Operation
 WMTS operations, might move to other services (e.g. WMS 1.4)
 #1: DescribeDomains
 Which dimensions do you have in your domain?
 xx?REQUEST=DescribeDomains&Version=1.0.0&Layer=test:flexpart_time_series&TileMatrix
Set=EPSG:900913&elevation=0.0/500.0&time=2016-02-23T03:00:00.000Z/2016-02-
23T12:00:00.000Z&reference_time=2016-02-23T00:00:00.000Z

 #2) DescribeDomains
 Tell me which dimensions you expose
 xx?REQUEST=DescribeDomains&Version=1.0.0&Layer=test:flexpart_time_series&TileMatrix
Set=EPSG:900913&elevation=0.0/500.0&time=2016-02-23T03:00:00.000Z/2016-02-
23T12:00:00.000Z&reference_time=2016-02-23T00:00:00.000Z

 #3: GetHistogram
 Show me how much data you have in this spatiotemporal bin
 xx?REQUEST=GetHistogram&Version=1.0.0&Layer=test:flexpart_time_series&TileMatrixSet
=EPSG:4326&histogram=reference_time&resolution=PT1H

 #4: GetFeature
 Let do fine grain queries where the data is located
 xx?REQUEST=GetFeature&Version=1.0.0&Layer=test:B2&TileMatrixSet=EPSG:4326&time=2
016-07-01T00:00:00.000Z/2016-07-31T00:00:00.000Z&bbox=-118,34,-115,38

WCS 2.0
 Basics
 Core service
 KVP binding
 XML binding
 Common extensions
 CRS
 Scaling
 Interpolation
 Range subsetting
 GeoTiff
 GML
 NetCDF
 Add the output format extensions
 GeoTIFF
 GML Grid
 NetCDF

WCS 2.0 DescribeCoverage
4D BBOX
TIME details as
extension
ELEVATION
details as
extension
CUSTOM

WCS – custom filtering and sorting
&CQL_FILTER=platform=SENTINEL2  filter WCS on the fly
&sortBy=time A&sortBy=cloudCoverage D

Download ND NetCDF
http://localhost:8080/geoserver/wcs?request=GetCoverage
&service=WCS&version=2.0.1&coverageId=geosolutions__NO2
&Format=application/x-netcdf
&subset=Long(5,20)
&subset=Lat(40,50)
&subset=elevation(300,1250)
&subset=time("2013-03-01T10:00:00.000Z","2013-03-01T22:00:00.000Z")
• Can also handle custom
dimensions
• Described in DescribeCoverage
in a vendor metadata section
4D
BBOX

The new kid on the block:
Integrated model with OpenSearch

OpenSearch Support: Putting it all together (and finding it)
 Going beyond flat index ImageMosaic
 Support Collections and Granules  two steps
search
 Use case
 Lots of satellite imagery
 Multiple satellites
 Multiple sensors
 Deep time and space distribution
 What to do about it?
 “OpenSearch for EO“ to the rescue
 Geo and Time Extension to OpenSearch  keywords
for time and space
 Earth Observation extension  keywords for and EO
properties
 Cloud cover, Snow cover, Off nadir…
OpenSearch
OGC OpenSearch Geo and Time
OGC OpenSearch for EO

Integrated Model: Single Model with OGC Services Support
 Based on PostGIS  ElasticSearch is on the radar
 Core Elements  Collection, Product, Granule
 Single integrated model  Drives all services
 Exposing imagery catalog as OpenSearch for discovery
 Exposing imagery catalog as WFS for discovery
 Different collections in different FeatureTypes
 Simpler (?) Alternative to OpenSearch for discovery
 Exposing Collections as WCS ImageMosaic layers
 Exposing Collections as WMS ImageMosaic layers
 Support for CQL_Filter and SortBy  Rich query lang
 1st discovery then jump to the other OGC services!
 Full Admin REST Interface with automation
Swagger documentation here

 http://cloudsdi.geo-solutions.it/geoserver/oseo/search?parentId=SENTINEL2&cloudCover=30]
 Output is RSS with description, thumb and links
 Links to metadata
 ISO metadata for collection
 GML O&M for products
 Links to OGC services (cross linking)
 WMS/WMTS to see collection/product
 WFS to get “masks” (validity, sea, cloud, snow, …)
 WCS to extract raster
 Direct download to get original package
 One “feature type” per collection
 Can be used as the index to a mosaic
 One mosaic per collection
 Filter using CQL and sort by attribute on the WMS/WCS service
 Use all the attributes already searchable by OpenSearch
Simple search + Tight integration with mosaic

GeoServer – REST Interface
 Administrative Programmatic Interface
 Manage EO Model Content
 Create/Update/Remove a collection
 Add/Update/Remove products with support for mass
unpublish/delete
 Swagger Description here
 Collections
 Listing
 Listing with paging
 Create a new collection
 Collection
 Inspect a collection
 Delete a collection
 Update a collection
 Manage individual elements (ogcLinks, description, metadata, thumbnail)

GeoServer – REST Interface
 Products
 Listing
 Listing with paging
 Add new product
 Product
 View Single Product
 Update a product
 Delete a product
 Manage individual elements (searchable elements, ogcLinks, description,
metadata, thumbnail, granules)
 ogcLinks use CQL_FILTER to filter individual elements in the time series
(WMS, WFS and WCS)

GeoServer – WFS
 Exposing imagery catalog as WFS
 Different collections in different FeatureTypes
 Alternative to OpenSearch for discovery
 Lower resistence path for web clients
 Support for CQL_Filter and SortBy
 Rich query syntax (might not work over time as data changes in the prototype over time)
 All Sentinel 1 products
 Sentinel 1, OrbitDirection ASCENDING
 Get Single Product by ID (uses CQL)
 Sentinel 1, OrbitDirection ASCENDING, SortBy startTime
 Once discovery is done we can jump to the other services!

GeoServer – WCS
 Exposing Collections as WCS ImageMosaic layers
 Different collections in different layer
 Support for CQL_Filter and SortBy for filtering
 Rich query syntax (might not work over time as data changes)
 All Sentinel 1 products, SortBy timeStart
 Ascending
 Descending
 Get Single Product by ID (uses CQL)
 Ascending
 Descending

 Exposing Collections as WMS ImageMosaic layers
 Different collections in different layer
 Support for CQL_Filter and SortBy for filtering
 Rich query syntax (might not work over time as data changes)
 All Sentinel 1 products
 Sentinel 1, OrbitDirection ASCENDING
 View Single Product by ID (uses CQL)
 Sentinel 1, OrbitDirection DESCENDING, SortBy startTime
Descending
Ascending
 Sentinel 1, OrbitDirection DESCENDING and startTime BEFORE
2017
GeoServer – WMS

Bonus Track:
Multiresolution, MultiCRS Mosaicks

WMS – heterogeneous CRS mosaic
UTM60N
UTM1N
WGS84
 Support for mosaicking
data in multiple CRS
 Support for mosaicking
data over the dateline

WMS – heterogeneous resolution view
 Sentinel2 bands have variable resolution, e.g., 10m, 20m, 60m
 N separate single band mosaics merged into a single view

WMS – heterogeneous resolution view
 Sentinel2 bands have variable resolution, e.g., 10m, 20m, 60m
 Merged into a single view

That’s all!
Questions?
info@geo-solutions.it

Serving earth observation data with GeoServer: addressing real world requirements with Open Source

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Serving earth observation data with GeoServer: addressing real world requirements with Open Source

Similar to Serving earth observation data with GeoServer: addressing real world requirements with Open Source (20)

More from GeoSolutions

More from GeoSolutions (12)

Recently uploaded

Recently uploaded (20)

Serving earth observation data with GeoServer: addressing real world requirements with Open Source