Thematic Exploitation Platforms 2

In the ESA/GSTP ‘ESE’ project (till Dec 2015) VITO and partners develop an end-to-end solution to drastically improve the exploitation of the SPOT-VEGETATION and PROBA-V EO-data archive (and derived products from the Copernicus Global Land Service) by both researchers and end-users. The analysis of time series of data is addressed, as well as the large scale on-demand processing of near real-time data.

From October 2015 an operational Mission Exploitation Platform (MEP) on these EO-data available at VITO will be gradually deployed, based on these results. This MEP PROBA-V will be a first testbed of such operational mission exploitation platforms.  

From early 2016, several applications will be released to the users, e.g.

-         A time series viewer, showing the evolution of vegetation parameters for an area of interest of choice.

-         Full-resolution viewing services for PROBA-V and Copernicus Global Land vegetation parameters.

-         A first on-demand processing chain for the calculation of N-daily composites.

-         A Virtual Machine will be provided with access to PROBA-V data and tools to work with this data, e.g. the ESA SNAP toolbox and support for R and Python.

After this initial release  a research platform will gradually be deployed allowing users to design, debug and test applications on the platform. In several iterations new services will be deployed and several projects will be supported to work on the platform.

From the MEP PROBA-V, access to e.g. Landsat, Sentinel-2 and Sentinel-3 data will be addressed as well since these data is demanded by the intended user community for vegetation research and services.

Users can make use of powerful Web based tools and can self-manage virtual machines to perform their work on the infrastructure at VITO with access to the complete data archive. To realise this, private cloud technology (openStack) is used and a distributed processing environment is built based on Hadoop. The Hadoop ecosystem offers a lot of technologies (Spark, Yarn, Accumulo, etc.) which we integrate with several open-source components.

The impact of this MEP on the user community will be high and will completely change the way of working with the data and hence open the large time series to a larger community of users. The presentation will address these benefits for the users and discuss on the technical challenges in implementing this MEP.

In the coming years, new EO missions will offer a large amount of new data useful for water resource management purposes. These data shall be accessed, processed, analysed and visualized for users and service providers but sometimes these users and providers do not want to invest ICT resources or do not have skills for handling these large volumes of data. The Hydrology Thematic Exploitation Platform (TEP) is an ESA project that aims at providing hydrology services such as flood mapping/forecasting, water quality and level monitoring, hydrology modeling and small water bodies mapping based on EO data. In addition to that, Hydrology TEP will also provide a collaborative framework where scientific users, river basin organisations and service providers could rapidly and easily access to a large number of EO data, integrate their own data and tools (in-situ data, socioeconomic data, analysis tools...) and process their processors (service prototypes, hydrological models, meteorological models) within a user-friendly environment. In summary, Hydrology TEP project wants to build a community and offer to this community an ensemble of services to facilitate and simplify their daily work.

The volume and variety of Earth Observation data available for the polar regions is growing rapidly, providing the opportunity for ever more complex investigations and exploitation in support of polar communities. The European Space Agency (ESA) and other satellite operators have been at the forefront of collecting, analysing, processing and disseminating new data and information from EO satellites. With this increase in volume of data and range of uses, there are new challenges in order to fully utilise and exploit this capacity.

ESA is establishing a series of Thematic Exploitation Platforms (TEP) which will provide the necessary collaborative environment which will deliver the resources and capabilities required for users’ exploitation work. The TEP projects will address present challenges and opportunities in scientific data exploitation and operational applications by collocating data, processing capabilities and ICT infrastructure, providing a complete cloud based work-environment for users. The Polar TEP will focus on the requirements and challenges of the diverse polar user community.

This presentation will describe the Polar TEP concept and the range of potential uses it might support. We will present details of the working environment where users can bring their algorithms, applications and development activities directly to the data. It will also cover the rich set of polar themed EO and complimentary datasets, relevant toolboxes and processing capabilities, plus functionality to allow deployment of user defined workflows and processing environments.

We will also outline the initial pilot project which will demonstrate the potential of Polar TEP to investigate current and future iceberg risk in Baffin Bay. The pilot project will integrate a diverse set of data, processors and models to allow users to investigate linkages between iceberg trajectories, changes in ice sheet velocity, glacier calving rates and ocean circulation. The integration of these components and toolsets will allow Polar TEP users to easily investigate questions about changing iceberg populations in support of regional climate change studies, evaluate risk assessments and inform infrastructure and ship routing decisions.

The coastal environment is a challenging area for Earth Observation because it is at the limit of two very heterogeneous domains. For instance, Ocean Colour data are more difficult to obtain near the coast, as the observed reflectances are affected by the presence of bright sediments, complex aerosol mixtures, as well as stray light from bright land surfaces. Coastal satellite altimetry is also challenging because the proximity of the land affects radar and microwave radiometry measurements.  Surface temperature measurements are similarly impacted at the boundary between the land and the sea. On the other hand, in-situ data is easier to access than in the open ocean, and this can help scientists to improve retrieval algorithms.

The Coastal Thematic Exploitation Platform (CTEP) is an on-going ESA project to develop a web-service dedicated to the observation of the coastal environment to support management and monitoring; this paper explains how the innovations of the CTEP will provide new means to handle the aforementioned technical challenges and contribute to improved understanding and decision-making with respect to coastal resources and environments.

The coastline is an object which naturally calls for a multi-resolution approach. While interactions with the open ocean need to be studied at a large scale, exchanges with rivers and shores need to be tackled at high resolution. When studying water constituents and bio-optical properties, the medium resolution images of OLCI on Sentinel-3 are therefore are ideally complemented by the high resolution images of MSI on Sentinel-2. The CTEP will provide access to OLCI L1 and L2 products, as well as L3 daily and weekly products at native resolution. For Sentinel-2, L1c and L2 images will be accessible over European coastal areas. Using extraction and format conversion tools, users will be able to obtain S2 and S3 BoA reflectances with compatible formats over the same area of interest in a given time frame. A major objective for the Coastal TEP is therefore to ease the connection between EO and in-situ data, thanks to a common data access portal and a set of automatic extraction and match-up tools.

Finally, the coastal environment is a particularly sensitive to anthropogenic pressure, pollution and climate change. Long-term monitoring can be supported by the generation of homogeneous time series (multi-mission data alignment), but also by automatic trend detection tools. The CTEP will allow users to perform custom processing over a complete data archive without actually downloading the products. In addition, the CTEP will create a semantic database of features and events (localized in space and time), which can be used for example to record oil spills, flooding and storm surges and toxic algal blooms. The semantic database will be fed manually or by automatic feature detection on EO data, and will be a key tool for a multi-disciplinary approach. For instance, the semantic database will make it easy to look for optical data acquired over flooded areas identified from radar images. Finally, CTEP subscription services will allow users to perform automatic monitoring of some key indicators (water quality, water level, vegetation stress…) from Near Real Time data.

In conclusion, the CTEP aims to provide new services and tools to enable a step change in the monitoring of coastal data, support a multi-disciplinary approach, and the provision of long term data series and innovative services.

The objective of the Forest Thematic Exploitation Platform (F-TEP) is to change the concept of earth observation in forestry. Currently everyone works on an individual basis and in scattered projects. F-TEP’s vision is of a one-stop shop for forestry remote sensing services for both the academic and commercial sectors. The services will offer pre-processed Copernicus and other satellite data, ancillary data, third party data and computing power. They will provide users access to appropriate image and data processing tools and GIS software. Easy to use push-button functionalities can provide simple value added products such as vegetation indices.  The F-TEP will also be a place to find information on what is happening in forestry earth observation globally and a place to network and discuss with the colleagues.

F-TEP will provide free access with free software tools and data to everyone interested. The Platform will also offer a self-service environment for the user to develop own tools and services. This will drastically reduce the present burden on data acquisition from miscellaneous sources with varying formats and processing levels.

In addition to the free an open tools and software the F-TEP will offer access to commercial software and services. Our aim is to evolve the Platform and introduce this in the marketplace, offering services provided by the European value adding SME’s.

F-TEP will serve user groups from academics and EO professionals to novice users, who are not experts in EO. Identified main user groups include:

-        Copernicus core services

-        UNREDD and other international programs such as the Global Forest Observation Initiative (GFOI)

-        National forest inventories

-        Universities and research centers

-        Forest managers

-        Value adding industry

-        Land use planning and nature conservation agencies

-        Sustainable development NGO’s

Forestry TEP will serve three basic scenarios: Data exploitation, service development and product development. All the three scenarios aim eventually at data exploitation.

In data exploitation the user employs tools that already are available on the platform. These tools range from browsing for pre-processed imagery, reference data and existing third party products to computing forest biomass maps and assessing their accuracy, and finally downloading just the final product for further use, for instance. The data exploitation user can be a novice in remote sensing, an institutional user or scientist who uses remote sensing product as part of his research. User-friendly interface is of central importance to the exploitation user. Acceptance by the exploitation users is the key to the success of the F-TEP.

A service developer user programs his own biomass estimation service to F-TEP with the help of his own tools and tools on the Platform. Once the service runs he can share or license the tool as Software-as-a-Service or use it by himself in which case he turns to an exploitation user.

A product developer differs from the service developer so that he computes final biomass products on F-TEP and distributes them to the exploitation users including possibly himself. The distribution can be commercial or for free.

The project is conducted by:

-        VTT Technical Research Centre of Finland Ltd, coordination, user interface, pilots

-        CGI UK, technical implementation of the platform

-        Science and Technology Facilities Council (STFC) (UK), cloud computing environment, technical implementation

-        Spacebel (B), sustainability analysis and planning

-        Arbonaut (FI), value adding service development

Two extensive pilots will be conducted in 2016-2017: one for Mexico on carbon monitoring for climate change reporting and another in Finland on operational forest management. F-TEP is closely coordinated with the FAO and JRC forest monitoring activities and Global Forest Observation Initiative (GFOI).

The project started in March 2015 and we are currently finalizing the initial development phase. It is planned that the pre-operational services will come online around September 2017, although we hope to provide some limited functionality to the users at an earlier date. In the Living Planet Symposium a live demonstration on the function of the TEP will be presented.

The F-TEP is being developed under a contract of the European Space Agency.

The upcoming suite of Sentinel satellites in combination with their free and open access data policy will open new perspectives for establishing a spatially and temporally detailed monitoring of the Earth’s surface. The Sentinel fleet will provide a so-far unique coverage with Earth observation (EO) data and new possibilities with respect to the implementation of innovative methodologies, techniques and geo-information products and services. However, the capability to effectively and efficiently access, process, analyze and distribute the mass data streams from the Sentinels and high-level information products derived from them poses a key challenge. This is also true with respect to the necessity of flexibly adapting the processing and analysis procedures to new or changing user requirements and technical developments. Hence, the implementation of operational, modular and highly automated processing chains, embedded in powerful hard- and software environments and linked with effective distribution functionalities, is of central importance.

This contribution introduces the TEP Urban project that aims at the utilization of modern information technology functionalities and services to bridge the gap between the technology-driven EO sector and the information needs of environmental science, planning, and policy. Key components of such a system are currently developed in the TEP Urban project. This includes the implementation of an open, web-based platform employing distributed high-level computing infrastructures (Platform as a Service – PaaS) as well as providing key functionalities for i) high-performance access to thematic data (Information as a Service – InaaS), ii) modular and generic state-of-the art pre-processing, analysis, and visualization (Software as a Service – SaaS), iii) customized development and dissemination of algorithms, products and services, and iv) networking and communication. These services and functionalities are supposed to enable any interested user to easily exploit and generate thematic information on the status and development of the environment based on EO data and technologies.

The TEP Urban platform is supposed to initiate a step change in the use of EO data by providing an open and participatory platform based on modern ICT technologies and services that enables any interested user to easily exploit and generate thematic information on the status and development of the built environment.