Data Preservation and Valorisation

Long term data preservation (LTDP) has been an overlooked endeavor for a long time as it was mostly considered as keeping the data in a safe place, while effort and resources were more dedicated to fresh and more exciting new data from the latest sensors. As a result, long-term archives turned out to become cemeteries from which information and data were very difficult – or even not possible - to extract.

However needs for long-term climatological studies, improvement on geophysical parameter inversion algorithms, innovating synergy and datamining applications merging or crossing observations from multiple sensor do require easy and lively access to these legacies from past missions. Besides, new arising concepts inherited from big data and cloud technologies now make possible providing online archiving and remote processing capabilities that could boost the revisiting rate and enhancement of the existing historical archives.

This is all the purpose of the ongoing effort at Ifremer/CERSAT to provide to user such thematic exploitation services, which has been demonstrated in the last years in particular in the context of ERS and Envisat (but not only) ocean data, associating EO data, processing capabilities, traceability, quality assessment and intercomparison resources (model, in situ).

This platform has helped supporting:

the consolidation of complete mission archives (from raw data to related ancillary and auxiliary information)

one-stop shop for specific mission archive and complementary sources of observations (third-party missions, collections of in situ measurements, auxiliary information,...) that can be combined and intercompared to each other

multiple reprocessing campains from L0 to L2 for improved and more consistent datasets (REAPER project)

the inter-calibration of the different missions available to provide long consistent time series (such as wave measurements from altimetry)

the re-visiting of these data archives  to derive new information such as ship traffic and iceberg detection from multiple altimeter missions

This concept should be extended further and end-user requirements will be presented for future opportunities.

Historical data access is required in many fields of long term environmental analysis. In particular, the long term SAR data preservation is vital to the analysis and the comprehension of surface deformation phenomena with natural or anthropogenic origin. A number of SAR images with almost the same acquisition geometry is necessary for detecting natural ground reference points (Permanent Scatterers - PS) in correspondence of which the surface motion can be detected with a reduced impact from the atmospheric artifacts. Stacks of multiple SAR images are then collected in repeated satellite passes. Thus, depending on the particular SAR missions, a rolling archiving period ranging from a few months (e.g. for Sentinel, Cosmo-SkyMed and TerraSAR-x) to more than one year (e.g. for ERS,  ENVISAT, ALOS) is the minimum requirement for any ground deformation analysis.  The need for longer term SAR data archives comes from situations where the comprehension of the present surface deformation needs an analysis of what happened in the past. In many practical cases the historical data analysis is necessary to understand whether the observed ground subsidence has been originated by natural or anthropogenic causes. In other cases the historical data analysis is exploited to monitor the outcome of interventions aimed at solving problems of deformation (typically subsidence or sliding phenomena). Another good reason to have long term SAR data preservation is that most of the historical data set should be periodically re-processed to conform to the latest revisions of the processing algorithms that continuously improve their capability to squeeze out of the data all the available information. For example the density of detected ground measurement radar targets has been dramatically improved passing from the original Permanent Scatterers (1999) to the SQUEESAR technique (2005). In this regards there are several emblematic examples where re-processing the same data stack with SQUEESAR has revealed new deformation phenomena that were invisible during the original processing.  In this presentation a selection of examples to demonstrate the role played by the historical data for the actual understanding of the deformation phenomena will be shown.

The Along Track Scanning Radiometer series has a long and glorious history, the ATSR instruments being known to be well calibrated, stable, and characterized by relatively high spatial resolution.

After the premature departure of ENVISAT (April 2012), the ATSR community started looking into Level 1 interesting features that had not been in the focus for many years.

In this context, ESA initiated a small project, called "ATSR Long Term Stability" (ALTS), in the framework of the Long Term Data Preservation programme.

The aim of this project is to start looking into some ATSR characteristics from innovative perspectives. The ALTS consortium (TELESPAZIO-VEGA, CNR-ISAC, RHEA, EOSense, SERCO) put together a team of very critical and motivated researchers, never before involved in ATSR studies but with recognized international experience in other fields of remote sensing.

A number of issues which up to now have had minor impact on the baseline mission objective of SST retrieval have been investigated, such as:

1) the geolocation and the nadir-forward view collocation quality

2) the radiometric calibration stability (all channels)

3) the (official) cloud mask reliability

4) the documentation availability across the whole instrument series  (technical notes, scientific articles, project deliverables ...)

In the first two years of ALTS, the investigations led to interesting advances (e.g. a new water vapour retrieval), or to the development of new tools (e.g. transformation from "gridded" to "ungridded" Level 1 products). The extensive work on the radiometric calibration is now producing its results.

It should be emphasised that improvements of any aspect of Level 1 products could improve the quality of existing algorithm retrievals or facilitate the development of new products.

In this contribution, we will provide details on ALTS achievements, demonstrating  the paramount importance of the revision of "old" data for future perspectives, and on the possible outlooks.

The Scatterometer Instrument Competence Centre (SCIRoCCo) has been initiated by the European Space Agency (ESA) to serve as an international and interdisciplinary competence centre for ERS Scatterometer (ESCAT) mission data valorisation and continued exploitation.

The two ESCAT instruments collected a unique C-band backscatter data base of the Earth surface at a spatial resolution of 25km and with long-term (20 years) availability. This data is invaluable  for a wider set of applications ranging from current and future sensors characterization and inter-calibration to climate applications. 

The paper presents, in the first part, the main SCIRoCCo activities organized in five areas which cover: i) the consolidation of the scatterometer data processing knowledge and calibration methodology following GCOS principles, ii) the scientific support to the ESCAT data reprocessing campaign, iii) the relationship with the scatterometer data user community to foster the usage of ESCAT data and to exchange information and knowledge about ESCAT via a web-portal: http://scirocco.sp.serco.eu/, iv) to initiate, foster, and facilitate the creation of an educational program for the maturation of the next-generation scientific community. For the latter a rewarding system (grants, stages, and training initiatives) for scatterometer data exploitation and evolution has been set up with special attention to those proposals and activities that, based on scatterometry, will demonstrate innovation and cross-domain applicability. (e.g. Land, Oceanography...), v) the improvement of the current processing algorithms and geophysical models, with particular attention to the instrument design, the level 1 processing, including inter/cross calibration among different instruments, the level 2 retrieval algorithms for soil moisture, sea surface wind vector, and sea ice characteristics, and finally the validation and assimilation of the results.

In the second part, the paper will present achievements within the main project’s area. In particular results for the ERS-1, ERS-2 sensors inter-calibration, sensor characterization, data validation and outcome of the educational activities will be described.

The SCIRoCCo consortium includes European industry, research centres, universities, and operational meteorological agency, which have recognized international expertise in the Scatterometer area. The Members of the consortium are: the Royal Military Academy of Belgium (RMA), the Koninklijk Nederlands Meteorologisch Instituut (KNMI), the Technische Universität Wien (TUWien), the European Centre for Medium Weather Forecast (ECMWF) and Serco spa (Italy) as prime contractor and industrial party. The consortium is funded by ESA  for a two year contract, starting in June 2014.

Changing climate is indoubtedly influencing our environment with increasing pace. Systematic observations to monitor its impact on many essential climate variables (ECVs) are therefore strongly required. Complementary to ground measurements and climate models, satellite remote sensing represents a valuable tool to get comprehensive information that can not only be used for time series analysis on its own, but also serves as an independent reference to evaluate results from climate models. One sensor with high potential for long term monitoring is the Advanced Very High Resolution Radiometer (AVHRR) onboard NOAA- and MetOp-satellites. Dating back to 1980 it offers an unprecedented data set for global monitoring with a daily resolution. The University of Bern receives and archives AVHRR data over Europe since 1984 and has meanwhile established a unique AVHRR-archive with the potential to serve as a fundamental climate data record (FCDR). This record will be made publicly accessible in the framework of ESA’s long-term data preservation (LTDP) activity. The data record covers more than the last 30 years and has been used for the retrieval of a set of ECVs such as snow, albedo, aerosols, or lake surface water temperature. Major drawbacks of the sensor suite are the missing on-board calibration of the channels 1, 2 and 3, and the orbit drift of the earlier NOAA-satellites. Hence, appropriate pre-processing including state-of-the-art calibration and sophisticated geocoding is the critical step on the way to generate reliable information of our environment.

The talk will show some details of our AVHRR-archive, the importance of pre-processing, validation, and the product retrieval including time series of snow extent, albedo and lake surface water temperature. Finally, some recommendations will be given on the usage of AVHRR for climate applications and the next steps to make our archive accessible via ESA web site will be presented.