Methods Optical Quality

The objective of FIDUCEO is to create new climate datasets from Earth Observations with a new level of metrological rigour. This responds to the need for enhanced credibility for climate data, in support of rigorous science, decision-making and climate services.

Thus, there are two strands to what we propose to develop: “methods” and “data”.

Regarding “methods”, FIDCUEO will define a complete, rigorous metrological approach to creating climate data from Earth Observations, including past missions. This will allow uncertainty to be characterised and propagated from the fundamental radiance measurements to derived geophysical products in a traceable manner. Methods for harmonising the calibration of sensor radiances will be developed that support estimation of all components of uncertainty along complete series of sensors. (In some cases, additional mutual constraints on harmonisation will be developed between complementary sensor-series.) FIDUCEO will then establish methods of propagation of radiance/harmonisation uncertainty through to essential climate variables, such that their stability can be traceably estimated and that ensemble datasets can be created. A software “toolkit” will give non-specialists practical access to rigorous, sophisticated methods for propagating uncertainty information, assessing the stability of datasets and validating uncertainty estimates.

Regarding data, FIDUCEO will (applying the above methods) create both Fundamental Climate Data Records (FCDRs) and, from these, [Thematic] Climate Data Records (CDRs) for essential climate variables. The FCDRs are chosen because they have a length relevant to climate (>20 years) and each can support several important CDRs. Not all CDRs potentially derived from the FCDRs can be addressed within FIDUCEO. Selected CDRs will be created that illustrate new capabilities (such as equi-probable ensembles) and the impact of the new FCDRs on derived CDRs (such as improved stability).

Specifically, the datasets to be created by FIDUCEO are:

Harmonised radiances for the sensor series: AVHRR, HIRS, AMSU-B/MHS and MVIRI

Geophysical products, with uncertainties, for: sea surface temperature, lake surface water temperature, tropospheric humidity, aerosol optical depth and surface albedo

The CDRs are carefully chosen to be complementary in various ways to datasets generated with existing programmes (Climate Change Initiative, Climate Monitoring SAF, etc), and to demonstrate positive impact arising from the FCDR improvements. The infra-red wavelength FCDRs will be traceably linked to new observations in the Copernicus space programme (Sentinel 3).

All data, software tools and methods will be freely and openly accessible. Methods will be disseminated in a variety of forms, including e-learning modules. FCDRs and CDRs will be available in both a common “easy” format (for general users) and community-standard/specialist formats. Two workshops will allow dialogue with users and present FIDUCEO methods and data externally. FIDUCEO will liaise explicitly with relevant programmes (Copernicus Climate Change Service, NOAA Climate Data Records programme, SCOPE-CM, re-analysis initiatives etc).

By creating valuable datasets and defining, applying and disseminating rigorous new metrological methods, FIDUCEO aims for a broad and lasting impact in the field of climate data from space. 

As the number of Earth-observation satellites is ever increasing, one of the challenges for the scientific community is to ensure that the absolute radiometric calibration of these sensors resides on the same SI-traceable scale. Individual teams, equipment, and sites are typically used to assess the post-launch radiometric calibration of instrument by simulating top-of-atmosphere signals from in-situ surface and atmosphere measurements. Such assessment based on a single site/instrumentation can lead to radiometric biases between satellite sensor radiometry. The Radiometric Calibration Network (RadCalNet) working group (WG) (formerly known as Landnet) is currently working on a prototype methodology that seeks to minimize calibration biases by creating a standardized network of sites and data processing protocols. Such a network approach will also greatly improve the temporal frequency at which an EO sensor radiometric accuracy can be assessed. The first RadCalNet WG meeting was in January 2014, and the duration of the initial prototyping project is two years. After the two-year prototyping phase, RadCalNet will become an operational network of instrumented sites that are used for the absolute radiometric calibration, intercalibration, and validation of Earth-observing sensors.

The RadCalNet concept has been on the agenda of the Committee on Earth Observation Satellites (CEOS) Working Group on Calibration and Validation (WGCV) Infrared Visible Optical Sensors (IVOS) for years, and in 2013 it was determined that there were sufficient resources to assemble the RadCalNet WG. The current RadCalNet WG consists of members from the Academy of Opto Electronics (AOE, China), the Centre National d’Etudes Spatiales (CNES, France), the European Space Agency (ESA), the National Aeronautics and Space Administration (NASA, USA), the National Physical Laboratory (NPL, UK) and the University of Arizona (USA).,. Four radiometric calibration test sites in China, France, Namibia and the USA are being used as test cases for the collection of surface reflectance and atmospheric data, which are then converted to top-of-atmosphere (TOA) reflectance for comparison with a limited number of satellite sensors. The objectives of the RadCalNet WG during the two-year preliminary phase are to 1) Define the architecture of RadCalNet, 2) Demonstrate the operational concept using currently-available satellite sensors, and 3) Provide recommendations to CEOS/WGCV for the evolution of RadCalNet towards an operational project and 4) Provide guidelines for the addition of additional sites. As the network progresses from a prototype to being operational, the current plan for a deliverable product will be a daily hyperspectral (350 nm –2500 nm) TOA reflectance at 30-minute intervals for a nadir-viewing sensor.

Currently, the RadCalNet WG is focused on setting up the framework to move from prototype to operational status. Maintenance and upgrades to the instruments at the four test sites are ongoing, and a data processing chain common to all groups is being developed. The current strategy is to have all groups forward their surface reflectance and atmospheric data to a central location for final processing to TOA reflectance. A comparison will be made between each group’s calibration coefficients that are determined for such sensors such as Landsat 8, SPOT–5 (and possibly DMC and Sentinel-2). This will support the understanding of biases that may occur between the three groups, as well as providing guidance for future work. NPL will provide support with the harmonization, traceability of measurements, and instrument calibration for the RadCalNet work. In addition, the RadCalNet WG is working on a technical note that describes the concepts, instrumented sites, measurements, data processing, and quality assurance protocols. 

System Vicarious Calibration (SVC) is generally introduced in the processing of ocean colour missions for meeting the stringent requirement of 5% accuracy and 0.5% stability per decade in the retrieved marine radiometry at sea level, today unachieved through instrumental calibration only. It consists in adjusting the top of atmosphere (TOA) radiometry through the use of high-quality in-situ marine reflectance concurrent with space acquisitions. Such approach is considered by the ESA Ocean-Colour Climate Change Initiative (OC-CCI, phase 2) whose goal is to produce long-term times-series of Essential Climate Variables, based on several satellite-borne ocean colour sensors.

Historically, SVC has been successfully implemented for so called classical atmospheric corrections (AC) characterized by 1) the use of NIR bands only to detect aerosols and 2) the sequential and linear correction of each band in the visible for atmospheric effect. In practice SVC produces gains at each validation point and each wavelength in the visible, which make the {sensor + Level-2 processing chain} system exactly match the field measurements, where the processor is mainly driven by the AC. By construction using the same reference in-situ measurements to calibrate different missions should harmonise all Climate Data Records, at least for marine and atmospheric conditions under which the processing chain performs similarly.

Vicarious calibration in the frame of the ESA OC-CCI requires a complete revisit because of the use of non-standard AC, such as POLYMER, characterized by spectrally coupled and non-linear inversion of the whole TOA spectrum. We reformulate the SVC as a general sensitivity problem between top and bottom of atmosphere radiometry and demonstrate that it cannot be solved by usual method when considering spectral matching AC. We propose a numerical method able to compute gains in an optimal sense and applicable to any Level-2 chain thanks to automatic differentiation. The method can be seen as a generalization of the historical SVC and yields back to classical gains for classical AC. We illustrate the approach for SeaWiFS, MODIS, MERIS and VIIRS times-series processed by POLYMER over the MOBY buoy and discuss the interest and limit of SVC for non-standard AC.

The 4^th MERIS data reprocessing should start in the first quarter of 2016. It is focused on the alignment, as far as possible, of the MERIS and Sentinel-3/OLCI data processing chains in order to ensure a continuity of the products derived from these two instruments.

A major evolution of this reprocessing is related to the data formatting: the Envisat data format (.N1) is given up in favour to the Sentinel-3 SAFE format based on a folder of netCDF files including a xml manifest file.

Moreover, following recommendations by the MERIS QWG, a number of algorithm evolutions have been implemented.  At level 1, the evolutions are:

The L1 calibration is updated based on a reanalysis of the complete mission in-flight calibration dataset. It includes in particular a revised diffuser ageing methodology accounting also for the ageing of the reference diffuser.

Improvement of geolocation: precise latitude, longitude and altitude are given for each pixel.

The a priori surface classification masks (land/sea, tidal areas and in-land-waters) are significantly upgraded and are in line with those used by the OLCI data processing.

At level 2, a large number of algorithms evolutions have also been implemented at each step of the data processing:

Cloud screening is improved thanks to the used of NN derived from the ESA Climate Change Initiative (CCI) programme, and enriched with semi-transparent cirrus clouds detection using O2 absorption;

The computations of H2O, O2 and O3 transmissions have been revised and the NO2 absorption has been added in the total gaseous atmospheric transmittance;

The Water Vapour retrieval has been upgraded through the use of the 1D-var  algorithm;

Handling of molecular scattering is improved by the use of accurate pixel elevation, better modelling of the relationship between pressure and elevation, better modelling of the Rayleigh optical thickness (Bodhaine et al., 1999) and improved correction of meteorological variation of atmospheric pressure;

For the atmospheric corrections over water, coastal waters are better handled thanks to the improvement of the Bright Pixel Atmospheric Correction (BPAC), new aerosols models (Ahmad et al., 2010) have been implemented in the LUTs that now have been extended to several pressure levels for use over in-land as well as oceanic waters. Vicarious gains have been recomputed to account for modified L1 calibration and atmosphere correction upgrades;

Aerosols retrieval over land has been improved with a better modelling of the surface BRDF;

The Case 2 Ocean Colour algorithm has upgraded;

Uncertainties have been added to ocean and land products.

This paper will review all the evolutions of the 4^th MERIS data reprocessing and present a set of validation results.

For the last decade, our understanding of oceans has grown up as the ARGO network developed. These ARGO floats equipped with temperature and salinity sensors provide new data on the water column, frequently and at global scale. The idea has been now to reproduce this success story with new parameters, biological parameters. Chlorophyll-A, Bbp and CDOM are some of the new variables available with the BioArgo floats deployed for the last three years. More than 12.000 profiles have been exploited so far, it is a chance for a field suffering of lack of data. In order to deliver high quality products, a strict Quality Check procedure has been defined (Spike, Range, offset, non-photochemical Quenching correction …) with IFREMER using a dedicated tool: Seaside Rendez-vous (have a look to seasiderendezvous.eu). QC statuses for each float are reported on a map and a panel with a set of different indicators is proposed for an operator to diagnose whether a particular measure is coherent or to be flagged.

For the ocean colour community, getting such a global network emerging is a chance for validation and quality control of ocean colour product. Indeed, comparing the Chl-A concentration in the upper layer of the water column to the surface concentration assessed by remote sensing already presents satisfying results (R² above 0.8) whereas these products are still being improved. Matchups between in-situ and ocean colour are also eased by SeasideRendezvous thanks to a tool enabling to produce dynamic scatter-plots and analyze statistics for a selected region of interest. The analysis focused on some region of interest is also a key point as it is believed and documented that oceans and seas can be split according to biological factors as large areas behave the same. A bioregional classification has already been proposed by D’Ortenzio and d’Alcalà (2009) and is also being studied with the help of  SeasideRendezVous.

The use of bioArgo for validation and Quality Control of Ocean Colour OLCI-Sentinel-3 in the frame of S3-Mission Performance Centre will be presented at the Symposium