Global RTOFS Class-1 Satellite Verification

General Information

Class-1 metrics are defined by the GODAE OceanPredict Intercomparison and Validation Task Team (IV-TT) as 2-D and 3-D model fields averaged to daily means and interpolated to common grid resolutions for intercomparison purposes (Hernandez 2007). For the purposes of these analyses, the surface model fields (SST, SSH, SSS) are compared with Level-3 and Level-4 gridded satellite analyses. The model nowcast and forecast fields are interpolated to the observational grids and averaged to match the different time periods of the satellite and analysis products.

The following observational data sets are used:

• SST GHRSST Level 4 AVHRR_OI Global Blended SST Analysis (GDS v2) from NCEI (daily product)
• SSH Copernicus Global Ocean SSH NRT (Level 4, 6-day mean)
• SSS SMAP via PO.DAAC (Level 3, 8-day mean)
• SSS SMOS via NESDIS (Level 3, 1-day composite)

The following climatology data sets are used in computing anomaly statistics:

• World Ocean Atlas 2013 version 2 for SST and SSS
• GOFS 3.1: 41-layer HYCOM + NCODA Global 1/12° 30-year Reanalysis for SSH

Class-1 Verification Data Processing Flow Chart

Click to enlarge

Statistics Definitions

Bias

The mean difference between the model and observations, measures the tendency of the model process to over- or under-estimate the value of a parameter. Smaller absolute bias values indicate better agreement between measured and calculated values. Positive bias means overprediction, negative means underprediction.

        diff = model_data - observations
        bias = diff.mean()       
        

Root-Mean-Square Error (RMS Error)

Also called the root-mean-squared deviation, it's a measure of the differences between the observed and predicted values. Smaller RMSE values indicate better agreement between measured and calculated values.

        rmse=(diff**2).mean()**0.5
        

Correlation Coefficient

The cross-correlation coefficient is a measure of the similarity of two time-series of model and observed values. The standard Pearson product-moment correlation coefficient is computed from a pair-wise correlation of the two fields (model, observation), disregarding any NaN or missing values.

        cc=numpy.ma.corrcoef(obs.flatten(),model.flatten())[0,1]
        

Scatter Index (SI)

Defined as the standard deviation of the difference between model and observations, normalised by the mean of the observations. Smaller values of SI indicate better agreement between the model and observations.

        scatter_index=100.0*(((diff**2).mean())**0.5 - bias**2)/observations.mean()
        

Structural Similarity Index Metric (SSIM)

"The structural similarity (SSIM) index is a method for predicting the perceived quality of digital television and cinematic pictures, as well as other kinds of digital images and videos. The early version of the model was developed in the Laboratory for Image and Video Engineering (LIVE) at The University of Texas at Austin and the full version was further developed jointly with the Laboratory for Computational Vision (LCV) at New York University. Further variants of the model were developed in the Image and Visual Computing Laboratory at University of Waterloo and have been commercially marketed." (Wikipedia link)

When comparing images, the mean squared error (MSE)–while simple to implement–is not highly indicative of perceived similarity. Structural similarity aims to address this shortcoming by taking texture into account. (Wang and Bovik 2009, Wang et al. 2004)

References

• Hernandez, F. (2007). List of internal metrics for the MERSEA GODAE global ocean: specification for implementation. MERSEA IP report MERSEA-WP05-MERCA-STR-0015-01C. doc.
• Zhou Wang; Bovik, A.C. (2009). Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures. Signal Processing Magazine, IEEE, vol. 26, no. 1, pp. 98-117, Jan. 2009.
• Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004.