Environmental Modeling Center Environmental Modeling Center Environmental Modeling Center United States Department of Commerce

GFS-Wave Buoy Verification



Buoy


Year


Month


Forecast Hour


Buoy Locations

SE US / Caribbean
Gulf of Mexico
NE US/Canada
West Coast US / AK
Hawaii





These statistics are based on all available buoys per day


These plots are based on all available buoys for the whole month. The color indicates the percentage of points, with red being the lowest percentage and purple being the highest percentage of points. Here the scales have been normalized and are the same for both models.


These statistics are based on all available buoys for the whole month



By month: Click through the forecast buttons

By forecast: Aggregate of all data from June 2019 through August 2020

This GFS-Wave buoy validation is for the Wave component of the GFSv16 proposed implementation. The page is continually being updated, and will cover data from:

  • Retro1: 20190608 - 20190831
  • Missing: 20190901 - 20190909
  • Retro2: 20190910 - 20191130
  • Missing: 20191201 - 20200131
  • Retro3: 20200201 - 20200519
  • Realtime: 20200519 forwards

The results are compared to the current operational wave model at NCEP:

  • Global Wave Model (Multi_1)

Both wave models compared here are WAVEWATCH III, albeit of different versions.

Using the Verification Archive

In order to browse the archive by individual buoy, follow these steps:

  1. Select a Buoy
  2. Select a Year
  3. Select a Month
  4. Select a Forecast

This will open up the tabs available for "Individual Buoys". Not all buoys have data for significant wave height (Hs), primary wave peak period (Tp), 10-m wind-speed (u10), and 10-m wind-direction (udir).


To browse the archive by aggregated buoys (i.e. all buoys available per day), follow these steps:

  1. Select a Year
  2. Select a Month
  3. Select a Forecast

Statistics are given on the aggregated buoys, i.e. all buoys available per day, which is usually between 100-200 buoys. For these statistics, the deterministic models are compared against the mean of the ensembles


Summary Statistics

Plots for the variables statistics calculated on a monthly basis, and for statistics against forecast hour will be added as they become available. Note that only the "Forecast Hour" selector will work with the monthly plots.

Technical Definitions


Wind Speed (u10) and Direction (udir)

All the wave models are driven by 10-m surface winds:

  • GFS-Wave is driven by 1/4 degree winds
  • GWES is driven by 1/2 degree GEFSv11 winds
  • Multi_1 is driven by 1/4 degree GDAS/GFSv15 winds

Power law estimation for NDBC buoy winds

NDBC buoy wind speeds are recorded at the buoy anemometer height and are not height adjusted by NDBC in the archive. We estimate the winds at 10 meters by using as method described by S. A. Hsu et al. (1994).

Although never used operationally by NDBC, the method was tested and found to compare favorably with the more elaborate method under near-neutral stability. This is the condition most frequently encountered at sea and occurs when air and water temperatures are not too far apart. The method, referred to as the Power Law Method, is offered here for those who may want to explore the nature of the marine wind speed profile without having to deal with the complexity of the above method. The relationship is:

        u2 = u1 (z2/z1)^P
        
where u2 is the wind speed at the desired reference height, z2, and u1 is the wind speed measured at height z1. A value for the exponent, P, equal to 0.11 was empirically determined to be applicable most of the time over the ocean.

Reference
Hsu, S. A., Eric A. Meindl, and David B. Gilhousen, 1994: Determining the Power-Law Wind-Profile Exponent under Near-Neutral Stability Conditions at Sea, Applied Meteorology, Vol. 33, No. 6, June 1994.

Significant Wave Height (Hs)

The significant wave height is defined as the mean wave height (trough to crest) of the highest third of the waves. Note that the highest wave height of an individual wave will be significantly larger. Significant wave height values are in meters (m).

Peak Wave Period (Tp)

The peak wave period is estimated as the period corresponding to the highest peak in the one dimensional frequency spectrum of the wave field. The wave field generally consists of a set of individual wave fields. The peak period identifies either the locally generated "wind sea" (in cases with strong local winds) or the dominant wave system ("swell") that is generated elsewhere. Peak wave period values are in seconds (s).

Bias

The mean difference between the model and observations, measures the tendency of the model process to over- or under-estimate the value of a parameter. Smaller absolute bias values indicate better agreement between measured and calculated values. Positive bias means overprediction, negative means underprediction.

        diff = model_data - buoy_data
        bias = diff.mean()       
        

Root-Mean-Square Error (RMS Error)

Also called the root-mean-squared deviation, it's a measure of the differences between the observed and predicted values. Smaller RMSE values indicate better agreement between measured and calculated values.

        rmse=(diff**2).mean()**0.5
        

Scatter Index (SI)

Defined as the standard deviation of the difference between model and observations, normalised by the mean of the observations. Smaller values of SI indicate better agreement between the model and observations. Note that low wave heights or areas where wind seas dominate can result in high SI values.

        scatter_index=100.0*(((diff**2).mean())**0.5 - bias**2)/buoy_data.mean()
        

Scatter Plot

Shows the relationship of a specific parameter (u10, udir, Hs, Tp) by plotting the buoy value on the x-axis and the model value on the y-axis. The ideal agreement would line up all the points along the 45 degree dashed line. The solid line represents the Ordinary Least Squares (OLS) fit to the data. Additionally, the Coefficient of Determination is given: it is the square of the correlation between the predicted and actual values, and thus ranges from 0 to 1.

Quantile-Quantile (Q-Q) Plot

A Q-Q plot is a plot of the quantiles of the first data set against the quantiles of the second data set. This is a graphical technique for comparing two probability distributions - if the two distributions agree, then the Q-Q plot follows some line. Perfect agreement would yield the y=x line. If the general trend of th Q-Q plot is flatter than the line y=x, the distribution plotted on the x-axis is more dispersed than the distribution plotted on the y-axis, and vice versa.

Taylor Diagram

Taylor diagrams are usually used to show how a variety of different models do compared to the same data source. However, we want to show how one model​ ​ does at a variety of data sources (different buoys). In order to put the data from many different buoys on the same Taylor diagram, first you have to​ ​ normalize the model and observation values by the standard deviation of the​ ​ observation. In this case the model buoys standard deviation is divided by​ ​ the observed buoys standard deviation. But you must also normalize the observed values standard deviation: this means that the observations standard deviation will be set to 1, cross-correlation is 1, and the RMS is 0. So you've basically replaced all the observed points by the star at 1 along the x-axis of the Taylor diagram, and you just plot all of the model buoys statistics.

Aggregated Buoys

The "Aggregated Statistics" and "Taylor Diagram" tabs contains time series for all the variables gathered across all reporting buoys and analyzed per-day for the full month.