A Neural Network Approach to Parameterizing Nonlinear Interactions in Wind Wave Models.

Vladimir Krasnopolsky, Dmitry V. Chalikov, and Hendrik L. Tolman²,

(1)

where F is the spectrum, S_in is the input source term, S_nl is the nonlinear interaction source term, S_ds is the dissipation or 'whitecapping' source term, and � represents additional (shallow water) source terms. The SWAMP study (SWAMP Group 1985) identified the critical role of the nonlinear interactions, and the need for explicit modeling of S_nl in wave models. State-of-the-art or so-called third generation wave models therefore explicitly model this source term.

In its full form (e.g., Hasselmann and Hasselmann 1985), the calculation of the interactions S_nl requires the integration of a six-dimensional Bolzmann integral. This integration requires roughly 10³ to 10⁴ times more computational effort than all other aspects of the wave model combined. Present operational constraints require that the computational effort for the estimation of S_nl should be of the same order of magnitude as the remainder of the wave model. This requirement was met with the development of the Discrete Interaction Approximation (DIA, Hasselmann et al 1985).

The development of the DIA allowed for the successful development of the first third generation wave model WAM (WAMDI Group 1988, Komen et al. 1994). Nearly two decades of experience with the WAM model and its derivatives have identified shortcomings of the DIA. The DIA tends to unrealistically increase the directional width of spectra, has a systematic spurious impact on the shape of the spectrum near the spectral peak frequency, and has a much too strong signature at high frequencies. In present third generation wave models, these deficiencies can be countered at least in part by the dissipation source term S_ds, which is generally used as the closure term in the equation (1). Although this approach gives good results, it is counterproductive, because it prohibits development of dissipation source terms based on solid physical considerations. With our increased understanding of other source terms, this becomes a bigger obstacle for the further development of third-generation wave models.

Considering the above, it is of crucial importance for the development of third generation wave models to develop a cheap yet accurate approximation for S_nl. We present here a Neural Network Interaction Approximation (NNIA) to achieve this goal.

Applying Neural Networks to nonlinear interactions

Neural networks (NNs) are well suited for a very broad class of nonlinear approximations and mappings. Neural networks consist of layers of uniform processing elements, nodes, units, or neurons. The neurons and layers are connected according to a specific architecture or topology. A so-called multilayer perceptron is a simple architecture which is sufficient for any continuous nonlinear input to output mapping. The number of input neurons (n) in the input layer is equal to the dimension of input vector X. The number of output neurons (m) in the output layer is equal to the dimension of the output vector Y. A multilayer perceptron always has at least one hidden layer with k neurons. Without going into detail, NNs perform a nonlinear mapping of an input vector X Î Â ⁿ on an output vector Y Î Â ^m

Y = f_NN(X) (2)

where f_NN denotes the neural network mapping which nonlinearly links each individual component of Y to all components of X (e.g., Chen 1996). It has been shown (e.g., Chen and Chen 1995a,b, Hornik 1991, Funahashi 1989, Gybenko, 1989) that a NN with one hidden layer can approximate any continuous mapping defined on compact sets in  ⁿ.

Thus, any problem which can be mathematically reduced to a nonlinear mapping can be solved using a NN. NN solutions for different problems will differ in several important ways. For each particular problem, n and m are determined by the dimensions of the input and output vectors X and Y. The number of hidden neurons k in each particular case should be determined taking into account the complexity of the problem. The more complicated the mapping, the more hidden neurons that are required (Attali and Pages 1997). After these topological parameters are defined, the weights and biases can be found, using a procedure called NN training (e.g., Beale and Jackson 1990, Chen 1996). Although NN training is often time consuming, NN application is not. After the training is finished (it usually needs to be performed only once), each application of the trained NN is practically instantaneous and yields an estimate for Eq. (2).

The nonlinear interaction source term can be considered as a nonlinear mapping between a continuous source term S_nl and a continuous spectrum F

S_nl = T(F) , (3)

where T in is the exact nonlinear operator given by the full Bolzmann interaction integral (Hasselmann and Hasselmann 1985, Resio and Perrie 1991). Discretization of S and F (as is necessary in any numerical approach) reduces (3) to continuous mapping of two vectors of finite dimensions.

In order to convert the mapping (3) to a continuous mapping of two finite vectors (independent of the actual spectral discretization), the spectrum F and source function S_nl are expanded using systems of two-dimensional functions each of which (F _i and Y _q) creates a complete and orthogonal two-dimensional basis

(4)

where for x_i and y_q we have

, (5)

where the double integral identifies integration over the spectral space. Because both sets of basis functions

{F _i }_i=1,�,� and {Y _q}_q=1,�,� are complete, increasing n and m in (4) improves the accuracy of approximation, and any spectrum F and source function S_nl can be approximated by (4) with a required accuracy. Substituting (4) into Eq. ( 3) we can get

Y = T (X) , (6)

which represents a continuous mapping of the finite vectors X Î Â ⁿ and Y Î Â ^m , and where T still represents the full nonlinear interaction operator. As described in the previous section, this operator can be approximated with a NN with n inputs and m outputs and k neurons in the hidden layer

Y � T_NN (X) . (7)

The accuracy of this approximation (T_NN) is determined by k, and can generally be improved by increasing k (see above).

To train the NN approximation T_NN of T, a training set has to be created which consists of pairs of vectors X and Y. To create this training set, a representative set of spectra F_p has to be generated with corresponding (exact) interactions S_nl,p. For each pair (F, S_nl)_p, the corresponding vectors (X,Y)_p are determined using Eq. (5). All pairs of vectors are then used to train the NN to obtain T_NN.

After T_NN has been obtained by training, the resulting NN Interaction Approximation (NNIA) algorithm consists of three steps :

• Decompose F by applying Eq. (5) to calculate X.
• Estimate Y from X using Eq. (7).
• Compose S_nl from Y using Eq. (4).

The above describes the general procedure for developing an NNIA. Development of an actual NNIA requires the following steps:

• Select basis functions
F _i and Y _q and the number of each (n,m).
• Design a NN topology (number of neurons k).
• Construct a representative training set.
• Select training strategies.

The first three points all have a significant impact on both accuracy and economy of a NNIA. Unfortunately, there is no pre-defined way to tackle these issues. It is therefore unavoidable that the development of a NNIA involves many iterations. The first requirement of an NNIA to be potentially useful in operational wave modeling, is that the exact interactions S_nl are closely reproduced for computational costs comparable to that of the DIA. The following section shows the potential of this approach with the design of a simple ad-hoc NNIA.

To address the basic feasibility of a NNIA, we have considered an NNIA to estimate the nonlinear interactions S_nl(f,q ) as a function of frequency f and direction q from the corresponding spectrum F(f,q ). We first will also consider deep water only. To train and test this NNIA, we used a set of about 20,000 simulated realistic spectra for F(f,q ), and the corresponding exact estimates of S_nl(f,q ) (Van Vledder, Herbers, Jensen, Resio, and Tracy 2000). Simulation has been performed using a generator that calculated a spectral function composed of several Pierson-Moskowitz (1964) spectra for different peak frequencies oriented randomly in [0,2p ] interval. Comparison of simulated spectra with spectra simulated by WAVEWATCH model (Tolman 1999, Tolman and Chalikov 1996) shows that this approach allowed us to simulate sufficiently realistic and complicated spectra describing a broad range of wave systems. Spectra with four peaks were used in calculations below. Separate data sets have been generated for training and validation.

As is common in parametric spectral descriptions, we choose separable basis functions where frequency and angular dependence are separated. For F _i this implies

  (8)

A similar separation is used for Y _q. Considering the strongly suppressed behavior of F and S_nl for f ® 0, and the exponentially decreasing asymptotic for f ® � , generalized Laguerre's polynomials (Abramowitz and Stegun 1964) are used to define f _f and y _f. Considering that no directional preferences exist in F and S_nl, a Fourier decomposition is used for f q and y q . The number of base functions is chosen to be n = 51 and m = 64 to keep the accuracy of approximation for F on average better than 2% and for S_nl - better than 5-6%. The number of hidden neurons was taken k = 30 which allows a satisfactory approximation (7) for the mapping (6).

Table 1. RMSE statistics for 10,000 S_nl

Fig. 1. RMSE as functions of frequency f and angle. Dashed line � error of approximation (lower bound for all other errors). Solid line � DIA, line with squares � NNIA (51:20:64), and line with triangles � NNIA (51:30:64)

Figure 1 shows mean RMSE as function of the frequency f (left) and the angle q (right). Numbers in Table 1 correspond to NNIA with NN with 30 neurons in the hidden layer (51:30:64).

Fig2 shows 3 pairs (one row in the figure corresponds to one pair) of one dimensional, integrated over q , source functions S_nl (f) (left column) and one dimensional, integrated over f, source functions S_nl(q ) (right column) from the validation data set. Thick solid curves correspond to the exact S_nl. Dashed curves correspond to DAI of S_nl. Curves with triangles correspond to the NNIA estimate of S_nl. Numbers inside the panels show DIA and NNIA errors in percents with respect to exact solution.

Fig.2. See explanations in the text above.

The results in Fig. 2 are fairly representative for the validation data set. In general, the NNIA reproduces the exact S_nl accurately. Even if clear oscillations are present in the decomposed spectrum (e.g., line in middle panel on left), the NNIA shows no spurious oscillations, and gives reasonable results. Note that many DIA source functions exhibit complicated behavior and spurious oscillations. Major peaks in these functions coexist with more or less random small-scale fluctuations. These fluctuations are probably an artifact produced by a simplified nature of DIA. Exact interactions are the result of averaging over much lager number of resonant sets of wave numbers, and are therefore much smoother than the results of the DIA.