"cugb410"

"The activation function also plays an important role in the optimization of the network. We use a locally adaptive inverse tangent function for all hidden layers except the final layer, which has a linear activation function. Locally adaptive activation functions have recently shown better learning capabilities than the traditional or fixed activation functions in achieving higher convergence rate and solution accuracy (Jagtap et al., 2020). Using a scalable parameter in the activation function for each neuron changes the slope of the activation function and, therefore, alters the loss landscape of the network, yielding improved performance.It is also worth emphasizing that the proposed approach is different from traditional (or non-physics constrained) deep learning techniques. The training of the network here refers to the tuning of weights and biases of the network such that the resulting solution minimizes the loss function J on a given set of training points. The training set here refers to the collocation points, usually chosen randomly, from within the computational domain. The number of collocation points needed to obtain a sufficiently accurate solution increases with the heterogeneity of the velocity model."