I tried to reduce the overfitting problem in previous neural network by reducing the number of nodes and layers and by varying the learning rate, beta factors (exponential decay rates of moving first and second moments) of Nadam optimizer assuming error of 5% is reasonable.
32 * 32 image frames (converted to 1d array & pixel values of 0 to 255 normalized) of simulated video by applying sine signal to move beam spot in pitch with frequency 0.2Hz and at 10 frames per second.
Total: 300 cycles , Train: 60 cycles, Validation: 90 cycles, Test: 150 cycles
Input --> Hidden layer --> Output layer
4 nodes 1 node
Activation function: selu linear
Batch size = 32, Number of epochs = 128, loss function = mean squared error
Learning rate = 0.00001, beta_1 = 0.8 (default value in Keras = 0.9), beta_2 = 0.85 (default value in Keras = 0.999)
Plot of predicted output by neural network, applied input signal & residual error given in 1st attachment.
Changed number of nodes in hidden layer from 4 to 8. All other parameters same.
These plots show that when residual error increases basically the output of neural network has a smaller amplitude compared to the applied signal. This kind of training error is unclear to me.
When beta parameters of optimizer is changed farther from 1, error increases.