Aim: To develop a neural network that resolves mirror motion from video.

I tried to reduce the overfitting problem in previous neural network by reducing the number of nodes and layers and by varying the learning rate, beta factors (exponential decay rates of moving first and second moments) of Nadam optimizer assuming error of 5% is reasonable.

Input:

32 * 32 image frames (converted to 1d array & pixel values of 0 to 255 normalized) of simulated video by applying sine signal to move beam spot in pitch with frequency 0.2Hz and at 10 frames per second.

Total: 300 cycles ,           Train: 60 cycles,    Validation: 90 cycles,    Test: 150 cycles

Model topology:

                                          Input               -->                  Hidden layer               -->                    Output layer                                  

                                                                                          4 nodes                                              1 node

Activation function:                                  selu                                             linear

Batch size = 32, Number of epochs = 128, loss function = mean squared error

Optimizer: Nadam

Case 1:

Learning rate = 0.00001,    beta_1 = 0.8 (default value in Keras = 0.9),  beta_2 = 0.85 (default value in Keras = 0.999)

Plot of predicted output by neural network, applied input signal & residual error given in 1st attachment.

Case 2:

Changed number of nodes in hidden layer from 4 to 8. All other parameters same.

These plots show that when residual error increases basically the output of neural network has a smaller amplitude compared to the applied signal. This kind of training error is unclear to me.

When beta parameters of optimizer is changed farther from 1, error increases.