40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Message ID: 14070     Entry time: Fri Jul 13 23:23:49 2018
Author: pooja 
Type: Update 
Category: Cameras 
Subject: Update in developing neural networks 

Aim: To develop a neural network that resolves mirror motion from video.

I tried to reduce the overfitting problem in previous neural network by reducing the number of nodes and layers and by varying the learning rate, beta factors (exponential decay rates of moving first and second moments) of Nadam optimizer assuming error of 5% is reasonable.


32 * 32 image frames (converted to 1d array & pixel values of 0 to 255 normalized) of simulated video by applying sine signal to move beam spot in pitch with frequency 0.2Hz and at 10 frames per second.

Total: 300 cycles ,           Train: 60 cycles,    Validation: 90 cycles,    Test: 150 cycles

Model topology:

                                          Input               -->                  Hidden layer               -->                    Output layer                                  

                                                                                          4 nodes                                              1 node

Activation function:                                  selu                                             linear

Batch size = 32, Number of epochs = 128, loss function = mean squared error

Optimizer: Nadam

Case 1:

Learning rate = 0.00001,    beta_1 = 0.8 (default value in Keras = 0.9),  beta_2 = 0.85 (default value in Keras = 0.999)

Plot of predicted output by neural network, applied input signal & residual error given in 1st attachment.

Case 2:

Changed number of nodes in hidden layer from 4 to 8. All other parameters same.

These plots show that when residual error increases basically the output of neural network has a smaller amplitude compared to the applied signal. This kind of training error is unclear to me.

When beta parameters of optimizer is changed farther from 1, error increases.

Attachment 1: nn_simulation_2_nodes4_lr0p00001_beta1_0p8_beta2_0p85.pdf  215 kB  Uploaded Sat Jul 14 01:29:02 2018  | Hide | Hide all
Attachment 2: nn_simulation_2_nodes8_lr0p00001_beta1_0p8_beta2_0p85.pdf  252 kB  Uploaded Sat Jul 14 01:29:30 2018  | Hide | Hide all
ELOG V3.1.3-