40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Sun Jul 22 14:01:07 2018, pooja, Update, Cameras, Developing neural networks on simulated video nn_simulation_2_normalized_mult_sin_nodes8_128epochs_lr0p00001_beta1_0p8_beta2_0p85_0p4train_0p1valid_marked.pdfnn_simulation_normalizedtarget_128epochs_mult_sin_load_wt_varyingtest_nodes8_lr0p00001_beta1_0p8_beta2_0p85_0p4train_0p1valid_marked.pdfnn_simulation_2_normalized_varying_mult_sin_nodes8_128epochs_lr0p00001_beta1_0p8_beta2_0p85_0p4train_0p1valid_marked.pdf
    Reply  Tue Jul 24 06:11:50 2018, rana, Update, Cameras, Developing neural networks on simulated video 
       Reply  Tue Jul 24 09:47:51 2018, gautam, Update, Cameras, Developing neural networks on simulated video 
Message ID: 14097     Entry time: Sun Jul 22 14:01:07 2018     Reply to this: 14100
Author: pooja 
Type: Update 
Category: Cameras 
Subject: Developing neural networks on simulated video 

Aim: To develop a neural network that resolves mirror motion from video.

Since error was high for the same input as in my previous elog http://nodus.ligo.caltech.edu:8080/40m/14089

I modified the network topology by tuning the number of nodes, layers and learning rate so that the model fitted the sum of 4 sine waves efficiently, saved weights of the final epoch and then in a different program, loaded saved weights & tested on simulated video that's produced by moving beam spot from the centre of image by sum of 4 sine waves whose frequencies and amplitudes change with time.

Input : Simulated video of beam spot motion in pitch by applying 4 sine  waves of frquencies 0.2, 0.4, 0.1, 0.3 Hz  and amplitude ratios to frame size to be 0.1, 0.04, 0.05, 0.08. This is divided into train (0.4), validation (0.1) and test (0.5).

Model topology:

                                          Input               -->                  Hidden layer               -->                    Output layer                                  

                                                                                          8 nodes                                              1 node

Activation function:                                  selu                                             linear

Batch size = 32, Number of epochs = 128, loss function = mean squared error

Optimizer: Nadam ( learning rate = 0.00001, beta_1 = 0.8, beta_2 = 0.85)

Normalized the target sine signal of NN by dividing by its maximum value.

Plot of predicted output by neural network, applied input signal & residual error given in 1st attachment. These weights of the model in the final epoch have been saved to h5 file and then loaded & tested with simulated data of 4 sine waves with amplitudes and frequencies changing with time from their initial values by random uniform noise ranging from 0 to 0.05. Plot of predicted output by neural network, target signal of sine waves & residual error given in 2nd attachment. The actual signal can be got from predicted output of NN by multiplication with normalization constant used before. However, even though network fits training  & validation sets efficiently, it gives a comparatively large error on test data of varying amplitude & frequency.

Gautam suggested to try training on this noisy data of varying amplitudes and frequencies. The results using the same model of NN is given in Attachment 3. It was found that tuning the number of nodes, layers or learning rate didn't improve fitting much in this case.



Attachment 1: nn_simulation_2_normalized_mult_sin_nodes8_128epochs_lr0p00001_beta1_0p8_beta2_0p85_0p4train_0p1valid_marked.pdf  222 kB  Uploaded Sun Jul 22 15:53:11 2018  | Hide | Hide all
Attachment 2: nn_simulation_normalizedtarget_128epochs_mult_sin_load_wt_varyingtest_nodes8_lr0p00001_beta1_0p8_beta2_0p85_0p4train_0p1valid_marked.pdf  257 kB  Uploaded Sun Jul 22 15:53:47 2018  | Hide | Hide all
Attachment 3: nn_simulation_2_normalized_varying_mult_sin_nodes8_128epochs_lr0p00001_beta1_0p8_beta2_0p85_0p4train_0p1valid_marked.pdf  215 kB  Uploaded Sun Jul 22 15:54:01 2018  | Hide | Hide all
ELOG V3.1.3-