I had created a python code to find the combination of hyperparameters that trains the neural network. The code (nn_hyperparam_opt.py) is present in the github repo. It's running in cluster since a few days. In the meanwhile I had just tried some combination of hyperparameters.
These give a low loss value of approximately 1e-5 but there is a large error bar for loss value since it fluctuates a lot even after 1500 epochs. This is unclear.
Input: 64*64 image frames of simulated video by applying beam motion sine wave of frequency 0.2Hz and at 10 frames per sec. This input data is given as an hdf5 file.
Train : 100 cycles, Test: 300 cycles, Optimizer = Nadam (learning rate = 0.001)
256 -> 128 -> 1
Activation : selu selu linear
Case 1: batch size = 48, epochs = 1000, loss function = mean squared error
Plots of output predicted by neural network (NN) & input signal has been shown in 1st graph & variation in loss value with epochs in 2nd graph.
Case 2: batch size = 32, epochs = 1500, loss function = mean squared logarithmic error
Plots of output predicted by neural network (NN) & input signal has been shown in 3rd graph & variation in loss value with epochs in 4th graph.