Pete, Rob

After looking at some oplev noise spectra in DTT, we discovered that the ETMY quad (serial number 115)  was noisy.  Particularly, in the XX_OUT and XX_IN1 channels, quadrants 2 (by a bit more than an order of magnitude over the ETMX ref) and 4 (by a bit less than an order of mag).  We went out and looked at the signals coming out of the oplev interface board; again, channels 2 and 4 were noise compared to 1 and 3 by about these same amounts.  I popped in the ETMX quad and everything looked fine.  I put the ETMX quad back at ETMX, and popped in Steve's scatterometer quad (serial number 121 or possibly 151, it's not terribly legible), and it looks fine.  We zeroed via the offsets in the control room, and I went out and centered both the ETMX and ETMY quads. 

Attached is a plot.  The reference curves are with the faulty quad (115).  The others are with the 121.

 I adjusted the ETMY quad gains up by a factor of 10 so that the SUM is similar to what it was before.

It seems that the MC3 problem is intermittent (one-day trend attached).  I tried to take advantage of a "clean MC3" night, but the watch script would usually fail at the transition to DC CARM and DARM.  It got past this twice and then failed later, during powering up.   I need to check the handoff.

I checked the four rear coils on ETMX by exciting XXCOIL_EXC channel in DTT with amplitude 1000@ 500 Hz and observing the oplev PERROR and YERROR channels.  Each coil showed a clear signal in PERROR, about 2e-6 cts.  Anyway, the coils passed this test.

I found some neat signal analysis software for my mac (http://www.faberacoustical.com/products/), and took a spectrum of the ambient noise coming from the cryopump.  The two main noise peaks from that bad boy were nowhere near 3.7 kHz.

 I also made xfer fctns of the 4 piston coils on ETMY and ETMX with OL_PIT.  (I looked at all 4 even though the attached plot only shows three.)  So it looks ike the coils are OK.

Looks like something went nuts in late April.  We have yet to try a hard reboot.

alberto, pete

We worked on tuning the DD handoff tonight.  We checked the DD PD alignments and they looked fine.  First I tuned the 3 demod phases to minimize offsets.  Then I noticed that the post-handoff MICH xfer function needed an increase in gain to look like the pre-handoff xfer function (which has a UGF of about 25 Hz).  I increased the MICH PD9_Q gain from 2 to 7 in the input matrix.   But, the handoff to PRC still failed, so tomorrow we will try to find out why.

In the plot, ref0 is before MICH handoff, and ref1 is after MICH handoff.  There is also a PRC trace (before PRC handooff).

rob, alberto, rana, pete

we reset this computer, which was out of sync (16384 in the FE_SYNC field instead of 0)

Rana, Alberto, Pete

We have the DD handoff nominally working.  Sometimes, increasing the SRC gain at the end makes MICH get unstable.  This could be due to a non-diagonal term in the matrix, or possibly because the DRM locks in a funky mode sometimes. 

To get the DD handoff working, first we tuned demod phases in order to zero the offsets in the PD signals handed-off-to.  Based on transer function measurements, I set the PRC PD6_I element to 0.1, and set the PD8_I signal to 0, since it didn't seem to be contributing much.  We also commented out the MICH gain increase at the end of the DD_handoff script.

It could still be more stable, but it seems to work most of the time.

I played with the DD handoff during the day.  The DRM dark port was flickering like a candle flame in Dracula's castle.  The demod offsets for the handoff signals looked fine.  After MICH handoff, the MICH_CTRL started to get unstable at some low frequency, maybe 3 Hz (I didn't measure).  So I increased the MICH gain from 0.1 to 0.17 and it settled down.  PRC and SRC went fine.  Then the DD_handoff script raised the MICH gain to 0.7, and an instability started to grow in MICH_CTRL (at some higher frequency).  I decreased the MICH gain from 0.7 to 0.5, and it settled down and stayed stable.

Looks like yesterday was particularly noisy.  It's unclear to me why diurnal variation much more visible in MC1_Y, and why the floor wanders.

The first plot shows 5 days.  The second plot shows 20 days.

After fixing the tp problem, I tried locking again.  Grabbing and DD handoff, no problem.  Died earlier than last night, handing off CARM to REFL_DC, around arm power of 4 or so.  Seems to happen after turning off the moving zero, Rob says it might be touchy in daytime.

Last night Rob ran senseDRM and loadDRMImatrixData and came up with the following for the input matrix:

tdswrite C1:LSC-ITMTRX_b2 0.065778 \
C1:LSC-ITMTRX_d2 2.2709 \
C1:LSC-ITMTRX_f2 2.9361 \
C1:LSC-ITMTRX_122 0.42826 \
C1:LSC-ITMTRX_b3 -0.064839 \
C1:LSC-ITMTRX_d3 -0.016913 \
C1:LSC-ITMTRX_f3 -0.021576 \
C1:LSC-ITMTRX_123 -0.0025243 \
C1:LSC-ITMTRX_b5 0.3719 \
C1:LSC-ITMTRX_d5 1.3109 \
C1:LSC-ITMTRX_f5 -0.16412 \
C1:LSC-ITMTRX_125 0.39574 \
C1:LSC-ITMTRX_33 0 \
C1:LSC-ITMTRX_42 0 \
C1:LSC-ITMTRX_155 0

Today, I reran these and got the following, and DD_handoff remained happy:

tdswrite C1:LSC-ITMTRX_b2 -0.10329 \
C1:LSC-ITMTRX_d2 2.0344 \
C1:LSC-ITMTRX_f2 3.2804 \
C1:LSC-ITMTRX_122 0.22516 \
C1:LSC-ITMTRX_b3 -0.076292 \
C1:LSC-ITMTRX_d3 -0.014603 \
C1:LSC-ITMTRX_f3 -0.12101 \
C1:LSC-ITMTRX_123 0.0054128 \
C1:LSC-ITMTRX_b5 0.33521 \
C1:LSC-ITMTRX_d5 1.1425 \
C1:LSC-ITMTRX_f5 -0.32759 \
C1:LSC-ITMTRX_125 0.25877 \
C1:LSC-ITMTRX_33 0 \
C1:LSC-ITMTRX_42 0 \
C1:LSC-ITMTRX_155 0

I wanted to remeasure with the canonical output matrix (-0.7 from MICH to PRM and 0.7 from MICH to SRM), but the DRM freaked out when MICH to PRM went below -0.3.

I added a temporary channel, to input 9 on the PEM ADCU.    Beware the 30, 31, and 32 inputs.  I tried 32 and it only gave noise.

I compiled and ran a simple (i.e. empty) front end controller on scipe12 at wilson house.  I hooked a signal into the ADC and watched it in the auto-generated medm screens. 

There were a couple of gotchas:

1. Add an entry SYS to the file /etc/rc.local, to the /etc/setup_shmem.rtl line, where the system file is SYS.mdl.

2. If necessary, do a BURT restore.  Or in the case of a mockup set the BURT Restore bit (in SYS_GDS_TP.adl) to 1.

Koji, Pete 

Yesterday, Jay brought over the IO box for megatron, and got it working.  We plan to firewall megatron this afternoon, with the help of Jay and Alex, so we can set up GDS there and play without worrying about breaking things.  In the meantime, we went to Wilson House to get some breakout boards so we can take transfer functions with the 785, for an ETMX controller.  We put in a sine wave, and all looks good on the auto-generated epics screens, with an "empty" system (no filters on). Next we'll load in filters and take transfer functions.

Unfortunately we promised to return the breakout boards by 1pm today.  This is because, according to denizens of Wilson House, Osamu "borrowed" all their breakout boards and these were the last two!  If we can't locate Osamu's cache, they expect to have more in a day or two.

Here is the transfer function of the through filter working at 16KHz sampling. It looks fine except for the fact that the dc gain is ~0.8. Koji is going to characterize the digital down sampling filter in order to try to compare with the generated code and the filter coefficients.

Alex has firewalled megatron.  We have started a framebuilder there and added testpoints.  Now it is possible to take transfer functions with the shared memory MDC+MDP sandbox system.  I have also copied filters into MDC (the controller) and made a really ugly medm master screen for the system, which I will show to no one.

Koji, Pete

Yesterday we found that the channel C1:MDP-POS_EXC looked distorted and had what appeared to be doubled frequency componenets, in the dataviewer.  This was because the dcu_rate in the file /caltech/target/fb/daqdrc was set to 16K while the adl file was set to 32K.  When daqdrc was corrected it was fixed.  I am going to recompile and run all these models at 16K.  Once the 40 m moves over to the new front end system, we may find it advantageous to take advantage of the faster speeds, but maybe it's a good idea to get everything working at 16K first.

Koji, Peter

We put a simple pendulum into the MDP model, and everything communicates.  We're still having some kind of TP or daq problem, so we're still in debugging mode.  We went back to 32K in the .adl's, and when driving MDP,  the MDC-ETMX_POS_OUT is nasty, it follows the sine wave envelope but goes to zero 16 times per second.

The breakout boards have arrived.  The plan is to fix this daq problem, then demonstrate the model MDC/MDP system.  Then we'll switch to the "external" system (called SAM) and match control TF to the model.  Then we'd like to hook up ETMX, and run the system isolated from the rest of the IFO.  Finally we'd like to tie it into the IFO using reflective memory.

The daq on megatron was nuts.  Alex and I discovered that there was no gds installation for site_letter=C (i.e. Caltech) so the default M was being used (for MIT).  Apparently we are the first Caltech installation.  We added the appropriate line to the RCG Makefile and recompiled and reinstalled (at 16K).  Now DV looks good on MDP and MDC, and I made a transfer function that replicates  bounce-roll filter.  So DTT works too.

Today I was able to make low frequency transfer function with DTT on megatron.  There seems to have been a timing problem, perhaps Alex fixed it or it is intermittent.

I have attached the open loop transfer function for the un-optimized system, which is at least stable to step impulses with the current filters and gains.  The next step is to optimize, transfer this knowledge to the ADC/DAC version, and hook it up to isolated ETMX.

I've added the PIT and YAW dofs to the MDC and MDP systems.  The pendula frequencies in MDP are 0.8, 0.5, 0.6 Hz for POS, PIT, and YAW respectively.  The three dofs are linear and uncoupled, and stable, but there is no modeled noise in the system (yet) and some gains may need bumping up in the presence of noise.  The MDC filters are identical for each dof (3:0.0 and Cheby). The PIT and YAW transfer functions look pretty much like the one Rana recently took of POS, but of course with the different pendulum frequencies.  I've attached one for YAW.

Pete, Koji

We discussed a preliminary game plan for this project.  The thing I really want to see is an ETMX RCG controller hooked into the existing frontend via reflective memory, and the 40 m behaving normally with this hybrid system, and my list is geared toward this.  I suspect the list may cause controversy.

+ copy the MDC filters into SAM, and make sure everything looks good there with DTT and SR785.

+ get interface / wiring boards from Wilson House, to go between megatron and the analog ETMX system

+ test tying the ETMX pendulum and bare-bones SAM together (use existing watchdogs, and "bare-bones" needs defining)

+ work some reflective memory magic and create the hybrid frontend

In parallel with the above, the following should also happen:

+ MEDM screen design

+ add non-linear bits to the ETMX MDP/MDC model system

+ make game plan for the rest of the RCG frontend

 Joe and I moved megatron and its associated IO chassis from 1Y3 to 1Y9, in preparations for RCG tests at ETMY.

  Koji, Joe, and I are planning to try controlling the ETMY, on Monday or Tuesday.  Our plan is to try to do this with megatron out of the RFM loop.   The RCG system includes pos, pit, yaw, side, and oplevs.  I will use matrix elements as currently assigned (i.e. not the ideal case of +1 and -1 everywhere).   I will also match channel names to the old channels.   We could put buttons on the medm screen to control the analog DW by hand.  

 I've added the side coil to the model controller and plant, and the oplev quad to the model controller and plant.  After the megatron wipe, the code now lives in /home/controls/cds/advLigo/src/epics/simLink.  The files are mdc.mdl (controller) and mdp.mdl (plant).  These RCG modules go at 16K with no decimation (no_oversampling=1 in the cdsParameters block) so hopefully will work with the old (16K) timing.

I've loaded many of the filters, there are some eft to do.  These filters are simply copied from the current frontend.  

Next I will port to the SUS module (which talks to the IO chassis).  This means channel names will match with the current system, which will be important when we plug in the RFM.

The .mdl code for the mdc and mdp development modules is finished.  These modules need more filters, and testing.  Probably the most interesting piece left to do is putting in the gains and filters for the oplev model in mdp.  It might be OK to simply ignore oplevs and first test damping of the real optic without them.   However, it shouldn't be hard to get decent numbers for oplevs, add them to the mdp (plant) module, and make sure the mdc/mdp pair is stable.  In mdp, the oplev path starts with the SUSPIT and SUSYAW signals. Kakeru recently completed calibration of the oplevs from oplev cts to radians:   1403  .  From this work we should find the conversion factors from PIT and YAW to oplev counts, without making any new measurements.  (The measurements wouldn't be hard either, if we can't simply pull numbers from a document.)  These factors can be added to mdp as appropriate gains.

I've also copied mdc to a new module, which I've named "sas" to address fears of channel name collisions in the short term, and replaced the cpu-to-cpu connections with ADC and DAC connections.  sas can be the guy for the phase I ETMY test.  When we're happy with mdc/mdp, we hopefully can take the mdc filter file from chans, replace all the "MDC" strings with "SAS", and use it.

/cvs/cds/caltech/target/fb/daqd -c daqdrc

This starts the FB.

Now the dataviewer and DTT work!

 Instead of doing RCG stuff, I went to Millikan to work on data analysis as I couldn't stand the fumes from the construction.  (this morning, 8am) 

the command

tdsavg 5 C1:LSC-PD4_DC_IN1

was causing grievous woe in the cm_step script.  It turned out to fail intermittently at the command line, as did other LSC channels.  (But non-LSC channels seem to be OK.)  So we power cycled c1lsc (we couldn't ssh).

Then we noticed that computers were out of sync again (several timing fields said 16383 in the C0DAQ_RFMNETWORK screen).  We restarted c1iscey, c1iscex, c1lsc, c1susvme1, and c1susvme2.  The timing fields went back to 0.  But the tdsavg command still  intermittently said "ERROR: LDAQ - SendRequest - bad NDS status: 13".

The channel C1:LSC-SRM_OUT16 seems to work with tdsavg every time.

Let us know if you know how to fix this. 

attached plot shows MC_IN1/MC_IN2.  needs work.

This is supposed to be a measurement of the relative gain of the MCL and AO paths in the CM servo. We expect there to

be a more steep slope (ideally 1/f). Somehow the magnitude is very shallow and so the crossover is not stable. Possible

causes? Saturations in the measurement, broken whitening filters, extremely bad delay in the digital system? needs work.

Today Steve and I tried to to capture the image of scattering of light by dust particles on the surface of ETMX using GigE camera. The image ( at gain =100, exposure time = 125000) obtained has been attached. Unlike the previous images, a creepy shape of bright spots was seen. Gautam helped us lock infrared light and see the image. A similar less intense shape was seen. This may be because of the dust on the lens.

Aim: To synchronize data from the captured video and the signal applied to ETMX

In order to correlate the intensity fluctuations of the scattered light with the motion of the test mass, we are planning to use the technique of neural network. For this, we need a synchronised video of scattered light with the signal applied to the test mass. Gautam helped me capture 60sec video of scattering of infrared laser light after ETMX was dithered in PITCH at ~0.2Hz..

I developed a python program to capture the video and convert it into a time series of the sum of pixel values in each frame using OpenCV to see the variation. Initially we had tried the same with green laser light and signal of approximately 11.12Hz. But in order to see the variation clearly, we repeated with a lower frequency signal after locking IR laser today. I have attached the plots that we got below. The first graph gives the intensity fluctuations from the video. The third and fourth graphs are that of transmitted light and the signal applied to ETMX to shake it. Since the video captured using the camera was very noisy and intensity fluctuations in the scattered light had twice the frequency of the signal applied, we captured a video after turning off the laser. The second plot gives the background noise probably from the camera. Since camera noise is very high, it may not be possible to train this data set in neural network.

Since the videos captured consume a lot of memory I haven't uploaded it here. I have uploaded the python code 'sync_plots.py' in github (https://github.com/CaltechExperimentalGravity/GigEcamera/tree/master/Pooja%20Sekhar/PythonCode).

Just to inform, I'm working in optimus to develop python code to train the neural network since it requires a lot of memory.

Aim: To develop a neural network in order to correlate the intensity fluctuations in the scattered light to the angular motion of the test mass. A block diagram of the technique employed is given in Attachment 1.

I have used Keras to implement supervised learning using neural network (NN). Initially I had developed a python code that converts a video (59 sec) of scattered light, after an excitation (sine wave of frequency 0.2 Hz) is applied to ETMX pitch, to image frames (of size 480*720)  and stores the 2D pixel values of 1791 images frames captured into an hdf5 file. This array of shape (1791,36500) is given as an input to the neural network. I have tried to implement regular NN only, not convolution or recurrent NN. I have used sequential model in Keras to do this. I have tried with various number of dense layers and varied the number of nodes in each layer. I got test accuracy of approximately 7% using the following network. There are two dense layers, first one with 750 nodes with a dropout of 0.1 ( 10% of the nodes not used) and second one with 500 nodes. To add nonlinearity to the network, both the layers are given an activation function of tanh. The output layer has 1 node and expects an output of shape (1791,1). This model has been compiled with a loss function of categorical crossentropy, optimizer = RMSprop. We have used these since they have been mostly used in the image analysis examples. Then the model is trained against the dataset of mirror motion. This has been obtained by sampling the cosine wave fit to the mirror motion so that the shapes of the input and output of NN are consistent. I have used a batch size ( number of samples per gradient update) = 32 and epochs (number of times entire dataset passes through NN) = 20. However, using this we got an accuracy of only 7.6%. 

I think that the above technique gives overfitting since dense layers use all the nodes during training apart from giving a dropout. Also, the beam spot moves in the video. So it may be necessary to use convolution NN to extract the information.

The video file can be accesses from this link https://drive.google.com/file/d/1VbXcPTfC9GH2ttZNWM7Lg0RqD7qiCZuA/view.

Gabriele told us that he had used the beam spot motion to train the neural network. Also he informed that GPUs are necessary for this. So we have to figure out a better way to train the network.  

gautam noon 11Jun: This link explains why the straight-up fully connected NN architecture is ill-suited for the kind of application we have in mind. Discussing with Gabriele, he informed us that training on a GPU machine with 1000 images took a few hours. I'm not sure what the CPU/GPU scaling is for this application, but given that he trained for 10000 epochs, and we see that training for 20 epochs on Optimus already takes ~30minutes, seems like a futile exercise to keep trying on CPU machines.

Aim: To calibrate CCD of GigE using LED1050E.

The following table shows some of the specifications for LED1050E as given in Thorlabs datasheet.

 The circuit diagram is given in Attachment 1.

Considering a power supply voltage V_cc = 15V, current I = 20mA & forward voltage of led V_F = 1.25V, resistance in the circuit is calculated as,

R = (V_cc - V_F)/I = 687.5

Attachment 2 gives a plot of resistance (R) vs input voltage (V_cc) when a current of 20mA flows through the circuit. I hope I can proceed with this setup soon.

Today I made the led (1050nm) circuit inside a box as given in my previous elog. Steve drilled a 1mm hole in the box as an aperture for led light.

Resistance (R) used = 665 .

We connected a power supply and IR has been detected using the card.

Later we changed the input voltage and measured the optical power using a powermeter.

Input voltage (Vcc in V)	Optical power
0 (dark reading)	60 nW
15	68 W
18	82.5 W
20	92 W

Since the optical power values are very less, we may need to drill a larger hole.

Now the hole is approximately 7mm from led, therefore aperture angle is approximately 2*tan^-1(0.5/7) = 8deg. From radiometric curve given in the datasheet of LED1050E, most of the power is within 20 deg. So a hole of size 2* tan(10) *7 = 2.5mm may be required.

I have also attached a photo of the led beam spot on the IR detection card.

Aim : To develop a neural network on simulated data.

I developed a python code that generates a 64*64 image of a white Gaussian beam spot at the centre of black background. I gave a sine wave of frequency 0.2Hz that moves the spot vertically (i.e. in pitch). Then I simulated this video at 10 frames/sec for 10 seconds. Then I saved this data into an hdf5 file, reshaped it to a 1D array and gave as input to a neural network. Out of the 100 image frames, 75 were taken as training dataset and 25 as test data. I varied several hyperparameters like learning rate of the optimizer, number of layers, nodes, activation function etc. Finally, I was successful in reducing the mean squared error with the following network model:

• Sequential model of 2 fully connected layers with 256 nodes each and a dropout of 0.1
• loss function = mean squared error, optimizer = RMSprop (learning rate = 0.00001) and activation function that adds nonlinearity = relu
• batch size = 32 and number of epochs = 1000

I have attached the plot of the output of neural network (NN) as well as sine signal applied to simulate the video and their residula error in Attachment 1. The plot of variation in mean squared error (in log scale) as number of epochs increases is given in Attachment 2.

I think this network worked easily since there is no noise in the input. Gautam suggested to try the working of this network on simulated data with a noisy background.