40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Mon Jan 16 23:25:07 2017, gautam, Update, SUS, MC1 SUS electronics investigation 20170116_231625.pngIMG_7175.JPGIMG_7174.JPG
    Reply  Tue Jan 17 21:29:52 2017, gautam, Update, SUS, MC1 SUS electronics investigation 
       Reply  Wed Jan 18 11:40:54 2017, gautam, Update, SUS, MC1 SUS electronics investigation MC1glitches.pngMC1coilGlitches.pngMC1CoilGlitchZoomed.png
          Reply  Wed Jan 18 14:23:47 2017, gautam, Update, SUS, MC1 SUS electronics investigation 
             Reply  Wed Jan 18 18:44:53 2017, gautam, Update, SUS, MC1 SUS electronics investigation MCsatBoxSwitch.png
                Reply  Thu Jan 19 08:25:12 2017, Steve, Update, SUS, MC1 SUS electronics investigation MC1_MC3_ITMY_ETMX_sensors.pngsensors_UL.png
                   Reply  Thu Jan 19 12:00:10 2017, gautam, Update, SUS, MC1 SUS electronics investigation SatBoxSwitchGlitch.png
                      Reply  Thu Jan 19 19:56:09 2017, rana, Update, SUS, MC1 SUS electronics investigation 
                         Reply  Fri Jan 20 11:16:30 2017, gautam, Update, SUS, MC1 SUS electronics investigation MCglitchesReturn.png
                            Reply  Mon Jan 23 10:24:01 2017, Steve, Update, SUS, MC1 SUS electronics investigation 3glitchingSUS.png
Message ID: 12725     Entry time: Mon Jan 16 23:25:07 2017     Reply to this: 12728
Author: gautam 
Type: Update 
Category: SUS 
Subject: MC1 SUS electronics investigation 

[rana,gautam]

Summary:

  • MC1 glitchy behaviour is back
  • Found a broken LEMO cable, left unplugged for the night -> to be repaired tomorrow
  • Further diagnosis to follow

During the course of Rana's inspection of the general state of the IFO, he commented that there seemed to be several seismic-related IMC lock losses in the time that he had been observing it. This issue looked suspiciously like the the MC1 glitches I had noticed sometime late last year, especially since each time the IMC would unlock, we could see significant amounts of motion on MC REFL. To diagnose, we did the following:

  1. Closed PSL shutter
  2. Ramped down the gains of the MC1 damping loops by factor of 1000 in ~4 secs using z step
  3. Shut down the watchdog for MC1
  4. Observed dataviewer traces for glitches

Sure enough, there were several glitches that occurred in all 5 sensor channels. These glitches varied in size from a few counts (the smaller ones) to 60-70counts for the bigger ones. In the past, squishing the LEMO connector on the front of the PD whitening board (D000210) had apparently made the glitching go away. So tonight, for starters, we squished everything else - Sat. Box connectors, the breakout board from Sat. Box to whitening board on the back of 1X6, and the DB connector on the front of the whitening board. This had no effect - the glitching remained consistent.

Next, Rana pulled out two of the three 4pin LEMOs, and left only those coresponding to UL/LL plugged in - but the glitching persisted in these two channels. We then pulled out the board. It was installed in 1998, but has a sticker on it that says "fixed in 2003". Not sure what the fix was. Visual inspection of the circuit didn't show anything obviously faulty, but it did look like the two MAX333A quad switches (these control whether the whitening is bypassed or not) had been replaced at some point. There are other undesirable features, such as the use of thick film resistors, but nothing that would explain the glitchy behaviour.

Next, we re-inserted the whitening board back into its original slot in the Eurocrate, but switched the cables (both D sub and LEMO, but only on the whitening board end) between the boards for MC1 and MC3 (i.e. MC1 cables were routed through the whitening board that was originally used for MC3, and vice-versa). But the glitches remained consistent on the MC1 channels. So it looks like the board is not a likely culprit.

Finally, we went in and squished all the cables from the PD whitening board to the ADC (via an AA filter board). For some of the LEMO cables from the whitening board, the LEMO backshells were not properly tightened. Rana fixed these before putting them back in. Some of the connectors were also not pushed in tightly enough, Rana heard the click when he pushed them in. The cables from the adaptor board to the ADC itself looked fine, it was screwed on at both ends, and all these connections looked snug enough. In the interest of completeness, Rana also pushed in the backplane connectors on the Eurocrate (these supply the signals from the BIO cards to switch the whitening ON/OFF). The one corresponding to MC1 was indeed a little loose.

Coming back to the control room, we saw that the MC1 LR sensor was dead. After some investigation, Rana found that on the AA filter board end, one of the 4pin LEMOs from the whitening board had one of its wires come unstuck from where it was soldered (this presumably happened while we were squishing cables tonight, as the LR channel was fine before that). Also, there was no heat shrink used on any of the solder joints. Could this explain the glitchy behaviour? Perhaps, but the glitches remained in the 3 channels that were connected. Anyways, I will repair this cable tomorrow, and we can see if this has fixed the problem or not..


Some misc points:

  1. Regarding the adaptor boards that take the PD signals from the satellite box and route it to the whitening board, there are some clamps that hold the IDE connectors in place for MC1, MC2 and MC3 boards, but not for the others (see attached picture). Steve, can we install clamps for all of the boards? [taken care of, see here]
  2. The whitening boards are not screwed in place into the Eurocrate. This should be rectified.

PSL shutter is closed, MC1 watchdog is shutdown for the night.

Attachment 1: 20170116_231625.png  268 kB  | Hide | Hide all
20170116_231625.png
Attachment 2: IMG_7175.JPG  2.150 MB  | Hide | Hide all
IMG_7175.JPG
Attachment 3: IMG_7174.JPG  2.041 MB  | Hide | Hide all
IMG_7174.JPG
ELOG V3.1.3-