40m QIL Cryo_Lab CTN SUS_Lab CAML OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Sat Aug 5 11:04:07 2017, gautam, Update, SUS, MC1 glitches return MC1_glitch_Aug42017.pngMC1_glitch_zoomed.png
    Reply  Wed Aug 9 15:15:47 2017, gautam, Update, SUS, MC1 glitches return MC1_glitch.png
       Reply  Thu Aug 10 21:01:43 2017, gautam, Update, SUS, MC1 glitches debugging MC_QUAD_10AUG2017.jpgMCR_10AUG2017.jpg
          Reply  Fri Aug 11 12:32:46 2017, gautam, Update, SUS, MC1 glitches debugging MC1_freeswinging.pdfMC1_inmatrix.pngMC1_sensors.png
             Reply  Fri Aug 11 17:36:47 2017, gautam, Update, SUS, MC1 <--> MC3 MC1_glitch_watchdog_shutdown.png
                Reply  Sat Aug 12 14:09:36 2017, gautam, Update, SUS, Glitches stay on MC1 MC1_glitch.pngMC_12hr_trend.pngMC1_glitches_intermittent.png
                   Reply  Sat Aug 12 22:35:22 2017, rana, Update, SUS, Glitches stay on MC1 
                      Reply  Sun Aug 13 01:35:09 2017, Koji, Update, SUS, Glitches stay on MC1 
                         Reply  Mon Aug 14 20:01:38 2017, gautam, Update, SUS, Glitches stay on MC1 
                Reply  Wed Aug 16 19:50:17 2017, gautam, Update, SUS, MC1 <--> MC3 switched back 
                   Reply  Thu Aug 17 11:17:49 2017, gautam, Update, SUS, MC1 <--> MC3 switched back MC1_misaligned.pngMC1_glitch.png
                      Reply  Thu Aug 17 17:33:01 2017, gautam, Update, SUS, MC1 <--> MC3 switched back 
Message ID: 13187     Entry time: Thu Aug 10 21:01:43 2017     In reply to: 13178     Reply to this: 13195
Author: gautam 
Type: Update 
Category: SUS 
Subject: MC1 glitches debugging 

I have squished cables in all the places I can think of - but MC1 has been glitching regularly today. Before starting to pull electronics out, I am going to attempt a more systematic debugging in the hope I can localize the cause.

To this end, I've disabled the MC autolocker, and have shutdown the MC1 watchdog. I plan to leave it in this state overnight. From this, I hope to look at the free-swinging optic spectra to see that this isn't a symptom of something funky with the suspension itself.

Some possible scenarios (assuming the free swinging spectra look alright and the various resonances are where we expect them to be):

  1. With the watchdog shutdown, the PIT/YAW bias voltages still goto the coil (low-passed by 4 poles @1Hz). So if the glitching happens in this path, we should see it in both the shadow sensors and the DC spot positions on the WFS.
  2. If the glitching happens in the shadow sensor readout electronics/cabling, we should see it in the shadow sensor channels, but NOT in the DC spot positions on the WFS (as the watchdog is shutdown, so there should be no actuation to the coils based on OSEM signals).
  3. If we don't see any glitches in WFS spot positions or shadow sensors, then it is indicative of the problem being in the coil driver board / dewhitening board/anti-aliasing board.
  4. I am discounting the problem being in the Satellite box, as we have switched around the MC1 satellite box multiple times - the glitches remain on MC1 and don't follow a Satellite Box. Of course there is the possibility that the cabling from 1X5/1X6 to the Satellite box is bad.

MC1 has been in a glitchy mood today, with large (MC-REFL spot shifts by ~1 beam diameter on the CCD monitor) glitches happening ~every 2-3 hours. Hopefully it hasn't gone into an extended quiet period. For reference, I've attached the screen-grab of the MC-QUAD and MC-REFL as they are now.


GV 9.20PM: Just to make sure of good SNR in measuring the pendulum eigenfreqs, I ran /opt/rtcds/caltech/c1/scripts/SUS/freeswing MC1 in a terminal . The result looked rather violent on the camera but its already settling down. The terminal output:

The following optics were kicked:
MC1
Thu Aug 10 21:21:24 PDT 2017
1186460502
Quote:

Happened again just now, although the characteristics of the glitch are very different from the previous post, its less abrupt. Only actuation on MC1 at this point was local damping.

 

Attachment 1: MC_QUAD_10AUG2017.jpg  21 kB  Uploaded Thu Aug 10 22:06:02 2017  | Show | Hide all | Show all
Attachment 2: MCR_10AUG2017.jpg  16 kB  Uploaded Thu Aug 10 22:07:55 2017  | Hide | Hide all | Show all
MCR_10AUG2017.jpg
ELOG V3.1.3-