40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Mon Dec 7 11:25:31 2020, gautam, Update, SUS, MC1 suspension glitchy again 
    Reply  Wed Dec 9 16:22:57 2020, gautam, Update, SUS, Yet another round of Sat. Box. switcharoo WFS2.pngWFS_lineNoise.pngWFSchar.pdf
       Reply  Sun Jan 3 16:26:06 2021, Koji, Update, SUS, IMC WFS check (Yet another round of Sat. Box. switcharoo) Screen_Shot_2021-01-03_at_17.14.57.png
    Reply  Thu May 13 11:55:04 2021, Anchal, Paco, Update, SUS, MC1 suspension misbehaving Screenshot_from_2021-05-13_09-50-24.pngMC1_Glitches_Invest2.pdf
       Reply  Thu May 13 19:38:54 2021, Anchal, Update, SUS, MC1 Satellite Amplifier Debugged MC1_UL_Channel_Fixed.png
          Reply  Mon May 24 19:14:15 2021, Anchal, Paco, Summary, SUS, MC1 Free Swing Test set to trigger 
             Reply  Tue May 25 10:22:16 2021, Anchal, Paco, Summary, SUS, MC1 new input matrix calculated and uploaded SUS_Input_Matrix_Diagonalization.pdf
          Reply  Thu Jun 17 11:45:42 2021, Anchal, Paco, Update, SUS, MC1 Gave trouble again SummaryScreenShot.pngMC1_LL_SENSOR_DEAD.png
             Reply  Thu Jun 17 16:37:23 2021, Anchal, Paco, Update, SUS, c1susaux computer rebooted 
                Reply  Tue Jun 22 11:56:16 2021, Anchal, Paco, Update, SUS, ADC/Slow channels issues CDS_FE_Status.png
                   Reply  Tue Jun 22 16:52:28 2021, Paco, Update, SUS, ADC/Slow channels issues shake_and_damp.png
                      Reply  Wed Jun 23 09:05:02 2021, Anchal, Update, SUS, MC lock acquired back again 
                         Reply  Thu Jun 24 16:40:37 2021, Koji, Update, SUS, MC lock acquired back again P_20210624_163641_1.jpg
Message ID: 16218     Entry time: Tue Jun 22 11:56:16 2021     In reply to: 16210     Reply to this: 16219
Author: Anchal, Paco 
Type: Update 
Category: SUS 
Subject: ADC/Slow channels issues 

We checked back in time to see how the BS and PRM OSEM slow channels are zero. It was clear that they became zero when we worked on this issue on June 17th, Thursday. So we simply went back and power cycled the c1susaux acromag chassis. After that, we had to log in to c1susaux computer and run

sudo /sbin/ifdown eth1
sudo /sbin/ifup eth1

This restarted the ethernet port acromag chassis is connected to. This solved this issue and we were able to see all the slow channels in BS and PRM.

But then, we noticed that the OPLEV of ITMX is unable to read the position of the beam on the QPD at all. No light was reaching the QPD. We went in, opened the ITMX table cover and confirmed that the return OPLEV beam is way off and is not even hitting one of the steering mirrors that brings it to the QPD. We switched off the OPLEV contribution to the damping.

We did burt restore to 16th June morning using
burtwb -f /opt/rtcds/caltech/c1/burt/autoburt/snapshots/2021/Jun/16/06:19/c1susaux.snap -l /tmp/controls_1210622_095432_0.write.log -o /tmp/controls_1210622_095432_0.nowrite.snap -v

This did not solve the issue.

Then we noticed that the OSEM signals from ITMX were saturated in opposite directions for Left and Right OSEMs. The Left OSEM fast channels are saturated to 1.918 um for UL and 1.399 um for LL, while both right OSEM channels are bottomed to 0 um. On the other hand, the acromag slow PD monitors are showing 0 on the right channels but 1097 cts on UL PDMon and 802 cts in LL PD Mon. We actually went in and checked the DC voltages from the PD input monitor LEMO ports on the ITMX dewhitening board D000210-A1 and measured non-zero voltages across all the channels. Following is a summary:

ITMX OSEM readouts
  C1-SUS-ITMX_XXSEN_OUT
(Fast ADC Channels) (um)
C1-SUS-ITMX_xxPDMon
(Slow Acromag Monitors) (cts)
Multimeter measurements at input to Dewhitening Boards
(V)
UL 1.918 1097 0.901
LL 1.399 802 0.998
UR 0 0 0.856
LR 0 0 0.792
SD 0.035 20 0.883

We even took out the 4-pin LEMO outputs from the dewhitening boards that go to the anti-aliasing chassis and checked the voltages. They are same as the input voltages as expected. So the dewhitening board is doing its job fine and the OSEMs are doing their jobs fine.

It is weird that both the ADC and the acromags are reading these values wrong. We believe this is causing a big yaw offset in the ITMX control signal causing the ITMX to turn enough make OPLEV go out of range. We checked the CDS FE status (attachment 1). Other than c1rfm showing a yellow bar (bit 2 = GE FANUC RFM card 0) in RT Net Status, nothing else seems wrong in c1sus computer. c1sus FE model is running fine. c1x02 (the lower level model) does show a red bar in TIM which suggests some timing issue. This is present in c1x04 too.


Bottomline:

Currently, the ITMX coil outputs are disabled as we can't trust the OSEM channels. We're investigating more why any of this is happening. Any input is welcome.

 

 

 

Attachment 1: CDS_FE_Status.png  30 kB  Uploaded Tue Jun 22 13:37:37 2021  | Hide | Hide all
CDS_FE_Status.png
ELOG V3.1.3-