40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 302 of 341  Not logged in ELOG logo
ID Date Author Type Category Subjectdown
  16769   Mon Apr 11 11:00:30 2022 JancarloUpdateVACC1VAC Reboot and Nitrogen tanks

[Paco, JC, Ian, Jordan, Chub]

Checking in the morning, I walked over to the Nitrogen tanks to check the levels. Noticed one tank was empty, so I swapped it out. Chub came over to check the levels and to take note of how many tanks were left available for usage (None). Chub continued to put in a work order for a set of full Nitrogen tanks. We should be set on Nitrogen until Thursday this week (4/14/22).

As for C1VAC, this morning, Paco and I attempted to open the PSL shutter, but the interlock system was tripped so we didn't get any light into the IFO. We traced the issue down to C1VAC being unresponsive. We discussed this may have interlocked as a result of the Nitrogen tanks running out, but we do not believe this was the issue since we would have recieved an email. We tried troubleshooting as much as possible avoiding a reboot, but were unable to solve the issue. In response, we ran the idea of a reboot across Jordan and Ian, where everyone was in agreement, and fixed the system. Restarting c1vac seems to have closed V4, but this didn't cause any issues with the current state of the vacuum system.

After opening the PSL shutter again, we see the laser down the IFO, so we resume alignment work

  17149   Wed Sep 21 10:10:22 2022 YehonathanUpdateCDSC1SU2 crashed

{Yehonathan, Anchal}

Came this morning to find that all the BHD optics watchdogs tripped. Watchdogs were restored but the all ADCs were stuck.

We realized that the C1SU2 model crashed.

We run /scripts/cds/restartAllModels.sh to reboot all machines. All the machines rebooted and burt restored to yesterday around 5 PM.

The optics seem to have returned to good alignment and are damped.

  3921   Mon Nov 15 14:36:37 2010 KojiUpdatePSLC1PSL rebooted?

Has C1PSL rebooted? Has burtrestore been forgotten? Even without elog?

We found some settings are wrong and the PMC has pretty low gain.

  3922   Mon Nov 15 14:42:01 2010 AidanUpdatePSLC1PSL rebooted?

Yeah. Joe and I rebooted c1psl a couple of times this morning. I didn't realize the burtrestore wasn't automatic.

 

Quote:

Has C1PSL rebooted? Has burtrestore been forgotten? Even without elog?

We found some settings are wrong and the PMC has pretty low gain.

 

  15256   Thu Mar 5 19:45:23 2020 JonSummaryPSLC1PSL in-situ test results

We've completed almost all of the in-situ testing of the c1psl channels. During this process, we identified several channels which needed to be rewired to different Acromags (BIO sinking v. sourcing). We also elected to change the connector type of a few channels for practical advantages. Those modifications and other issues found during testing are detailed below. Also attached are the updated channel assignments, with a column indicating the in-situ testing status of each channel.

Post-installation modifications

  • All four channels connected to the sourcing BIO module were found to in fact require sinking I/O. They were reassigned to sinking BIO modules. Affected channels:
    • C1:PSL-FSS_FASTSWEEP
    • C1:PSL-FSS_SW1
    • C1:PSL-FSS_SW2
    • C1:PSL-PSL_Shutter
  • Added a new AI channel:
    • C1:PSL-FSS_MIXERM
  • Removed an unneccessary AI channel:
    • C1:PSL-FSS_LODET
  • Moved two AI channels from BNC connectors to a new Dsub connector (labelled DB25M-2 in the spreadsheet).
    • C1:PSL-FSS_RCTEMP
    • C1:PSL-FSS_RMTEMP_VOLTS

Issues identified during testing

  • Digital calibration. The following channels work, but we need to verify their EPICS calibration parameters (EGUF/EGUL):
    • C1:PSL-FSS_FASTGAIN
    • C1:PSL-FSS_FAST
    • C1:PSL-PMC_RFADJ
    • C1:PSL-PMC_MODET
  • IMC servo board. The Acromag channels themselves were found to work, but the linearity of the mbbo gain stages are in question (i.e., a potential problem with the board). GV is currently testing the servo board.
  • PSL QPD board apears to be dead. We connected a scope directly to the test points on the board and measured a high level of noise and no signal (for all four of the QPD channels). I understand this QPD has not been used in some time, so it may not have been noticed before.
  • WFS DC channels are saturating when the IMC is unlocked. The acceptance range of the Acromag ADC is only +/-10 V, but we measured sensor voltages as high as ~14 V. It appears that the old ADCs were somehow accepting a range of 0 to +20 V instead of -10 to +10 V. However, the Acromags do not support the input range 0-20 V. Since SNR is not critical for these channels (they're used only for initial alignment), I propose we simply install a voltage divider inside the chassis, just before the Acromag, for each of these signals.
Attachment 1: c1psl_feedthrough_wiring_-_By_Connector_(3).pdf
c1psl_feedthrough_wiring_-_By_Connector_(3).pdf c1psl_feedthrough_wiring_-_By_Connector_(3).pdf c1psl_feedthrough_wiring_-_By_Connector_(3).pdf c1psl_feedthrough_wiring_-_By_Connector_(3).pdf
  12924   Mon Apr 3 17:09:47 2017 gautamUpdateCDSC1PSL burt-restored

When I came in this morning, Steve had re-locked the PMC and IMC - but I could see a ~1Hz intensity fluctuation on the PMC REFL video monitor. I unlocked the PMC and tried to re-lock it, but couldn't using the usual prescription of turning the servo gain down and moving the DC bias slider around. I checked the status of the slow machines - all were responding to pings and could be telnet'ed into, so that didn't seem to be the problem. In the past, this sort of behaviour was characteristic of the infamous "sticky slider" problem - so I simply burt-restored c1psl using a snapshot from 29 March, after which I could easily re-lock the PMC. The transmitted light level looked normal on the scope on the PSL table, and the PMC REFL video monitor also look normal now.

  15251   Wed Mar 4 20:42:53 2020 gautamUpdateElectronicsC1PSL acromag crate is sitting on the floor

Jon is going to write up the details of todays adventures. But the C1PSL Acromag chassis is sitting on the floor between the IMC beamtube and the 1X1 electronics rack, and is very much a trip hazard. Be careful if youre in that area.

  15254   Thu Mar 5 11:27:48 2020 gautamUpdateElectronicsC1PSL acromag crate is no longer sitting on the floor

[jordan, gautam]

The C1PSL crate has now been installed in a more permanent way in the rack.

  • Top and bottom covers were re-attached after work yesterday.
  • +/- 24 V DC and +15 V DC power connectors were screwed on for better robustness (I had removed the fuse for the -24V supply as part of debugging yesterday, this was reconnected).
  • PSL diagnostics DB 25 cable was re-run appropriately over the cable tray and connected to the unit.
  • The chassis sits on some rails - these rails are mounted to the rack using rack nuts but that means the ears on the Acromag chassis no longer line up with any rack nut slots, and so the chassis is not bolted on to the rack.
  • We also took this opportunity to remove the c1iool0 VME chassis from 1X2 - given that the DAC and BIO cards of c1ioo (rtcds system) are unused, I felt comfortable disconnecting them and that made the removal relatively easy. The CDS overview MEDM screen reports no errors after this work.

After this work, I disabled logging and restarted the modbus service (and copied the current version of the systemd service file to the target directory for backup). The PMC and IMC lock alright. The system is now ready to be tested in-situ. I will separately continue my IMC Servo board tests in the evening.

One thought about how to protect against this kind of silent failure - how about we always run the modbus service with logging enabled, and then send out a warning email and stop the service if the logfile size suddenly blows up (which is characteristic of when the communications process dies)? This should be done in addition to the ping-ing of the individual IPs.

Regarding the burt-restore step that the systemd service runs after starting up the IOC - this is not even that useful, at least in the way it is currently setup (restore the "latest" burt snapshot file). If the maintenance takes >1hour as it often does, the "latest" snapshot for the system under maintenance is just garbage. So either the burt-restore should be for a "known good time" (dangerous because this will require frequent updates of the systemd service every time we find a new safe state) or we should just do it manually (my preference). Then there is no need to install custom packages on the server machine. Anyway, for now, I have not commented this step out.

Jordan is going to take pictures of all the electronics racks and update the relevant wiki pages.

Quote:

Jon is going to write up the details of todays adventures. But the C1PSL Acromag chassis is sitting on the floor between the IMC beamtube and the 1X1 electronics rack, and is very much a trip hazard. Be careful if youre in that area.

  6068   Mon Dec 5 02:55:30 2011 DenUpdateAdaptive FilteringC1OAF

I've added filter banks for correction path in the C1OAF model to use AA filters.  I compiled and installed the new version. I runs but does not sync. Probably, I've made a mistake in the some names of epics channels. Leave it for now, figure out tomorrow. If someone needs an old version, it is in the /opt/rtcds/caltech/c1/userapps/trunk/isc/c1/models/c1oaf_BACKUP20111204.mdl file. Corresponding medm screen is in the /opt/rtcds/caltech/c1/userapps/trunk/isc/common/medm/OAF_OVERVIEW.adl file.

  6070   Mon Dec 5 10:13:13 2011 JenneUpdateAdaptive FilteringC1OAF

Quote:

I've added filter banks for correction path in the C1OAF model to use AA filters.  I compiled and installed the new version. I runs but does not sync. Probably, I've made a mistake in the some names of epics channels. Leave it for now, figure out tomorrow. If someone needs an old version, it is in the /opt/rtcds/caltech/c1/userapps/trunk/isc/c1/models/c1oaf_BACKUP20111204.mdl file. Corresponding medm screen is in the /opt/rtcds/caltech/c1/userapps/trunk/isc/common/medm/OAF_OVERVIEW.adl file.

 The general rule in the 40m is that if it's not an 'emergency', i.e. something is wrong with the computers and preventing the main locking work, no model recompiling-type activities at nighttime. 

Also, if you do things and recompile, you need to do an svn check-in.  That's where backups are kept.  We don't want to clutter folders with backups anymore.

  6093   Fri Dec 9 13:28:09 2011 DenUpdateAdaptive FilteringC1OAF

I tried to figure out why red NO SYNC label became present in the C1OAF_GDS_TP screen after I added AA filters to the C1OAF model.

C1OAF model contains 8 libraries C1OAF_ADAPT for 8 DOF. I changed C1OAF_ADAPT library to C1OAF_ADAPT_AA library where I added 28 AA filters for 28 witness channels. It turns out that if I use this library for all 8 DOF then I see NO SYNC label, if only for one DOF (MCL) then I see green IOP label. This means that using AA filters for each DOF too much channels of filters are created for online system to operate. I think there is some number inside the code that one can not exceed. Analyzing compilation output after "make c1oaf" I figured out that without using AA filters we have 632 filters and using AA we have 856 filters.

For now I'll use AA filters for MCL only.

  6094   Fri Dec 9 14:33:16 2011 Alex IvanovUpdateAdaptive FilteringC1OAF

Quote:

I tried to figure out why red NO SYNC label became present in the C1OAF_GDS_TP screen after I added AA filters to the C1OAF model.

C1OAF model contains 8 libraries C1OAF_ADAPT for 8 DOF. I changed C1OAF_ADAPT library to C1OAF_ADAPT_AA library where I added 28 AA filters for 28 witness channels. It turns out that if I use this library for all 8 DOF then I see NO SYNC label, if only for one DOF (MCL) then I see green IOP label. This means that using AA filters for each DOF too much channels of filters are created for online system to operate. I think there is some number inside the code that one can not exceed. Analyzing compilation output after "make c1oaf" I figured out that without using AA filters we have 632 filters and using AA we have 856 filters.

For now I'll use AA filters for MCL only.

 I have a feeling we are not fitting into pre-allocated memory space in the shared memory between the front-end process and the epics process. Filter module data is overwriting some other data and that's why we are not getting a sync light. I suggest we upgrade to 2.4 code first and then we will figure out a way to expand memory areas to fit 856 filters.

  6100   Fri Dec 9 17:53:31 2011 DenUpdateAdaptive FilteringC1OAF

[Jenne, Den]

AA filters for witness channels are added to the oaf model. It is working now and the number of memory used is not critical.  NO SYNC is not present any more.

  7225   Sat Aug 18 17:09:01 2012 SashaUpdateSimulationsC1LSP MEDM Screens Added

C1LSP has been added to the site map. I'll work on filling in the structure some more today and tomorrow (as well as putting up PDH and REFL/AS MEDM screens).

NOTE: Does anyone know how to access channels (or if they're even there) for straight Simulink inputs and outputs (i.e. I have some sort of input, do something to it in the simulink model, then get some output)? I've been trying to add ADC MEDM screens to c1lsp, but channels along the lines of C1LSP-ADC0_0_Analog_Input or C1LSP-ADC0_A0 don't seem to exist.

  7227   Sat Aug 18 19:40:47 2012 SashaUpdateSimulationsC1LSP MEDM Screens Added

Quote:

C1LSP has been added to the site map. I'll work on filling in the structure some more today and tomorrow (as well as putting up PDH and REFL/AS MEDM screens).

NOTE: Does anyone know how to access channels (or if they're even there) for straight Simulink inputs and outputs (i.e. I have some sort of input, do something to it in the simulink model, then get some output)? I've been trying to add ADC MEDM screens to c1lsp, but channels along the lines of C1LSP-ADC0_0_Analog_Input or C1LSP-ADC0_A0 don't seem to exist.

 NVM. Figured out that I can just look in dataviewer for the channels. It looks like there aren't any channels for ADC0...I'll try reinstalling the model and restarting the framebuilder.

  14187   Tue Aug 28 18:39:41 2018 JonUpdateCDSC1LSC, C1AUX reboots

I found c1lsc unresponsive again today. Following the procedure in elog #13935, I ran the rebootC1LSC.sh script to perform a soft reboot of c1lsc and restart the epics processes on c1lsc, c1sus, and c1ioo. It worked. I also manually restarted one unresponsive slow machine, c1aux.

After the restarts, the CDS overview page shows the first three models on c1lsc are online (image attached). The above elog references c1oaf having to be restarted manually, so I attempted to do that. I connect via ssh to c1lsc and ran the script startc1oaf. This failed as well, however.

In this state I was able to lock the MICH configuration, which is sufficient for my purposes for now, but I was not able to lock either of the arm cavities. Are some of the still-dead models necessary to lock in resonant configurations?

Attachment 1: CDS_FE_STATUS.png
CDS_FE_STATUS.png
  7279   Sun Aug 26 21:47:50 2012 KojiUpdateCDSC1LSC ooze

I came in to the lab in the evening and found c1lsc had "red" for FB connection.
I restarted c1lsc models and it kept hung the machine everytime.

I decided to kill all of the model during the startup sequence right after the reboot.
Then run only c1x04 and c1lsc. It seems that c1oaf was the cause, but it wasn't clear.

  1028   Mon Oct 6 12:45:41 2008 AlbertoDAQLSCC1LSC in coma
Alberto, Joe,

The C1LSC medm screen is frozen and the C1LSC computer is down. We tried to reboot and to restart it first turning off the power and then just rebooting remotely. None of them worked. We check whether any of the cable was unplugged but they were ok. Also all the led turned on to green after rebooting.
Trying to reboot we get the following error message: init_module: device or resource busy.

We called Alex who first suggested to check all the connection and then to swap the timing cable between two Pentek boards but the computer was still down.
It is possible that the board is dead. Alex and Rolf are going to look into this problem and for any spare board.

by now we can't lock any DOF of the IFO.
  1029   Mon Oct 6 16:41:33 2008 AlbertoDAQLSCC1LSC in coma

Quote:
Alberto, Joe,

The C1LSC medm screen is frozen and the C1LSC computer is down. We tried to reboot and to restart it first turning off the power and then just rebooting remotely. None of them worked. We check whether any of the cable was unplugged but they were ok. Also all the led turned on to green after rebooting.
Trying to reboot we get the following error message: init_module: device or resource busy.

We called Alex who first suggested to check all the connection and then to swap the timing cable between two Pentek boards but the computer was still down.
It is possible that the board is dead. Alex and Rolf are going to look into this problem and for any spare board.

by now we can't lock any DOF of the IFO.


Alex, Rob, Alberto,

Alex replaced the Pentek board used by C1LSC with a spare one that they had at the Wilson house. That fixed the DMA failure but since the board had a channel broken, one of the channels (POY) was still not available.

Looking at the wiring diagram of the ASC crate, we found that one of the Pentek boards in it was actually not used by anything and thus available to replace the bad one in LSC. We switched the two boards so that now the one that Alex installed is mounted in the ASC crate and it is connected to the cable labeled 1x2-ASC 6.

C1LSC is up again and all the channels in the C1LSC screen, including POY, now seem to be working properly.
  5579   Fri Sep 30 02:56:56 2011 kiwamuUpdateCDSC1IOO.ini and C1LSC.ini files reverted

[Suresh/Kiwamu]

We found that the C1LSC.ini and C1IOO.ini file had been refreshed and there were a few recorded channels in the files.

So we manually recovered C1LSC.ini file using the daqconfig GUI screen.

For the C1IOO.ini file we simply replaced it by the archived one which had been saved in 22nd of September.

Then we restarted daqd.

  5327   Tue Aug 30 17:31:55 2011 SureshUpdateIOOC1IOO model reverted and fb restarted

I reverted the C1IOO model to the last working version and restarted the fb at this time..Tue Aug 30 17:28:38 PDT 2011

  5732   Tue Oct 25 01:14:15 2011 SureshUpdateIOOC1IOO model modified to include new WFS lockin structure

A while back we faced the problem that when we use several lockins to excite the MC degrees of freedom, their relative phase was not known.  The solution suggested was to use one oscillator and several demodulators.

I have now modified the C1IOO.mdl so that this can be implemented.  Previously we were using the MC_ASS lockins for WFS work.  I have now separated the WFS and MC_ASS structures. 

Other jobs to be done in this context are:

1) The medm screens associated with WFS lockins need to be updated with new channel names.

2) The scripts associated with both MC_ASS decentering and WFS ouput matrix determination have to be updated with the new channel names.

3) I also deleted all medm screens in the $screens$/c1ioo/ directory after copying them to $screens$/c1ioo/bak/.  After installing the new c1ioo model $screens$/c1ioo directory now contains just the automatically created screens.  All other user made screens should go into $screens$/c1ioo/master/ directory

This is a pic of the new c1ioo model:

 

 c1ioo20111025.png

 

  5735   Tue Oct 25 16:24:58 2011 SureshUpdateIOOC1IOO model modified to include new WFS lockin structure

I forgot to mention another change I made to the C1IOO model.

The location of the WFS global switch and the WFS_GAIN have been shifted. The switch now cuts off signals just before the WFS servo filters.

I have also added some test points just before the switch and the so that we can monitor the WFS error signals which would be unaffected even if the WFS_GAIN is changed..

 

 

Quote:

A while back we faced the problem that when we use several lockins to excite the MC degrees of freedom, their relative phase was not known.  The solution suggested was to use one oscillator and several demodulators.

I have now modified the C1IOO.mdl so that this can be implemented.  Previously we were using the MC_ASS lockins for WFS work.  I have not separated the WFS and MC_ASS structures. 

Other jobs to be done in this context are:

1) The medm screens associated with WFS lockins need to be updated with new channel names.

2) The scripts associated with both MC_ASS decentering and WFS ouput matrix determination have to be updated with the new channel names.

3) I also deleted all mdem screens in the $screens$/c1ioo/ directory after copying them to $screens$/c1ioo/bak/.  After installing the new c1ioo model $screens$/c1ioo model now contains just the automatically created screens.  All other user made screens should to into $screens$/c1ioo/master/ directory

This is a pic of the new c1ioo model:

 

 c1ioo20111025.png

 

 

  5737   Tue Oct 25 18:50:22 2011 SureshUpdateIOOC1IOO model modified to include new WFS lockin structure

Some small fixes to the c1ioo model.

1) I edited the WFS lockin modules to make use of new library part called demod.

2) c1ioo model has been compiled and restarted.

3) fb was restarted at Tue Oct 25 18:43:55 PDT 2011

 

Quote:

I forgot to mention another change I made to the C1IOO model.

The location of the WFS global switch has been shifted. It now cuts off signals just before the WFS servo filters.

I have also added some test points just before the switch so that we can monitor the WFS sensor signals even if the switch is off.

 

 

Quote:

A while back we faced the problem that when we use several lockins to excite the MC degrees of freedom, their relative phase was not known.  The solution suggested was to use one oscillator and several demodulators.

I have now modified the C1IOO.mdl so that this can be implemented.  Previously we were using the MC_ASS lockins for WFS work.  I have not separated the WFS and MC_ASS structures. 

Other jobs to be done in this context are:

1) The medm screens associated with WFS lockins need to be updated with new channel names.

2) The scripts associated with both MC_ASS decentering and WFS ouput matrix determination have to be updated with the new channel names.

3) I also deleted all mdem screens in the $screens$/c1ioo/ directory after copying them to $screens$/c1ioo/bak/.  After installing the new c1ioo model $screens$/c1ioo model now contains just the automatically created screens.  All other user made screens should to into $screens$/c1ioo/master/ directory

This is a pic of the new c1ioo model:

 

 c1ioo20111025.png

 

 

 

  5880   Sat Nov 12 02:27:00 2011 SureshUpdateComputersC1IOO front end suddenly froze. Was restarted remotely

[Koji Suresh]

No one was messing with the c1ioo or any other machine.   The medm screens for WFS and MC alignment froze while I was working on Rossa.

There were number of red lights pertaining to c1ioo machine on the CDS_FR_STATUS screen.  So we logged into c1ioo   from Rossa and restarted it with 'sudo shutdown -r now'.  It came back up but the C1IOO_MC_TRANS_SUM, P and Y signals were not available on the C1IOO LOCKMC screen.

I saw several messages similar to the one here


Sat Nov 12 02:09:14 PST 2011

  medmCAExceptionHandlerCb: Channel Access Exception:
  Channel Name: Unavailable
  Native Type: Unavailable
  Native Count: 0
  Access: Unavailable
  IOC: Unavailable
  Message: Virtual circuit disconnect
  Context: c1ioo.martian:43553
  Requested Type: TYPENOTCONN
  Requested Count: 0
  Source File: ../cac.cpp
  Line number: 1126

 

The MC autolocker script wasnt running.  The heartbeat bit was not blinking on the MC_LOCKMC screen.  So we manually restarted the script.  Hopefully it will return to normal operation.

I restarted the fb at Sat Nov 12 02:12:19 PST 2011  in an attempt to see this resolves the problem.

It didnt.

 

  5881   Sat Nov 12 02:44:18 2011 SureshUpdateComputersC1IOO front end suddenly froze. Was restarted remotely

Quote:

[Koji Suresh]

No one was messing with the c1ioo or any other machine.   The medm screens for WFS and MC alignment froze while I was working on Rossa.

There were number of red lights pertaining to c1ioo machine on the CDS_FR_STATUS screen.  So we logged into c1ioo   from Rossa and restarted it with 'sudo shutdown -r now'.  It came back up but the C1IOO_MC_TRANS_SUM, P and Y signals were not available on the C1IOO LOCKMC screen.

I saw several messages similar to the one here


Sat Nov 12 02:09:14 PST 2011

  medmCAExceptionHandlerCb: Channel Access Exception:
  Channel Name: Unavailable
  Native Type: Unavailable
  Native Count: 0
  Access: Unavailable
  IOC: Unavailable
  Message: Virtual circuit disconnect
  Context: c1ioo.martian:43553
  Requested Type: TYPENOTCONN
  Requested Count: 0
  Source File: ../cac.cpp
  Line number: 1126

 

The MC autolocker script wasnt running.  The heartbeat bit was not blinking on the MC_LOCKMC screen.  So we manually restarted the script.  Hopefully it will return to normal operation.

I restarted the fb at Sat Nov 12 02:12:19 PST 2011  in an attempt to see this resolves the problem.

It didnt.

 

 The problem was resolved after I burtrestored (c1mcs c1ioo and c1rfm) epics snapshots.

 

  5687   Tue Oct 18 20:50:19 2011 SureshUpdateIOOC1IOO and WFS associated screens

In keeping with the current protocol,  I have started to move all the user-built medm screens associated with C1IOO into the $screens$/c1ioo/master/ directory. 

I then edited the menu button in the sitemap.adl to point to the screens in the ..c1ioo/master/ directory.  All the screens in $screens$/c1ioo/ directory have been backed up into bak/.  I plan to edit the c1ioo model soon and at that time I will delete all the screens in the $screens$/c1ioo directory and let only the automatically regenerated screens  stay there.   If there are broken links to user-built screens associated with c1ioo, please copy the relevant screen to the master/ directory and edit the path in the menus.

 

  11641   Thu Sep 24 17:06:14 2015 ericqUpdateCalibration-RepairC1CAL Lockins

Just a quick note for now: I've repopulated C1CAL with a limited set of lockin oscillators/demodulators, informed by the aLIGO common LSC model. Screens are updated too. 

Rather than trying to do the whole magnitude phase decompostion, it just does the demodulation of the RFPD signals online; everything beyond that is up to the user to do offline. 

Briefly testing with PRMI, it seems to work as expected. There is some beating evident from the fact that the MICH and PRCL oscillation frequencies are only 2Hz apart; the demod low pass is currently at an arbitrary 1Hz, so it doesn't filter the beat much. 

Screens, models, etc. all svn'd.

  15285   Thu Mar 26 22:31:34 2020 YehonathanUpdateCDSC1AUXEY wiring + channel list

I have made a wiring + channel list that need to be included in the new C!AUXEY Acromag.

It was mostly copied from C1AUXEX

I ignored the IPANG channels since it is going to be removed from the table.

  15292   Thu Apr 2 16:31:33 2020 JonUpdateCDSC1AUXEY wiring + channel list
Quote:

I have made a wiring + channel list that need to be included in the new C!AUXEY Acromag.

I used Yehonathan's wiring assignments to lay the rest of groundwork for the final slow controls machine upgrade, c1auxey. Actions completed:

  • Created an internal wiring diagram for assembling the Acromag chassis (log in with LIGO.ORG credentials to view/edit)
  • Created a new target directory on the network drive:
/cvs/cds/caltech/target/c1auxey1

The "1" will be dropped after the new system is permanently installed.

  • Populated the target directory with files:
    • modbusIOC.service - wraps the EPICS IOC as a systemd service
    • ETMYaux.env - defines the EPICS environment variables
    • ETMYaux.cmd - command file to set up the EPICS IOC
    • ETMYaux.sh - enables DAC outputs to the suspension (executed lastly)
  • Created the EPICS channel databases:
    • ETMYaux.db - migration of the existing database
    • c1auxey_state.db - contains logic for loopback monitoring of the IOC "alive" state (visible from Sitemap > CDS > Slow Controls Status)

Hardware-wise, this system will require:

  • 2 Acromag XT-1221 units (ADC)
  • 1 Acromag XT-1541 unit (DAC)
  • 1 Acromag XT-1111 unit (sinking BIO)

I know that we do have these quantities left on hand. The next steps are to set up the Supermicro host and begin assembling the Acromag chassis. Both of these activities require an in-person presence, so I think this is as far as we can advance this project for now.

  15293   Thu Apr 2 22:19:18 2020 KojiUpdateCDSC1AUXEY wiring + channel list

We want to migrate the end shutter controls from c1aux to the end acromags. Could you include them to the list if not yet?

This will let us remove c1aux from the rack, I believe.

 

  15294   Fri Apr 3 12:09:53 2020 JonUpdateCDSC1AUXEY wiring + channel list
Quote:

We want to migrate the end shutter controls from c1aux to the end acromags. Could you include them to the list if not yet?

This will let us remove c1aux from the rack, I believe.

Yehonathan's list does include C1:AUX-GREEN_Y_Shutter and I copied its definition from /cvs/cds/caltech/target/c1aux/ShutterInterlock.db into the new ETMYaux.db file.

I noticed ShutterInterlock.db still contains about a dozen channels. Some of them appear to be ghosts (like the C1:AUX-PSL_Shutter[...] set, which has since become C1:PSL-PSL_Shutter[...] hosted on c1psl) but others like C1:AUX-GREEN_X_Shutter appear to still be in active use.

  5518   Thu Sep 22 13:56:56 2011 kiwamuUpdateASCC1ASS : status update

The output matrix in the C1ASS servo were coarsely readjusted and the servos seemed working.

However it is difficult to say the servo is very good or so-so,

because the ETMY suspension moves a lot and hence the cavity eigen axis moves a lot too.

 


(to do)

 + optimization of the ETMY oplevs and OSEM damping.
 + evaluation of the performance of the C1ASS with a good damping.

(Background)

 Since we have installed the new mid-HV amplifier for the PZT1 mirror (#5450) it changed the response of the PZT1 (gains from EPICS to the actual angles).
Therefore the C1ASS output matrix needs be adapted to the new PZT1 response.
 
(What I have done)
  What I was measuring was a coupling from each PZT mirror to both beam angle and beam position by looking at the output from the LOCKINs.
So this measurement eventually gives us a nicely diagonalized output matrix by inverting the coupling.
However the measurement turned out to be difficult because the ETMY moved too much.
In fact the cavity eigen axis also moves and the fluctuation was larger than the intentionally introduced beam angle/translation offsets, which are for the coupling measurement.
 
 Instead of measuring the couplings, I put some numbers into the matrix based on a guess.
Since the PZT1 HV amp became weaker than that of PZT2, the elements in the output matrix should be amplified by some number.
Right now the PZT1 amp can drive the mirror in a range of -5 -30 V with EPICS range of +/-10 counts, and for PZT2 it is about 0 -150V with EPICS range of +/-5 counts.
So the difference of the responses in unit of V/counts is about 8.5.
The PZT1 elements in the matrix were multiplied by this number and I became able to close the servos.

Quote from #5455

  + Modification of C1ASS (Kiwamu)

  5543   Mon Sep 26 12:41:27 2011 kiwamuUpdateASCC1ASS : status update

Quote from #5518
(to do)
 + optimization of the ETMY oplevs and OSEM damping.
 + evaluation of the performance of the C1ASS with a good damping.

The servo for aligning the Y arm is working fine with the coarse gain coefficients.

However then I found the ASS_Xarm servo was not healthy.

So the next step is to refine the X arm servo in C1ASS.

 

(some notes)

  + With the ETMY oplev the Y arm became quieter after we recovered the oplev whitening filter (#5523)

  + The Y arm alignment scripts can be run from the usual C1IFO_CONFIGURE screen.

   It will servo the spot positions on ITMY and ETMY, and align the input beam pointing. It brings the Y arm power to about 1.

 + The X arm servo is doing something funny. It doesn't bring the arm power up to 1.

   I thought the X arm didn't need any modifications because the X arm servo doesn't include PZT1 and PZT2.

   So it maybe a simple bug (for example, some switches are disable and so on)

 

  5578   Fri Sep 30 01:18:39 2011 kiwamuUpdateASCC1ASS : status update

Now the C1ASS servos are working fine.

However at the end of the scripts sometimes it changes the DC force (e.g. C1:SUS-ITMX_PIT_COMM and so on) by a wrong amount.

So for this bug, it misaligns the suspensions a lot. I will take a look at the script tomorrow.

Quote from #5543

However then I found the ASS_Xarm servo was not healthy.

  8028   Thu Feb 7 19:25:22 2013 yutaUpdateCDSC1ALS filters reloaded

Filters for C1ALS were all gone. So, I copied /opt/rtcds/caltech/c1/chans/C1GCV.txt and renamed it as C1ALS.txt.

I also fixed links in the medm screens; C1ALS.adl and C1ALS_COMPACT.adl.
I'm not sure what happened to C1SC{X,Y} screens.

Quote:

I decided to rename the c1gcv model to be c1als.  This is in an ongoing effort to rename all the ALS stuff as ALS, and get rid of the various GC{V,X,Y} named stuff.

(...snip...)

The above has been done.  Still todo:

  • FIX SCRIPTS!  There are almost certainly scripts that point to GC{V,X,Y} channels.  Those will have to be fixed as we come across them.
  • Fix the c1sc{x,y}/master/C1SC{X,Y}_GC{X,Y}_SLOW.adl screens.  I need to figure out a more consistent place for those screens.
  • Fix the C1ALS_COMPACT screen
  • ???

 

 

  11699   Mon Oct 19 16:16:01 2015 ericqUpdateCDSC1ALS crashes

Gautam was working on his digital frequency counter stuff when the c1als model crashed. I had trouble bringing it back until I realized that, for reasons unknown to me, the safe.snap file that the model looks for at boot had been deleted. (This file lives in /target/c1als/c1alsepics/burt). I copied over this morning's version from Chiara's local backup. 

At the sites, these files are under version control in the userapps svn repository, presumably symlinked into the target directory. We should definitely do something along these lines. 

  5666   Fri Oct 14 16:20:11 2011 ZachUpdateSUSC1:SUS-ETMX_SPDMon fixed

I offered to help Kiwamu with some of the 40m work. The first task was to figure out why the ETMX side OSEM monitor was so low, since we know that the depth is about right. It was showing ~0.13 V to the others' ~0.7 V.

TL,WR: There was a wire disconnected from the breakout panel on the side of the rack

I started by pulling the board out and checking to make sure that it was working properly. I injected a sine wave to the SIDE IN and found that it showed up in the signal coming out of the back (into the crate) just fine (see below). One strange thing I noticed while testing the board is that both inputs for each used channel of the MAX333 switches on the board are shorted to their respective outputs. That is, the switches seem to be open to BOTH 0 and 1 logic states. This seems counterintuitive, but perhaps there's something about how these work that I don't know.

board_works.png

 

Then I went about tracing the signal from the back of the crate to the breakout panel on the side of the rack. I opened it up, verified that the ribbon cable was transmitting correctly, and as I went to plug it back in I noticed that one of the wires---the correct one---had come completely undone.

rut_roh_raggie.png

The screw clamp appeared to be a bit finicky, as I had to loosen and tighten it a few times before it finally seemed to grab hold of the wire. It probably just wasn't tight in the first place and the wire was pulled out. Anyway, things are working now:

Screen_shot_2011-10-14_at_4.09.45_PM.png

  5667   Fri Oct 14 18:38:41 2011 kiwamuUpdateSUSC1:SUS-ETMX_SPDMon fixed

Quote from #5666

 Anyway, things are working now:

 Good job ! Thank you so much

  735   Thu Jul 24 19:29:26 2008 YoichiConfigurationPSLC1:PSL-STAT_FSS_NOM_C_GAIN is changed from 30 to -0.7
Koji, Yoichi

Since the light power going to the ref. cavity is now significantly increased (see Janne's elog later),
C1:PSL-STAT_FSS_NOM_C_GAIN
is changed from 30 to -0.7.
Otherwise, the MC did not lock.
  950   Tue Sep 16 13:04:22 2008 YoichiConfigurationPEMC1:PSL-FSS_RMTEMP alarm level changed
At the request of Steve, I modified the HIGH value of C1:PSL-FSS_RMTEMP from 21.27 to 23.0.
The HIHI is set to 23.50.
  1292   Wed Feb 11 10:52:22 2009 YoichiConfigurationDAQC1:PEM-OSA_APTEMP and C1:PEM-OSA_SPTEMP disconnected

During the cleanup of the lab. Steve found a box with two BNCs going to the ICS DAQ interface and an unconnected D-SUB on the floor under the AP table.  It seemed like a temperature sensor.

The BNCs were connected to C1:PEM-OSA_APTEMP and C1:PEM-OSA_SPTEMP.

Steve removed the box from the floor. These channels can be now used as spare DAQ channels. I labeled those cables.

  1338   Thu Feb 26 00:36:53 2009 YoichiSummaryComputersC1:LSC-TRX_OUT broken (and fixed later).
Today, Kakeru tried to convert C1:LSC-TRX_OUT and C1:LSC-TRY_OUT to DAQ channels.
He edited C1LSC.ini in the chans/daq directory to add the channel but it did not work.
Then he reverted the file back to the original one.
But after we still could not access these channels from dataviewer nor tds tools.
We restarted daqd and tpman on fb40m, but the problem persisted. Even rebooting the whole fb40m did not help.
After inspecting the log file of daqd, it was clear that tpman was failing to create test points for those channels.
I rebooted c1daqawg and then restarted tpman and daqd on fb40m again.
This time, the problem went away.
  4664   Mon May 9 12:33:40 2011 kiwamuUpdateLSCC1:LSC-TRIG_MTRX : wrong matrix size

I found that C1:LSC-TRIG_MTRX has a wrong matrix size. It needs to be fixed.

It is designed to have a 11x8 matrix in the simlink model file, but it's been compiled as a 6x8 matrix.

I recompiled c1lsc but it didn't fix the issue. Here below is the matrix statement in the C file :c1lsc.c. Indeed it is 6x8 matrix ....

// MuxMatrix:  LSC_TRIG_MTRX
for(ii=0;ii<8;ii++)
{
  lsc_trig_mtrx[ii] =
          pLocalEpics->lsc.LSC_TRIG_MTRX[ii][0] * lsc_imux_trigger[0] +
          pLocalEpics->lsc.LSC_TRIG_MTRX[ii][1] * lsc_imux_trigger[1] +
          pLocalEpics->lsc.LSC_TRIG_MTRX[ii][2] * lsc_imux_trigger[2] +
          pLocalEpics->lsc.LSC_TRIG_MTRX[ii][3] * lsc_imux_trigger[3] +
          pLocalEpics->lsc.LSC_TRIG_MTRX[ii][4] * lsc_imux_trigger[4] +
          pLocalEpics->lsc.LSC_TRIG_MTRX[ii][5] * lsc_imux_trigger[5];
 }

  4665   Mon May 9 13:14:48 2011 josephbUpdateLSCC1:LSC-TRIG_MTRX : wrong matrix size

[Joe, Kiwamu]

There is a feature/bug of the RCG code that you can only have 1 receiving tag for every sending tag.  There were 5 tags which were being received by two tags each, for two different matrices.  Only the first tag was receiving, the second was apparently ignored.

This has been fixed temporarily by putting in direct lines in place of these 5 tags.

Quote:

I found that C1:LSC-TRIG_MTRX has a wrong matrix size. It needs to be fixed.

It is designed to have a 11x8 matrix in the simlink model file, but it's been compiled as a 6x8 matrix.

 

  5901   Tue Nov 15 23:44:44 2011 MirkoUpdateCDSC1:LSC & C1:SUS restarted

Earlier this evening C1:LSC died then I hit the DAQ reload after adding an OAF channel to be recorded. No change to any model. Had to restart C1:SUS too. Reloaded burts from this morning 5am, except for C1:IOO, which I loaded from 16:07.

  5006   Wed Jul 20 20:04:54 2011 JamieUpdateCDSC1:DAQ-FB0_C1XXX_STATUS sometimes unexplainably goes red

I have been noticing this happening occasionally, but I don't understand what is causing:

status-fb-red1.png

The channel in question above is C1:DAQ-FB0_C1SCX_STATUS.  This channel is (I believe) reporting some status of the front end model communication with the frame builder, but I'm not sure exactly what.

Usually this problem goes away when I restart the model or the frame builder, but it didn't work this time.  Tomorrow I will figure out what this channel means, why it's sporadically going red, and how to correct it.

  4961   Tue Jul 12 10:18:05 2011 JamieUpdateCDSC1:DAQ-FB0_C1???_STATUS indicators red, restored after controller restarts

Yesterday I found the C1:DAQ-FB0_C1???_STATUS lights to be red for the SUS, MCS, SCX, and SCY controllers.  I know this has something to do with model communication with the framebuilder, but I unfortunately don't remember exactly what it is.  I decided to try restarting the affected models to see if that cleared up the problem.  It did.  After restarting c1scx, c1scy, c1sus, and c1mcs everything came back up green.

We need some better documentation about what all of these status indicators mean.

  8672   Tue Jun 4 17:28:09 2013 AnnalisaUpdateLSCC1:ALS-TRY_OUT filter and green progress

 [Annalisa, Gautam, Rana]

I made the anti-whitening filter for the C1:ALS-TRY_OUT channel.

zpk [[150],[15],1] Hz

Now we can look at the picks of this signal to align the green into the cavity.

We already had some 00 flash, but a better alignment has to be done.

TO DO:

- put the shutter along the beam path

- check the polarization (we have a new PBS for visible)

 

Attachment 1: green.JPG
green.JPG
Attachment 2: QUAD1_1054358327.mp4
  8674   Tue Jun 4 21:50:23 2013 AnnalisaUpdateLSCC1:ALS-TRY_OUT filter and green progress

 

 [Annalisa, Gautam]

 The green beam alignment has been improved, so we see much more 00 bright flashing. We checked the polarization and the Ygreen shutter is back in place.

A mirror is already in place to steer the rejected beam from the green Faraday into a PD, tomorrow morning we'll put a lens and the PD to take the signal for PDH locking.

 

ELOG V3.1.3-