40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 60 of 328  Not logged in ELOG logo
ID Date Author Type Category Subject
  13493   Thu Dec 28 17:22:02 2017 gautamUpdateGeneralpower outage - CDS recovery
  1. I had to manually reboot c1lsc, c1sus and c1ioo.
  2. I edited the line in /etc/rt.sh (specifically, on FB /diskless/root.jessie/etc/rt.sh) that lists models running on a given frontend, to exclude c1dnn and c1oaf, as these are the models that have been giving us most trouble on startup. After this, I was able to bring back all models on these three machines using rtcds restart --all. The original line in this file has just been commented out, and can be restored whenever we wish to do so.
  3. mx_stream processes are showing failed status on all the frontends. As a result, the daqd processes are still not working. Usual debugging methods didn't work.
  4. Restored all sus dampings.
  5. Slow computers all seem to be responsive, so no action was required there.
  6. Burtrestored c1psl to solve the "sticky slider" problem, relocked PMC. I didn't do anything further on the PSL table w.r.t. the manual beam block Steve has placed there till the vacuum situation returns to normal.

@Steve: I noticed that we are down to our final bottle of N2, not sure if it will last till 2 Jan which is presumably when the next delivery will come in. Since V1 is closed and the PSL beam is blocked, perhaps this doesn't matter.

from Steve: there are spare full N2 bottles at the south end outside and inside. I replaced the N2 on Sunday night. So the system should be Ok as is.

I also hard-rebooted megatron and optimus as these were unresponsive to ping.

*Seems like the mx_stream errors were due to the mx process not being started on FB. I could fix this by running sudo systemctl start mx on FB. After which I ran sudo systemctl restart daqd_*. But the DC errors persist - not sure how to fix this. Elogging suggests that "0x4000" errors are connected to timing problems on FB, but restarting the ntp service on FB (which is the suggested fix in said elogs) didn't fix it. Also unsure if mx process is supposed to automatically start on FB at startup.

Attachment 1: 28.png
28.png
  13492   Tue Dec 26 17:24:24 2017 SteveUpdateGeneralpower outage

There was a power outage.

The IFO pressure is 12.8 mTorr-it and it is not pumped. V1 is still closed. TP1 is not running. The Rga is not powered.

The PSL output shutter is still closed. 2W Innolight turned on and manual beam block placed in its beampath.

3 AC units turned on at room temp 84F

Attachment 1: powerOutage.png
powerOutage.png
  13491   Fri Dec 22 09:40:19 2017 ranaSummaryGeneralDAC noise contribution to squeezing noise budget
  1. Should not count the ITMs. On those we can use big resistors / filters to cut out the noise.
  2. For the initial LIGO, we used 7 K resistors and the mass was 10 kg. But...the output driver went +/- 150 V.

So we had a max F/m = (20 mA * 0.064 N/A)/(10 kg) = 0.0001. For the 40m, to get the same thing, we would need 40x less current (~0.5 mA). At the moment we have (12 V / 400 Ohm) = 30 mA.

We need to get a spectrum and times series of the required coil current for acquiring and holding the DRMI, and also the single arm. Then we can see where to make noise reductions to allow this drastic force reduction. 

Coil Driver Upgrade wiki here.

  13490   Thu Dec 21 19:25:48 2017 KevinSummaryGeneralDAC noise contribution to squeezing noise budget

Gautam and I redid our calculations, and the updated plot of squeezing as a function of DAC current noise per coil is shown in the attachment. The current noise is calculated as the maximum of the filtered DAC noise and the Johnson noise of the series resistor. The total noise is for four optics with four coils each.

The numbers are worse than we quoted before: according to these calculations we can get to 0 dBvac for current noise per coil of about 2.4 pA/rtHz at 100 Hz.

Quote:

Gautam and I looked into the DAC noise contribution to the noise budget for homodyne detection at the 40m. DAC noise is currently the most likely limiting source of technical noise.

Several of us have previously looked into the optimal SRC detuning and homodyne angle to observe pondermotive squeezing at the 40m. The first attachment summarizes these investigations and shows the amount of squeezing below vacuum obtainable as a function of homodyne angle for an optimal SRC detuning including fundamental classical sources of noise (seismic, CTN, and suspension thermal). These calculations are done with an Optickle model. According to the calculations, it's possible to see 6 dBvac of squeezing around 100 Hz.

The second attachment shows the amount of squeezing obtainable including DAC noise as a function of current noise in the DAC electronics. These calculations are done at the optimal -0.45 deg SRC detuning and 97 deg homodyne angle. Estimates of this noise are computed as is done in elog 13146 and include de-whitening. It is not possible to observe squeezing with the current 400 Ω series resistor which corresponds to 30 pA/rtHz current noise at 100 Hz. We can get to 0 dBvac for current noise of around 10 pA/rtHz (1.2 kΩ series resistor) and can see 3 dBvac of squeezing with current noise of about 5 pA/rtHz at 100 Hz (2.5 kΩ series resistor). At this point it will be difficult to control the optics however.

So it seems reasonable to reduce the DAC noise to sufficient levels to observe squeezing, but we will need to think about the controls problem more.

 

Attachment 1: 40mDAC_squeezing.pdf
40mDAC_squeezing.pdf
  13489   Wed Dec 20 00:43:58 2017 KevinSummaryGeneralDAC noise contribution to squeezing noise budget

Gautam and I looked into the DAC noise contribution to the noise budget for homodyne detection at the 40m. DAC noise is currently the most likely limiting source of technical noise.

Several of us have previously looked into the optimal SRC detuning and homodyne angle to observe pondermotive squeezing at the 40m. The first attachment summarizes these investigations and shows the amount of squeezing below vacuum obtainable as a function of homodyne angle for an optimal SRC detuning including fundamental classical sources of noise (seismic, CTN, and suspension thermal). These calculations are done with an Optickle model. According to the calculations, it's possible to see 6 dBvac of squeezing around 100 Hz.

The second attachment shows the amount of squeezing obtainable including DAC noise as a function of current noise in the DAC electronics. These calculations are done at the optimal -0.45 deg SRC detuning and 97 deg homodyne angle. Estimates of this noise are computed as is done in elog 13146 and include de-whitening. It is not possible to observe squeezing with the current 400 Ω series resistor which corresponds to 30 pA/rtHz current noise at 100 Hz. We can get to 0 dBvac for current noise of around 10 pA/rtHz (1.2 kΩ series resistor) and can see 3 dBvac of squeezing with current noise of about 5 pA/rtHz at 100 Hz (2.5 kΩ series resistor). At this point it will be difficult to control the optics however.

So it seems reasonable to reduce the DAC noise to sufficient levels to observe squeezing, but we will need to think about the controls problem more.

Attachment 1: 40m_squeezing.pdf
40m_squeezing.pdf
Attachment 2: 40mDAC_squeezing.pdf
40mDAC_squeezing.pdf
  13488   Mon Dec 18 20:37:18 2017 gautamUpdatePSLPMC MEDM cleanup

There are fewer lies on this screen now. For reference, the details of the electronics modifications made are in this elog.

  1. Error and control signals are now in units of nm, the appropriate filter switches have been SDF'ed.
  2. I think it's useful to see the control voltage to the PZT in volts as well, so I've made two readbacks available at the control point, one in V and one in nm.
  3. Indicated that the on-board LO mon readback, which reads "nan", is no longer meaningful, as the mixer is off the demod board.
  4. Indicated that the PMC Trans readback of "0" is because of a dead ADC.
Quote:

I think many of the readbacks on the PMC MEDM screen are now bogus and misleading since the PMC RF upgrade that Gautam did awhile ago. We ought to fix the screen and clearly label which readbacks and actuators are no longer valid.

 

Attachment 1: PMC_revamped.png
PMC_revamped.png
  13487   Mon Dec 18 17:48:09 2017 ranaUpdateComputersrossa: SL7.3 upgrade continues

Following instructions from LLO-CDS fo the rossa upgrade. Last time there were some issues with not being to access the LLO EPEL repos, but this time it seems to be working fine.

After adding font aliases, need to run 'sudo xset fp rehash' to get the new aliases to take hold. Afterwards, am able to use MEDM and sitemap just fine.

But diaggui won't run because of a lib-sasl error. Try 'sudo yum install gds-all'.

diaggui: error while loading shared libraries: libsasl2.so.2: cannot open shared object file: No such file or directorycrying (have contacted LLO CDS admins)

X-windows keeps crashing with SL7 and this big monitor. Followed instructions on the internet to remove the generic 'Nouveau' driver and install the proprietary NVDIA drivers by dropping to run level 3 and runnning some command line hoodoo to modify the X-files. Now I can even put the mouse on the left side of the screen and it doesn't crash. laugh

  13486   Mon Dec 18 16:45:44 2017 gautamUpdateIOOIMC lockloss correlated with PRM alignment?

I stopped the test earlier today morning around 11:30am. The log file is located at /opt/rtcds/caltech/c1/scripts/SUS/FaradayIsolationTest/PRM_stepping.txt. It contains the times at which the PRM was aligned/misaligned for lookback, and also the number of MC unlocks during every 30 minute period that the PRM alignment was toggled. This was computed by:

  • continuously reading the current value of the EPICS record for MC Trans.
  • comparing its current value to its values 3 seconds ago.
  • If there is a downward step in this comparison greater than 5000 counts, increment a counter variable by 1.
  • Reset counter at the end of 30 minute period.

I think this method is a pretty reliable proxy, because the MC autolocker certainly takes >3 seconds to re-acquire the lock (it has to run mcdown, wait for the next cavity flash, and run mcup in the meantime).

Preliminary analysis suggests no obvious correlation between MC lock duty cycle and PRM alignment.

I leave further analysis to those who are well versed in the science/art of PRM/IMC statistical correlations.

  13485   Fri Dec 15 19:09:49 2017 gautamUpdateIOOIMC lockloss correlated with PRM alignment?

Motivation:

To test the hypothesis that the IMC lock duty cycle is affected by the PRM alignment. Rana pointed out today that the input faraday has not been tuned to maximize the output->input isolation in a while, so the idea is that perhaps when the PRM is aligned, some of the reflected light comes back towards the PSL through the Faraday and hence, messes with the IMC lock.

A script to test this hypothesis is running over the weekend (in case anyone was thinking of doing anything with the IFO over the weekend).

Methodology:

I've made a simple script - the pseudocode is the following:

  • Align PRM
  • For the next half hour, look for downward transitions in the EPICS record for MC TRANS > 5000 cts - this is a proxy for an MC lockloss
  • At the end of 30 minutes, record number of locklosses in the last 30 minutes
  • Misalign PRM, repeat the above 3 bullets

The idea is to keep looping the above over the weekend, so we can expect ~100 datapoints, 50 each for PRM misaligned/aligned. The times at which PRM was aligned/misaligned is also being logged, so we can make some spectrograms of PC drive RMS (for example) with PRM aligned/misaligned. The script lives at /opt/rtcds/caltech/c1/scripts/SUS/FaradayIsolationTest/FaradayIsolCheck.py. Script is being run inside a tmux session on pianosa, hopefully the machine doesn't crash over the weekend and MC1/CDS stays happy.

A more direct measurement of the input Faraday isolation can be made by putting a photodiode in place of the beam dump shown in Attachment #1 (borrowed from this elog). I measured ~100uW of power leaking through this mirror with the PRM misaligned (but IMC locked). I'm not sure what kind of SNR we can expect for a DC measurement, but if we have a chopper handy, we could put a chopper (in the leaked beam just before the PD so as to allow the IMC to be locked) and demodulate at that frequency for a cleaner measurement? This way, we could also measure the contribution from prompt reflections (up to the input side of the Faraday) by simply blocking the beam going into the vacuum. The window itself is wedged so that shouldn't be a big contributor.

Attachment 1: PSL_layout.JPG
PSL_layout.JPG
  13484   Fri Dec 15 18:24:46 2017 ranaSummaryOptical LeversPRM

Today Angelina and I looked at the PRM OL with an eye towards installing a 2nd QPD. We want to try out using 2 QPDs for a single optic to see if theres a way to make a linear combination of them to reduce the sensitivity to jitter of the HeNe laser or acoustic noise on the table.

The power supply for the HeNe was gone, so I took one from the SP table.

There are WAY too many optics in use to get the beam from the HeNe into the vacuum and then back out. What we want is 1 steering mirror after the laser and then 1 steering mirror before the QPD. Even though there are rumors that this is impossible, I checked today and in fact it is very, very possible.

More optics = more noise = bad.

  13483   Fri Dec 15 18:23:03 2017 ranaUpdatePEMTrillium seismometer DC offset

UVW refers to the 3 internal, orthogonal velocity sensors which are not aligned with the vertical or horizontal directions. XYZ refers to the linear combinations of UVW which correspond to north, east, and up.

  13482   Fri Dec 15 17:05:55 2017 gautamUpdatePEMTrillium seismometer DC offset

Yesterday, while we were bringing the CDS system back online, we noticed that the control room wall StripTool traces for the seismic BLRMS signals did not come back to the levels we are used to seeing even after restarting the PEM model. There are no red lights on the CDS overview screen indicative of DAQ problems. Trending the DQ-ed seismometer signals (these are the calibrated (?) seismometer signals, not the BLRMS) over the last 30 days, it looks like

  1. On ~1st December, the signals all went to 0 - this is consistent with signals in the other models, I think this is when the DAQ processes crashed.
  2. On ~8 December, all the signals picked up a DC offset of a few 100s (counts? or um/s? this number is after a cts2vel calibration filter). I couldn't find anything in the elog on 8 December that may be related to this problem.

I poked around at the electronics rack (1X5/1X6) which houses the 1U interface box for these signals - on its front panel, there is a switch that has selectable positions "UVW" and "XYZ". It is currently set to the latter. I am assuming the former refers to velocities in the xyz directions, and the latter is displacement in these directions. Is this the nominal state? I didn't spend too much time debugging the signal further for now.

 

Attachment 1: Trillium.png
Trillium.png
  13481   Fri Dec 15 11:19:11 2017 gautamUpdateCDSCDS recovery, NFS woes

Looking at the dmesg on c1iscex for example, at least part of the problem seems to be associated with FB1 (192.168.113.201, see Attachment #1). The "server" can be unresponsive for O(100) seconds, which is consistent with the duration for which we see the MEDM status lights go blank, and the EPICS records get frozen. Note that the error timestamped ~4000 was from last night, which means there have been at least 2 more instances of this kind of freeze-up overnight.

I don't know if this is symptomatic of some more widespread problem with the 40m networking infrastructure. In any case, all the CDS overview screen lights were green today morning, and MC autolocker seems to have worked fine overnight.

I have also updated the wiki page with the updated daqd restart commands.

Unrelated to this work - Koji fixed up the MC overview screen such that the MC autolocker button is now visible again. The problem seems to do with me migrating some of the c1ioo EPICS channels from the slow machine to the fast system, as a result of which the EPICS variable type changed from "ENUM" to something that was not "ENUM". In any case, the button exists now, and the MC autolocker blinky light is responsive to its state.

Quote:

I don't think the problem is fb1.  The fb1 NFS is mostly only used during front end boot.  It's the rtcds mount that's the one that sees all the action, which is being served from chiara.

 

Attachment 1: NFS.png
NFS.png
Attachment 2: MCautolocker.png
MCautolocker.png
  13480   Fri Dec 15 01:53:37 2017 jamieUpdateCDSCDS recovery, NFS woes
Quote:

I would make a detailed post with how the problems were fixed, but unfortunately, most of what we did was not scientific/systematic/repeatable. Instead, I note here some general points (Jamie/Koji can addto /correct me):

  1. There is a "known" problem with unloading models on c1lsc. Sometimes, running rtcds stop <model> will kill the c1lsc frontend.
  2. Sometimes, when one machine on the dolphin network goes down, all 3 go down.
  3. The new FB/RCG means that some of the old commands now no longer work. Specifically, instead of telnet fb 8087 followed by shutdown (to fix DC errors) no longer works. Instead, ssh into fb1, and run sudo systemctl restart daqd_*.

This should still work, but the address has changed.  The daqd was split up into three separate binaries to get around the issue with the monolithic build that we could never figure out.  The address of the data concentrator (DC) (which is the thing that needs to be restarted) is now 8083.

Quote:

UPDATE 8:20pm:

Koji suggested trying to simply retsart the ASS model to see if that fixes the weird errors shown in Attachment #2. This did the trick. But we are now faced with more confusion - during the restart process, the various indicators on the CDS overview MEDM screen froze up, which is usually symptomatic of the machines being unresponsive and requiring a hard reboot. But we waited for a few minutes, and everything mysteriously came back. Over repeated observations and looking at the dmesg of the frontend, the problem seems to be connected with an unresponsive NFS connection. Jamie had noted sometime ago that the NFS seems unusually slow. How can we fix this problem? Is it feasible to have a dedicated machine that is not FB1 do the NFS serving for the FEs?

I don't think the problem is fb1.  The fb1 NFS is mostly only used during front end boot.  It's the rtcds mount that's the one that sees all the action, which is being served from chiara.

  13479   Fri Dec 15 00:26:40 2017 johannesUpdateCDSRe: CDS recovery, NFS woes
Quote:

Didn't touch Xarm because we don't know what exactly the status of ETMX is.

The Xarm is currently in its original state, all cables are connected and c1auxex is hosting the slow channels.

  13478   Thu Dec 14 23:27:46 2017 johannesUpdateDAQaux chassis design

Made a front and back panel and slot panels for DSub and IDC breakouts. I want to send this out soon, are there any comments? Preferences for color schemes?

Attachment 1: auxdaq_40m_4U_front.pdf
auxdaq_40m_4U_front.pdf
Attachment 2: auxdaq_40m_4U_rear.pdf
auxdaq_40m_4U_rear.pdf
Attachment 3: auxdaq_40m_4U_DSub37x2.pdf
auxdaq_40m_4U_DSub37x2.pdf
Attachment 4: auxdaq_40m_4U_IDC50.pdf
auxdaq_40m_4U_IDC50.pdf
  13477   Thu Dec 14 19:41:00 2017 gautamUpdateCDSCDS recovery, NFS woes

[Koji, Jamie(remote), gautam]

Summary: The CDS system seems to be back up and functioning. But there seems to be some pending problems with the NFS that should be looked into.

We locked Y-arm, hand aligned transmission to 1yes. Some pending problems with ASS model (possibly symptomatic of something more general). Didn't touch Xarm because we don't know what exactly the status of ETMX is.

Problems raised in elogs in the thread of 13474 and also 13436 seem to be solved.


I would make a detailed post with how the problems were fixed, but unfortunately, most of what we did was not scientific/systematic/repeatable. Instead, I note here some general points (Jamie/Koji can addto /correct me):

  1. There is a "known" problem with unloading models on c1lsc. Sometimes, running rtcds stop <model> will kill the c1lsc frontend.
  2. Sometimes, when one machine on the dolphin network goes down, all 3 go down.
  3. The new FB/RCG means that some of the old commands now no longer work. Specifically, instead of telnet fb 8087 followed by shutdown (to fix DC errors) no longer works. Instead, ssh into fb1, and run sudo systemctl restart daqd_*.
  4. Timing error on c1sus machine was linked to the mx_stream processes somehow not being automatically started. The "!mxstream restart" button on the CDS overview MEDM screen should run the necessary commands to restart it. However, today, I had to manually run sudo systemctl start mx_stream on c1sus to fix this error. It is a mystery why the automatic startup of this process was disabled in the first place. Jamie has now rectified this problem, so keep an eye out.
  5. c1oaf persistently reported DC errors (0x2bad) that couldn't be addressed by running mxstream restart or restarting the daqd processes on FB1. Restarting the model itself (i.e. rtcds restart c1oaf) fixed this issue (though of course I took the risk of having to go into the lab and hard-reboot 3 machines).
  6. At some point, we thought we had all the CDS lights green - but at that point, the END FEs crashed, necessitating Koji->EX and Gautam->EY hard reboots. This is a new phenomenon. Note that the vertex machines were unaffected.
  7. At some point, all the DC lights on the CDS overview screen went white - at the same time, we couldn't ssh into FB1, although it was responding to ping. After ~2mins, the green lights came back and we were able to connect to FB1. Not sure what to make of this.
  8. While trying to run the dither alignment scripts for the Y-arm, we noticed some strange behaviour:
    • Even when there was no signal (looking at EPICS channels) at the input of the ASS servos, the output was fluctuating wildly by ~20cts-pp.
    • This is not simply an EPICS artefact, as we could see corresponding motion of the suspension on the CCD.
    • A possible clue is that when I run the "Start Dither" script from the MEDM screen, I get a bunch of error messages (see Attachment #2).
    • Similar error messages show up when running the LSC offset script for example. Seems like there are multiple ports open somehow on the same machine?
    • There are no indicator lights on the CDS overview screen suggesting where the problem lies.
    • Will continue investigating tomorrow.

Some other general remarks:

  1. ETMX watchdog remains shutdown.
  2. ITMY and BS oplevs have been hijacked for HeNe RIN / Oplev sensing noise measurement, and so are not enabled.
  3. Y arm trans QPD (Thorlabs) has large 60Hz harmonics. These can be mitigated by turning on a 60Hz comb filter, but we should check if this is some kind of ground loop. The feature is much less evident when looking at the TRANS signal on the QPD.

UPDATE 8:20pm:

Koji suggested trying to simply retsart the ASS model to see if that fixes the weird errors shown in Attachment #2. This did the trick. But we are now faced with more confusion - during the restart process, the various indicators on the CDS overview MEDM screen froze up, which is usually symptomatic of the machines being unresponsive and requiring a hard reboot. But we waited for a few minutes, and everything mysteriously came back. Over repeated observations and looking at the dmesg of the frontend, the problem seems to be connected with an unresponsive NFS connection. Jamie had noted sometime ago that the NFS seems unusually slow. How can we fix this problem? Is it feasible to have a dedicated machine that is not FB1 do the NFS serving for the FEs?

Attachment 1: CDS_14Dec2017.png
CDS_14Dec2017.png
Attachment 2: CDS_errors.png
CDS_errors.png
  13476   Thu Dec 14 19:33:20 2017 gautamFrogsASSc1ass slow channel offloading scripts with small

I don't think this is really a problem - we offload to the fast channels and not to the slow (although we really should offload to the slow channels). I think the best approach is to use the ezcaservo utility to offload the DC part of the ASS control signals to the slow channels, so as to not waste fast channel DAC counts on DC offsets. In principle, this approach should be somewhat immune to the slow channel calibration not being perfect.

Quote:

While staring at epics records all day I noticed something about the PIT/YAW offset sliders and ASS offset offloading to slow channels scripts that I'm not sure others are aware off, so I'll briefly discuss it in this post.

The PIT and YAW sliders directly control soft channels that are hosted on the slow machine. Secondary epics records disentangle them for the individual coils:

  • UL = PIT+YAW
  • LL = -PIT+YAW
  • UR = PIT-YAW
  • LR = -PIT-YAW

These channels are the direct input for the physical output channels that generate the control voltage.

The fast channels for PIT and YAW have a numerical correction factor built in that accounts for differences between the OSEMs, but the slow channels don't. This means that the slow PIT/YAW controls are not entirely orthogonal but have crosstalk on the order of 10 percent. This in itself is not that dramatic, however the offload offsets scripts for the dither alignment use the fast PIT/YAW values as inputs, which represent the necessary adjustments to the OSEMs only after the individual correction factors have been applied. The offloading to slow knows nothing of this calibration difference between the OSEMs. The result is that there is a ~10 percent of the offset correction error on the mirror alignment AFTER offloading. This will of course converge after a few iterations, but in any case it is recommendable to run the dither alignment again after offloading and not offload the new offsets to the fast channels.

 

  13475   Thu Dec 14 08:59:17 2017 SteveUpdateGeneralwe are here
Attachment 1: 8_days.png
8_days.png
  13474   Thu Dec 14 07:07:09 2017 ranaUpdateIOOLots of red on the FE status screen

I had to key the c1psl crate to get the PMC locking again. Without this, it would still sort of lock, but it was very hard to turn on the loop; it would push itself off the fringe. So probably it was stuck in some state with the gain wrong. Since the RF stuff is now done in a separate electronics chain, I don't think the RF phase can be changed by this. Probably the sliders are just not effective until power cycling.

Quote:

Once the RT machines were back, we launched only the five IOPs. They had bunch of red lights, but we continued to run essential models for the IFO. SOme of the lights were fixed by "global diag reset" and "mxstream restart".

The suspension were damped. We could restore the IMC lock. The locking became OK and the IMC was aligned. The REFL spot came back.

At least, I could confirm that the WFS ASC signals were not transmitted to c1mcs. There must be some disconneted links of IPC.

I then tried to get the MC WFS back, but running rtcds restart --all would make some of the computers hang. For c1ioo I had to push the reset button on the computer and then did 'rtcds start --all' after it came up. Still missing IPC connections.

I'm going to get in touch with Rolf.

  13473   Thu Dec 14 00:32:56 2017 johannesUpdateASSAcromag new crate; c1auxex2 configured as gateway server for acromag

This splicing in of fast binary channels we discussed at yesterday's and today's meetings is getting messy with the current chassis. Cleaning up the cable mess was a key point, so I got a 4U height DEEP chassis from Rich and drew up a front panel for a modular approach that we can use at the other 40m locations as well. The front panel will have slots for smaller slot panels to which we can mount the breakout boards as before, so all the wiring that I've done can be transfered to this design. If some new connector standard is required it will be easy to draw a new slot panel from a template, for now I'll make some with two DSub37 and IDC50. Since this chassis is so huge it will have ample space for cross-connects.

I also moved the communication of c1auxex2 with the Acromag units off the martian network, connecting them with a direct cable connection out of the second ethernet port. To test if this works I configured the second ethernet port of c1auxex2 to have the IP address 192.168.114.1 and one of the acromag units to have 192.168.114.11, and initialized an IOC with some test channels. Much to my surprise this actually worked straight out of the box, and the test channels can be accessed from the control room computers without having a direct ethernet link to the acromag modules. huzzah!

Steve: it would be nice to have all plugs- connectors lockable

 

Attachment 1: fp_mod_4U.pdf
fp_mod_4U.pdf
Attachment 2: IMG_20171213_171541850_HDR.jpg
IMG_20171213_171541850_HDR.jpg
  13472   Wed Dec 13 17:46:08 2017 Udit KhandelwalSummaryGeneralSummary of Current Tasks

40m Lab CAD

1. 40m_bldg.dwg has 2D drawing of the 40m building

  • After importing file as a 2D sketch into solidworks, make sure to retrace all the lines before performing any 3D extrusion stuff.
  • Made walls 3m high

2. 40m_VE.dwg has the Vaccuum Envelope.

  • Divided the file into individual sketches for the tubes, test mass, and beam splitter chambers (so they can be individually modified later if required).

3. 40melev.dwg has the relative positioning between (1) and (2).

  • Using this file to position objects inside building cad.

4. All files can be found in Dropbox folder [40m SOS Modeling], which should be renamed to [40m CAD].

5. Next step would be to add the optical table, mirrors.

Tip-Tilt Suspension

1. Current objective: (refer to D070172) - Increase the length of the side arms (so it matches the dimensions of D960001), while keeping the test mass subassembly at the same height.
2. Future objective: Resonant frequency FEM of the frame (sans the test mass), and then change height to get the desired frequency.

Past Work

  • Completed solidworks model of SOS (D960001). I understand this is not the focus right now so this is for reference that the model is ready to be used.

Comments

  • I will be in India from 16th December until 6th January so this is my final visit for this year. I have enough material to work from home, and will correspond with Koji over email regarding Lab CAD and tip-tilt suspension.
  13471   Wed Dec 13 09:49:23 2017 johannesUpdateASSwiring diagram

I attached a wiring schematic from the slow DAQ to the eurocrate modules. Of these, pins 1-32 (or 1A-16C) and pins 33-64 (17A-32C) are on separate DSub connectors. Therefore the easiest solution is to splice the slow DIO channels into the existing breakouts so we can proceed with the transition. This will still remove a lot of the current cable salad. For the YEND we can start thinking about a more elegant solution (For example a connector on the front panel of the Acromag chassis for the fast DIO) now that the problem is better defined.

Attachment 1: 1Y9.pdf
1Y9.pdf
  13470   Fri Dec 8 23:31:31 2017 johannesFrogsASSc1ass slow channel offloading scripts with small

While staring at epics records all day I noticed something about the PIT/YAW offset sliders and ASS offset offloading to slow channels scripts that I'm not sure others are aware off, so I'll briefly discuss it in this post.

The PIT and YAW sliders directly control soft channels that are hosted on the slow machine. Secondary epics records disentangle them for the individual coils:

  • UL = PIT+YAW
  • LL = -PIT+YAW
  • UR = PIT-YAW
  • LR = -PIT-YAW

These channels are the direct input for the physical output channels that generate the control voltage.

The fast channels for PIT and YAW have a numerical correction factor built in that accounts for differences between the OSEMs, but the slow channels don't. This means that the slow PIT/YAW controls are not entirely orthogonal but have crosstalk on the order of 10 percent. This in itself is not that dramatic, however the offload offsets scripts for the dither alignment use the fast PIT/YAW values as inputs, which represent the necessary adjustments to the OSEMs only after the individual correction factors have been applied. The offloading to slow knows nothing of this calibration difference between the OSEMs. The result is that there is a ~10 percent of the offset correction error on the mirror alignment AFTER offloading. This will of course converge after a few iterations, but in any case it is recommendable to run the dither alignment again after offloading and not offload the new offsets to the fast channels.

  13469   Fri Dec 8 12:06:59 2017 johannesOmnistructureComputersc1auxex2 ready - but need more cables

The new slow machine c1auxex2 is ready to deploy. Unfortunately we don't have enough 37pin DSub cables to connect all channels. In fact, we need a total of 8, and I found only three male-male cables and one gender changer. I asked Steve to buy more.

Over the past week I have transferred all EPICS records - soft channels and physical ones - from c1auxex to c1auxex2, making changes where needed. Today I started the in-situ testing

  1. Unplugged ETMX's satellite box
  2. Unplugged the eurocrate backplane DIN cables from the SOS Driver and QPD Whitening filter modules (the ones that receive ao channels)
  3. Measured output voltages on the relevant pins for comparison after the swap
  4. Turned off c1auxex by key, removed ethernet cable
  5. Started the modbus ioc on c1auxex2
  6. Slow machine indicator channels came online, ETMX Watchdog was responsive (but didn't have anything to do due to missing inputs) and reporting. PIT/YAW sliders function as expected
  7. Restoring the previous settings gives output voltages close to the previous values, in fact the exact values requested (due to fresh calibration)
  8. Last step is to go live with c1auxex2 and confirm the remaining channels work as expected.

I copied the relevant files to start the modbus server to /cvs/cds/caltech/target/c1auxex2, although kept local copies in /home/controls/modbusIOC/ from which they're still run.

I wonder what's the best practice for this. Probably to store the database files centrally and load them over the network on server start?

  13468   Thu Dec 7 22:24:04 2017 johannesOmnistructureComputersAcromag XEND progress

 

Quote:
 
  • Need to calibrate the +/- 10V swing of the analog channels via the USB utility, but that requires wiring the channels to the connectors and should probably be done once the unit sits in the rack
  • Need to wire power from the Sorensens into the chassis. There are +/- 5V, +/- 15V and +/- 20V present. The Acromags need only +12V-32V, for which I plan to use the +20V, and an excitation voltage for the binary channels, for which I'm going to wire the +5V. Should do this through the fuse rails on the side.
  • The current slow binary channels are sinking outputs, same as the XT1111 16-channel module we have. The additional 4 binary outputs of the XT1541 are sourcing, and I'm currently not sure if we can use them with the sos driver and whitening vme boards that get their binary control signals from the slow system.
  • Confirm switching of binary channels (haven't used model XT1111 before, but I assume the definitions are identical to XT1121)
  • Setup remaining essential EPICS channels and confirm that dimensions are the same (as in both give the same voltage for the same requested value)
  • Disconnect DIN cables, attach adapter boards + DSUB cables
  • Testing

Getting the chassis ready took a little longer than anticipated, mostly because I had not looked into the channel list myself before and forgot about Lydia's post which mentions that some of the switching controls have to be moved from the fast to the slow DAQ. We would need a total of 5+5+4+8=22 binary outputs. With the existing Acromag units we have 16 sinking outputs and 8 sourcing outputs. I looked through all the Eurocrate modules and confirmed that they all use the same switch topology which has sourcing inputs.

While one can use a pull-down resistor to control a sourcing input with a sourcing output,

pulling down the MAX333A input (datasheet says logic low is <0.8V) requires something like 100 Ohms for the pull down resistor, which would require ~150mA of current PER CHANNEL, which is unreasonable. Instead, I asked Steve to buy a second XT1111 and modified the chassis to accomodate more Acromag units.

I have now finished wiring the chassis (except for 8 remaining bypass controls to the whitening board which need the second XT1111), calibrated all channels in use, confirmed all pin locations via the existing breakout boards and DCC drawings for the eurocrate modules, and today Steve and I added more fuses to the DIN rail power distribution for +20V and +15V.

There was not enough contingent free space in the XEND rack to mount the chassis, so for now I placed it next to it.

c1auxex2 is currently hosting all original physical c1auxex channels (not yet calc records) under their original name with an _XT added at the end to avoid duplicate channel names. c1auxex is still in control of ETMX. All EPICS channels hosted by c1auxex2 are in dimensions of Volts. The plan for tomorrow is to take c1auxex off the grid, rename the c1auxex2 hosted channels and transfer ETMX controls to it, provided we can find enough 37pin DSub cables (8). I made 5 adapter boards for the 5 Eurocrate modules that need to talk to the slow DAQ through their backplane connector.

  13467   Thu Dec 7 16:28:06 2017 KojiUpdateIOOLots of red on the FE status screen

Once the RT machines were back, we launched only the five IOPs. They had bunch of red lights, but we continued to run essential models for the IFO. SOme of the lights were fixed by "global diag reset" and "mxstream restart".

The suspension were damped. We could restore the IMC lock. The locking became OK and the IMC was aligned. The REFL spot came back.

At least, I could confirm that the WFS ASC signals were not transmitted to c1mcs. There must be some disconneted links of IPC.

  13466   Thu Dec 7 15:46:31 2017 johannesHowToComputer Scripts / ProgramsLots of red on the FE status screen

[Koji, Johannes]

The issue was partially fixed and the interferometer is in workable condition now.

What -probably- fixed it was restarting the dhcp server on chiara

sudo service isc-dhcp-server restart

Afterwards the frontends were restarted one by one. SSH access was possible and the essential models for IFO operation were started.

c1iscex reported initially that no DAQ card was found, and inside the IO chassis the LED indicator strip was red. Turning off the machine, checking the cables and rebooting fixed this.

Attachment 1: 04.png
04.png
  13465   Thu Dec 7 15:02:37 2017 KojiHowToComputer Scripts / ProgramsLots of red on the FE status screen

Once a realtime machine was rebooted, it did not come back. I suspect that the diskless hosts have a difficulty to boot up.

Attachment 1: DSC_0552.JPG
DSC_0552.JPG
  13464   Thu Dec 7 11:14:37 2017 johannesHowToComputer Scripts / ProgramsLots of red on the FE status screen

Since we're getting ready to put the replacement slow DAQ for c1auxex in I wanted to bring the IFO back to operating condition after the PMC hasn't been locked for days. Something seems wrong with the CDS system though, many of the frontent models have red background and don't seem to be responsive. I followed the instructions laid out in https://wiki-40m.ligo.caltech.edu/Computer_Restart_Procedures.

In the attached screenshot, initially all c1ioo models were red, and on c1iscex only c1x01 was blue, the other ones red. I was able to ssh into both machines and tried to restart indivitual models, which didn't work and instead turned their background white. Still following the wiki page, I restarted both machines but they don't respond to pinging anymore and thus I cannot use ssh to reach them. Not sure what to do, I also rebooted fb over telnet.

So far I couldn't find any records of how to fix this situation.

Attachment 1: 22.png
22.png
  13463   Mon Dec 4 22:06:07 2017 johannesOmnistructureComputersAcromag XEND progress

I wired up the power distribution, and ethernet cables in the Acromag chassis today. For the time being it's all kind of loose in there but tomorrow the last parts should arrive from McMaster to put everything in its place. I had to unplug some of the wiring that Aaron had already done but labeled everything before I did so. I finalized the IP configuration via USB for all the units, which are now powered through the chassis and active on the network.

I started transcribing the database file ETMXaux.db that is loaded by c1auxex in the format required by the Acromags and made sure that the new c1auxex2 properly functions as a server, which it does.

ToDo-list:

  • Need to calibrate the +/- 10V swing of the analog channels via the USB utility, but that requires wiring the channels to the connectors and should probably be done once the unit sits in the rack
  • Need to wire power from the Sorensens into the chassis. There are +/- 5V, +/- 15V and +/- 20V present. The Acromags need only +12V-32V, for which I plan to use the +20V, and an excitation voltage for the binary channels, for which I'm going to wire the +5V. Should do this through the fuse rails on the side.
  • The current slow binary channels are sinking outputs, same as the XT1111 16-channel module we have. The additional 4 binary outputs of the XT1541 are sourcing, and I'm currently not sure if we can use them with the sos driver and whitening vme boards that get their binary control signals from the slow system.
  • Confirm switching of binary channels (haven't used model XT1111 before, but I assume the definitions are identical to XT1121)
  • Setup remaining essential EPICS channels and confirm that dimensions are the same (as in both give the same voltage for the same requested value)
  • Disconnect DIN cables, attach adapter boards + DSUB cables
  • Testing

 

Quote:

[Aaron, Johannes]

We configured the AtomServer for the Martian network today. Hostname is c1auxex2, IP is 192.168.113.49. Remote access over SSH is enabled.

There will be 6 acromag units served by c1auxex2.

Hostname Type IP Address
c1auxex-xt1221a 1221 192.168.113.130
c1auxex-xt1221b 1221 192.168.113.131
c1auxex-xt1221c 1221 192.168.113.132
c1auxex-xt1541a 1541 192.168.113.133
c1auxex-xt1541b 1541 192.168.113.134
c1auxex-xt1111a 1111 192.168.113.135

Some hardware to assemble the Acromag box and adapter PCBs are still missing, and the wiring and channel definitions have to be finalized. The port driver initialization instructions and channel definitions are currently locally stored in /home/controls/modbusIOC/ but will eventually be migrated to a shared location, but we need to decide how exactly we want to set up this infrastructure.

  • Should the new machines have the same hostnames as the ones they're replacing? For the transition we simply named it c1auxex2.
  • Because the communication of the server machine with the DAQ modules is happening over TCP/IP and not some VME backplane bus we could consolidate machines, particularly in the vertex area.
  • It would be good to use the fact that these SuperMicro servers have 2+ ethernet ports to separate CDS EPICS traffic from the modbus traffic. That would also keep the 30+ IPs for the Acromag thingies off the Martian host tables.
  13462   Sun Dec 3 17:01:08 2017 KojiConfigurationComputerssendmail installed on nodus

An email has come at 5PM on Dec 3rd.

 

  13461   Sun Dec 3 05:25:59 2017 gautamConfigurationComputerssendmail installed on nodus

Pizza mail didn't go out last weekend - looking at logfile, it seems like the "sendmail" service was missing. I installed sendmail following the instructions here: https://tecadmin.net/install-sendmail-server-on-centos-rhel-server/

Except that to start the sendmail service, I used systemctl and not init.d. i.e. I ran systemctl start sendmail.service (as root). Test email to myself works. Let's see if it works this weekend. Of course this isn't so critical, more important are the maintenance emails that may need to go out (e.g. disk usage alert on chiara / N2 pressure check, which looks like nodus' responsibilities). 

  13460   Fri Dec 1 17:09:29 2017 Udit KhandelwalSummaryGeneral 

Current objectives and statuses:

  • CAD Model of 40m lab (facility, chambers, invacuum components etc)
    • Status: On hold since I'm unable to acquire general dimensions of 
  13459   Thu Nov 30 10:38:39 2017 SteveUpdateVACannuloses are not pumped

Annuloses are not pumped for 30 days, since TP2 failed.  IFO pressure 7e-6 Torr it,  Rga 2.6e-6 Torr

Valve configuration: Vacuum Normal as TP3 is the forepump of Maglev, annuloses are not puped at 1.1 Torr

TP3 50K rpm,  0.15A  24C, foreline pressure 16.1 mTorr

 

Quote:

The TP3 foreline pressure was 4.8 Torr, 50K rpm 0.54A and 31C........Maglev rotation normal 560 Hz.......    IFO pressure 7.2e- 6 Torrit was not effected

V1 closed ......replaced drypump.........V1 opened

IFO 6.9e-6 Torrit at 19:55, TP3fl 18 mT,  50Krpm 0.15A 24C

VM1 is still closed

 

 

Attachment 1: AnnulosesNotPumped.png
AnnulosesNotPumped.png
  13458   Wed Nov 29 21:40:30 2017 johannesOmnistructureComputersSlow DAQ replacement computer progress

[Aaron, Johannes]

We configured the AtomServer for the Martian network today. Hostname is c1auxex2, IP is 192.168.113.49. Remote access over SSH is enabled.

There will be 6 acromag units served by c1auxex2.

Hostname Type IP Address
c1auxex-xt1221a 1221 192.168.113.130
c1auxex-xt1221b 1221 192.168.113.131
c1auxex-xt1221c 1221 192.168.113.132
c1auxex-xt1541a 1541 192.168.113.133
c1auxex-xt1541b 1541 192.168.113.134
c1auxex-xt1111a 1111 192.168.113.135

Some hardware to assemble the Acromag box and adapter PCBs are still missing, and the wiring and channel definitions have to be finalized. The port driver initialization instructions and channel definitions are currently locally stored in /home/controls/modbusIOC/ but will eventually be migrated to a shared location, but we need to decide how exactly we want to set up this infrastructure.

  • Should the new machines have the same hostnames as the ones they're replacing? For the transition we simply named it c1auxex2.
  • Because the communication of the server machine with the DAQ modules is happening over TCP/IP and not some VME backplane bus we could consolidate machines, particularly in the vertex area.
  • It would be good to use the fact that these SuperMicro servers have 2+ ethernet ports to separate CDS EPICS traffic from the modbus traffic. That would also keep the 30+ IPs for the Acromag thingies off the Martian host tables.
  13457   Wed Nov 29 15:33:16 2017 ranaUpdatePSLPMC locking

PMC wasn't locking. Had to power down c1psl. Did burt restore. Still not great.

I think many of the readbacks on the PMC MEDM screen are now bogus and misleading since the PMC RF upgrade that Gautam did awhile ago. We ought to fix the screen and clearly label which readbacks and actuators are no longer valid.

Also, the locking procedure is not so nice. The output V adjust doesn't work anymore with BLANK enabled. Would be good to make an autolocker script if we find a visitor wanting to do something fun.

  13456   Tue Nov 28 17:27:57 2017 awadeBureaucracyCalibration-RepairSR560 return, still not charging

I brought a bunch of SR560s over for repair from Bridge labs. This unit, picture attached (SN 49698), appears to still not be retaining charge. I’ve brought it back. 

Attachment 1: 96B6ABE6-CC5C-4636-902A-2E5DF553653D.jpeg
96B6ABE6-CC5C-4636-902A-2E5DF553653D.jpeg
Attachment 2: image.jpg
image.jpg
  13455   Tue Nov 28 16:02:32 2017 ranaUpdatePEMseismometer can testing

I've ordered 4 of these from McMaster. Should be delivered to the 40m by noon tomorrow.

Quote:

For the insulation, I have decided to use this one (Buna-N/PVC Foam Insulation Sheets). We will need 3 of the 1 inch plain backing ones (9349K4) to wrap a few layers around it. I'll try two layers for now, since the insulation seems to be doing quite well according to initial testing.

Kira and I also discussed the issiue. It would be good if someone can hunt aroun on the web and get some free samples of non-shedding foam with R~4.

  13454   Sun Nov 26 19:38:40 2017 SteveUpdateVACTP3 drypump replaced again

The TP3 foreline pressure was 4.8 Torr, 50K rpm 0.54A and 31C........Maglev rotation normal 560 Hz.......    IFO pressure 7.2e- 6 Torrit was not effected

V1 closed ......replaced drypump.........V1 opened

IFO 6.9e-6 Torrit at 19:55, TP3fl 18 mT,  50Krpm 0.15A 24C

VM1 is still closed

 

Attachment 1: after_replacement.png
after_replacement.png
  13453   Thu Nov 23 18:03:52 2017 gautamUpdateOptical LeversOplev "noise budget"

Here are a couple of preliminary plots of the noise from a 20minute stretch of data - the new curve is the orange one, labelled sensing, which is the spectrum of the PIT/YAW error signal from the HeNe beam single bounce off a single steering mirror onto the QPD, normalized to account for the difference in QPD sum. The peaky features that were absent in the dark noise are present here.

I am a bit confused about the total sum though - there is ~2.5mW of light incident on the PD, and the transimpedance gain is 10.7kohm. So I would expect 2.5e-3 mW * 0.4A/W * 10.7 kV/A ~ 10.7V over 4 quadrants. The ADC is 16 bit and has a range +/- 10V, so 10.7 V should be ~35,000 cts. But the observed QPD sum is ~14,000 counts. The reflected power was measured to be ~250uW, so ~10% of the total input power. Not sure if this is factored into the photodiode efficiency value of 0.4A/W. I guess there is some fraction of the QPD that doesn't generate any photocurrent (i.e. the grooves defining the quadrants), but is it reasonable that when the Oplev beam is well centered, ~50% of the power is not measured? I couldn't find any sneaky digital gains between the quadrant channels to the sum channel either... But in the Oplev setup, the QPD had ~250uW of power incident on it, and was reporting a sum of ~13,000 counts with a transimpedance gain of 100kohm, so at least the scaling seems to hold...

I guess we wan't to monitor this over a few days, see how stationary the noise profile is etc. I didn't look at the spectrum of the intensity noise during this time.

Quote:

I've setup a test setup on the ITMY Oplev table. Details + pics to follow, but for now, be aware that

Here are some pics of the setup: https://photos.app.goo.gl/DHMINAV7aVgayYcf1.None of the existing Oplev input/output steering optics were touched. Steve can make modifications as necessary, perhaps we can make similar mods to the SRM Oplev QPD and the BS one to run the HeNe test for a few days...

 

 

Attachment 1: ITMY_P_noise.pdf
ITMY_P_noise.pdf
Attachment 2: ITMY_Y_noise.pdf
ITMY_Y_noise.pdf
  13452   Wed Nov 22 23:56:14 2017 gautamUpdateOptical LeversOplev "noise budget"

Do not turn on BS/ITMY/SRM/PRM Oplev servos without reading this elog and correcting the needful!

I've setup a test setup on the ITMY Oplev table. Details + pics to follow, but for now, be aware that

  1. I've turned off the HeNe that is used for the SRM and ITMY Oplev.
  2. Moved one of the HeNe's Steve setup on the SP table to the ITMY Oplev table.
  3. Output power was 2.5mW, whereas normal power incident on this PD was ~250uW.
  4. So I changed all transimpedance gains on the ITMY Oplev QPD from 100k to 10k thin film - these should be changed back when we want to use this QPD for Oplev purposes again. Note that I did not change the compensation capacitors C3-C6, as with 10k transimpedance, and assuming they are 2.2nF, we get a corner frequency of 6.7kHz. The original schematic recommends 0.1uF. In hindsight, I should have changed these to 22nF to keep roughly the same corner frequency of ~700Hz.
    I've implemented this change as of ~5pm Nov 23 2017 - C3-C6 are now 22nF, so the corner frequency is 676Hz, as opposed to 723Hz before... This should also be undone when we use this QPD as an Oplev QPD again...
  5. I marked the position of the ITMY Oplev QPD with sharpie and also took pics so it should be easy enough to restore when we are done with this test.
  6. I couldn't get the HeNe to turn on with any of the power supplies I found in the cabinet, so I borrowed the one used to power the BS/PRM. So these oplevs are out of commission until this test is done.
  7. There is a single steering mirror in a Polaris mount which I used to center the spot on the QPD.
  8. The specular reflection (~250uW, i.e. 10% of the power incident on the QPD) is dumped onto a clean razor beam stack. Steve can put in a glass beam dump on Monday.
  9. Just in case someone accidentally turns on some servos - I've disabled the inputs to the BS, PRM and SRM oplevs, and set the limiter on the ITMY servo to 0.

Here are some pics of the setup: https://photos.app.goo.gl/DHMINAV7aVgayYcf1.None of the existing Oplev input/output steering optics were touched. Steve can make modifications as necessary, perhaps we can make similar mods to the SRM Oplev QPD and the BS one to run the HeNe test for a few days...

Quote:

too complex; just shoot straight from the HeNe to the QPD. We lower the gain of the QPD by changing the resistors; there's no sane reason to keep the existing 100k resistors for a 2 mW beam. The specular reflection of the QPD must be dumped on a black glass V dump (not some flimsy anodized aluminum or dirty razor stack)

 

  13451   Wed Nov 22 19:20:01 2017 ranaUpdateOptical LeversOplev "noise budget"

too complex; just shoot straight from the HeNe to the QPD. We lower the gain of the QPD by changing the resistors; there's no sane reason to keep the existing 100k resistors for a 2 mW beam. The specular reflection of the QPD must be dumped on a black glass V dump (not some flimsy anodized aluminum or dirty razor stack)

  13450   Wed Nov 22 17:52:25 2017 gautamUpdateOptimal ControlVisualizing cost functions

I've attempted to visualize the various components of the cost function in the way I've defined it for the current iteration of the Oplev optimal control loop design code. For each term in the cost function, the way the cost is computed depends on the ratio of the abscissa value to some threshold value (set by hand for now) - if this ratio is >1, the cost is the logarithm of the ratio, whereas if the ratio is <1, the cost is the square of the ratio. Continuity is enforced at the point at which this transition happens. I've plotted the cost function for some of the terms entering the code right now - indicated in dashed red lines are the approximate value of each of these costs for our current Oplev loop - the weights were chosen so that each of the costs were O(10) for the current controller, and the idea was that the optimizer could drive these down to hopefully O(1), but I've not yet gotten that to happen.

Based on the meeting yesterday, some possible ideas:

  1. For minimizing the control noise injection - we know the transfer function from the Oplev control signal coupling to MICH from measurements, and we also have a model for the seismic noise. So one term could be a weighted integral of (coupling - seismic) - the weight can give more importance to f>30Hz, and even more importance to f>100Hz. Right now, I don't have any suc frequency dependant requirement on the control signal.
  2. Try a simpler problem first - pendulum local damping. The position damping controller for example has fewer roots in the complex plane. Although it too has some B/R notches, which account for 16 complex roots, and hence, 32 parameters, so maybe this isn't really a simpler problem?
  3. How do we pick the number of excess poles compared to zeros in the overall transfer function? The OL loop low-pass filters are elliptic filters, which achieve the fastest transition between the passband and stopband, but for the Oplev loop roll-off, perhaps its better to have a just have some poles to roll off the HF response?
Attachment 1: globalCosts.pdf
globalCosts.pdf
  13449   Wed Nov 22 16:40:00 2017 KojiUpdateOptical LeversOplev "noise budget"

You may want to consult with the cryo Q people (Brittany, Aaron) for a Si QPD. If you want the same QPD architecture, I can look at my QPD circuit stock.

  13448   Wed Nov 22 15:29:23 2017 gautamUpdateOptical LeversOplev "noise budget"

[steve, gautam]

What is the best way to set this test up?

I think we need a QPD to monitor the spot rather than a single element PD, to answer this question about the sensor noise. Ideally, we want to shoot the HeNe beam straight at the QPD - but at the very least, we need a lens to size the beam down to the same size as we have for the return beam on the Oplevs. Then there is the power - Steve tells me we should expect ~2mW at the output of these HeNes. Assuming 100kohm transimpedance gain for each quadrant and Si responsivity of 0.4A/W at 632nm, this corresponds to 10V (ADC limit) for 250uW of power - so it would seem that we need to add some attenuating optics in the way.

Also, does anyone know of spare QPDs we can use for this test? We considered temporarily borrowing one of the vertex OL QPDs (mark out its current location on the optics table, and move it over to the SP table), but decided against it as the cabling arrangement would be too complicated. I'd like to use the same DAQ electronics to acquire the data from this test as that would give us the most direct estimate of the sensor noise for supposedly no motion of the spot, although by adding 3 optics between the HeNe and the QPD, we are introducing possible additional jitter couplings...

Quote:

For the OL NB, probably don't have to fudge any seismic noise, since that's a thing we want to suppress. More important is "what the noise would be if the suspended mirrors were no moving w.r.t. inertial space".

For that, we need to look at the data from the OL test setup that Steve is putting on the SP table.

 

Attachment 1: OplevTest.jpg
OplevTest.jpg
  13447   Wed Nov 22 14:47:03 2017 KiraUpdatePEMseismometer can testing

For the insulation, I have decided to use this one (Buna-N/PVC Foam Insulation Sheets). We will need 3 of the 1 inch plain backing ones (9349K4) to wrap a few layers around it. I'll try two layers for now, since the insulation seems to be doing quite well according to initial testing.

Quote:

Updated some values, most importantly, the k-factor. I had assumed that it was in the correct units already, but when converting it to 0.046 W/(m^2*K) from 0.26 BTU/(h*ft^2*F), I got the following plot. The time constant is still a bit larger than what we'd expect, but it's much better with these adjustments.

For our next steps, I will measure the time constant of the heater without any insulation and then decide how many layers of it we will need. I'll need to construct and calibrate a temperature sensor like the ones I've made before and use it to record the values more accurately.

 

 

  13446   Wed Nov 22 12:13:15 2017 KiraUpdatePEMseismometer can testing

Updated some values, most importantly, the k-factor. I had assumed that it was in the correct units already, but when converting it to 0.046 W/(m^2*K) from 0.26 BTU/(h*ft^2*F), I got the following plot. The time constant is still a bit larger than what we'd expect, but it's much better with these adjustments.

For our next steps, I will measure the time constant of the heater without any insulation and then decide how many layers of it we will need. I'll need to construct and calibrate a temperature sensor like the ones I've made before and use it to record the values more accurately.

Quote:

I performed a test with the can last week with one layer of insulation to see how well it worked. First, I soldered two heaters together in series so that the total resistance was 48.6 ohms. I placed the heaters on the sides of the can and secured them. Then I wrapped the sides and top of the can in insulation and sealed the edges with tape, only leavng the handles open. I didn't insulate the bottom. I connected the two ends of the heater directly into the DC source and drove the current as high as possible (around 0.6A). I let the can heat up to a final value of 37.5C, turned off the current and manually measured the temperature, recoding the time every half degree. I then plotted the results, along with a fit. The intersection of the red line with the data marks the time constant and the temperature at which we get the time constant. This came out to be about 1.6 hours, much longer than expected considering that onle one layer instead of four was used. With only one layer, we would expect the time constant to be about 13 min, while for 4 layers it should be 53 min (the area A is 0.74 m^2 and not 2 m^2).

Quote:

I made a model for our seismometer can using actual data so that we know approximately what the time constant should be when we test it out. I used the appendix in Megan Kelley's report to make a relation for the temperature in terms of time.

\frac{dQ}{dt}=mc\frac{dT(t)}{dt} so T(t)=\frac{1}{mc}\int \frac{dQ}{dt}dt=\frac{1}{mc}\int P_{net}dt and P_{net}=P_{in}-P_{out}

In our case, we will heat the can to a certain temerature and wait for it to cool on its own so P_{in}=0

We know that P_{out}=\frac{kA\Delta T}{d} where k is the k-factor of the insulation we are using, A is the area of the surface through which heat is flowing, \Delta T is the change in temperature, d is the thickness of the insulation.

Therefore,

T(t)=\frac{1}{mc}\int_{0}^{t}\frac{kA}{d}[T_{lab}-T(t')]dt'=\frac{kA}{mcd}(T_{lab}t-\int_{0}^{t}T(t')dt')

We can take the derivative of this to get

T'(t)=\frac{kAT_{lab}}{mcd}-\frac{kA}{mcd}T(t), or T'(t)=B-CT(t) 

We can guess the solution to be

T(t)=C_{1}e^{-t/\tau}+C_{2} where tau is the time constant, which we would like to find.

The boundary conditions are T(0)=40 and T(\infty)=T_{lab}=24. I assumed we would heat up the can to 40 celcius while the room temp is about 24. Plugging this into our equations,

C_{1}+C_{2}=40, C_{2}=24, so C_{1}=16

We can plug everything back into the derivative T'(t)

T'(t)=-\frac{16}{\tau}e^{-t/\tau}=B-C[16e^{-t/\tau}+24]

Equating the exponential terms on both sides, we can solve for tau

\frac{16}{\tau}e^{-t/\tau}=16Be^{-t/\tau}, \frac{1}{\tau}=B, \tau=\frac{1}{B}=\frac{mcd}{kA}

Plugging in the values that we have, m = 12.2 kg, c = 500 J/kg*k (stainless steel), d = 0.1 m, k = 0.26 W/(m^2*K), A = 2 m^2, we get that the time constant is 0.326hr. I have attached the plot that I made using these values. I would expect to see something similar to this when I actually do the test.

To set up the experiment, I removed the can (with Steve's help) and will place a few heating pads on the outside and wrap the whole thing in a few layers of insulation to make the total thickness 0.1m. Then, we will attach the heaters to a DC source and heat the can up to 40 celcius. We will wait for it to cool on its own and monitor the temperature to create a plot and find the experimental time constant. Later, we can use the heatng circuit we used for the PSL lab and modify the parts as needed to drive a few amps through the circuit. I calculated that we'd need about 6A to get the can to 50 celcius using the setup we used previously, but we could drive a smaller current by using a higher heater resistance.

 

 

Attachment 1: cooling_fit_1.png
cooling_fit_1.png
  13445   Wed Nov 22 11:51:38 2017 gautamConfigurationComputersnodus post OS migration admin

Confirmed that this crontab is running - the daily backup of the crontab seems to have successfully executed, and there is now a file crontab_nodus.ligo.caltech.edu.20171122080001 in the directory quoted below. The $HOSTNAME seems to be "nodus.ligo.caltech.edu" whereas it was just "nodus", so the file names are a bit longer now, but I guess that's fine...

Quote:

I restored the nodus crontab (copied over from the Nov 17 backup of the same at /opt/rtcds/caltech/c1/scripts/crontab/crontab_nodus.20171117080001. There wasn't a crontab, so I made one using sudo crontab -e.

This crontab is supposed to execute some backup scripts, send pizza emails, check chiara disk usage, and backup the crontab itself.

I've commented out the backup of nodus' /etc and /export for now, while we get back to fully operational nodus (though we also have a backup of /cvs/cds/caltech/nodus_backup on the external LaCie drive), they can be re-enabled by un-commenting the appropriate lines in the crontab.

 

 

  13444   Wed Nov 22 05:41:32 2017 ranaUpdateOptical LeversOplev "noise budget"

For the OL NB, probably don't have to fudge any seismic noise, since that's a thing we want to suppress. More important is "what the noise would be if the suspended mirrors were no moving w.r.t. inertial space".

For that, we need to look at the data from the OL test setup that Steve is putting on the SP table.

ELOG V3.1.3-