40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 150 of 344  Not logged in ELOG logo
ID Date Author Type Category Subjectup
  1891   Wed Aug 12 12:08:16 2009 StephanieUpdateGeneralMultiply Resonant EOM Update

I measured the magnitude of modulation as a function of frequency using the optical spectrum analyzer and an oscilloscope while generating signals using a Marconi signal generator; the results are shown in the attached plot and are compared to the expected modulation given the measured transfer function of the circuit and the nominal modulation index of the EOM used (13 mrad/V). Using the oscilloscope, I found the resonant peaks to be at 11.11 MHz, 29.57 MHz, and 54.70 MHz. There are several different colors on the plot; this is because I had to take the data in several different segments and had to switch to measuring a different sideband partway through the measurment. I also separately found the modulation at each resonant peak for each sideband. The magnitude of modulation was measured  by finding the ratio between the magnitude of the carrier and sideband powers using an oscilloscope, and calculating the magnitude of modulation from this. This method was also used to quantify the dependence of modulation magnitude on input power at each resonant peak; these results are also attached. These same results can also be plotted as modulation magnitude as a function of voltage into the resonant circuit; this is also attached (I'm not sure which is more useful).

In order to produce these results (get the measurements in mrad/V) it was necessary to measure the gain of the amplifier. I used the signal generator to input signals of varying power and measured the output signal voltage using the oscilloscope; I then repeated this process at each resonant frequency. From this I was able to calculate the gain of the amplifier to be 28.1 dB at 11.11 MHz, 27.4 dB at 29.57 MHz, and 25.7 dB at 54.70 MHz. These values are in the same ballpark as the values in the Mini Circuits data sheet (all values are ~25-28 MHz).

  8148   Sat Feb 23 16:16:11 2013 Max HortonUpdateSummary PagesMultiprocessing

Calendars:  The calendar issue discussed previously (http://nodus.ligo.caltech.edu:8080/40m/8098) where the numbers are squished together is very difficult for me to find.  I am not going to worry about it for the time being.

Multiprocessing:   Reviewed the implementation of Multiprocessing in python (using Multiprocessing package).  Wrote a simple test function and ran it on megatron, to verify that multiprocessing could successfully take advantage of megatron’s multiple cores – it could.  Now, I will work on implementing multiprocessing in the program.  I began testing at a section in the program where a for loop calls process_data() (which has a long runtime) multiple times.  The megatron terminals I had open began to run very slowly.  Why?  I believe that the process_data() function loads data into global variables to accomplish its task.  The global variables in the original implementation were cleared before the subsequent calls to process_data().  But in the multiprocessing version, the data is not cleared, meaning the memory fills quickly, which drastically reduces performance.  In the short term, I could try generating fewer processes at a time, wait for them to finish, then clearing the data, then generating more processes, etc.  This will probably generate a nominal performance boost.  In the long-term, restructuring of the way the program handles data may help (but not for sure).  In the coming week I will experiment with these techniques and try to decrease the run time of the program.

  8194   Wed Feb 27 22:46:53 2013 Max HortonUpdateSummary PagesMultiprocessing Implementation

Overview: In order to make the code more maintainable, I need to factor it into different well-documented classes.  To do this carefully and rigorously, I need to run tests every time I make changes to the code.  The runtime of the code is currently quite high, so I will work on improving the runtime of the program before factoring it into classes.  This will be more efficient (minimize testing time) and allow me to factor more quickly.  So, my current goal is to improve runtime as much as possible.

Multiprocessing Implementation:

I invented a simple way to implement multiprocessing in the summary_pages.py file.  Here is an example: in the code, there is a process_data() function, which is run 75 times and takes rather long to run.  I created multiple processes to run these calls concurrently, as follows:

Original Code: (around line 7840)

for sec in datasections:
      for run in run_opts:
        run_opt = 'run_%s_time' % run
        if hasattr(tabs[sec], run_opt) and getattr(tabs[sec], run_opt):

          process_data(cp, ifo, start, end, tabs[sec],\
                       cache=datacache, segcache=segcache, run=run,\
                       veto_def_table=veto_table[run], plots=do['plots'],\
                       subplots=do['subplots'], html_only=do['html_only'])

  #
  # free data memory
  #
          keys = globvar.data.keys()
          for ch in keys:
            del globvar.data[ch]

 

The weakness in this code is that process_data() is called many times, and doesn't take advantage of megatron's multiple threads.  I changed the code to:

Modified Code: (around line 7840)
  import multiprocessing

  if do['dataplot']:
  ... etc... (same as before)       
    if hasattr(tabs[sec], run_opt) and getattr(tabs[sec], run_opt):

          # Create the process
          p = multiprocessing.Process(target=process_data, args=(cp, ifo, start, end, tabs[sec], datacache, segcache, run, veto_table[run], do['plots'], do['subplots'], do['html_only']))
          # Add the process to the list of processes
          plist += [p]

 

Then, I run the process in groups of size "numconcur", as follows:
    numconcur = 8
    curlist = []
    for i in range(len(plist)):
      curlist += [plist[i]]
      if (i % numconcur == (numconcur - 1)):
        for item in curlist:
          item.start()
        for item in curlist:
          item.join()
          item.terminate()
        keys = globvar.data.keys()
        for ch in keys:
          del globvar.data[ch]
        curlist = []

 

The value of numconcur (which defines how many threads megatron will use concurrently to run the program) greatly effects the speed of the program!  With numconcur = 8, the program runs in ~45% of the time of the original code!  This is the optimal value -- megatron has 8 threads.  Several other values were tested - numconcur = 4 and numconcur = 6 had almost the same performance as numconcur = 8, but numconcur = 1 (which is essentially the same as the unmodified code) has a much worse performance.

Improvement Cap:

Why does numcores = 4 have almost the same performance as numcores = 8?  I monitored the available memory of megatron, and it is quickly consumed during these runs.  I believe that once 4 or more cores are being used, the fact that the data can't all fit in megatron's memory (which was entirely filled during these trials) counteracts the usefulness of additional threads.

Summary of Improvements:

Original Runtime of all process_data() statements: (approximate): 8400 sec

Runtime with 8 processes (approximate): 3842 sec

This is about a 55% improvement for speed, in this particular sector (not in the overall performance of the entire program).  It saves about 4600 seconds (~1.3 hours) per run of the program.  Note that these values are approximate (since other processes are running on megatron during my tests.  They might be inflated or deflated by some margin of error).

 

Next Time:

This same optimization method will be applied to all repetitive processes with reasonably large runtimes.

  8195   Wed Feb 27 23:19:54 2013 ranaUpdateSummary PagesMultiprocessing Implementation

  At first I thought that this was goofy, but then I logged in and saw that Megatron only has 8GB of RAM. I guess that used to be cool in the old days, but now is tiny (my laptop has 8 GB of RAM). I'll see if someone around has some free RAM for a 4600; in the meantime, I've killed a MEDM that was running on there and using up a few hundred MB.

Run your ssh-MEDMs elsewhere or else I'll make a cronjob to kill them periodically.

  8201   Thu Feb 28 14:19:20 2013 Max HortonUpdateSummary PagesMultiprocessing Implementation

Okay, more memory would definitely be good.  I don't think I have been using MEDM (which Jamie tells me is the controls interface) so making a cronjob would probably be a good idea.

  8218   Mon Mar 4 10:41:18 2013 Max HortonUpdateSummary PagesMultiprocessing Implementation

Update:

Upon investigation, the other methods besides process_data() take almost no time at all to run, by comparison.  The process_data() method takes roughly 2521 seconds to run using Multiprocessing with eight threads.  After its execution, the rest of the program only takes 120 seconds to run.  So, since I still need to restructure the code, I won't bother adding multiprocessing to these other methods yet, since it won't significantly improve speed (and any improvements might not be compatible with how I restructure the code).  For now, the code is just about as efficient as it can be (in terms of multiprocessing).  Further improvements may or may not be gained when the code is restructured.

  8227   Mon Mar 4 21:05:49 2013 Max HortonUpdateSummary PagesMultiprocessing and Crontab

Multiprocessing:  In its current form, the code uses multiprocessing to the maximal extent possible.  It takes roughly 2600 seconds to run (times may vary depending on what else megatron is running, etc.).  Multiprocessing is only used on the process_data() function calls, because this by far takes the longest.  The other function calls after the process_data() calls take a combined ~120 seconds.  See http://nodus.ligo.caltech.edu:8080/40m/8218 for details on the use of Multiprocessing to call process_data().

Crontab:  I also updated the crontab in an attempt to fix the problem where data is only displayed until 5PM.  Recall that previously (http://nodus.ligo.caltech.edu:8080/40m/8098) I found that the crontab wasn't even calling the summary_pages.py script after 5PM.  I changed it then to be called at 11:59PM, which also didn't work because of the day change after midnight.

I decided it would be easiest to just call the function on the previous day's data at 12:01AM the next day.  So, I changed the crontab.

Previous Crontab:

59 5,11,17,23 * * * /users/public_html/40m-summary/bin/c1_summary_page.sh 2>&1

New Crontab:

0 6,12,18 * * * /users/public_html/40m-summary/bin/c1_summary_page.sh 2>&1
1 0 * * * /users/public_html/40m-summary/bin/c1_summary_page.sh $(date "+%Y/%m/%d" --date="1 days ago") 2>&1

For some reason, as of 9:00PM today (March 4, 2013) I still don't see any data up, even though the change to the crontab was made on February 28.  Even more bizarre is the fact that data is present for March 1-3.  Perhaps some error was introduced into the code somehow, or I don't understand how crontab does its job.  I will look into this now.

Next:

Once I fix the above problem, I will begin refactoring the code into different well-documented classes.

  9626   Wed Feb 12 11:57:55 2014 JenneUpdateLSCMusings on RFPDs

[Rana, Jenne, Manasa]

We looked at the I vs. Q separation in several of the Refl PDs, while driving the PRM, while the  PRMI was locked on sidebands.

For REFL 55, we adjusted the demod phase to try to minimize the peak in the Q signal, and were only able to get it to be about 1/10th the size of the I peak.  This is not good, since it should be more like 1/100, at least.

For both REFL 11 and REFL 165, we were able to get the Q peaks to less than 1/100 of the I peak height. 

We changed the REFL55 phase from 17 to 16, and the REFL165 phase from -160.5 to -162.5. 

Since we believed that we had done a good job of setting the demod phase for REFL165, we used it to also check the balance of BS/PRM for MICH locking.  I drove the BS with an arbitrary number (0.5), which creates a peak in the I phase of REFL165, and then I put in a drive on the PRM and tweaked it around until that peak was minimized.  I came up with the same ratio as Koji had last Friday:  BS=0.5, PRM=-0.2625.  (The old ratio we were using, up until ~December when we started locking MICH with the ITMs, was BS=0.5, PRM=-0.267). 

Also, while we were locked using REFL55 I&Q, we noticed that the other REFL PDs had lots of broadband noise in their I signals, as if some noise in the REFL55 diode is being injected into the PRM, that we are then seeing in the other PDs. 

Some checks that we need to do:

* Inject a calibration line, set all the peak heights equal, and look at the noise floors of each PD.

* Use the calibration line to calibrate the PDs (especially REFL165) into meters, so that we know that it's noise is low enough to hold the PRC through the CARM offset reduction.

* Check out the state of the transmission QPDs - what is their noise, and is it good enough to use for holding the arms after we transition from green beatnote locking?  Does the whitening switching do anything?  What is the state of the whitening?

  5865   Thu Nov 10 19:41:24 2011 JenneUpdateSUSMusings on SUS dewhitening, and MC ELP28's

The following will be a stream-of-consciousness, approximately chronological story of my last hour or so of looking at screens....

In the old OAF days, we used to bypass the analog dewhitening in the coil driver path, using the XYCOMS.  See, ex. elog 2548.

I began to wonder if we needed to do the same thing now.  I checked several optics, to see how the switching works. 

For the whitening on the OSEM sensor input, FM1 is linked to the Contec binary I/O.  FM1 is the inverse whitening filter.  Turn it on, and the analog whitening is on (bit in the binary I/O screen turns red).  Turn it off, and the analog whitening is bypassed (bit in the binary I/O screen turns gray).  Good.  Makes sense.  Either way, the net transfer function is flat.

The dewhitening is not so simple.  In FM9 of the Coil Output filter bank, we have "SimDW", and in FM10, we have "InvDW".  Clicking SimDW on makes the bit in the binary I/O screen gray (off?), while clicking it off makes it red (on?).  Clicking InvDW does nothing to the I/O bits.  So.  I think that for dewhitening, the InvDW is always supposed to be on, and you either have Simulated DW, or analog DW enabled, so that either way your transfer function is flat.  Fine.  I don't know why we don't just tie the analog to the InvDW filter module, and delete the SimDW, but I'm sure there's a reason.

All optics have this setup, except MC1 and MC3.  They don't have the SimDW or InvDW filter modules.  Instead, in FM9 (which on all the other suspensions is SimDW, and controls the binary I/O) there is a 28Hz Elliptic Low Pass filter.  The only thing I can find about these is elog 1405 where Rana talks about implementing ELP28's in MC2.  But right now there is no ELP in the MC2 coil output filters.  So, if Rana's old elog is to be believed, we need to fix up the ELP28 situation.  But that elog was from a long time ago, so maybe things are different now?  If MC1 and MC3 need the SimDW and InvDW (why wouldn't they?) then the ELP28 needs to move to another filter module.  Because right now, when I click the ELP28's on and off, it changes bits in the binary I/O.  Which I don't think it should.  Maybe.  I don't really know.

Okay. So. Now we know where everything is, and which buttons do what.  Maybe not why, but at least what.

In the old world, Rob had lots and lots of trouble (elog 2027) with locking when the analog dewhitening was bypassed.  But right now, I think that all of the analog dewhitening filters are bypassed, for every single optic we have.  So.  Which way do we want things?  What's the new game plan.  What's going on??

\end{stream-of-consciousness}

  5873   Fri Nov 11 13:26:24 2011 jamieUpdateSUSMusings on SUS dewhitening, and MC ELP28's

Quote:

For the whitening on the OSEM sensor input, FM1 is linked to the Contec binary I/O.  FM1 is the inverse whitening filter.  Turn it on, and the analog whitening is on (bit in the binary I/O screen turns red).  Turn it off, and the analog whitening is bypassed (bit in the binary I/O screen turns gray).  Good.  Makes sense.  Either way, the net transfer function is flat.

The dewhitening is not so simple.  In FM9 of the Coil Output filter bank, we have "SimDW", and in FM10, we have "InvDW".  Clicking SimDW on makes the bit in the binary I/O screen gray (off?), while clicking it off makes it red (on?).  Clicking InvDW does nothing to the I/O bits.  So.  I think that for dewhitening, the InvDW is always supposed to be on, and you either have Simulated DW, or analog DW enabled, so that either way your transfer function is flat.  Fine.  I don't know why we don't just tie the analog to the InvDW filter module, and delete the SimDW, but I'm sure there's a reason.

The input/whitening filters used to be in a similarly confused state as the output filters, but they have been corrected.  There might have been a reason for this setup in the past, but it's not what we should be doing now.  The output filters all need to be fixed.  We just haven't gotten to it yet.

As with the inputs, all output filters should be set up so that the full output transfer function is always flat, no matter what state it's in.  The digital anti-dewhitening ("InvDW") and analog dewhitening should always be engaged and disengaged simultaneously.   The "SimDW" should just be removed.

This is on my list of things to do.

  9638   Fri Feb 14 02:33:09 2014 manasaUpdateGeneralMy IFO time summary

MC tuning

Although the morning MC tuning looked stable, Koji pointed out that the MC_REFL_OFFSET was changed from its nominal value.

The offset was reset and this caused drift in the MC_TRANS_SUM.

To fix this:

- disabled the WFS servo

- aligned MC using MC1 and MC3

- centered beam on the MC_REFL

- reset WFS offsets

- locked MC

MC looks happy now.

__________________________________________________________________________________________________________________

ALS locking

ALS is in a very different state from a couple of days ago when we could successfully lock the arms and scan.

The green alignment to the arms had drifted.

PSL green alignment on the PSL table was off. The PSL green was not even on the steering mirror. Did anyone work around the PSL table in the last couple of days?

After aligning and finding the beat note, I found the ALS servo very noisy. The error signal had 10 times more rms noise than what was achieved earlier this week and there were some new 60Hz peaks as well.

Overall, we could not do any PRMI+ALS arms today

 

 

 

 

  1276   Thu Feb 5 21:42:28 2009 YoichiUpdatePSLMy thoughts on ISS

Today, I worked with Kakeru on ISS.

The problem is sort of elusive. Some time, the laser power looks fine, but after a while you may see many sharp drops in the power. Some times, the power drops happen so often that they look almost like an oscillation.

We made several measurements today and Kakeru is now putting the data together. Meanwhile, I will put my speculations on the ISS problem here.

The other day, Kakeru took the transfer function of the ISS feedback filter (he is supposed to post it soon). The filter shape itself has a large phase margin ( more than 50deg ?) at the lower UGF (~3Hz) if we assume the response of the current shunt to be flat. However, when we took the whole open loop transfer function of the ISS loop, the phase margin was only 20deg. This leads to the amplification of the intensity noise around the UGF. The attached plot is the spectrum of the ISS monitor PD. You can see a broad peak around 2.7Hz. In time series, this amplified intensity noise looks like semi-oscillation around this frequency.

Since it is very unlikely that the PD has a large phase advance at low frequencies, the additional phase advance has to be in the current shunt. We measured the response of the current shunt (see Kakeru's coming post). It had a slight high-pass shape below 100Hz (a few dB/dec). This high-pass response produces additional phase advance in the loop.

There seems to be no element to produce such a high-pass response in the current shunt circuit ( http://www.ligo.caltech.edu/docs/D/D040542-A1.pdf )

This Jamie's document shows a similar high-pass response of the current ( http://www.ligo.caltech.edu/docs/G/G030476-00.pdf  page 7 )

Now the question is what causes this high-pass response. Here is my very fishy hypothesis :-)

The PA output depends not only on the pump diode current but also on the mode matching with the NPRO beam, which can be changed by the thermal lensing. If the thermal lensing is in such a condition that an increase in the temperature would reduce the mode matching, then the temperature increase associated with a pump current increase could cancel the power increase. This thermal effect would be bigger at lower frequencies. Therefore, the intensity modulation efficiency decreases at lower frequencies (high-pass behavior). If this model is true, this could explain the elusiveness of the problem, as the cancellation amount depends on the operation point of the PA. 

To test this hypothesis, we can change the pump current level to see if the current shunt response changes. However, the PA current slider on the MEDM screen does not work (Rob told me it's been like this for a while). Also the front panel of the MOPA power supply does not work (Steve told me it's been like this for a while). We tried to connect to the MOPA power supply from a PC through RS-232C port, which did not work neither. We will try to fix the MEDM slider tomorrow.

  8631   Thu May 23 17:51:39 2013 KojiSummaryLSCMy usual locking procedure

For purpose of the automation and my record, I summarized my locking procedure as a chart.

  17085   Wed Aug 17 07:35:48 2022 yutaBureaucracyGeneralMy wish list for IFO commissioning

FPMI related
- Better suspension damping HIGH
 - Investigate ITMX input matrix diagonalization (40m/16931)
 - Output matrix diagonalization
 * FPMI lock is not stable, only lasts a few minutes for so. MICH fringe is too fast; 5-10 fringes/sec in the evening.
- Noise budget HIGH
 - Calibrate error signals (actually already done with sensing matrix measurement 40m/17069)
 - Make a sensitivity curve using error and feedback signals (actuator calibration 40m/16978)
 * See if optical gain and actuation efficiency makes sense. REFL55 error signal amplitude is sensitive to cable connections.
- FPMI locking
 - Use CARM/DARM filters, not XARM/YARM filters
 - Remove FM4 belly
 - Automate lock acquisition procedure
- Initial alignment scheme
 - Investigate which suspension drifts much
 - Scheme compatible with BHD alignment
 * These days, we have to align almost from scratch every morning. Empirically, TT2 seems to recover LO alignment and PR2/3 seems to recover Yarm alignment (40m/17056). Xarm seems to be stable.
- ALS
 - Install alignment PZTs for Yarm
 - Restore ALS CARM and DARM
 * Green seems to be useful also for initial alignment of IR to see if arms drifted or not (40m/17056).
- ASS
 - Suspension output matrix diagonalization to minimize pitch-yaw coupling (current output matrix is pitch-yaw coupled 40m/16915)
 - Balance ITM and ETM actuation first so that ASS loops will be understandable (40m/17014)
- Suspension calibrations
 - Calibrate oplevs
 - Calibrate SUSPOS/PIT/YAW/SIDE signals (40m/16898)
 * We need better understanding of suspension motions. Also good for A2L noise budgeting.
- CARM servo with Common Mode Board
 - Do it with single arm first

BHD related
- Better suspension damping HIGH
 - Invesitage LO2 input matrix diagonalization (40m/16931)
 - Output matrix diagonalization (almost all new suspensions 40m/17073)
 * BHD fringe speed is too fast (~100 fringes/sec?), LO phase locking saturates (40m/17037).
- LO phase locking
 - With better suspensions
 - Measure open loop transfer function
 - Try dither lock with dithering LO or AS with MICH offset (single modulation)
 - Modify c1hpc/c1lsc so that it can modulate BS and do double demodulation, and try double demodulation
- Noise Budget HIGH
 - Calibrate MICH error signal and AS-LO fringe
 - Calibrate LO1, LO2, AS1, AS4 actuation using ITM single bounce - LO fringe
 - Check BHD DCPD signal chain (DCPD making negative output when fringes are too fast; 40m/17067)
 - Make a sensitivity curve using error and feedback signals
- AS-LO mode-matching 
 - Model what could be causing funny LO shape
 - Model if having low mode-matching is bad or not
 * Measured mode-matching of 56% sounds too low to explain with errors in mode-matching telescope (40m/16859, 40m/17067).

IMC related
- WFS loops too fast (40m/17061)
- Noise Budget
- Investigate MC3 damping (40m/17073)
- MC2 length control path

  7156   Mon Aug 13 00:33:06 2012 MashaUpdateGeneralMysterious banging on emergency door

[Masha, Sasha]

Sorry to spam the e-log, but did someone come knock loudly on the emergency exit door a few moments ago? It gave Sasha and I quite a fright, and we are rather worried.

  7157   Mon Aug 13 01:33:55 2012 DenUpdateGeneralMysterious banging on emergency door

Quote:

[Masha, Sasha]

Sorry to spam the e-log, but did someone come knock loudly on the emergency exit door a few moments ago? It gave Sasha and I quite a fright, and we are rather worried.

 Probably, security. You can call 5555 and ask them. Otherwise you can ask them to come and check everything.

  7158   Mon Aug 13 09:59:05 2012 KojiUpdateGeneralMysterious banging on emergency door

You mean 5000?

Quote:

Probably, security. You can call 5555 and ask them. Otherwise you can ask them to come and check everything.

 

  9885   Wed Apr 30 21:31:25 2014 ranaHowToIOOMystery Alignment again

Looks like there was some mysterious MC alignment shift around 5:30 PM today, but no elog.....?? Now things are drifting much more than this morning or yesterday. Who did what and why???

I think I'll blame Jamie since he just got back and did some computer fiber voodoo today.

http://www.lawsome.net/no-throwing-rotten-tomatoes-a-repealed-kentucky-law/

  4333   Mon Feb 21 17:29:57 2011 ranaSummaryIOOMyterious data loss: FB needs investigation

Looks like there was a mysterious loss of data overnight; since there's nothing in the elog I assume that its some kind of terrorism. I'm going to call Rolf to see if he can come in and work all night to help diagnose the issue.

Untitled.png

  4343   Wed Feb 23 10:37:02 2011 josephbSummaryIOOMyterious data loss: FB needs investigation

Friday: 

In addition to the other fixes, Alex rebuilt the daqd process. I failed to elog this. When he rebuilt it, he needed change the symmerticom gps offset in the daqdrc file (located in /opt/rtcds/caltech/c1/target/fb). 

On Friday night, Kiwamu contacted me and let me know the frame builder had core dumped after a seg fault.  I had him temporarily disable the c1ass process (the only thing we changed that day), and then replaced Alex's rebuilt daqd code with the original daqd code and restarted it.  However, I did not change the symmetricom offset at this point.  Finally, I restarted the NDS process.  At that point testpoints and  trends seemed to be working.

Sunday:

The daqd process was restarted sometime on Sunday night (by Valera i believe).  Apparently this restart finally had the symmetricom gps offset kick in (perhaps because it was the first restart after the NDS was restarted?).  So data was being written to a future gps time.

Monday:

Kiwamu had problems with testpoints and trends and contacted me.  I tracked down the gps offset and fixed it, but the original daqd process only started once successfully, after that is was segfault, core dump non-stop. I tried Alex's rebuilt daqd (along with putting the gps offset to the correct value for it), and it worked.  Test points, trends, excitations were checked at the point and found working.

I still do not understand the underlying causes of all these segmentation faults with both the old and new daqd codes.  Alex has suggested some new open mx drivers be installed today.

Quote:

Looks like there was a mysterious loss of data overnight; since there's nothing in the elog I assume that its some kind of terrorism. I'm going to call Rolf to see if he can come in and work all night to help diagnose the issue.


 

  17154   Fri Sep 23 10:05:38 2022 JCUpdateVACN2 Interlocks triggered

[Chub, Anchal, Tega, JC]

After replacing an empty tank this morning, I heard a hissing sound coming from the nozzle area. It turns out that this was from the copper tubing. The tubing we slightly broken and this was comfirmed with soapy water bubbles. This caused the N2 pressure to drop and the Vac interlocks to be triggered. Chub and I went ahead and replaced the fitting in connected this back to normal. Anchal and Tega have used the Vacuum StartUp procedures to restore the vacuum to normal operation.

Adding screenshot as the pressure is decreasing now.

  11678   Fri Oct 9 11:41:09 2015 ericqUpdateVACN2 Pressure fell

At 10:02AM, the N2 Pressure fell below 60 PSI. The watch script saw this happen, but I did not recieve the email it is supposed to send frown

C1:Vac-P1_pressure reads 7e-4, which is the same as it has for the past ~2 days, so the V1 interlock worked fine. 

I've put some fresh N2 into the system, and Bob will pop in over the weekend to check it. I'll stay on top of it until Steve gets back. 

After consulting ELOGs and the 40m wiki, I reasoned it was ok to open the V1 to reconnect the turbo pump to the main IFO volume and VM1 to reconnect the RGA, and have now done so. 

  15354   Tue May 26 10:04:54 2020 JordanUpdateGeneralN2 Replacement

Replaced empty N2 tank, left tank at ~2000 psi, right tank ~2600 psi.

  15403   Tue Jun 16 16:05:26 2020 JordanUpdateGeneralN2 Replacement

I replaced an empty N2 cylinder, there are now two empty tanks in the outside rack.

  12293   Tue Jul 12 18:13:19 2016 KojiUpdateVACN2 bottle replaced

Gautam, Koji

We replaced the right N2 bottle as it was empty.

  15314   Thu Apr 30 07:29:01 2020 ChubUpdateVACN2 delivered.

Hi All,

The new nitrogen cylinders were delivered to the rack at the west entrance.  We only get one Airgas delivery per week during the stay-at-home order, but so far they've not let us down.

  14331   Tue Dec 4 18:24:05 2018 gautamOmnistructureGeneralN2 line disconnected

[jon, gautam]

In the latest installment in this puzzler: turns out that maybe the trend of the "N2 pressure" channel increasing over the ~3 day timescale it takes a cylinder of N2 to run out is real, and is a feature of the way our two N2 cylinder lines/regulators are setup (for the automatic switching between cylinders when one runs out). In order to test this hypothesis, we'd like to have the line pressure be 0 initially, and then just have 1 cylinder hooked up. When we went into the drill-press area, we heard a hiss, turns out that one of the cylinders is leaking (to be fair, this was labelled, but i thought it isn't great to have a higher N2 concentration in an enclosed space). Since we don't need any actuation ability, I valved off the leaky cylinder, and disconnected the other properly functioning one. Attachment #1 shows the current state.

  14333   Thu Dec 6 17:33:33 2018 JonOmnistructureGeneralN2 line disconnected

I believe I finally have the N2 gauge working correctly. The wiring is unchanged from its original state and the controller has been recalibrated.

After letting the line pressure drop to 0 PSI as indicated by the analog gauge in the drill-press room, I recorded the number of counts read by the Omega controller. Then I pressurized the line to 80 PSI, again indicated by the analog gauge, and recorded the Omega counts again. I entered these two reference points into the controller (automatically determines the gain and offset from these), then confirmed the readings to agree with the anaog gauge as I varied the line pressure.

The two reference points are:

0 PSI  :  13 counts
80 PSI : 972 counts

 

Quote:

[jon, gautam]

In the latest installment in this puzzler: turns out that maybe the trend of the "N2 pressure" channel increasing over the ~3 day timescale it takes a cylinder of N2 to run out is real, and is a feature of the way our two N2 cylinder lines/regulators are setup (for the automatic switching between cylinders when one runs out). In order to test this hypothesis, we'd like to have the line pressure be 0 initially, and then just have 1 cylinder hooked up. 

 

  1212   Thu Jan 1 01:15:45 2009 YoichiSummaryVACN2 line leak ?
I've been replacing the N2 bottles recently.
I noticed that the consumption is too high. I had to replace them every two days.
Normally the interval is three or more days.
I suspect there is some leak in the line.

Strangely, it is always the left hand bottle which goes empty. The right hand bottle has been
there for more than a week at about 1000 psi.

We should check it when Steve is back.
  14377   Fri Dec 21 11:13:13 2018 gautamOmnistructureVACN2 line valved off

Per the discussion yesterday, I valved off the N2 line in the drill press room at 11 am PST today morning so as to avoid any accidental software induced gate-valve actuation during the holidays. The line pressure is steadily dropping...

Attachment #1 shows that while the main volume pressure was stable overnight, the the pumpspool pressure has been steadily rising. I think this is to be expected as the turbo pumps aren't running and the valves can't preserve the <1mtorr pressure over long timescales?

Attachment #2 shows the current VacOverview MEDM screen status.

  14379   Fri Dec 21 12:57:10 2018 KojiOmnistructureVACN2 line valved off

Independent question: Are all the turbo forelines vented automatically? We manually did it for the main roughing line.

 

  14383   Fri Jan 4 10:25:19 2019 JonOmnistructureVACN2 line valved off

Yes, for TP2 and TP3. They both have a small vent valve that opens automatically on shutdown.

Quote:

Independent question: Are all the turbo forelines vented automatically? We manually did it for the main roughing line.

 

 

  11260   Mon Apr 27 14:37:55 2015 SteveUpdateVACN2 pneumatic pressure watch

 

Quote:

Based on Jenne's chiara disk usage monitoring script, I made a script that checks the N2 pressure, which will send an email to myself, Jenne, Rana, Koji, and Steve, should the pressure fall below 60psi. I also updated the chiara disk checking script to work on the new Nodus setup. I tested the two, only emailing myself, and they appear to work as expected. 

The scripts are committed to the svn. Nodus' crontab now includes these two scripts, as well as the crontab backup script. (It occurs to me that the crontab backup script could be a little smarter, only backing it up if a change is made, but the archive is only a few MB, so it's probably not so important...)

This watch script gives little time to replace N2 cylinder. When the regulated supply drops below 60 psi the cylinder pressure is 60 psi too.

It is more of a statement that V1 is closed and act accordingly. It's only practical if you are in the lab.

Rana pointed it out correctly that we need this message 24 hrs before it happens. This requires monitoring  of total supplies , not the regulated one.

So we need pressure transducers on each nitrogen cylinder, before the regulator. The sum of the two N2 cylinder when they are full 4000 - 4500 psi

The first email should be send out at 1000 psi as sum of the two cylinders. This means that you have  1 day to replace nitrogen cylinder.

 Most of the time the daily consumption is 750 +-50 psi   

However sometimes  this variation goes up   ~750 +-150 psi

  11264   Thu Apr 30 16:30:25 2015 SteveUpdateVACN2 pneumatic pressure watch set up

We have 2 transduser PX303-3KG5V   http://www.omega.com/pressure/pdf/PX303.pdf They  will be installed on the out put of the N2 cylinders to read the supply pressure.

I will order one DC power supply  http://www.omega.com/pptst/PSU93_FPW15.html      PSU-93

One full cylinder pressure is ~ 2400 PSI max so two of them will give us ~9Vdc

The email reminder should be send at 1000 PSI  =  1.8 V

 

 

 

 

  14325   Fri Nov 30 15:53:52 2018 JonOmnistructureGeneralN2 pressure gauge fix

I've made a repair to the N2 pressure monitor. I don't believe the polarity of the analog signal into the controller actually was reversed. I found the data sheet (attached) for the transducer model we have installed. Its voltage should read ~0 at 0 PSI and 100mV at 100 PSI. As wired, the input voltage reads +80 mV as it should.

The controller calibrates the sensor voltage to PSI (i.e., applies a scale and offset) based on two settable reference points which appeared to be incorrect. I changed them to:

  1. 0 mV = 0 PSI (neglecting the small dark bias)
  2. 100 mV = 100 PSI

After the change, the pressure reads 80 PSI. Let's see if the time history now shows a sensible trend. 

Quote:

[koji, gautam, jon, steve]

  • We suspect analog voltage from N2 pressure gauge is connected to interfacing Omega controller with the 'wrong' polarity (i.e pressure is rising over ~4 days and then rapidly falling instead of the other way around). This should be fixed.

 

  12195   Fri Jun 17 15:22:31 2016 gautamUpdateVACN2 supply line restored after retiling
Quote:

The drill room floor will be retiled Thursday, June 16. Temporary nitrogen line set up will allow emptying the hole area.

 

Ifo room entry will be through control room.

 

The retiling work has finished, Steve and I restored the N2 supply configuration to its normal state. The sequence of steps followed was:

  1. Went to the X end and closed the following valves, roughly in this order: VAEE, VAEV, VABS, VABSCI, VASV, VASE, V4, V1.
  2. Checked the RPM on the various turbo pump controllers to make sure they were in their nominal states
  3. Disconnect the electrical connections to V1, V4, V5 and VA6 - just to make sure some spurious signal doesn't unintentionally open any of these valves while we are mucking around with the N2 supply
  4. Close the valves on the N2 cylinders in the drill room. Disconnect the temporary nitrogen line (at this point, the N2 pressure to the IFO valves goes down from ~7-PSI to 0), reconnect the old supply chain, taking care that we aren't unintentionally loosening any of the Swagelock connections while unscrewing stuff
  5. Replaced one of the N2 cylinders that was running low.
  6. Reopen the cylinders, restore N2 pressure to IFO valves to ~70PSI.
  7. Do steps 1-3 in reverse: i.e. reconnect power to all valves, open them in the reverse order we closed them while monitoring the state of the various turbo pumps. 
  8. Acknowledged the error message on the C0VAC medm screen

Note: the valve isolating the RGA automatically shutoff during this work, possibly because it detected a pressure above its threshold - after checking the appropriate pressure gauges, we reopened this valve as well. 

The attached screenshot suggests that everything went as planned and that the vacuum system is back to normal...

 

 

  1226   Wed Jan 14 09:22:43 2009 steveHowToVACN2 supply pressure lowered for vac valves

Quote:
I've been replacing the N2 bottles recently.
I noticed that the consumption is too high. I had to replace them every two days.
Normally the interval is three or more days.
I suspect there is some leak in the line.

Strangely, it is always the left hand bottle which goes empty. The right hand bottle has been
there for more than a week at about 1000 psi.

We should check it when Steve is back.



All vac valves operated by N2
The two cylinders are located in entry room 103
There is an auto switch over valve between them to insure continuous supply.
The pressure regulator should be set 70 PSI on the gauge
This pressure we keep constant.
All vac valves will close in case of running out of N2 or losing ac power so
it is essential that one replaces empty N2 cylinder.

Simply disconnect large CGA580 fitting with a crescent wrench.
Swap in full cylinder from outside, reconnect fitting tightly and open cylinder valve.
Now you should be reading the full cylinder pressure.
Write each cylinder pressure and date-time on the board so one can see if there is a leak
  10245   Mon Jul 21 10:51:06 2014 SteveUpdateVACN2 supply run out

Interlock closed valve V1, V4, V5 and VM1 when the nitrogen supply run out. The IFO pressure rose to P1 1 mTorr

In order to recover Vacuum Normal valve configuration I did the following:

Replaced both nitrogen cylinders. Confirmed pneumatic nitrogen pressure 70 PSI.   Opened valves V4 and V5

At P2 < 1 mTorr, Maglev rotation 560 Hz , V1 was opened.

VM1 was opened when CC1 pressure dropped below < 1e-5 torr

 

Please  take a look at the N2 cylinders pressure on Friday to insure that there is enough for the week end.

The daily consumption is 600-700 PSI

  542   Wed Jun 18 18:32:09 2008 Max JonesUpdateComputer Scripts / ProgramsNB Update
I am reconfiguring the the noisebudget code currently in use at the sights. To that end, I have done the following things (in addition to the elog I posted earlier)

In get_dtt_dataset.m - I added C1 specific cases for DARM_CTRL, SEIS, and ITMTRX changing the specific channels to make those in use at caltech

In LocalizeSite.m - I changed the NDS_PATH to match caltechs. I left NDS_HOST untouched.

Since I am trying to get SEIS and DARM to work initially I added C1 specific cases to both of these.

Better documentation may be found in /users/mjones/DailyProgressReport/06_18_08. Suggestions are appreciated. Max.
  12929   Wed Apr 5 16:05:47 2017 gautamUpdateGeneralNB code checkout

[evan, gautam]

We spent some time trying to get the noise-budgeting code running today. I guess eventually we want this to be usable on the workstations so we cloned the git repo into /ligo/svncommon. The main objective was to see if we had all the dependencies for getting this code running already installed. The way Evan has set the code up is with a bunch of dictionaries for each of the noise curves we are interested in - so we just commented out everything that required real IFO data. We also commented out all the gwpy stuff, since (if I remember right) we want to be using nds2 to get the data. 

Running the code with just the gwinc curves produces the plots it is supposed to, so it looks like we have all the dependencies required. It now remains to integrate actual IFO data, I will try and set up the infrastructure for this using the archived frame data from the 2016 DRFPMI locks..

  13097   Wed Jul 5 19:10:36 2017 gautamUpdateGeneralNB code checkout - updated

I've been making NBs on my laptop, thought I would get the copy under version control up-to-date since I've been negligent in doing so.

The code resides in /ligo/svncommon/NoiseBudget, which as a whole is a git directory. For neatness, most of Evan's original code has been put into the sub-directory  /ligo/svncommon/NoiseBudget/H1NB/, while my 40m NB specific adaptations of them are in the sub-directory /ligo/svncommon/NoiseBudget/NB40. So to make a 40m noise budget, you would have to clone and edit the parameter file accordingly, and run python C1NB.py C1NB_2017_04_30.py for example. I've tested that it works in its current form. I had to install a font package in order to make the code run (with sudo apt-get install tex-gyre ), and also had to comment out calls to GwPy (it kept throwing up an error related to the package "lal", I opted against trying to debug this problem as I am using nds2 instead of GwPy to get the time series data anyways).

There are a few things I'd like to implement in the NB like sub-budgets, I will make a tagged commit once it is in a slightly neater state. But the existing infrastructure should allow making of NBs from the control room workstations now.

Quote:

[evan, gautam]

We spent some time trying to get the noise-budgeting code running today. I guess eventually we want this to be usable on the workstations so we cloned the git repo into /ligo/svncommon. The main objective was to see if we had all the dependencies for getting this code running already installed. The way Evan has set the code up is with a bunch of dictionaries for each of the noise curves we are interested in - so we just commented out everything that required real IFO data. We also commented out all the gwpy stuff, since (if I remember right) we want to be using nds2 to get the data. 

Running the code with just the gwinc curves produces the plots it is supposed to, so it looks like we have all the dependencies required. It now remains to integrate actual IFO data, I will try and set up the infrastructure for this using the archived frame data from the 2016 DRFPMI locks..

 

  2976   Mon May 24 16:34:22 2010 KevinUpdatePSLND Filters for 2W Beam Profile

I tried to measure the beam profile of the 2W laser today but ran into problems with the ND filters. With the measurements I made a few weeks ago, I used a reflective ND 4.0 filter on the PD. The PD started to saturate and Koji and I noticed that a lot of the metallic coating on the filter had been burnt off. Koji told me to use an absorptive ND 4.0 filter in front of a reflective ND 0.6 filter. I tried this today but noticed that a few holes were being burned into the absorptive filter and that the coating on the reflective filter behind it was also being burned off in spots. I don't think we wanted to use a polarizing beam splitter to reduce the power before the PD but I didn't want to ruin any more filters.

  14352   Thu Dec 13 18:12:47 2018 gautamUpdateIOOND filter on AS camera changed

In order to see the AS beam a bit more clearly in our low-power config, I swapped out the ND=1.0 filter on the AS camera for ND=0.5.

  6360   Mon Mar 5 23:47:15 2012 keikoConfigurationLSCND2 at REFL OSA

ND filter ND3 (which is at the REFL port to the REFL OSA) is removed. Don't forget to put it back when you restore PRM!!!

  3716   Thu Oct 14 12:45:47 2010 josephbUpdateComputersNDS 2 server installation on mafalda

[Joe, John Zweizig]

John stopped by around noon today to install the NDS 2 server.  He installed it /cvs/cds/caltech/users/jzweizig/nds2-server/.

Once John is done, I will be moving this to a more sensible install location that is not his user directory, but its there for the moment.

We had to install a couple more packages including bzip2, bzip2-devel, gcc-c++, openssl, and openssl-devel. 

We mounted the /frames directory from the fb machine to mafalda by modifying the /etc/fstab file with the line:

fb:/frames              /frames                 nfs     bg,ro           0 0

 

If we change channels recorded by the frame builder, we need to update a channel list file for the NDS 2 server.  There's an excutable located at:

/cvs/cds/caltech/users/jzweizig/nds2-server/bin/buildChannelList

This builds the file list if given a .gwf file.  These are written by the frame builder, and can be found in /frames/full/####, where #### are the first 4 gps digits of the gravity files contained in that directory.

Upon questing about when we get to GPS time 1000000000, he said there's some updates he needs to do so it rather throws away the last 5 digits, rather than keeping the first 4.

An example command run on the fb or mafalda machine is:

/cvs/cds/caltech/users/jzweizig/nds2-server/bin/buildChannelList  /frames/full/9711/C-R-971119728-16.gwf > nds2-mafalda/C-R-ChanList.txt

For a seconds trends file (located in /frames/trends/second/ instead of /frames/full)

/cvs/cds/caltech/users/jzweizig/nds2-server/bin/buildChannelList  /frames/trends/second/9711/ C-T-971106780-60.gwf > nds2-mafalda/C-T-ChanList.txt

For a minute trends file (located in /frames/trends/minute)

/cvs/cds/caltech/users/jzweizig/nds2-server/bin/buildChannelList  /frames/trends/minute/9711/C-T-971106780-60.gwf  > nds2-mafalda/C-M-ChanList.txt

In these cases, John was putting the lists in the /cvs/cds/caltech/users/jzweizig/nds2-mafalda/ directory.

 

Both the  C-raw-cache.txt file and the nds2.conf files need to be configured to point at the correct files in the nds2-mafalda directory.

 

  3720   Thu Oct 14 14:09:30 2010 josephbUpdateCDSNDS 2 server status

[Joe, John]

The nds 2 server is about 50% of the way there.  You can connect to it and get channel lists, but its having issues actually serving the data.  The errors we're getting basically say it can't find the source data for the channel

John had to go get lunch at 2pm, but he said he'd log in remotely later and try to configure it properly.

  14104   Wed Jul 25 22:46:15 2018 gautamConfigurationComputersNDS access from outside

After this work, I've been having some trouble getting data with Python NDS. Eventually, I figured out that the nds connection request has to be pointed at '131.215.115.200' (the address of the NAT router which faces the outside world), port 31200 (it used to work with 'nds40.ligo.caltech.edu' or '131.215.115.189'). So the following snippet in python allows a connection to be opened. Offline access of frame data via NDS2 now seems possible.

import nds2
conn = nds2.connection('131.215.115.200',31200)
Quote:
 

So far, ssh (22), web services (30889), and elog (8081, 8080) were tested. We also need to test megatron NDS port forwarding and rsync for nodus, too.Finally I turned off the firewall rules of shorewall on nodus as it is no longer necessary.

  1485   Wed Apr 15 03:52:27 2009 ranaUpdateDMFNDS client32 updated for DMF
 Since our seisBLRMS.m complains about 'can't find hostname' after a few hours, even though matlab is able to ping fb40m, 
I have recompiled the NDS mex client for 32-bit linux on mafalda and stuck it into the nds_mexs/ directory. This time I 
 

 compiled using the 'gcc' compiler instead of the 'ANSI C' compiler that is recommended in the README (which, I notice,

 

 is now missing from Ben Johnsons web page!). Let's see how long this runs.

  5847   Wed Nov 9 13:44:04 2011 MirkoUpdateComputersNDS1 missing channels in matlab

The Matlab NDS1 access seems to only work for some channels. With other channels it just hangs 'Busy' and does nothing.
Below you can see some channels that make matlab hang. Everyting happened on allegra. I tried compiling the NDS1 sources (from https://www.gravity.phy.syr.edu/dokuwiki/doku.php?id=ligodv:nds1_ligodv_install ) into mex files myself. Same result. I
 

a=NDS_GetChannels('fb:8088'); %/cvs/cds/caltech/apps/linux64/share/matlab/NDS_GetChannels.m
%data=NDS_GetData({'C1:IOO-MC_F_DQ'},1004826500,100,'fb:8088',a)     %Works
%data=NDS_GetData({'C1:IOO-WFS1_PIT_IN1_DQ'},1004826500,100,'fb:8088',a)     %Works
data=NDS_GetData({'C1:LSC-AS11_I_OUT'},1004826500,100,'fb:8088',a)         %Doesn't work, hangs
%%%which NDS_GetData.m: /cvs/cds/caltech/apps/linux64/share/matlab/NDS_GetData.m

  3657   Wed Oct 6 00:32:01 2010 ranaSummaryDAQNDS2

This is the link to the NDS2 webpage:

https://www.lsc-group.phys.uwm.edu/daswg/wiki/NetworkDataServer2

We should install this so that we can use this modern interface to get 40m data from outside and inside of the 40m.

ELOG V3.1.3-