ID |
Date |
Author |
Type |
Category |
Subject |
1304
|
Sat Feb 14 16:53:26 2009 |
rob | Update | LSC | Locking status |
Quote: | Yoichi, Jenne, Alberto, Rob
Last night, the locking proceeded until the CARM -> MC_L hand-off.
However, the MC_F gets saturated (as expected) and the IFO loses lock soon after the hand-off.
So we need to offload MC_F.
We ran the offloadMCF script, but it did not work, i.e. just waiting for CARM mode.
Looks like an EPICS flag is not set right. |
I found a '$<' in the offloadMCF script. I don't know precisely what that construct means, but I think it caused the script to wait for input when it shouldn't. It probably got in there accidentally. We need to be careful when we're opening scripts just to look at how they work that we don't accidentally change them. I like to use the command 'less' for this purpose.
With this gone, the script worked properly, although the lock didn't last long. I don't know if the next stage in the process is failing or if it's just a bit too noisy in the afternoon. I didn't get a chance to do much testing since the sus controller (susvme1) went nuts. In retrospect, this could be due to something in the script, so maybe we should try a burt restore to Friday afternoon next time someone wants to look at it. |
1305
|
Sun Feb 15 09:35:00 2009 |
Yoichi | Update | LSC | Locking status |
Quote: |
I found a '$<' in the offloadMCF script. I don't know precisely what that construct means, but I think it caused the script to wait for input when it shouldn't.
|
'$<' acts like 'read' in csh. I might have put it in the offloadMCF script to debug the behavior of the script.
Sorry I probably forgot to remove it from the script when I left. |
1306
|
Sun Feb 15 15:53:21 2009 |
Rob | Update | LSC | Locking status |
Quote: |
I didn't get a chance to do much testing since the sus controller (susvme1) went nuts. In retrospect, this could be due to something in the script, so maybe we should try a burt restore to Friday afternoon next time someone wants to look at it. |
I tried the burt restore today, it didn't work. Also tried some switching of timing cables, and multiple reboots, to no avail. This will require some more debugging. We might try diagnosing the clock driver and fanout modules, the penteks, and we can also try rebooting the whole FE system. |
1307
|
Mon Feb 16 00:43:46 2009 |
rana | Update | Computers | medm directory wiped on nodus |
I accidentally did an 'rm -rf' on the medm directory in nodus, instead of on my laptop as was intended.
I then did an svn checkout. So everything should be current as of the last update, but I am sure that
we have not done a checkin on all of the latest screen enhancements. So...we may have to revert to the
Sunday morning tar to get the latest changes back. |
1308
|
Mon Feb 16 10:18:13 2009 |
Alberto | Update | LSC | FE system rebooted |
Quote: |
Quote: |
I didn't get a chance to do much testing since the sus controller (susvme1) went nuts. In retrospect, this could be due to something in the script, so maybe we should try a burt restore to Friday afternoon next time someone wants to look at it. |
I tried the burtrestore today, it didn't work. Also tried some switching of timing cables, and multiple reboots, to no avail. This will require some more debugging. We might try diagnosing the clock driver and fanout modules, the penteks, and we can also try rebooting the whole FE system. |
I rebooted the whole FE system and now c1susvme1 and c1susvme2 are back on.
I can't restart the MC autolocker on c1susvme2 because it doesn't let me ssh in. I tried to reboot it a few times but it didn't work. Once you restart it, it becomes inaccessible and doesn't even respond to pinging. Although the controls for the MC mirrors are on.
The mode cleaner stays unlocked. |
1309
|
Mon Feb 16 14:12:21 2009 |
Yoichi | Update | LSC | FE system rebooted |
Quote: |
I can't restart the MC autolocker on c1susvme2 because it doesn't let me ssh in. I tried to reboot it a few times but it didn't work. Once you restart it, it becomes inaccessible and doesn't even respond to pinging. Although the controls for the MC mirrors are on.
The mode cleaner stays unlocked. |
MC autolocker runs on op340m, not on c1susvme2.
I restarted it and now MC locks fine.
Before that, I had to reboot c1iool0 and restore the alignment of the MC mirrors (for some reason, burt did not restore the alignment properly, so I used conlog). |
1310
|
Mon Feb 16 15:54:07 2009 |
Yoichi | Update | Computers | medm directory wiped on nodus |
Quote: | I accidentally did an 'rm -rf' on the medm directory in nodus, instead of on my laptop as was intended.
I then did an svn checkout. So everything should be current as of the last update, but I am sure that
we have not done a checkin on all of the latest screen enhancements. So...we may have to revert to the
Sunday morning tar to get the latest changes back. |
Indeed, some changes to the medm directory I made were lost.
It was my fault not to check-in those changes.
I asked Alan to restore the directory from the daily rsync backup.
However, the backup job executed this morning have already overwritten the previous (good) backup with the current (bad) medm directory, which Rana restored from the svn. Alan will ask Stuart and Phil if there is still older backup remaining somewhere.
Anyway, I realized that we should stop the backup cron job whenever you think you made a mistake on /cvs/cds/ directory to prevent unwanted overwriting.
The procedure is:
(1) Login to fb40m
(2) Type 'crontab -e'. Emacs will open up in the terminal.
(3) Comment out the backup job (insert # at the beginning of the line containing /cvs/cds/caltech/scripts/backup/rsync.backup ).
(4) Save the file (Ctrl-x Ctrl-s) and exit (Ctrl-x Ctrl-c).
I will post this information on the wiki. |
1311
|
Mon Feb 16 16:26:29 2009 |
rob | Update | Computers | medm directory wiped on nodus |
Quote: |
Quote: | I accidentally did an 'rm -rf' on the medm directory in nodus, instead of on my laptop as was intended.
I then did an svn checkout. So everything should be current as of the last update, but I am sure that
we have not done a checkin on all of the latest screen enhancements. So...we may have to revert to the
Sunday morning tar to get the latest changes back. |
Indeed, some changes to the medm directory I made were lost.
It was my fault not to check-in those changes.
I asked Alan to restore the directory from the daily rsync backup.
However, the backup job executed this morning have already overwritten the previous (good) backup with the current (bad) medm directory, which Rana restored from the svn. Alan will ask Stuart and Phil if there is still older backup remaining somewhere.
Anyway, I realized that we should stop the backup cron job whenever you think you made a mistake on /cvs/cds/ directory to prevent unwanted overwriting.
The procedure is:
(1) Login to fb40m
(2) Type 'crontab -e'. Emacs will open up in the terminal.
(3) Comment out the backup job (insert # at the beginning of the line containing /cvs/cds/caltech/scripts/backup/rsync.backup ).
(4) Save the file (Ctrl-x Ctrl-s) and exit (Ctrl-x Ctrl-c).
I will post this information on the wiki. |
We should change the rsync script so that it does not delete stuff. Maybe it can keep deleted stuff for 6 months or something. |
1313
|
Mon Feb 16 21:49:06 2009 |
Kakeru, Rana | Update | IOO | WFS |
We centerd the input of WFS QPD. |
1315
|
Mon Feb 16 23:09:52 2009 |
rana | Update | Electronics | MC Servo Board offset gone bad! |
The attached plot shows that someone broke the MC_SUM_MON channel around 10:30 AM this past Wednesday the 11th. This is the EPICS monitor of the MC error point.
Come forward now with your confession and I promise that I won't let Steve hurt you. |
1316
|
Tue Feb 17 05:20:11 2009 |
Yoichi | Update | LSC | Locking |
Since we excluded *.snap and *.req files from the svn control in the medm directory and these were not restored by the svn co, the burt part of the align/mis-align scripts were not working correctly this evening. So I recreated .req files and cooked up some mis-aligned .snap files.
After some cut-and-try work, I was able to run the dither alignment scripts fine.
Due to the above mentioned delay, the locking work started around midnight.
Tonight, the DD hand-off was not robust. I spent sometime to optimize this.
After the optimization, the locking proceeded to the DC CARM/DARM control state stably.
The CARM->MCL hand-off failed because the LSC-MC offset button was off.
I added a line to turn on the button in the ontoMCL script.
Today, the offloadMCF script worked fine.
Next, the cm_step script stumbled on the "ENGAGERIZING" of the AO path.
I got a hunch that the AO path might not be connected to the MC board.
Indeed, OMC_OSC_FM was connected to the IN2 of the MC board. Looks like it was used for the optimization of the modulation frequencies.
Probably I had the hunch because I did it 
I was able to increase the arm power up to 3.9.
The script failed when it tried to switch the CARM signal from TR_DC to SPOB_DC.
I haven't tackled on this issue yet. |
1317
|
Wed Feb 18 03:17:40 2009 |
Yoichi | Update | LSC | Locking |
Yoichi, Kakeru,
Last night, the cm_step script failed at the hand-off of CARM error signal from TR_DC to PO_DC.
This was fixed by reducing the PO_DC gain by a factor of 2.
Currently the script fails when changing C1:LSC-DEMOD_GAIN to zero.
To be honest, I don't fully understand the purpose of this step. |
1318
|
Wed Feb 18 03:25:25 2009 |
Yoichi | Update | Computers | medm directory back |
I restored the medm directory from the backup on the tape.
The directory had an svn property svn:ignore set and the value of the property included *.snap and *.req.
This resulted in the exclusion of those files from the repository.
I fixed this problem by changing the property of all the directories under /cvs/cds/caltech/medm.
After fixing several other svn problems, the current medm directory contents were checked in to the repository. |
1321
|
Wed Feb 18 21:03:22 2009 |
rana | Update | Cameras | ETMY Camera work not elogged! |
The control room video is showing us a false ETMY image. Who worked on the ETMY camera or video today??!! |
1322
|
Wed Feb 18 21:10:21 2009 |
rana | Update | IOO | MC Drumhead mode lost again |
In early December, Caryn and I noticed that the MC Drumhead mode was visible at the Qmon point of
the MC demod board using a spectrum analyzer and no external excitation of the MC mirrors. We then
started tracking the MC Drumhead modes.
Today I found that it is gone again. It also wasn't there when I looked for it in 2007. 
I looked at the MC error point spectrum and it seemed reasonable. Changing the gains in the MZ, ISS, PMC, & FSS
had no good effect on the noise spectrum.
The voltage noise above 10 kHz in the MC error point is increasing like ~f. I think that this means that
the leftover is the noise from the FSS. Below 10 kHz it is the noise of the VCO (10 mHz/rHz).
One possibility is that the high frequency noise changes with the mood of the NPRO. There should be no
frequency noise induced by the decay of the PA diode power. We can do an NPRO SLOW scan to see if there
is some kind of mode hop noise happening. |
1323
|
Thu Feb 19 04:16:17 2009 |
Yoichi | Update | LSC | Locking status |
Rob, Yoichi
We checked the CM-MC cross over just before turning off the moving zero.
There was a slight bump in the gain of the MC_L loop at (I believe) the optical spring freq. (~400Hz) just below 0 dB. The phase margin there was very thin.
Removing the moving zero will increase the bump more and make the loop unstable.
Rob suggested to increase the AO gain a bit more.
To see if the AO path is really working, I connected the OUT2 of the MC board to a spare DAQ channel (C1:PEM-OSA_APTEMP).
I confirmed that the PO_DC signal is actually coming to the AO path input of the MC board.
I also hooked up the SR785 to the A excitation channel of the common mode board, so that we can measure the loop gain of the AO path.
After these preparation, the lock acquisition process became somewhat unstable. The ifo loses lock randomly at various places in the lock acquisition steps.
So, as of 4:00 am, I have not gotten a chance to try Rob's suggestion nor the TF measurement with SR785 yet.
I will continue the work tomorrow (i.e. tonight ??).
|
1324
|
Thu Feb 19 11:51:56 2009 |
steve | Update | MOPA | HTEMP variation is too much |
The C1:PSL-MOPA_HTEMP variation is more than 0.5 C daily
Normally this temp stays well within 0.1 C
This 80 days plots shows that we have just entered this unstable region some days ago.
The control room temp set unchanged at 70 F, actual temp at ac-mon 69-70 with occasional peaks at 74 F
Water temp at chiller repeatedly around 20.6 C at 8 am
This should be rock solid at 20.00C +- 0.02C
|
1325
|
Thu Feb 19 16:29:43 2009 |
Yoichi | Update | Computers | Martian wireless router bad |
The Martian wireless router is dead.
I rebooted it several times, but it hangs up in a minute.
I will ask steve to buy a new one. |
1326
|
Thu Feb 19 22:40:33 2009 |
Kiwamu | Update | Electronics | PSL angle QPD |
I checked a broken QPD, which was placed for PSL angle monitor, and finally I cocluded one segment of the quadrant diode was broken.
The broken segment has a offset voltage of -0.7V after 1st I-V amplifier. It means the diode segment has a current offset without any injection of light.
Tomorrow I will check a new QPD for replacement.
Kiwamu IZUMI
|
1327
|
Thu Feb 19 23:50:31 2009 |
pete | Update | Locking | aligned pd's on AP table |
Yoichi, Peter
While continuing our efforts to lock, we noticed the procedure failed at a point it had gotten past last night: turning on the bounce/roll filters in MICH, PRC, and SRC. We checked the MICH transfer function and noticed that the unity gain point was ~10 Hz, well below the bounce modes. We tried increasing the gain but found saturation, and Rob suggested that there could be misalignment on the AP table, which Steve worked on today. We went out and found two of the PDs (ASDD133 and AS166) to be badly misaligned probably due to a bumped optic upstream. We re-aligned.
|
1328
|
Fri Feb 20 01:54:18 2009 |
Kakeru | Update | Computer Scripts / Programs | tdsdata might have a bug |
I found a strange jump of value in my data taken with tdsdata.
I couldn't find same jump in a playback of DataViewer, so I think this is a problem of tdsdata.
Be careful when you use tdsdata!
The attached file is an example of jumped data.
I try to get data with allegra and op440m, and both has same kind of jump.
(A downsampling or interpolation may be wrong.)
Rana said there is a fixed version of tdsdata in some PC, but 64bit linux may not have.
I try it tomorrow. |
1329
|
Fri Feb 20 03:52:23 2009 |
Yoichi | Update | Locking | Locking Tonight |
Yoichi, Peter
Tonight, we had a problem with the DD hand off.
It failed when the RG filters of MICH for the bounce-roll modes are engaged.
The reason for the failure was that the MICH UGF was too low (~10Hz).
As in the Peter's elog entry, we found that the AS PDs are mis-centered.
Even after we fixed the centering, the MICH UGF was still too low. So we increased the MICH feedback gain by a factor of 10.
The reason for the gain decrease is unknown. It seems almost like the BS coils get weaker.
I checked the UGF of the BS OL loops. These are around 4Hz, so fine. We should check the HWP on the AP table tomorrow.
After the DD hand-off goes ok, the switching of DARM signal from DC to RF failed.
I found that the gain and the polarity of the RF signal were wrong.
AS166 is one of the PDs we found mis-centered (and re-centered). But how can you flip the sign of the signal ?
After this, the cm_step script goes until the activation of the moving zero, but fails when the arm power is increased to 0.7.
Also the ontoMCL script succeeds only 50% of the time. |
1330
|
Fri Feb 20 19:31:16 2009 |
Yoichi | Update | LSC | MICH low gain problem |
Last night, we found that MICH UGF was too low. Even after re-aligning the PDs, it was still too low.
Today, I compared the UGFs of MICH and PRC when in the DRMI configuration locked with the single demod. signals.
In this configuration, MICH signal comes from REFL33Q and the PRC signal comes from REFL33I (the same PD).
The PRC UGF was about 100Hz whereas MICH was only ~10Hz.
Since they uses the same PD, the low gain is not caused by the PD.
I checked conlog history and confirmed there is no change in the MICH->BS path in the last few days.
I also checked the svn history of chans directory for changes in filters. Nothing problematic found.
Then I noticed that the susvme computers were overloaded.
This time, I rebooted all the FE computers just in case.
Then the MICH gain was somewhat recovered (by a factor of 3 or so). Don't know why.
I adjusted the DD_handoff script to set the MICH gain to 0.7 before the bounce-roll filter is engaged.
|
1334
|
Tue Feb 24 02:23:40 2009 |
Yoichi | Update | Locking | Locking - MC board bad |
Rob, Yoichi, Alberto, Kiwamu, Kakeru
We found that the OMC alignment feedback was on for the POS X loop even though the OMC was not locked.
This caused the PZT mirror to be tilted in yaw a lot. This was probably the reason for the mysterious shift in the AS beam last week, because the AS RF beam is picked up after this PZT mirror.
Rob aligned the OMC and we re-centered the AS PDs and the CCD.
This changed the DARM RF gain, so we changed it from 3 to 1. This gain used to be -1. It is still not understood why the polarity was changed.
The MC length was changed ? We should check the sideband transmission.
After this, we reached to the arm power 4. But the IFO loses lock immediately after the moving zero is turned off.
At this stage, the CARM loop bandwidth is supposed to be high enough that the moving zero is no longer necessary.
However, when we measured the MCL loop gain with several different AO path gains, the loop shape did not change at all.
This led us to suspect the AO path may not be connected. The cabling from the common mode board to the MC board seemed ok.
We tested the signal flow in the MC board using a signal generator and an oscilloscope.
Then we found that a signal injected to the IN2 (AO path) does not reach to the TP1A (right after the boost stages), though the signal is visible in the OUT2 (monitor BNC right after the initial amplifier (B-amp) for the AO path). The signal from IN1 (MC REFL) can be observed at TP1A. This means something is broken between the B-amp and the sum-amp in the AO path.
We will check the MC board tomorrow. |
1335
|
Tue Feb 24 18:42:15 2009 |
pete | Update | Locking | mc board repair |
Peter, Yoichi
Last night:
Quote: | However, when we measured the MCL loop gain with several different AO path gains, the loop shape did not change at all. This led us to suspect the AO path may not be connected. The cabling from the common mode board to the MC board seemed ok. We tested the signal flow in the MC board using a signal generator and an oscilloscope. Then we found that a signal injected to the IN2 (AO path) does not reach to the TP1A (right after the boost stages), though the signal is visible in the OUT2 (monitor BNC right after the initial amplifier (B-amp) for the AO path). The signal from IN1 (MC REFL) can be observed at TP1A. This means something is broken between the B-amp and the sum-amp in the AO path. We will check the MC board tomorrow.
|
Today we examined the MC board. With the extension board in place everything seemed fine. Without the extension board we could replicate the problem. Jiggling the IN2 jack caused the injected signal observed at TP1A to come and go. These jacks are unfortunately mounted directly on the board. We traced the problem to a resistor in this path (R30) which looked fishy. We soldered on a new 2K resistor with OWC and it fixed the problem. |
1336
|
Wed Feb 25 03:10:24 2009 |
Yoichi | Update | Locking | Locking status |
Rob, Yoichi, Kakeru, Kiwamu
Tonight, CARM -> MCL hand off was not stable. The MCF signal monotonically went up to +2V after CARM and MCL gain was turned down to zero.
This was repeatable and it only goes up (not down).
After a while, we found that putting sleep (~5sec) between the zeroing of CARM gain and MCL gain prevents this problem.
Handing off of CARM error signal from TR to PODC was also not robust.
It seems that the suitable gain changes every time.
tdsavg started to exit with errors. We rebooted fb40m.
When tdsavg returns an error, the cm_step script tries to write NaN into SPOB DC offset.
To prevent this, I put the tdsavg in a while loop which runs until tdsavg returns something other than NaN.
I was able to hand off to PODC several times, but could not proceed further because the IFO lost lock soon.
|
1337
|
Wed Feb 25 11:48:02 2009 |
Jenne | Update | PEM | Wiener filtering update - work on filtering some S5 DARM_CTRL data |
Quick update on my wiener filtering status:
Joe has been helping me get on the GRID, so I now have a grid certificate, and accounts on most/all of the clusters.
Joe also helped me get menkar to get S5 data so that I can do wiener filtering to the back-data.
I've been running the wiener filtering algorithm, and right now, it doesn't do anything to improve the DARM_CTRL data. I am confident that this is because something is funky in the wiener filtering algorithm somewhere. The indicator of this is that the wiener filtering calculation takes the same amount of time (~95 seconds) to calculate a filter for 64 seconds of data as for 1 hour of data (both for N = 2000 taps).
For reference, attached are my plots for the wiener filtering result for (1) 64 seconds of S5 data, and for (2) 3600 seconds of S5 data.
These plots were made using H1:DARM_CTRL as the signal to minimize, with 4 seismometers as the witness channels (EX_SEISX, EY_SEISY, LVEA_SEISX, LVEA_SEISY)
I'm working on figuring out what's going on with the filtering algorithm, and why it does work for C1:MC_L minimization, but does not work for H1:DARM_CTRL minimization.
|
1339
|
Thu Feb 26 01:24:44 2009 |
Yoichi | Update | Computers | Martian wireless is back |
Today, a new wireless router arrived.
I configured and installed it. Now the martian wireless network is back.
I updated the wiki page about the wireless network.
http://lhocds.ligo-wa.caltech.edu:8000/40m/Network |
1341
|
Thu Feb 26 19:59:23 2009 |
Yoichi | Update | Locking | Daytime locking |
Osamu, Yoichi
We tried locking today from about 2PM.
It took about 1000sec on average to acquire the initial lock.
After the initial lock is achieved, the hand-off/ramp-up steps were reasonably robust, although the AS beam sometimes fluctuates a lot (not good for mental health).
Like last night, the IFO loses lock at around arm-power=8.
We measured the CARM AO path loop gain at arm-power=4. We used the SR785 connected to the A-excitation channel of the common mode board through my TFSR785.py script.
The first attachment is the transfer function measured right after the arm power was ramped up to 4.
The overall bandwidth of the CM servo is only 400Hz. Note that since this is the loop gain of only the AO path, the low frequency gain is eaten by the MCL path.
The second attachment is the same transfer function measured after the AO path gain was increased by 6dB.
It is evident that the AO path is working.
We increased both the AO path and MCL gain by 18dB. The third attachment is the AO path TF in this state.
We then increased the arm power but lost lock at arm-power=6. We should have checked the DARM loop too.
BTW, these plots are automatically generated when you use TFSR785.py for transfer function measurements.
I added -notickle option to c1_watch_dr_bang, since tickling seems to be not necessary during the daytime (actually the initial lock was easier with no tickling).
As the construction work in the next building is now calmed down, I think it is ok to do locking during the day time, though I still plan to come at night.
The improvement of my brain efficiency during the day time may compensate for the longer wait time for initial lock.
|
1343
|
Fri Feb 27 13:49:19 2009 |
rob | Update | Locking | thurs night |
Could not get past arm power of ~11 or so. I was suspicious of the transmon high-gain/low-gain PD handover, so I ran the matchTransMon scripts, but that did not help. I also removed the line in the cm_step script that increased the CM gain by 18dB at an arm power of 4. The gain of the CM servo will increase naturally as the power in the IFO builds up, so it may not be good to crank it right away. I tried several other CM gains, and watched the DARM loop, but still could not get past an arm power of ~10-11. I'm not sure what's wrong, but it may be that mysterious CM-servo/McWFS conspiracy, so we can try turning down the McWFS gain next time. |
1344
|
Mon Mar 2 03:57:44 2009 |
Yoichi | Update | Locking | Sunday night locking |
Tonight's locking started with a boot fest of the FE computers which were all red when I came in.
It also took me sometime to realize that C1:IOO-MC_F was returning always zero to tdsavg, causing the offloadMCF script to do nothing.
I fixed this by rebooting c1iovme and c1iool0.
Like Rob on the thursday night, I was only able to reach arm power around 10.
This time, I turned down the MC WFS gain to 0.02 (from 0.3).
I also checked gains of most of the loops (MICH, PRC, SRC, DARM, CARM-MCL, CARM-AO).
All the loops looked fine until the lock was lost suddenly. Also the spectrum of MC_F did not change as the arm power was ramped up.
Actually, I was able to reach arm power=10 only once because I spent a long time checking the loop gains and spectrum at fine steps of the arm power.
So it is quite possible that this loss of lock was just caused by a seismic kick. |
1346
|
Mon Mar 2 21:16:32 2009 |
Yoichi | Update | Locking | Low-gain High-gain PD switching may not be working well |
Osamu, Yoichi
This afternoon, I run the locking script while doing calculations for the upgrade.
The IFO lost lock even at lower arm powers (around 6) if it was operated for a while (~ 5min).
It seemed as if there were some intermittent glitches (seismic? laser?) causing the lock losses.
We also saw once the TRX and TRY signals saturated at around arm power = 11 when there was a large fluctuation in the arm power.
Osamu suggested that it looked like the high-gain to low-gain PD switching was not working.
I won't come tonight as I may have caught a cold, but if someone comes tonight, it is worth checking the PD switching. |
1347
|
Tue Mar 3 08:44:31 2009 |
steve | Update | PEM | air cond. maintenance today |
IFO room 104 air conditions will be shut down for maintenance today.
This should be finished by noon.
The temperature and particle count variation can be more than usual. |
1348
|
Tue Mar 3 10:39:07 2009 |
Alberto | Update | LSC | c1lsc discontinued functioning |
The c1lsc has been unstable since last night. Its status on the DAQ screen was oscillating from green to red every minute.
Yesterday, I power recycled it. That brought it back but the MC got unclocked and the autolocker could not get engaged. I think it's because the power recycling also turned c1iscaux2 off which occupies the same rack crate.
Killing the autolocker on op340 e restarting didn't work. So I rebooted also c1dcuepis and burt-restored almost all snapshot files. To do that, as usual, I had to edit the snapshot files of c1dcuepics to move the quotes from the last line.
After that I restarted the autolocker that time it worked.
This morning c1lsc was again in the same unstable status as yesterday. This time I just reset it (no power recycling) and then I restarted it. It worked and now everything seems to be fine. |
1350
|
Tue Mar 3 19:26:44 2009 |
Yoichi | Update | Locking | Low-gain High-gain PD switching may not be working well |
I checked the switching of the QPDX from high gain to low gain.
Switching happens as expected, but the low gain QPDX output was very low compared to QPDY.
Also the digital gain for the high gain TRX was not matched with the low gain one. So when the switching happens, there is a large jump in the TRX.
I also found that the offset values for the low gain QPDX were totally wrong. I adjusted it.
Then I removed a beam splitter in front of the QPDX to increase the power falling on it.
But still the low gain QPDX output is four times lower than the low gain QPDY.
I'm still working on it. So don't expect the switching to work correctly at this moment.
I'm planning to be back after the dinner.
Quote: | Osamu, Yoichi
This afternoon, I run the locking script while doing calculations for the upgrade.
The IFO lost lock even at lower arm powers (around 6) if it was operated for a while (~ 5min).
It seemed as if there were some intermittent glitches (seismic? laser?) causing the lock losses.
We also saw once the TRX and TRY signals saturated at around arm power = 11 when there was a large fluctuation in the arm power.
Osamu suggested that it looked like the high-gain to low-gain PD switching was not working.
I won't come tonight as I may have caught a cold, but if someone comes tonight, it is worth checking the PD switching. |
|
1351
|
Tue Mar 3 23:59:26 2009 |
Yoichi | Update | Locking | Low-gain High-gain PD switching may not be working well |
Quote: | I checked the switching of the QPDX from high gain to low gain.
Switching happens as expected, but the low gain QPDX output was very low compared to QPDY.
Also the digital gain for the high gain TRX was not matched with the low gain one. So when the switching happens, there is a large jump in the TRX.
I also found that the offset values for the low gain QPDX were totally wrong. I adjusted it.
Then I removed a beam splitter in front of the QPDX to increase the power falling on it.
But still the low gain QPDX output is four times lower than the low gain QPDY.
I'm still working on it. So don't expect the switching to work correctly at this moment.
I'm planning to be back after the dinner.
|
This sounds like the QPD whitening gain sliders may be stuck. The slider twiddling script should be run, or the sliders should be twiddled by hand. |
1352
|
Wed Mar 4 03:37:51 2009 |
Yoichi | Update | Locking | Low-gain High-gain PD switching may not be working well |
Quote: |
This sounds like the QPD whitening gain sliders may be stuck. The slider twiddling script should be run, or the sliders should be twiddled by hand. |
Yes, it was indeed the whitening gain slider problem.
I moved them and the QPDX gain went up suddenly. I reinstalled the BS in front of the QPDX and adjusted the offsets, gains accordingly.
Now the high-gain to low-gain switching is fine. |
1353
|
Wed Mar 4 03:50:17 2009 |
Yoichi | Update | Locking | Still won't go above arm power 10 |
Yoichi, Kentaro
The IFO still loses lock at around arm power 10.
I attached time series of various error signals when losing lock.
In all of the three cases, both of the arm powers go up rapidly just before losing lock.
(The first attachment was taken before I fixed the QPDX switching, so you can see saturation in TRX.)
But PD1_DC (the DC power in the PRC) did not go up in the third case.
I should also check correlations with laser power, CARM length (OSEM signals), seismic noise etc. |
1354
|
Wed Mar 4 12:38:07 2009 |
Alberto | Update | Computer Scripts / Programs | 3f locking simulations |
I simulated the REFL signals demodulated at the differential frequencies of the sidebands (f2-f1), at their summed frequencies (f2+f1). I also simulated their combination as in the Double Demodulation.
I repeated the simulation for:
- Old (current) 40m
- 40m Upgrade
- AdvLIGO
I'm attaching the results to this elog entry.
The plots show how the signal varies exploring the two-dimensional space of the demodulation frequencies (differential and sum).
Both the Upgrade and the Old40m's signals look anomalous since the zero-crossing point does not change with the demodulation phases.
I suspect there's is a problem with the optickle model of the 40m. |
1355
|
Wed Mar 4 17:20:04 2009 |
josephb | Update | Cameras | Camera code upgrades |
I've updated the digital camera python code as well as changed the network topology.
At the moment, both cameras are connected to a small gigabit switch which only talks to Ottavia. This means all camera servers must be run on Ottavia, allow camera output is still UDP multicast so any machine capable of running gstreamer can pick up the images.
The server and client programs now have the ability to read a configuration file for the setup of the cameras. They default to pcameraSettings.ini, but this can default can be changed with a -c or --config option
For example, "serverV3.py --config pcam1.ini" will run the server using the pcam1.ini settings file. Similarly, "client.py --config pcam1.ini" will also take the IP settings from the config file so that it knows at which port and IP to listen.
These programs and .ini files have been placed in /cvs/cds/caltech/apps/linux64/python/pcamera/
I've updated the cshrc.40m aliases so that it uses the new configuration file options, so now pcam1 calls "client.py -c pcam1.ini" in the above directory.
So to start a client use pcam1 or pcam2 (for the 32223 camera in PSL looking at MC trans or 44026 looking at an analog moniter in the control room respectively). These can be run on Allegra, Rosalba or Ottavia at the moment.
To start a server, use pserv1 or pserv2. These *must* be run on Ottavia.
I've also added a -n or --no-gui option at Yoichi's request, one which just starts up and plays, with no graphical gui.
Lastly, I've made some changes to the base pcamerasrc.py file, which should make display more robust. After a failed transmission of an image from the camera to Ottavia, it should re-attempt up to 10 times before giving up. I'm hoping this will make it more robust against packet loss. The change in network topology has also helped this, allowing 640x480 to be transmitted on both cameras before tens of minutes before a packet loss causes a stop. |
1358
|
Thu Mar 5 00:06:32 2009 |
Kakeru, Rana | Update | IOO | WFS centering |
We found that the MC REFL image was no longer round and that the MCWFS DC quadrant spots were mostly
in one quadrant. So we re-centered the MCWFS beams in the following way:
1) We unlocked the MZ and adjusted the PZT voltage to keep the beam on the WFS from saturating.
2) Re-aligned the black hole beam dump to center its beam in its aperture.
3) centered the beam on the MCWFS optics and MCWFS QPD displays.
4) Relocked MC.
Below is the image of the IOO Strip tool. You can see that the MC REFL DC is now more flat. The
MC pointing has also been changed (see the MC TRANS HOR & VERT channels). The MC transmitted
light is also now more stable and higher.
We tried to center the QPD, and we found that there were a few hundred mV of dark offset for each
quadrant of QPD. We adjusted them with this scripts:
/cvs/cds/caltech/scripts/MC/WFS/McWFS_dc_offsets |
1359
|
Thu Mar 5 01:09:29 2009 |
rana, alberto | Update | DMF | still not working |
We tried to run DMF on mafalda, but it didn't work. I tried to compile it using Rob's elog instructions.
On mafalda, I started matlab and ran the mdv_config.m to set up mDV. I tested that the seisBLRMS.m
script ran and correctly produced changes in the seisBLRMS strip tool. but when I tried to compile it I got:
>> mcc -v -m -R -nojvm seisBLRMS.m
Warning: Duplicate directory name: /cvs/cds/caltech/apps/linux/matlab/toolbox/local.
Compiler version: 4.6 (R2007a)
Processing /cvs/cds/caltech/apps/linux/matlab/toolbox/matlab/mcc.enc
Processing /cvs/cds/caltech/apps/linux/matlab/toolbox/signal/mcc.enc
Processing /cvs/cds/caltech/apps/linux/matlab/toolbox/control/mcc.enc
Processing /cvs/cds/caltech/apps/linux/matlab/toolbox/filterdesign/mcc.enc
Processing /cvs/cds/caltech/apps/linux/matlab/toolbox/shared/controllib/mcc.enc
Processing /cvs/cds/caltech/apps/linux/matlab/toolbox/ident/mcc.enc
Warning: an error occurred while parsing class FilterDesignDialog.AbstractEditor:
Undefined function or variable 'DAStudio.Object'.
> In /cvs/cds/caltech/apps/linux/matlab/toolbox/shared/filterdesignlib/@FilterDesignDialog/@CoeffEditor/schema.p>schema at 9
Warning: an error occurred while parsing class FilterDesignDialog.CoeffEditor:
Invalid superclass handle.
Processing /cvs/cds/caltech/apps/linux/matlab/toolbox/fixedpoint/mcc.enc
terminate called after throwing an instance of 'ApplicationRedefinedException*'
Abort (core dumped)
"/cvs/cds/caltech/apps/linux/matlab/bin/mcc" -E "/tmp/fileRnU5Qj_31324": Aborted
??? Error executing mcc, return status = 134.
In the meantime, I've started up a green terminal on allegra which is ssh'd into megatron.
On megatron there is a regular matlab session which is running seisBLRMS.m as a matlab script
and the seis DMF channels are getting updated. |
1361
|
Thu Mar 5 05:07:09 2009 |
Yoichi | Update | Locking | Wednesday night locking |
Tonight, I was having a problem with the PO_DC hand-off.
It fails most of the time.
I increased the averaging time for the PD1_DC offset measurement.
I also wrote a script to match the gain of the transmission DC and the PO_DC signals.
This script (/cvs/cds/caltech/scripts/CM/matchPODCGain ) measures the gains of the old (TRX+TRY) and new (PO_DC) signals at 150Hz and returns the optimal value to be put into the input matrix.
cm_step script calls matchPODCGain to determine the matrix element value for the PO_DC signal.
Even with this script, the hand-off was still unreliable.
I checked the AO path loop gain just before the hand off. It looked normal.
Then I realized that the oscilloscope I hooked up to the PO_DC signal using a T-BNC may be introducing some noise into the channel.
So I removed it. Then the PO_DC hand off went well at least once.
The IFO still loses lock at around arm power 10.
I attached time series of the latest lock loss. The second attachment is a zoom of the first one.
This time, there is a glitch in the ETM feedback signals, which is also present in the DARM and CARM and error signals.
I saw this kind of glitches several times today. |
1364
|
Fri Mar 6 05:44:14 2009 |
Yoichi | Update | Locking | Locking distracted by the QPD whitenning problem again |
Tonight, I was able to ramp up the arm power to around 20. Then the DARM loop started to oscillate and the IFO lost lock in a few seconds.
I repeated this several times, then realized that the transmission QPDs were not working properly again due to the well known sticky slider problem.
I should have run slider_twiddle script. Since the DARM RF signal is normalized by the sqrt(TRX+TRY), it is reasonable that the DARM loop got unstable.
The fact that I was able to go up to arm power = 20 means there is nothing saturating below this power level.
|
1365
|
Fri Mar 6 15:23:39 2009 |
Yoichi | Update | Locking | Locking distracted by the QPD whitenning problem again |
By looking at the time series of DARM signal at the time of a lock loss, the oscillation frequency was about 3.5kHz (see the attm1 and its zoomed version attm2).
I will measure the DARM loop gain around this frequency next.
Quote: | Tonight, I was able to ramp up the arm power to around 20. Then the DARM loop started to oscillate and the IFO lost lock in a few seconds.
I repeated this several times, then realized that the transmission QPDs were not working properly again due to the well known sticky slider problem.
I should have run slider_twiddle script. Since the DARM RF signal is normalized by the sqrt(TRX+TRY), it is reasonable that the DARM loop got unstable.
The fact that I was able to go up to arm power = 20 means there is nothing saturating below this power level.
|
|
1366
|
Fri Mar 6 18:14:58 2009 |
Yoichi | Update | Computers | awg not working |
Starting from this afternoon, the awg is not working.
I rebooted FE computers, c0daqawg as well as tpman and daqd processes on fb40m several times.
But the problem is still there.
I sent an email to Alex. |
1369
|
Sat Mar 7 16:50:25 2009 |
Yoichi | Update | Computers | Not even data retrieval working |
Now our digital system is really in trouble.
We can't even get data from tp channels.
I did another round of computer reboots, this time including the RFM bypass switch, c0daqctrl, c0dcu1 and fb40m itself.
But the problem still persists.
I guess there is nothing I can do until Alex comes in. |
1370
|
Sun Mar 8 23:09:26 2009 |
rana | Update | Computers | Not even data retrieval working |
Although getting the regular DAQ data works, we can't get any testpoints.
I tried restarting tpman several times; there's no inittab on fb40m for this so we should get Alex to set one up when he comes.
I also tried various power cycles and reboots: daqawg, daqctrl, etc. I also notice that Osamu's setup of new stuff is connected to
the same rack and power strips as all of our sensitive DAQ machines. We should find out if there was any hardware installed in the
last couple days; it would be easy to accidentally unplug or damage on of our fibers.
I moved the old tpman.log over to tpman.log.090308. It starts out with a header and then just lists when each TP is requested.
When restarting tpman it puts the following into the terminal:fb:controls>./tpman &
[1] 1037
fb:controls>VMIC RFM 5565 (0) found, mapped at 0x2868c90
VMIC RFM 5579 (1) found, mapped at 0x2868c90
Could not open 5565 reflective memory in /dev/daqd-rfm1
16 kHz system
Spawn testpoint manager
Channel list length for node 0 is 4168
Test point manager (31001001 / 1): node 0 which is OK?; its the same startup outputs that are in the old log file. It would be nice if there was not and error message about the RFM.
Requesting new testpoints via tdsdata, dtt, or the diag command line doesn't seem to work. tpman doesn't spit anything out although 'tp show 0'
does show that the TP is selected.
Once Alex fixes the 'tpman' issue, we should make sure to put an inittab or startup script in there so that tpman writes a log
file and also archives its old log files upon a restart. |
1371
|
Sun Mar 8 23:14:52 2009 |
rana | Update | Computer Scripts / Programs | tdsdata doesn't work |
Matt logged in and rebuilt the TDS stuff for us on Mafalda in /cvs/cds/caltech/apps/linux/tds_090304.
He says that he can't build his stuff on 64-bit because there's not a sanctioned 64-bit build of GDS yet.
This should have all the latest fixes in it. I tried using both the old and new code from allegra and they both are fine:
./tdsdata 16384 2 C1:IOO-MC_F > /users/rana/test.txt
I loaded the data I got with the above command and there were no data dropouts. Possibly the dropout problem is only
associated with testpoints and so we have to wait for the TP fix. |
1373
|
Mon Mar 9 11:09:33 2009 |
Alberto | Update | Computers | Re: Not even data retrieval working |
Quote: | Although getting the regular DAQ data works, we can't get any testpoints.
I tried restarting tpman several times; there's no inittab on fb40m for this so we should get Alex to set one up when he comes.
I also tried various power cycles and reboots: daqawg, daqctrl, etc. I also notice that Osamu's setup of new stuff is connected to
the same rack and power strips as all of our sensitive DAQ machines. We should find out if there was any hardware installed in the
last couple days; it would be easy to accidentally unplug or damage on of our fibers.
I moved the old tpman.log over to tpman.log.090308. It starts out with a header and then just lists when each TP is requested.
When restarting tpman it puts the following into the terminal:fb:controls>./tpman &
[1] 1037
fb:controls>VMIC RFM 5565 (0) found, mapped at 0x2868c90
VMIC RFM 5579 (1) found, mapped at 0x2868c90
Could not open 5565 reflective memory in /dev/daqd-rfm1
16 kHz system
Spawn testpoint manager
Channel list length for node 0 is 4168
Test point manager (31001001 / 1): node 0 which is OK?; its the same startup outputs that are in the old log file. It would be nice if there was not and error message about the RFM.
Requesting new testpoints via tdsdata, dtt, or the diag command line doesn't seem to work. tpman doesn't spit anything out although 'tp show 0'
does show that the TP is selected.
Once Alex fixes the 'tpman' issue, we should make sure to put an inittab or startup script in there so that tpman writes a log
file and also archives its old log files upon a restart. |
Alex fixed the problem. It was caused by the awgtpman running on kami1.martian which conflicted with the tpman in fb0.
Killing awgtpman on kami1 allowed for the tpman on tp0 to work properly again.
If more test points are needed, Alex suggested to tune the GDS settings accordingly.
What this actually means, I still have to understand it. |