I wish I could use a bigger font for this, but, the suspension work is totally done for the upgrade!!!
Now, nobody break any suspensions, or we're not going to be friends for a while.
Koji and I put ETMY back in its tower, and made sure that both scribe lines are at the correct height. We also confirmed that the balance is good (as Suresh mentioned in a previous elog, since we balanced using the AR surface, the HR surface is pointing downward a little bit, but it's well within the OSEMs ability to correct.
While we were in there, we also looked at Tip Tilt number 002. As mentioned in elog 3645, the pitch pointing was off by a little bit. Since the TTs don't have actuators, the pointing has to be pretty good. We tweaked the balancing, and now the reflected beam goes completely back into the laser aperture, so it's as balanced as it's going to get. This TT is now ready for installation onto the ITMY table as part of the SRC.
Kiwamu confirmed that he's going to install these optics tomorrow, since he's doing some other alignment work today.
Just for good measure, the Table:
Autoburt wouldn't restore settings for front ends on reboot
First I moved the burt directory over to the new directory structure.
This involved moving /cvs/cds/caltech/burt/ to /opt/rtcds/caltech/c1/burt.
Then I updated the burt.cron file in the new location, /opt/rtcds/caltech/c1/burt/autoburt/. This pointed to the new autoburt.pl script.
I created an autoburt directory in the /opt/rtcds/caltech/c1/scripts directory and placed the autoburt.pl script there.
I modified the autoburt.pl script so that it pointed to the new snapshot location. I also modified it so it updates a directory called "latest" located in the /opt/rtcds/caltech/c1/burt/autoburt directory. In there is a set of soft links to the latest autoburt backup.
Lastly, I edited the crontab on op340m (using crontab -e) to point to the new burt.cron file in the new location.
This was the easiest solution since the start script is just a simple bash script and I couldn't think of a quick and easy way to have it navigate the snapshots directory reliably.
I then modified the Makefile located in /opt/rtcds/caltech/c1/core/advLigoRTS/ which actually generates the start scripts, to point at the "latest" directory when doing restores. Previously it had been pointing to /tmp/ which didn't really have anything in it.
So in the future, when building code, it should point to the correct snapshots now. Using sed I modified all the existing start scripts to point to the latest directory when grabbing snapshots.
According to Keith directory documentation (see T1000248) , the burt restores should live in the individual target system directory i.e. /target/c1sus/burt, /target/c1lsc/burt, etc. This is a distinctly different paradigm from what we've been using in the autoburt script, and would require a fairly extensive rewrite of that script to handle this properly. For the moment I'm keeping the old style, everything in one directory by date. It would probably be worth discussing if and how to move over to the new system.
The dolphin RFM was not sending data between c1lsc and c1sus.
Dig into the controller.c code located in /opt/rtcds/caltech/c1/core/advLigoRTS/src/fe/. Find this bit of code on line 2173:
2173 #ifdef DOLPHIN_TEST
2174 #ifdef X1X14_CODE
2175 static const target_node = 8; //DIS_TARGET_NODE;
2177 static const target_node = 12; //DIS_TARGET_NODE;
2179 status = init_dolphin(target_node);
Replace it with this bit of code:
2173 #ifdef DOLPHIN_TEST
2174 #ifdef C1X02_CODE
2175 static const target_node = 8; //DIS_TARGET_NODE;
2177 static const target_node = 4; //DIS_TARGET_NODE;
2179 status = init_dolphin(target_node);
Basically this was hard coded for use at the site on their test stands. When starting up, the dolphin adapter would look for a target node to talk to, that could not be itself. So, all the dolphin adapters would normally try to talk to target_node 12, unless it was the X1X14 front end code, which happened to be the one with dolphin node id 12. It would try to talk to node 8.
Unfortunately, in our setup, we only had nodes 4 and 8. Thus, both our codes would try to talk to a nonexistent node 12. This new code has everyone talk to node 4, except the c1x02 process which talks to node 8 (since it is node 4 and can't talk to itself).
I'm told this stuff is going away in the next revision and shouldn't have this hard coded stuff.
Apparently, the only models which should have pciRfm=1 are the IOP models which have a dolphin connection. Front end models that are not IOP models (like c1lsc and c1rfm) should not have this flag set. Otherwise they include the dolphin drivers and causes them and the IOP to refuse to unload when using rmmod.
So pciRfm=1 only in IOP models using Dolphin, everyone else should not have it or should have pciRfm=-1.
Current CDS status:
[Jenne, Koji and Kiwamu]
We finished a coarse alignment of IP_ANG.
The beam for IP_ANG successfully reached to the ETMY chamber and is ready for the final alignment.
(Additionally we again tried looking for a resonance for TEM00 in the X arm, but we obtained only flashes of some higher order modes.)
--- what we did
* installed the steering mirrors for IP_ANG and IP_POS.
* checked PZT1 if it worked correctly or not. It was healthy.
* neutralized and realigned PZT1.
* flipped a window, which is standing before PRM, because the wedged side of the window was at wrong side.
* realigned PZT2 and checked the spot positions on the TTs.
* repositioned more carefully the TTs and aligned them to the correct angles.
* aligned the beam to the center of both the BS and ITMY by rotating the last TT.
* aligned the beam more precisely by tweaking PZT2 while looking at the spot at the Y end.
The beam is still hitting the center of ETMY.
* aligned the steering mirror for IP_ANG while looking at the spot at the Y end.
In fact IP_ANG is visible with a card.
* aligned the BS by looking at the spot on ITMX.
* covered ETMX with aluminum foil, and made a ~1cm hole on the foil as a target.
The hole was placed on the center of ETMX.
* more precisely aligned the BS by looking at the spot on the aluminum foil.
The spot was clearly visible on CCD monitor.
* aligned the ITMX by looking at a spot on the foil. The spot represented the beam reflected by ITMX back to ETMX.
* saw flashes on the foil but couldn't make it TEM00 because it was difficult to see any flashes on the surface of either ETMX or ITMX.
It means the flashes are visible only when the beam is hitting some scattering surface.
The mirror surface of the test masses are less lossy than that of the old test masses ??
The nightly backup of the frames and the /cvs/cds directories is back up and running. We are free again to do crazy stuff at will, and it will all be saved for eternity.
This graph shows 5 hours data in minute trend for ITMX and ETMX from 5am to 10 am today. ITM pitch drift is 3 times lager than ETM pitch if the OSEM sensitivity is assumed to be the same.
This graph is last 1 hour data of above graph in second trend.
It is clealy seen that ITM yaw is jumping between two stages. I guess ITM is something wrong, touching magnets or earthquake stops?
This plot shows ETM oplev and OSEM trend for 10 hours on day before yesterday as almost the same as plot shown this entry. I reported the 10-30minites fluctuations were seen, but I noticed it comes from not suspension but from oplev power fluctuation.
After Kiwamu fixed the ETM OSEM touch yesterday afternoon, still the same trend was seen, so we had thought what we fixed was not enough. This morning I looked at the yesterday's and day before yesterday's trend and noticed the simila trend both the pit and yaw in ETM oplev but not on the OSEM trend. Kiwamu suggested me to put the oplev sum on the same plot. It was!
So, ETMX is not bad, but in fact, still alignment fluctuation exist on the cavity. ITM?
Good news. I feel multi-chromatic-locking success is just around the corner.
By the way, there's a new presentation on the DCC from the ANU group where they've locked a short single cavity with both colors - G1000735:
[Koji, Osamu and Kiwamu]
We aligned the beam axis pointing down to both X and Y arm.
Now the beams are hitting the centers of both ETMX and ETMY.
Amazingly Osamu made X arm flashing by aligning the cavity.
(what we did)
- opened almost all the chambers except for the MC2 chamber.
- locked and aligned the MC.
We set Marconi to the right frequency, which had been set to the default values, probably due to the power outage in the last weekend.
Also we found a DAC cable disconnected from the IO chassis of c1sus. So we connected it in order to damp the MC suspensions.
- aligned MMT2 and PZT2 in order to let the beam go through the center of PRM.
- checked the beam centering at the two TTs (PR2, PR3).
- rotated PR3 to make the beam go through the centers of both ITMY and BS at the same time.
- tried finding the beam spot at the ETMY chamber, and successfully found it.
To see such faint beam spot, we used an IR viewer.
In addition to that, we put a large piece of aluminum foil as a screen in the chamber.
- aligned the beam to the center of ETMY by tweaking the PZT mirror (SM2).
- aligned the BS so that the reflected beam at the BS goes through the center of ITMX.
- tried finding the beam spot at the X end, and successfully found it hitting the wall in the chamber.
- aligned the BS in order to let the beam hit the center of ETMX.
- tried aligning ETMX and ITMX to the beam.
Eventually we made the X arm flashing.
However the flash was a bit too weak to completely align the cavity.
(plan for tomorrow)
- reinstall some steering mirrors into the BS chamber
- check and neutralize PZT1
- alignment of IP_ANG
We checked the ETMX suspension and found the UR OSEM was close to the magnet.
So we rotated the UR OSEM so that it won't touch the magnet any more.
We will check the resonant frequencies again by taking the spectra.
In fact the ETMX stacked when we applied a big angular offset to Yaw direction.
This was because that the magnet was actually touching the UR OSEM.
The earthquake stops were fine, they weren't touching the test mass.
Also we looked at the wire and the standoffs, they seemed fine.
On the fb machine in /etc/dis/ there are several configurations files that need to be set for our dolphin network.
First, we modify networkmanager.conf.
We set "-dimensionX 2;" and leave the dimensionY and dimensionZ as 0. If we had 3 machines on a single router, we'd set X to 3, and so forth.
We then modify dishosts.conf.
We add an entry for each machine that looks like:
#Keyword name nodeid adapter link_width
ADAPTER: c1sus_a0 4 0 4
The nodeids (the first number after the name) increment by 4 each time, so c1lsc is:
ADAPTER: c1lsc_a0 8 0 4
The file cluster.conf is automatically updated by the code by parsing the dishosts.conf and networkmanager.conf files.
We uncommented the following lines in the rc.local file in /diskless/root/etc on the fb machine:
# Initialize Dolphin
# Have to set it first to node 4 with dxconfig or dis_nodemgr fails. Unexplai ned.
/opt/DIS/sbin/dxconfig -c 1 -a 0 -slw 4 -n 4
/opt/DIS/sbin/dis_nodemgr -basedir /opt/DIS
For the moment we left the following lines commented out:
# Wait for Dolphin to initialize on all nodes
We were unsure of the effect of the dolphin_wait script on the front ends without Dolphin cards. It looks like the script it calls waits until there are no dead nodes.
In /etc/conf.d/ on the fb machine we modified the local.start file by uncommenting:
This starts the Dolphin network manager on the fb machine. The fb machine is not using a Dolphin connection, but controls the front end Dolphin connections via ethernet.
The Dolphin network manager can be interacted with by using the dxadmin program (located in /opt/DIS/sbin/ on the fb machine). This is a GUI program so use ssh -X when logging into the fb before use.
Each IOP model (c1x02, c1x04) that runs on a machine using the Dolphin RFM cards needs to have the flag pciRfm=1 set in the configuration box (usually located in the upper left of the model in Simulink). Similarly, the models actually making use of the Dolphin connections should have it set as well. Use the PCIE_SignalName parts from IO_PARTS in the CDS_PARTS.mdl file to send and receive communications via the Dolphin RFM.
We found that the ETMX free swinging spectra showed a strange resonant frequencies.
We are going to inspect the suspension today.
In a ideal case the SOS (Small Optic Suspension) is supposed to have the following resonant frequecies.
(Although we didn't carefully identify which corresponds to which)
f_POS ~ 0.98 Hz
f_PITCH ~ 0.66 Hz
f_YAW ~ 0.8 Hz
f_SIDE ~ 0.99 Hz
However ETMX showed the following resonant frequencies.
f_POS ~ 0.91 Hz
f_PITCH ~ 0.7 Hz
f_YAW ~ 0.93 Hz
f_SIDE ~ 1.0 Hz
Especially f_YAW looks pretty high. Also the others are not at the right frequencies.
So we are suspicious that something wrong is happening on the ETMX suspension.
After Kiwamu had set the free swinging mode for ITMX and ETMX, I found a big jump of ITMX pitch and yaw. This jump is shown on oplev and OSEM plots.
I talked with Kiwamu on the phone that a shutdown of suspensions does not add a big offset, and so that it should not make a big jump.
We were not sure that this jump was due to the shutdown or drift or something else. Anyway I put ITMY oplev on center again at 0;57am.
In this morning, the same thing happened but to opposit direction when Kiwamu activated ITMX and ETMX. Then it turned out that 1000ct offset was existing on pit of ITMX. Erasing the offset fixed ITMX to normal position.
However a big drift exists in 11hours plot on ITMX 0.1->-0.25 at OLPIT, -580->-605 at SUSPIT and 0.08->0.15 at OLYAW, no significan drift at SUSYAW. On the other hand, ETMX has no big drift but has 10-30minites order fluctuations.
After 6am both the drift and the fluctuation became, roughly saying, 10 times larger, probably due to the human activity.
Kevin was working on characterizing all of our RF photodiodes for the upgrade, and he discovered that REFL11 didn't work, as described in elog 3890. Rana was working on fixing it, but then he went off to Japan.
I visually inspected the components inside the RF cage on the REFL 11 circuit board inside the PD. Most of them were okay, but the connection between L5 and C33 (the big tunable inductor and the next capacitor in the path) was totally flaky. the leg of the inductor had been soldered directly to the trace on the PCB, and the inductor was a little bit tipped over, and pulling the trace off the board. I wiggled it a little while trying to see what was going on, and the trace broke. Since there is nothing going on between L5 and C33, just the trace, I used a piece of resistor lead to attach the two. The connection now seems very robust. I'm a little worried about the connection between the inductor and the board on the other side, but I can't see it since it's under the inductor itself.
Also, the soldering of L4 (a standard surface mount component type body) to the board seemed totally shoddy. I was desoldering the first side, and the whole inductor popped off. It was clear that the inductor was making a physical connection to the board, but not a nice solid electrical connection. So I resoldered it on. (On Alberto's schematics, it is listed as a 633nH inductor. I can't find any of this value, so I just put the same one back on. The best I could do to confirm the component was still okay was measure its resistance, and compare that to a similar inductor of a similar value. It seemed okay.)
After that I powered up the PD, and took an electrical transfer function, just to have a look-see. It seems kind of okay, although the resonance seems to be closer to 13MHz than 11MHz.
Since we would like to remove the capacitor that is in parallel with the diode itself, which will then change all of the resonant conditions on other components, I didn't worry too much about the resonant peak for tonight. We're going to have to look in on this though.
Also, I'm leaving the optical check-out for Kevin, so he will let us know if I magically fixed the PD, or if it needs some more work.
Photos of the circuit board (mostly Alberto's mods) before and after I fitzed with it are on Picasa
More testing. Probably more fixing.
We measured Open loop TF for oplev pitch on ITMX.
All feed back filter of oplev was on as same as before. Original notch filters which notches above 10Hz resonance should be modified with some measurements of present resonant frequency. Up to 10Hz, a simple f^2 filter is used, so the notch should not affect this measurement.
Measured upper UGF is about 2Hz with gain slider 1, and lower UGF is 1.3Hz. Phase margin is 40 degree, so it is not a good idea to increase the gain drastically.
I have measured the coherence also but I could not find a way to put it on this picture. Anyway coherence below 0.6Hz was not so good like ~0.95. This can be improved if larger excitation is used next time.
During this measurement around 0.2-0.3Hz, small earthquake happened but seemed OK for the control.
We will measure the other TF, yaw, ETMX or somthing, maybe tomorrow, due to free swinging ITMX and ETMX tonight.
Tonight, swing again.
Please do not restore the watchdogs until tomorrow (Dec.9) morning.
I am leaving ITMX and ETMX freely swinging, so that later I can take the spectra and diagonalize the input matrices.
Please don't restore the watchdogs until tomorrow morning.
The oplevs have been installed on ITMX and ETMX.
Now the oplev servos are running.
The lock of the green beam became more stable after the oplevs were activated.
(what I did)
- opened the ITMX and ETMX chamber.
- rearranged the oplev mirrors in the vacuum chambers so that we can have the reflected oplev beam coming out from the viewport.
At the ITMX table, I put the oplev mirrors approximately on the designed places.
- aligned the beam on the optical benches
- strung a ribbon cable at the 1X9 rack.
This cable connects the oplev interface board and the ADC blue golden board.
- modified c1scx simulink model.
Since the model didn't have proper connections to the ADC channels, I added four ADC channels and plugged them into oplev servo in the model.
- relaunched the c1scx code after building and installing it.
- activated the oplev servos. Amazingly the default gains did work (i.e. all the gain = 1)
- after aligning X arm to the green beam, I did final centering of oplev beams
- - - - - ADC connection for ETMX oplev signals :
ADC0_24 = segment_1
ADC0_25 = segment_2
ADC0_26 = segment_3
ADC0_27 = segment_4
We put in the fiber for use with the Dolphin reflected memory between c1sus and c1lsc (rack 1X4 to rack 1Y3). I still need to setup the dolphin hub in the 1X4 rack, but once that is done, we should be able to test the dolphin memory tomorrow.
c1pem model was reporting all zeros for all the PEM channels.
Two fold. On the software end, I added ADCs 0, 1, and 2 to the model. ADC 3 was already present and is the actual ADC taking in PEM information.
There was a problem noted awhile back by Alex and Rolf that there's a problem with the way the DACs and ADCs are number internally in the code. Missing ADCs or DACs prior to the one you're actually using can cause problems.
At some point that problem should be fixed by the CDS crew, but for now, always include all ADCs and DACs up to and including the highest number ADC/DAC you need to use for that model.
On the physical end, I checked the AA filter chassis and found the power was not plugged in. I plugged it in.
We now have PEM channels being recorded by the FB, which should make Jenne happier.
When adding the ETMX DAQ channels using the daqconfig gui (located in /opt/rtcds/caltech/c1/scripts/) on C1SCX.ini, we forgot to set the acquire flag to 1 from 0.
So the frame builder was receiving the data, but not recording it.
We have since then added ETMX and the C1SCX.ini file to Yuta's useful "activateDAQ.py" script in /opt/rtcds/caltech/c1/chans/daq/, so that it now sets the sensor and SUSPOS like channels to be acquired at 2k when run. You still need to restart the frame builder (telnet fb 8087 and then shutdown) for these changes to take effect.
The script now also properly handles files which already have had channels activated, but not acquired.
The input matrix of ITMX has been diagonalized.
The evaluation of this diagonalisation will be done tonight by freely swinging ITMX again.
(Somehow I couldn't get any data for ETMX from the DAQ channels. I will try it again tonight.)
For solving the matrix, I used Yuta's python code called inmartixoptimizer.py.
I took the transfer functions of UL->UR, UL->LL and UL->LR as described in this entry.
In the measurement, the frequency bin was set to 0.001 Hz and the data were 50 times averaged on dtt.
Here is the new input matrix.
[[ 0.87059649 1.14491977 1.07992057 0.90456317]
[ 0.64313916 0.55555661 -1.44997325 -1.35133098]
[ 1.13979571 -1.19186285 -0.89606597 0.77227546]]
This matrix should give a better performance than before.
To show Osamu how to setup a a front end as well as provide a test computer for Osamu's use, we used the new megatron (sunfire x4600 with 16 cores and 8 gigabytes of memory) as a front end without an IO chassis.
The steps we followed are in the wiki, here.
The new megatron's IP address is 192.168.113.209. It is running the c1x99 front end code.
I found that all the front end machine showed the red light indicators of DAQ on the XXX_GDS_TP.adl screens.
Also I could not get any data from both test points and DAQ channels.
First I tried fixing by telneting and rebooting fb, but it didn't help.
So I rebooted all the front end machines, and then everything became fine.
The ETMU05 has been removed from the suspension and put into the little foil house.
Before removing it I checked the position and pitch of the optic with reference to the table top.
Using the traveling microscope I checked the height of the scribe lines from the table top. They are at equal heights, centered on 5.5 inches, correct to about a quarter of the width of the scribe line.
The retro-reflection of the He-Ne laser beam is correct to within one diameter of the beam at a distance of about 1.5m. This is the reflection from the rear, AR coated, surface. The reflection from the front, HR coated, surface was down by about two diameters.
Jenne has checked with Bob and agreed on a date for baking the optic.
I moved the Pynds package from Yuta's local directory to the public place.
Now the package is living under :
Also I added the PATH on cshrc.40m, so you don't have to setenv every time.
(This package can not run on non-64bit linux)
Typing "import nds" on python allows you to use the nds functions.
Quote from Yuta's past entry
3. I installed pyNDS to /cvs/cds/caltech/users/yuta/pynds
I swapped out the IO chassis which could only handle 3 PCIe cards with the another chassis which has space for 17, but which previously had timing issues. A new cable going between the timing slave and the rear board seems to have fixed the timing issues.
I'm hoping to get a replacement PCI extension board which can handle more than 3 cards this week from Rolf and then eventually put it in the Y-end rack. I'm also still waiting for a repaired Host interface board to come in for that as well.
At this point, RFM is working to c1iscex, but I'm still debugging the binary outputs to the analog filters. As of this time they are not working properly (turning the digital filters on and off seems to have no effect on the transfer function measured from an excitation in SUSPOS, all the way around to IN1 of the sensor inputs (but before measuring the digital fitlers). Ideally I should see a difference when I switch the digital filters on and off (since the analog ones should also switch on and off), but I do not.
[Joe and Kiwamu]
We added some more DAQ channels on c1sus.
We wanted to try diagonalizing the input matrices of the ITMX OSEMs because the motion of ITMX looked noisier than the others
So for this purpose we tried adding DAQ channels so that we can take spectra anytime.
After some debugging, now they are happily running.
(DAQ activation code)
There is a code which activates DAQ channels written by Yuta in this October.
If you just execute this code, it is supposed to activate the DAQ channels automatically by editing C1AAA.ini files.
However there were some small bugs in the code, so we fixed them.
Now the code seems fine.
(reboot fb DAQ process)
When new DAQ channels are added, one has to reboot the DAQ process running on fb.
To do this, log in to a certain port on fb,
telnet fb 8088
telnet fb 8088
Then the process will automatically recover by itself.
After doing the above reboo job, we found tpman on C1IOO got down.
We don't fully understand why only C1IOO was affected, but anyway rebooting of the c1ioo front end machine fixed the problem.
ETMU05 (ETMY) had its wire winched to the correct height, was balanced, and had the standoff glued. Since it's kind of like final exam week at Caltech, Suresh had his suspension exam today, and did most of this work himself, with me hanging around and watching.
As you can see in my almost entirely green table, all that is left to do with the whole suspensions project is bake the optic (hopefully Bob has time / space this week), and then stick it in the chamber! Hooray!!! (Can you tell I'm excited to not spend too much more time in the cleanroom?)
I restarted the elog because i changed the configuration for the cryo-elog.
Used the "start-elog.csh" script in /cvs/cds/caltech/elog/
[Suresh and Kiwamu]
We aligned the green beam to the X arm cavity more carefully.
Now the green beam is hitting the centers of ETMX, ITMX and BS.
Also we confirmed that the green beam successfully comes out from the chamber to the PSL table.
- opened the BS, ITMX and ETMX chambers.
- checked the positions of the beam spots on ITMX, BS and ETMX
The spot position on ETMX was fine,
But at BS and ITMX, the spots were off downward.
We decided to move the beam angle by touching a steering mirror at the end green setup.
- changed the beam axis by touching the steering mirror at the end station.
- checked the spot positions again, they all became good. It looks the errors were within ~ 1mm.
- moved the position of a TT, which is sitting behind the BS, by ~10mm, because it was almost clliping the beam.
- aligned the green optics
- got the beam coming out from the chamber.
The c1lsc computer is running Gentoo off of the fb server. It has been connected to the DAQ network and is handling mx_streams properly (so we're not flooding the network error messages like we used to with c1iscex). It is using the old c1lsc ip address (192.168.113.62). It can ssh'd into.
However, it is not talking properly to the IO chassis. The IO chassis turns on when the computer turns on, but the host interface board in the IO chassis only has 2 red lights on (as opposed to many green lights on the host interface boards in the c1sus, c1ioo, and c1iscex IO chassis). The c1lsc IO processor (called c1x04) doesn't see any ADCs, DACs, or Binary cards. The timing slave is receiving 1PPS and is locked to it, but because the chassis isn't communicating, c1x04 is running off the computer's internal clock, causing it to be several seconds off.
Need to investigate why the computer and chassis are not talking to each other.
The c1sus and c1ioo computers are not talking properly to the frame builder. A reboot of c1iscex fixed the same problem earlier, however, as Kiwamu and Suresh are working in the vacuum, I'm leaving those computers alone for the moment, but a reboot and burt restore probably should be done later today for c1sus and c1ioo
The c1lsc computer has been moved over to the 1Y3 rack, just above the c1lsc IO chassis.
It will talking to the c1sus computer via a Dolphin PCIe reflected memory card. The cards have been installed into c1lsc and c1sus this morning.
It will talk to its IO chassis via the usual short IO chassis cable.
The Dolphin fiber still needs to be strung between c1sus and c1lsc.
The DAQ cable between c1lsc and the DAQ router (which lets the frame builder talk directly with the front ends) also needs t to be strung.
c1lsc needs to be configured to use fb as a boot server, and the fb needs to be configured to handle the c1lsc machine.
I checked the vacuum system and judged there is no apparent issue.
The chambers and annulus had been vented before the power failure.
So the matters are only on the TMPs.
TP1 showed the "Low Input Voltage" failure. I reset the error and the turbine was lift up and left not rotating.
TP2 and TP3 seem rotating at 50KRPM and the each lines show low pressur (~1e-7)
although I did not find the actual TP2/TP3 themselves.
Looks like there was a power outage. The control room workstations were all off (except for op440m). Rosalba and the projector's computer came back, but rossa and allegra are not lighting up their monitors.
linux1 and nodus and fb all appear to be on and answering their pings.
I'm going to leave it like this for the morning crew. If it
The monitors for allegra and rossa's seemed to be in a weird state after the power outage. I turned allegra and rossa on, but didn't see anything. However, I was after awhile able to ssh in. Power cycling the monitors did apparently got them talking with the computers again and displaying.
I had to power cycle the c1sus and c1iscex machines (they probably booted faster than linux1 and the fb machines, and thus didn't see their root and /cvs/cds directories). All the front ends seem to be working normally and we have damped optics.
The slow crates look to be working, such as c1psl, c1iool0, c1auxex and so forth.
Kiwamu turned the main laser back on.
Looks like there was a power outage.
The last time(Moonday) Jenne and I worked on the RF distribution unit's structure.
We are making RF distribution unit for RF upgrade which is designed by Alberto.
Rana, Koji, Jenne suggested a better design for RF Distribution unit.
So Jenne and I gathered information of parts and decided what parts will be used with specific numbers.
Specific circuit is shown in the attached picture.
Any suggestion would be really appreciated.
The front ends and fb computers were unresponsive this morning.
This was due to the fb machine having its ethernet cable plugged into the wrong input. It should be plugged into the port labeled 0.
Since all the front end machines mount their root partition from fb, this caused them to also hang.
The cable has been relabled to "fb" on both ends, and plugged into the correct jack. All the front ends were rebooted.
I tested the RFM connection between c1ioo and c1scx. Unfortunately, on the first test, it turns out the c1ioo machine had its gps time off by 1 second compared to c1sus and c1iscex. A second reboot seems to have fixed the issue.
However, it bothers me that the code didn't come up with the correct time on the first boot.
The test was done using the c1gcv model and by modifying the c1scx model. At the moment, the MC_L channel is being passed the MC_L input of the ETMX suspension. In the final configuration, this will be a properly shaped error signal from the green locking.
The MC_L signal is currently not actually driving the optic, as the ETMX POS MATRIX currently has a 0 for the MC_L component.
This morning (~0100) I started to redo some of the wiring in the rack with the FB in it. This was in an effort to activate the new Megatron (Sun Fire 4600) which we got from Rolf.
Its sitting right above the Frame Builder (FB). The fibers in there are a rats nest. Someone needs to team up with Joe to neaten up the cabling in that rack - its a mini-disaster.
While fooling around in there I most probably disturbed something, leading to the FB troubles today.
I succeeded in locking the end green laser to X arm with the new ETM.
Though the lock is still not so stable compared to the previous locking with the old ETM. Also the beam centering is quite bad now.
So I will keep working on the end green lock a little bit more.
Once the lock gets improved and becomes reasonably stiff, we will move onto the corner PLL experiment.
- beam centering on ITMX
- check the mode matching
- revise the control servo
We finished the installation of ETMX into the chamber.
In order to clear the issue of the side OSEM, we put a spacer such that the OSEM can tilt itself and accommodate the magnet.
Though we still don't fully understand why the side magnet is off from the center.
Anyway we are going to proceed with this ETMX and perform the REAL green locking.
(what we did)
- took the ETM tower out from the chamber, and brought it to the clean room again.
- checked the rotation of the ETM by using a microscope. It was pretty good.
The scribe lines at the both sides are at the same height within the diameter of the scribe line.
- checked the height of the ETM by measuring the vertical distance from the table top to the scribe line. This was also quite good.
The height is correctly 5.5 inch within the diameter of the scribe line.
- checked the magnet positions compared with the OSEM holder holes.
All the face magnets are a little bit off upward (approximately by 1mm or less).
The side magnet is off toward the AR surface by ~ 1-2mm.
(yesterday we thought it was off downward, but actually the height is good.)
- raised the position of the OSEM holder bar in order to correct the miscentering of the face magnets.
Now all the face magnets are well centered.
- brought the tower back to the chamber again
- installed the OSEMs
We put a folded piece of aluminum foil in between the hole and the side OSEM as a spacer.
- leveled the table and set the OSEMs to their mid positions.
- slided the tower to place
We had timing problems across the front ends.
Noticing that the 1PPS reference was not blinking on the Master Timing Distribution box. It was supposed to be getting a signal from the c0dcu1 VME crate computer, but this was not happening.
We disconnected the timing signal going into c0dcu1, coming from c0daqctrl, and connected the 1PPS directly from c0daqctrl to the Ref In for the Master Timing distribution box (blue box with lots of fibers coming out of it in 1X5).
We now have agreement in timing between front ends.
After several reboots we now have working RFM again, along with computers who agree on the current GPS time along with the frame builder.
RFM is back and testpoints should be happy.
We still don't have a working binary output for the X end. I may need to get a replacement backplane with more than 4 slots if the 1st slot of this board has the same problem as the large boards.
I have burt restored the c1ioo, c1mcs, c1rms, c1sus, and c1scx processes, and optics look to be damped.
The front ends seem to have different gps timestamps on the data than the frame builder has when receiving them.
One theory is we have fairly been doing SVN checkouts of the code for the front ends once a week or every two weeks, but the frame builder has not been rebuilt for about a month.
Alex is currently rebuilding the frame builder with the latest code changes.
It also suggests I should try rebuilding the frame builder on a semi-regular basis as updates come in.
[Kiwamu, Jenne, Koji, Suresh]
The following steps in this process were completed.
1) Secured the current ETMX (Old ETMY) with the earth quake stops.
2) Removed the OSEMs and noted the Sl no. of each and its position
3) Placed four clamps to mark the location of the current ETMX tower (Old ETMY's position on the table)
4) Moved the ETMX (Old ETMY) tower to the clean table flow bench. In the process the tower had to be tilted during removal because it was too tall to pass upright through the vacuum chamber port. It was scary but nothing went wrong.
5) Koji calculated the location of the new ETMX and told us that it should be placed on the north end of the table.
6) Moved the OSEM cables, counter balancing weights and the 'chopper' out of the way. Had to move some of the clamps securing the cables.
7) Moved the ETMU07 tower from the clean room to the ETMX table
8) Positioned the OSEMs as they were placed in the earlier tower and adjusted their position to the middle of the range of their shadow sensors. The four OSEMs on the face did not give us any trouble and were positioned as required. But the side OSEM could not be put in place. The magnet on the left side, which we are constrained to use since the tower is not designed to hold an OSEM on the right side, seems a little too low (by about a mm) and does not interrupt the light beam in the shadow sensor. The possible causes are
a) the optic is rotated. To check this we need to take the tower back to the clean room and check the location of the optic with the traveling microscope. If indeed it is rotated, this is easy to correct.
b) the magnet is not located at the correct place on the optic. This can also be checked on the clean room optical bench but the solution available immediately is to hold the OSEM askew to accommodate the magnet location. If time permits the magnet position can be corrected.
We have postponed the testing of the ETMU07 tower to 1st of Nov Dec.
Restarted the elog again, this time using .../elog/start-elog.csh, which Joe pointed out works just fine. I have amended the wiki instructions to point to this script, instead.
We put ETMX back in its tower, and confirmed its balance. It might be pointing a teensy bit upward, but it is way less than the DC pointing offset we see when we put the OSEMs in the towers (since the PDs and LEDs have some magnetic bits to them).
Discussions are ongoing as to where the ETM should sit on its table, but we'll probably toss it into the chamber later this evening.
I took ETMY out of the magnet gluing fixture, and put it in a ring, in the foil house. It is ready to have the wire winched and get balanced at our convenience.
The updated status table:
1) Turns out the /opt/rtcds/caltech/c1/target/gds/param/testpoint.par file had been emptied or deleted at one point, and the only entry in it was c1pem. This had been causing us a lack of test points for the last few days. It is unclear when or how this happened. The file has been fixed to include all the front end models again. (Fixed)
2) Alex and I worked on tracking down why there's a GPS difference between the front ends and the frame builder, which is why we see a 0x4000 error on all the front end GDS screens. This involved several rebuilds of the front end codes and reboots of the machines involved. (Broken)
3) Still working on understanding why the RFM communication, which I think is related to the timing issues we're seeing. I know the data is being transferred on the card, but it seems to being rejected after being red in, suggesting a time stamp mismatch. (Broken)
4) The c1iscex binary output card still doesn't work. (Broken)
Alex and I will be working on the above issues tomorrow morning.
Currently, the c1ioo, c1sus and c1iscex computers are running with their front ends. They all still have 0x4000 error. However, you can still look at channels on dataviewer for example. However, there's a possibility of inconsistent timing between computer (although all models on a single computer will be in sync).
All the front ends where burt restorted to 07:07 this morning. I spot checked several optic filter banks and they look to have been turned on.
I didn't realize that the script in .../elog was configured to start 2.8.0 already. I will revert the instructions on the wiki to point to that