We were stymied tonight by a problem which began late this afternoon. The MC would periodically go angularly unstable, breaking lock and tripping the MC2 watchdogs. Suspicion fell naturally upon McWFS.
Eventually I traced the problem to the MC3 SIDE damping, which appeared to not work--it wouldn't actually damp, and the Vmon values did not correspond to the SDSEN outputs. Suspicion fell on the coil driver.
Looking at the LEMO monitors on the MC3 coil driver, with the damping engaged, showed clear bit resolution at the 100mV level, indicating a digital/DAC problem. Rebooting c1sosvme, which acquires all the OSEM sensor signals and actually does the side damping, resolved the issue.
Lies! The problem was not resolved. The plot shows a 2-day trend, with the onset of the problem yesterday clearly visible as well as the ineffectiveness of the soft-reboot done yesterday. So we'll try a hard-reboot.
So, it appears that one doesn't even have to change the Marconi set frequency to alter the phase of the output signal. It appears that other front panel actions (turning external modulations on/off, changing the modulation type) can do it as well. At least that's what I conclude from earlier this morning, when after setting up the f2 Marconi (166MHz) for external AM, the double-demod handoff in the DRMI no longer worked. Luckily this isn't a real problem now that we have the setDDphases and senseDRM scripts.
The real problem is that we are using frequency synthesizers to make the beat signals (133 and 199) instead of mixers. Luckily, the future 40m will not use beat signals (?) or synthesizers.
I ran the armLoss script for both Xarm and Yarm. The results are confidential, pending the completion of Alberto's cavity pole/finesse measurement due to the 'bet' as to what the new losses are after the drag wiping.
If you're the kind of person who likes to look at their Chrismas presents, the log files with the results are in the usual place for this script: /scripts/LSC/loss-ARM-GPStime.log (loss-Y-944865071.log and loss-X-944865946.log)
I checked the situation from my home and the problem was solved.
The main problem was undefined state of the autolocker and the strange undefined switch states, being associated with the bootfest and burtrestore.
- MC UP/DOWN status shows it was up and down. So I ran scripts/MC/mcup and scripts/MC/mcdown. These cleared the MC autolocker status.
- I had a problem handling the FSS. After mcup/mcdown above, I randomly pushed the "enable/disable" buttons and others, and with some reason, it recovered the handling. Actually it acquired the lock autonomously. Kiwamu may have also been working on it at the same time???
- Then, I checked the PSL loop. I disconnected the loop by pushing the "test" button. The DC slider changes the PZT voltage only 0~+24V. This is totally strange and I started pushing the buttons randomly. As soon as I pushed the "BLANK"/"NORMAL" button, the PZT output got back under the control.
- Then I locked the PMC, MZ, and MC as usual.
Alberto: You must be careful as the modulations were restored.
It's been an iffy last few hours here at the 40m. Kiwamu, Koji and I were all sitting at our desks, and the computers / RFM network decided to crash. We brought all of the computers back, but now the RefCav and PMC don't want to lock. I'm a wee bit confused by this. Both Kiwamu and I have given it a shot, and we can each get the ref cav to sit and flash, but we can't catch it. Also, when I bring the PMC slider rail to rail, we see no change in the PMC refl camera. Since c1psl had been finicky coming back the first time, I tried soft rebooting, and then keying the crate again, but the symptoms remained the same. Also, I tried burt restoring to several different times in the last few days, to see if that helped. It didn't. I did notice that MC2 was unhappy, which was a result of the burtrestores setting the MCL filters as if the cavity were locked, so I manually ran mcdown. Also, the MC autolocker script had died, so Kiwamu brought it back to life.
Since we've spent an hour on trying to relock the PSL cavities (the descriptive word I'm going to suggest for us is persistent, not losers), we're giving up in favor of waiting for expert advice in the morning. I suppose there's something obvious that we're missing, but we haven't found it yet......
This is a (sort of) known problem with the EPICS computers: it's generally called the 'sticky slider' problem, but of course it applies to buttons as well. It happens after a reboot, when the MEDM control/readback values don't match the actual applied voltages. The solution (so far) is just to `twiddle' the problematic sliders/button. There's a script somewhere called slider_twiddle that does this, but I don't remember if it has PSL stuff in it. A better solution is probably to have an individual slider twiddle script for each target machine, and add the running of that script to the reboot ritual in the wiki.
Steve showed me how to send an international fax today:
Alberto is visiting us from Australia. He brought some terrific presents. It is going to be very demanding task to wait for the rest of the 40m team
to return from Wales to taste coffee: PNG Peaberry of Wagonga, Monsooned Malabar of Jindebah and Signature Blue Blend of Cosmorex.
We have placed some sweet giant strawberries in the fridge; free for eating for anyone working in the lab today or tomorrow:
Trying to take an image or movie of the ETMY Transmon cam, we got instead this attached image.
I think it is just some scattered green light, but others in the control room think that it is a message from somewhere or someone...
It is not an angel, it is clearly a four leaf clover (also known as "quadrifoglio"). It is very rare, it brings good luck!
Last week Rana and I struggled to figure out how to un-full-screen windows on the Ubuntu workstations that appeared to be stuck in some sort of full screen mode such that the "Titlebar" was not on the screen. Nothing seemed to work. We were in despair.
Well, there is now hope: it appears that this really is a "fullscreen" mode that can be activated by hitting F11. It can therefore easily be undone by hitting F11 again.
The summery pages are working at a slow motion speed. It's response time 12 minutes.
this afternoon we centered the optical levers for all the optics.
To do that we first ran the alignment scripts for all the cavities.
ITMX Pitch: 142 microrad/counts
ITMX Yaw: 145 microrad/counts
ITMY Pitch: 257 microrad/counts
ITMY Yaw: 206 microrad/counts
ETMX Pitch: 318 microrad/counts
ETMX Yaw: 291 microrad/counts
ETMY Pitch: 309 microrad/counts
ETMY Yaw: 299 microrad/counts
BS Pitch: 70.9 microrad/counts
BS Yaw: 96.3 microrad/counts
PRM Pitch: 78.5 microrad/counts
PRM Yaw: 79.9 microrad/counts
SRM Pitch: 191 microrad/counts
SRM Yaw: 146 microrad/counts
After looking at some oplev noise spectra in DTT, we discovered that the ETMY quad (serial number 115) was noisy. Particularly, in the XX_OUT and XX_IN1 channels, quadrants 2 (by a bit more than an order of magnitude over the ETMX ref) and 4 (by a bit less than an order of mag). We went out and looked at the signals coming out of the oplev interface board; again, channels 2 and 4 were noise compared to 1 and 3 by about these same amounts. I popped in the ETMX quad and everything looked fine. I put the ETMX quad back at ETMX, and popped in Steve's scatterometer quad (serial number 121 or possibly 151, it's not terribly legible), and it looks fine. We zeroed via the offsets in the control room, and I went out and centered both the ETMX and ETMY quads.
Attached is a plot. The reference curves are with the faulty quad (115). The others are with the 121.
I adjusted the ETMY quad gains up by a factor of 10 so that the SUM is similar to what it was before.
Tonight I centered the oplevs for ITMX/Y, SRM, PRM, BS.
After doing that I noticed that the BS drifted a little from where I had set it.
I measured the ETMY oplev beam size at a couple different distances away from the HeNe by taking out the steering mirror and letting the light propagate a ways. I put the steering mirror back, aligned the oplev, and was able to relock the Yarm, so I think it's all back as it has been the last couple of weeks.
Now I need t o do some geometry and ray-tracing matrices to decide what focal length lens to buy, then we'll have a shiny new ETMY oplev.
ETMY oplev is currently a work in progress. The HeNe beam is hitting the photodiode, but the spot size there is pretty much the size of the entire QPD. Thus, the ETMY oplev isn't really useful right now. I'm re-figuring things out (note to self: close to the laser, you have to use Gaussian optics...regular ray tracing doesn't really work), and hopefully will have the oplev back under control by the time Alberto is finished realigning the IFO, so this doesn't keep anyone from doing any exciting locking work.
ETMY oplev is still out of order. Hopefully I'll get it under control by tomorrow.
We aligned the full IFO, and centered all of the oplevs and the IP_POS and IP_ANG QPDs. During alignment of the oplevs, the oplev servos were disabled.
Koji updated all of the screenshots of 10 suspension screens. I took a screenshot (attached) of the oplev screen and the QPD screen, since they don't have snapshot buttons.
We ran into some trouble while aligning the IFO. We tried running the regular alignment scripts from the IFO_CONFIGURE screen, but the scripts kept failing, and reporting "Data Receiving Error". We ended up aligning everything by hand, and then did some investigating of the c1lsc problem. With our hand alignment we got TRX to a little above 1, and TRY to almost .9 . SPOB got to ~1200 in PRM mode, and REFL166Q got high while in DRM (I don't remember the number). We also saw a momentary lock of the full initerferometer: On the camera view we saw that Yarm locked by itself momentarily, and at that same time TRX was above 0.5 - so both arms were locked simultaneously. We accepted this alignment as "good", and aligned all of the oplevs and QPDs.
It seems that C1LSC's front end code runs fine, and that it sees the RFM network, and the RFM sees it, but when we start running the front end code, the ethernet connection goes away. That is, we can ping or ssh c1lsc, but once the front end code starts, those functions no longer work. During these investigations, We once pushed the physical reset button on c1lsc, and once keyed the whole crate. We also did a couple rounds of hitting the reset button on the DAQ_RFMnetwork screen.
A "Data Receiving Error" usually indicates a problem with the framebuilder/testpoint manager, rather than the front-end in question. I'd bet there's a DTT somewhere that's gone rogue.
We restarted daqd and it did restored the problem
Then restart the 'daqd' process:'telnet fb40m 8087', type "shutdown" at the prompt. The framebuilder will restart itself in ~20s.
telnet fb40m 8087
It did not related to the problem, but we also cleaned the processes related to dtt, dataviewer by pkill
After that the alignment scripts started to work again. As a result, we got some misalignment of the oplevs.
I am going to come on Sunday
- Align the optics
- Align the oplevs again
- Take snapshots for the suspensions
- Align the IP_POS, IP_ANG
- Align the aux laser for the absolute length
- Align PSL table QPDs, and MCT QPD
NOTE: HEPA is on at its full.
[[[OK]]] Align the suspended optics (by Rob)
[[[OK]]] Align the oplevs again
[[[OK]]] Take snapshots for the suspensions/QPDs/IO QPDs/PZT strain gauges
[[[OK]]] Align the IP_POS, IP_ANG
[[[OK]]] Align the PSL table QPDs, the MC WFS QPDs, and the MCT QPD
[[[OK]]] Align the aux laser for the absolute length
Align the suspended optics (by Rob)
Align the oplevs again
Take snapshots for the suspensions/QPDs/IO QPDs/PZT strain gauges
Align the IP_POS, IP_ANG
Align the PSL table QPDs, the MC WFS QPDs, and the MCT QPD
Align the PSL table QPDs, the MC WFS QPDs, and the MCT QPD
Align the aux laser for the absolute length
o Go to only ITMX mode:
Save the alignment of the mirrors. Activate X-arm mode. Misalign ITMY and ETMX.
o Inject the aux beam:
Open the shutter of the aux NPRO. Turn the injection flipper on.
o Look at the faraday output:
There are several spots but only one was the right one. Confirm the alignment to the thorlabs PD. Connect the oscilloscope to the PD out with a 50Ohm termination.
Thanks to the Alberto's adjustment, the beat was already there at around 10MHz. After the PD adjustment, the DC was about 600mV, the beat amplitude was about 50mVpp.
o Adjust the aux beam alignment:
Adjust the alignment of the aux beam by the steering mirrors before the farady isolator. These only change the alignment of the aux beam independently from the IFO beam.
After the alignment, the beat amplitude of 100mVpp was obtained.
Close the shutter of the NPRO. Turn off the flipper mirror. Restore the full alignment of the IFO.
I set up instant green oplevs for ITMs.
A green laser pointer has been set on each end table. It illuminates the ITM center. The beam goea through the ETM substrate.
The reflected green beam returns to the ETM if the ITMs are aligned. Even though the reflected beam to the end is too big, this can
be a rough reference for each ITM.
Note: The green laser pointer at the ETMX were borrowed from Frank. We must return it to him when we finish the work.
The two acrylic optical table enclosures were moved from the carpenter shop to CES. I need to order windows. The latest quotes from Laseroptik are posted at the wiki / aux_optics page.
Things to do: order windows, draw and order window flange, install surgical tubing seals, buy and line enclosures with IR shield films.
For those of you who spend annoying amounts of time looking for tools, fear no more. Toolboxes for each optical table are coming!
They will probably have:
IR Viewer (a few optical tables will have IR viewers, these specific tables will be labeled in the diagram coming out later)
Ball screw drivers (3/16 in.) 6-8 in. handle
Various Connectors (I'll find out what's needed at some point)
Small flat screwdrivers (for adjusting camera gains)
Please suggest what else may be needed in these boxes.
The boxes will be held to the side of the tables, either by magnets or screws. A diagram of where they will be placed on each optical table in order to minimize obstruction of walkways will be distributed soon. Any objections can then be noted.
A heavy duty plastic box is the likeliest candidate for the optical table toolbox. It measures 5 9/16 in. x 11 5/8 in. x 4 5/8 in. and fits all the tools comfortably. ( http://www.mcmaster.com/#plastic-bin-boxes/=m4yh4m , under Heavy Duty Plastic Bin Boxes)
The list of tools has been updated to include a pen and a wire cutter as well as everything previously stated.
In addition, Steve has recommended that boxes should be secured to the walls or surfaces near the optical tables as opposed to the optical tables themselves, as to keep the tables from wobbling when tools are being exchanged.
A diagram of tentative box placements will go out soon.
I also took every allen key I can find so they can be sorted. They will be back in the appropriate drawer locations soon.
No, the small boxes must be attached to the optical tables. They won't be heavy enough to change the table tilt.
Also, all tools must be color coded according to the optical table using the 3M Vinyl table color code:
So the new tentative plan on the boxes is to bolt them (magnetic strips were proposed but overruled on the grounds that they're not strong enough to withstand being knocked down by accidents).
The boxes are going to be a mix of the Thorlabs Benchtop Organizer (http://www.thorlabs.com/thorProduct.cfm?partNumber=BT17) and the original box. The box will have a region covered in mesh, so tools can be placed and held there. The box will also have a spacer at the bottom, with another mesh right above it, lined up. However, this double-mesh will only cover half of the box. The other half of the box will be compartmentalized to hold things such as screws, connectors, etc. I will talk to Steve about building the boxes.
Also, using nail-polish to coat the Allen wrenches is not going to work. Nail polish does not stick easily enough. The tentative new plan is oil paint, but this is to be reviewed.
Finally, the diagram with the placement of the boxes relative to the optical tables has been put on paper, but needs to be computerized so it's easier to read. This will be done as soon as possible.
There are some tips for how to appy nail polish on YouTube from MKNails and MissJenFABULOUS. Their tips on how to prepare the site for a strong bonding strength are probably helpful for our gold/nickel coated tools. For chrome tools we may need to abrade the surface with a stone or fine sandpaper for it to take the layer better. IF the YouTube videos don't do it for you, then I suggest contacting Tom Evans at LLO to find out what kind of nail polish he uses.
This is the tentative box placement per optical table. The toolboxes are going to be color-coded by a combination of two colors (the order won't matter). The side of each toolbox will have a little panel to let you know which box corresponds to which set of colors.
On the diagram, the set of colors is simply the color of the box border and the color of the text.
If anyone has a problem with any of the colors or the box placement let me know before they are installed and become an annoyance:
ETMY: Box will be attached to the underside of the table by magnets. The box will be on the north side of the optical table.
POY: Box will be attached to the side of the optical table by magnets. The box will be on the west side of the optical table.
BSPRM: Box will be attached to the side of the optical table by magnets. The box will be on the west side of the optical table.
AS: Box will be attached to the side of the optical table by magnets. The box will be on the north side of the optical table.
PSL1: Box will be inside the optical table, in the northeast corner.
PSL2: Box will be inside the optical table, in the southwest corner.
POX: Box will be attached to the side of the optical table by magnets. The box will be on the south side of the optical table.
MC2: Box will be attached to the side of the optical table by magnets. The box will be on the south side of the optical table.
ETMX: Box will be attached to the side of the optical table by magnets. The box will be on the east side of the optical table.
I decided to go see what the electrical tape looks like on the other tools.
These are the tools I felt were necessary to label with tape: (the others don't seem to be terribly important in terms of not interchanging between boxes)
On another note I'm not sure why electrical tape can't be used on the Allen Wrenches too.
I also plan on ordering smaller flash lights for each table (this one is bulky and unwieldy), and filling in the gaps of the Allen Wrench sets as soon as I get the go-ahead.
The fibers should be routed beneath the electrical cables.
They should be fixed on the table for strain relieving.
The slack of the fibers should be nicely rolled and put together at the splitter side.
These are expected to be done next time when the fiber team work around the table.
We also expect to have the table photo every time the work of the day is finished.
Albert, our new undergrad work force received 40m specific- basic safety training last week. Please read and sign 40m procedures booklet.
These are the tentative box placements. Roughly. I don't actually have the box finalized yet, but the box should be around that size.
How do I perspective ._.
Recommended correction list:
1, refill- upgrade first aid boxes
2, maintain 18" ceiling to bookshelf clearance so the ceiling fire sprinklers are not blocked: room 101
3, label chilled water supply & return valves in IFO room
4, calibrate bake room hoods annually
5, update safety sign at fenced storage
40m still to do list:
1, clean and measure all safety glasses
2, annual crane inspection is scheduled for 8am March 19, 1013
3, make PSL encloser shelf earthquake proof
Do you see something that is not safe? Add it to this list please.
Restocked First Aid Kits Location:
Main entrance, room 100
Drill press - above N2 cylinders, room 103
Control room, next to fire extinguisher, room 102
Vertex-north wall, IFO room 104
ETMY - right on ends light switches, IFO room 104_ east end
ETMX - on vertical I-beam of crane, IFO room 104_south end
Behind 1X3 Rack, on south wall - under instrument breakers panel PC-1, IFO room 104
Last thing remaining to be fixed from 2013 Safety Audit:
replace book shelf with 83" height
Our early bird surf student Gautem has received 40m specific basic safety training today.
Updated laser inventory and operator list. They are posted in the 40m wiki and entry doors of the 40m IFO room.
Let me know if this list needs correction.
Lightwave M126N-1064-700 NPRO sn 337 in the PSL enclosure got connected to local emergency shut off switch. This is a LIGO operational safety requirement.