40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 27 of 341  Not logged in ELOG logo
ID Date Author Type Category Subjectup
  1736   Mon Jul 13 00:53:50 2009 AlbertoDAQComputersAll computers down

Quote:

Quote:

I popped by the 40m, and was dismayed to find that all of the front end computers are red (only framebuilder, DAQcontroler, PEMdcu, and c1susvmw1 are green....all the rest are RED).

 

I keyed the crates, and did the telnet.....startup.cmd business on them, and on c1asc I also pushed the little reset button on the physical computer and tried the telnet....startup.cmd stuff again.  Utter failure. 

 

I have to pick someone up from the airport, but I'll be back in an hour or two to see what more I can do.

 I think the problem was caused by a failure of the RFM network: the RFM MEDM screen showed frozen values even when I was power recycling any of the FE computers. So I tried the following things:

- resetting the RFM switch
- power cycling the FE computers
- rebooting the framebuilder
 
but none of them worked.  The FEs didn't come back. Then I reset C1DCU1 and power cycled C1DAQCTRL.
 
After that, I could restart the FEs by power recycling them again. They all came up again except for C1DAQADW. Neither the remote reboot or the power cycling could bring it up.
 
After every attempt of restarting it its lights on the DAQ MEDM  screen turned green only for a fraction of a second and then became red again.
 
So far every attempt to reanimate it failed.

 

After Alberto's bootfest which was more successful than mine, I tried powercycling the AWG crate one more time.  No success.  Just as Alberto had gotten, I got the DAQ screen's AWG lights to flash green, then go back to red.  At Alberto's suggestion, I also gave the physical reset button another try.  Another round of flash-green-back-red ensued.

When I was in a few hours ago while everything was hosed, all the other computer's 'lights' on the DAQ screen were solid red, but the two AWG lights were flashing between green and red, even though I was power cycling the other computers, not touching the AWG at the time.  Those are the lights which are now solid red, except for a quick flash of green right after a reboot.

I poked around in the history of the curren and old elogs, and haven't found anything referring to this crazy blinking between good and bad-ness for the AWG computers.  I don't know if this happens when the tpman goes funky (which is referred to a lot in the annals of the elog in the same entries as the AWG needing rebooting) and no one mentions it, or if this is a new problem.  Alberto and I have decided to get Alex/someone involved in this, because we've exhausted our ideas. 

  6997   Fri Jul 20 17:11:50 2012 JamieUpdateCDSAll custom MEDM screens moved to cds_users_apps svn repo

Since there are various ongoing requests for this from the sites, I have moved all of our custom MEDM screens into the cds_user_apps SVN repository.  This is what I did:

For each system in /opt/rtcds/caltech/c1/medm, I copied their "master" directory into the repo, and then linked it back in to the usual place, e.g.:

a=/opt/rtcds/caltech/c1/medm/${model}/master
b=/opt/rtcds/userapps/trunk/${system}/c1/medm/${model}
mv $a $b
ln -s $b $a

Before committing to the repo, I did a little bit of cleanup, to remove some binary files and other known superfluous stuff.  But I left most things there, since I don't know what is relevant or not.

Then committed everything to the repo.

 

  2386   Thu Dec 10 13:50:02 2009 JenneUpdateVACAll doors on, ready to pump

[Everybody:  Alberto, Kiwamu, Joe, Koji, Steve, Bob, Jenne]

The last heavy door was put on after lunch.  We're now ready to pump.

  7920   Sat Jan 19 15:05:37 2013 JenneUpdateComputersAll front ends but c1lsc are down

Message I get from dmesg of c1sus's IOP:

[   44.372986] c1x02: Triggered the ADC
[   68.200063] c1x02: Channel Hopping Detected on one or more ADC modules !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[   68.200064] c1x02: Check GDSTP screen ADC status bits to id affected ADC modules
[   68.200065] c1x02: Code is exiting ..............
[   68.200066] c1x02: exiting from fe_code()

Right now, c1x02's max cpu indicator reads 73,000 micro seconds.  c1x05 is 4,300usec, and c1x01 seems totally fine, except that it has the 02xbad.

c1x02 has 0xbad (not 0x2bad).  All other models on c1sus, c1ioo, c1iscex and c1iscey all have 0x2bad.

Also, no models on those computers have 'heartbeats'.

C1x02 has "NO SYNC", but all other IOPs are fine.

I've tried rebooting c1sus, restarting the daqd process on fb, all to no avail.  I can ssh / ping all of the computers, but not get the models running.  Restarting the models also doesn't help.

upside_down_cat-t2.jpg

c1iscex's IOP dmesg:

[   38.626001] c1x01: Triggered the ADC
[   39.626001] c1x01: timeout 0 1000000
[   39.626001] c1x01: exiting from fe_code()

c1ioo's IOP has the same ADC channel hopping error as c1sus'.

 

  7922   Sat Jan 19 18:23:31 2013 ranaUpdateComputersAll front ends but c1lsc are down

After sshing into several machines and doing 'sudo shutdown -r now', some of them came back and ran their processes.

After hitting the reset button on the RFM switch, their diagnostic lights came back. After restarting the Dolphin task on fb:

"sudo /etc/init.d/dis_networkmgr restart"

the Dolphin diagnostic lights came up green on the FE status screen.

iscex still wouldn't come up. The awgtpman tasks on there keep trying to start but then stop due to not finding ADCs.

 

Then power cycled the IO Chassis for EX and then awtpman log files changed, but still no green lights. Then tried a soft reboot on fb and now its not booting correctly.

Hardware lights are on, but I can't telnet into it. Tried power cycling it once or twice, but no luck.

Probably Jamie will have to hook up a keyboard and monitor to it, to find out why its not booting.

FE.png

P.S. The snapshot scripts in the yellow button don't work and the MEDM screen itself is missing the time/date string on the top.

  14241   Wed Oct 10 12:38:27 2018 yukiConfigurationLSCAll hardware was installed

I connected DAC - AIboard - PZTdriver - PZT mirrors and made sure the PZT mirrors were moving when changing the signal from DAC. Tomorrow I will prepare alignment servo with green beam for Y-arm.

  13885   Thu May 24 10:16:29 2018 gautamUpdateGeneralAll models on c1lsc frontend crashed

All models on the c1lsc front end were dead. Looking at slow trend data, looks like this happened ~6hours ago. I rebooted c1lsc and now all models are back up and running to their "nominal state".

  12284   Sun Jul 10 15:54:23 2016 ericqUpdateCDSAll models restarted

For some reason, all of the non-IOP models on the vertex frontends had crashed.

To get all of the bits green, I ended up restarting all models on all frontends. (This means Gautam's coil tests have been interrupted.)

  7724   Mon Nov 19 15:15:22 2012 JenneUpdateSUSAll oplev gains turned on

Quote:

Oplev values that were changed to zero:

PRM P=0.15, Y=-0.3

SRM P=-2.0, Y=2.0

BS P=0.2, Y=-0.2

ITMY P=2.1, Y=-2.0

ITMX P=1.0, Y=-0.5

ETMX P=-0.2, Y=-0.2

ETMY P=0.5, Y=0.6

Also, PRCL was changed in the LSC input matrix from REFL33I to AS55I, since there is no REFL beam out of the IFO :(

 Ayaka and I restored all of the oplev gains to these values.  The exception is ETMY, which has both gains negative.  I am unsure if this is a transcription error on my part, or if something physical has changed.  The layout of the ETMY oplev was modified (since Rana took out the offending lens) but that shouldn't affect the sign of the gains.

  7551   Mon Oct 15 22:16:09 2012 JenneUpdateSUSAll oplev gains turned to 0

Steve has promised to fix up all of the oplevs, but it hasn't happened yet, so I've turned all of the oplev gains to zero, so that when the optics are restored we don't have to quickly click them off.

Oplev values that were changed to zero:

PRM P=0.15, Y=-0.3

SRM P=-2.0, Y=2.0

BS P=0.2, Y=-0.2

ITMY P=2.1, Y=-2.0

ITMX P=1.0, Y=-0.5

ETMX P=-0.2, Y=-0.2

ETMY P=0.5, Y=0.6

Also, PRCL was changed in the LSC input matrix from REFL33I to AS55I, since there is no REFL beam out of the IFO :(

  11054   Fri Feb 20 12:29:01 2015 ericqUpdateCDSAll optics damped

I noticed some diagnostic bits in the c1sus IOP complaining about user application timing and FIFO under/overflow (The second and fourth squares next to the DACs on the GDS screen.) Over in crackle-cymac land, I've seen this correlate with large excess DAC noise. After restarting all models, all but one of these is green again, and the optics are now all damped. 

It seems there were some fishy BURT restores, as I found the TT control filters had their inputs and outputs switched off. Some ASS filters were found this way too. More undesired settings may still lurk in the mists...

The interferometer is now realigned, arms locked. 

  5007   Wed Jul 20 20:44:56 2011 JamieUpdateSUSAll sus models rebuilt and restarted

There were a couple of recent improvements to the sus_single_control model that had not been propagated to all of the suspension controllers.

Rebuilt and restarted c1mcs, c1sus, c1scx, and c1scy.  Everything seems to be working fine after restart.

  1975   Tue Sep 8 17:57:30 2009 JenneUpdatePEMAll the Acc/Seis working again

All of the accelerometers and seismometers are plugged in and functional again.  The cables to the back of the accelerometer preamp board (sitting under the BS oplev table) had been unplugged, which was unexpected.  I finally figured out that that's what the problem was with half of the accelerometers, plugged them back in, and now all of the sensors are up and running.

TheSEIS_GUR seismometer is under MC1, and all the others (the other Guralp, the Ranger which is oriented vertically, and all 6 accelerometers) are under MC2.

  1146   Wed Nov 19 10:32:11 2008 AlbertoConfigurationElectronicsAll the Marconi Set to the Rubidium Frequency Standard
I placed the SRS Rubidium FS275 over the PSL rack, next to the frequency counter. This one and the Marconi on the PSL rack have been connected to the 10MHz output of the frequency standard. I set also the first Marconi, the one that used to drive the others, to external, direct frequency reference. Now it reads 166981718 Hz versus 166981725 Hz measured by the frequency counter: 8 Hz difference.
  1147   Wed Nov 19 18:02:18 2008 ranaConfigurationElectronicsAll the Marconi Set to the Rubidium Frequency Standard
Not sure what was going on before. I changed the frequency counter to use an AC coupled input, and had it average
for 50 seconds. The output now agrees with the Marconi front panel to less than 1 Hz. Its still not 0.000 Hz,
but not bad.
  4084   Tue Dec 21 16:34:42 2010 kiwamuSummaryVACAll the test masses have been wiped

 [Jenne and Kiwamu]

 We wiped all the test masses with isopropyl alcohol.

They became apparently much cleaner.

(how to)

 At first we prepared the following stuff:

  * syringe

  * isopropyl alcohol 

  * lens papers

  * cleaned scissors

  Then we cut the lens papers into the half by the scissors such that the long side can remain.

This is because that the SOSs have relatively narrow spaces at their optic surfaces for putting a piece of paper. 

   We did vertical and horizontal wiping using the lens paper and appropriate amount of isopropyl alcohol.

Each wiping (vertical and horizontal) requires two or three times trials to appropriately remove dusts.

Amount of isopropyl:

   * vertical 15 [ul]

   * horizontal 10 [ul]

In addition to this, we also used the ionizer gun for blowing big dusts and fiber away from the surface.

 

 

(surface inspection)

   Before wiping them, all the test masses had small dusts uniformly distributed on the HR surfaces.

Especially ETMX was quite dirty, many small spots (dusts) were found when we shined the surface with the fiber illuminator.

ETMY was not so bad, only a couple of small dusts were at around the center.  ITMX/Y had several dusts, they were not as dirty as ETMX, but not cleaner than ETMY.

   After we wiped them,  we confirmed no obvious dusts were around the centers of the optics. They looked pretty good !

 

 

  8962   Fri Aug 2 22:51:10 2013 JenneUpdateGeneralAll vent tasks complete, just need oplev check

[Manasa, Koji, Jenne]

We went into the BS and IOO chambers, and aligned the green beams such that they came out of the vacuum chamber.  The idea here was to get the beams at the same height, but slightly offset in yaw.  This required moving the Periscope on BS table, PBS in front of that periscope, the Periscope on the IOO table, and 2 steering mirrors on the IOO table after the 2nd periscope.  The tables were not releveled, although we have aligned the full interferometer to this situation, so we do not want to touch the tables.  The MC spot positions are still consistent with those measured earlier this afternoon, before this work, so I'm not concerned.

We confirmed that both green beams are hitting a good place (centered in pitch, and just left and right of center in yaw) on the mirror in the OMC chamber, and are getting to the center of the first mirror on the PSL table.  We then coarsely aligned the beams on the PSL table.

We then relocked and aligned the arms for IR, and checked that the AS beam is centered on the mirrors in the BS chamber, and that the beam is coming out, and to the AS table.  I touched the last mirror before PZT3 a small amount in yaw, and then PZT3 in pitch and yaw, until we saw the beam recentered on the first mirror on the AS table.  At that point, we were also back to the center of the AS camera (which is good, since Koji had aligned all of that the other day).  So, the AS beam is good.

We checked IPPOS, and have centered the beam on all the mirrors, and aligned the beam onto the QPD. 

We checked IPANG, by looking through the viewports at the mirrors in the ETMY chamber.  We are now centered in yaw, but clipping a bit low.  This is what we want, since we always end up drifting high during the pump-down. 

We see a nice, unclipped REFL beam on the camera.

We see a beam a little high on the POP camera, but Koji looked on the table with a card, and saw the beam....we just need to do minor alignment on the out of vac mirrors.

We checked again that the green TEM00 beams from both arms come to the PSL table. 

We are getting POX and POY out, since we are using them to lock and align the arms.

Manasa and Koji recovered one clean allen key from the bottom of the chambers, but one remains, as a sacrifice to the vacuum gods. 

I believe that, with the exception of checking the oplevs and taking photos of PR3, and the green steering optics, we have finished all of our vent tasks.  We should do a quickie alignment on Monday, check the oplevs, take some photos, and put on the heavy doors.  Pumping can start either Monday afternoon or Tuesday morning. 

 

  15578   Wed Sep 16 17:44:27 2020 gautamUpdateCDSAll vertex FE models restarted

I had to make a CDS change to the c1lsc model in an effort to get a few more signals into the models. Rather than risk requiring hard reboots (typcially my experience if I try to restart a model), I opted for the more deterministic scripted reboot, at the expense of spending ~20mins to get everything back up and running.


Update 2230: this was more complicated than expected - a nuclear reboot was necessary but now everything is back online and functioning as expected. While all the CDS indicators were green when I wrote this up at ~1800, the c1sus model was having frequent CPU overflows (execution time > 60 us). Not sure why this happened, or why a hard power reboot of everything fixed it, but I'm not delving into this.

The point of all this was that I can now simultaneously digitize 4 channels - 2 DCPDs, and 2 demodulated quadratures of an RF signal.

  14591   Fri May 3 09:12:31 2019 gautamUpdateSUSAll vertex SUS watchdogs were tripped

I found the 8 vertex watchdogs tripped today morning. The ETMs were fine, suggesting this was not an actual earthquake. I suspect it was connected to this remote work? Was there a reason why they were left tripped?

On a side note - I don't think we log the watchdog state explicitly. We can infer whether the optic is damped by looking at the OSEM sensor time series, but do we want to record the watchdog state to frames?

  14596   Mon May 6 11:05:23 2019 JonUpdateSUSAll vertex SUS watchdogs were tripped

Yes, this was a consequence of the systemd scripting I was setting up. Unlike the old susaux system, we decided for safety NOT to allow the modbus IOC to automatically enable the coil outputs. Thus when the modbus service starts/restarts, it automatically restores all state except the watchdog channels, which are left in their default disabled state. They then have to be manully enabled by an operator, as I should have done after finishing testing.

Quote:

I found the 8 vertex watchdogs tripped today morning. The ETMs were fine, suggesting this was not an actual earthquake. I suspect it was connected to this remote work? Was there a reason why they were left tripped?

  305   Sat Feb 9 13:32:07 2008 JohnSummarySUSAll watchdogs tripped
When I arrived this afternoon the watchdogs had tripped on all optics. I reset them and enabled the coil currents.

I had to adjust the alignment of the mode cleaner to get it to lock.
  306   Sun Feb 10 20:47:01 2008 AlanSummarySUSAll watchdogs tripped
A moderate earthquake occurred at 11:12:06 PM (PST) on Friday, February 8, 2008.
The magnitude 5.1 event occurred 21 km (13 miles) NW of Guadalupe Victoria, Baja California, Mexico.
http://quake.wr.usgs.gov/recenteqs/Quakes/ci14346868.html
  15373   Wed Jun 3 19:19:11 2020 gautamUpdateSUSAll watchdogs tripped

This EQ seems to have knocked all suspensions out. ITMX was stuck. It is now released, and the IMC is locked again. It looks like there are some serious aftershocks going on so let's keep an eye on things.

  2467   Wed Dec 30 10:58:48 2009 AlbertoUpdateGeneralAll watchdogs tripped this morning

This morning I found all the watchdogs had tripped during the night.

I restored them all.

I can't damp ITMX. I noticed that its driving matrix is all 1s and -1s as the the right values had been lost in some previous burtrestoring.

  2468   Wed Dec 30 18:01:03 2009 Alberto, RanaUpdateGeneralAll watchdogs tripped this morning

WQuote:

This morning I found all the watchdogs had tripped during the night.

I restored them all.

I can't damp ITMX. I noticed that its driving matrix is all 1s and -1s as the the right values had been lost in some previous burtrestoring.

 

Rana fixed the problem. He found that the side damping was saturating. He lowered the gain a little for a while, waited for the the damping to slow down the optic and then he brought the gain back where it was.

He also upadted the MEDM screen snapshot.

  15155   Sun Jan 26 13:30:19 2020 gautamUpdateSUSAll watchdogs tripped, now restored

Looks like a M=4.6 earthquate in Barstow,CA tripped all the suspensions. ITMX got stuck. I restored the local damping on all the suspensions just now, and freed ITMX. Looks like all the suspensions damp okay, so I think we didn't suffer any lasting damage. IMC was re-aligned and is now locked.

  15335   Fri May 15 19:10:42 2020 gautamUpdateSUSAll watchdogs tripped, now restored

This EQ in Nevada seems to have tripped all watchdogs. ITMX was stuck. It was released, and all the watchdogs were restored. Now the IMC is locked.

  1294   Wed Feb 11 15:01:47 2009 josephbConfigurationComputersAllegra

So after having broke Allegra by updating the kernel, I was able to get it running again by copying the xorg.conf.backup file over xorg.conf in /etc/X11.  So at this point in time, Allegra is running with generic video drivers, as opposed to the ATI specific and proprietary drivers.

  2509   Tue Jan 12 11:34:26 2010 josephbUpdatePEMAllegra dataviewer

Quote:

So that we can use both Guralps for Adaptive stuff, and so that I can look at the differential ground motion spectra, I've reconnected the Guralp Seismometers to the PEM ADCU, instead of where they've been sitting for a while connected to the ASS ADC.  I redid the ASS.mdl file, so that the PEM and PEMIIR matricies know where to look for the Gur2 data.  I followed the 'make ass' procedure in the wiki.  The spectra of the Gur1 and Gur2 seismometers look pretty much the same, so everything should be all good.

There's a problem with DataViewer though:  After selecting signals to plot, whenever I hit the "Start" button for the realtime plots, DataViewer closes abruptly. 

When I open dataviewer in terminal, I get the following output:

allegra:~>dataviewer
Warning: communication protocol revision mismatch: expected 11.3, received 11.4
Warning: Not all children have same parent in XtManageChildren
Warning: Not all children have same parent in XtManageChildren
Warning: Not all children have same parent in XtManageChildren
Warning: Not all children have same parent in XtManageChildren
Warning: Not all children have same parent in XtManageChildren
Warning: communication protocol revision mismatch: expected 11.3, received 11.4
msgget: No space left on device
allegra:~>framer4 msgget:msqid: No space left on device

Does anyone have any inspiration for why this is, or what the story is?  I have GR class, but I'll try to follow up later this afternoon.

 This problem seems to be restricted to allegra.  Dataviewer works fine on Rosalba  and op440m, as well as using ssh -X into rosalba to run dataviewer remotely.  DTT seems to work fine on allegra.  The disk usage seems no where near full on allegra.  Without knowing which "device" its refering to (although it must be a device local to allegra), I'm not sure what to look at now. 

I'm going to do a reboot of allegra and see if that helps.

Update:  The reboot seems to have fixed the issue.

 

 

 

 

  3017   Sun May 30 17:51:04 2010 kiwamuHowToPEMAllegra dataviewer

I found the dataviewer didn't work only on Allegra. This thing sometimes happened as described in the past entry.

I rebooted Allegra, then the problem was fixed.

 

  15246   Wed Mar 4 11:10:47 2020 YehonathanUpdateComputersAllegra revival

Allegra had no network cable and no mouse. We found Allegra'snetwork cable (black) and connected it.

I found a dirty old school mouse and connected it.

I wiped Allegra and now I'm currently installing debian 10 on allegra following Jon's elog.

04/01 update: I forgot to mention that I tried installing cds software by following Jamie's instruction: I added the line in /etc/apt/sources.list.d/lscsoft.list: "deb http://software.ligo.org/lscsoft/debian/ stretch contrib". But this the only thing I managed to do. The next command in the instructions failed.

  5778   Tue Nov 1 18:45:48 2011 JenneUpdateComputersAllegra's screens

I was trying to give Allegra a second head, and it didn't quite work.  It's still in progress.  Steve, you might not like how I've 'mounted' the second monitor, so we can talk about something that might work tomorrow.

  14425   Fri Feb 1 01:24:06 2019 gautamUpdateSUSAlmost ready for pumpdown tomorrow

[koji, chub, jon, rana, gautam]

Full story tomorrow, but we went through most of the required pre close-up checks/tasks (i.e. both arms were locked, PRC and SRC cavity flashes were observed). Tomorrow, it remains to 

  1. Confirm clearance between elliptical reflector and ETMY
  2. Confirm leveling of ETMY table
  3. Take pics of ETMY table
  4. Put heavy door on ETMY chamber
  5. Pump down

The ETMY suspension chain needs to be re-characterized (neither the old settings, nor a +/- 1 gain setting worked well for us tonight), but this can be done once we are back under vacuum.

  17116   Wed Aug 31 01:22:01 2022 KojiUpdateGeneralAlong the X arm part 1

 

Attachment 5: RF delay line was accommodated in 1X3B. (KA)

  17129   Fri Sep 2 15:30:10 2022 AnchalUpdateGeneralAlong the X arm part 1

[Anchal, Radhika]

Attachment 2: The custom cables which were part of the intermediate setup between old electronics architecture and new electronics architecture were found.
These include:

  • 2 DB37 cables with custom wiring at their connectors to connect between vacuum flange and new Sat amp box, marked J4-J5 and J6-J7.
  • 2 DB15 to dual head DB9 (like a Hydra) cables used to interface between old coil drivers and new sat amp box.

A copy of these cables are in use for MC1 right now. These are spare cables. We put them in a cardboard box and marked the box appropriately.
The box is under the vacuum tube along Yarm near the center.

 

  17117   Wed Aug 31 01:24:48 2022 KojiUpdateGeneralAlong the X arm part 2

 

 

  17118   Wed Aug 31 01:25:37 2022 KojiUpdateGeneralAlong the X arm part 3

 

 

  17119   Wed Aug 31 01:30:53 2022 KojiUpdateGeneralAlong the X arm part 4

Behind the X arm tube

  17120   Wed Aug 31 01:53:39 2022 KojiUpdateGeneralAlong the Y arm part 1

 

 

  17121   Wed Aug 31 01:54:45 2022 KojiUpdateGeneralAlong the Y arm part 2
  17130   Fri Sep 2 15:35:19 2022 AnchalUpdateGeneralAlong the Y arm part 2

[Anchal, Radhika]

The cables in USPS open box were important cables that are part of the new electronics architecture. These are 3 ft D2100103 DB15F to DB9M Reducer Cable that go between coil driver output (DB15M on back) to satellite amplifier coil driver in (DB9F on the front). These have been placed in a separate plastic box, labeled, and kept with the rest of the D-sub cable plastic boxes that are part of the upgrade wiring behind the tube on YARM across 1Y2. I believe JC would eventually store these dsub cable boxes together somewhere later.

  6373   Wed Mar 7 13:59:07 2012 Ryan FisherSummaryComputer Scripts / ProgramsAlterations to base epics install for installing aLIGO conlog:

In order to install the necessary extensions to epics to make the aLIGO conlog work, I have edited one file in the base epics install that affects makefiles:

/cvs/cds/caltech/apps/linux64/epics/base/configure/CONFIG_COMMON

Jamie said he prefers diffs, so I regenerated the original file and did a diff against the current file:

megatron:configure>diff CONFIG_COMMON.orig.reconstructedMar72012 CONFIG_COMMON.bck.Mar72012
206,207c206,210
< USR_CPPFLAGS =
< USR_DBDFLAGS =
---
> USR_CPPFLAGS = -I $(EPICS_BASE)/include
> USR_CPPFLAGS += -I $(EPICS_BASE)/include/os/Linux/
> USR_CPPFLAGS += -I $(EPICS_BASE)/../modules/archive/lib/linux-x86_64/
> USR_DBDFLAGS = -I $(EPICS_BASE)/dbd
> USR_DBDFLAGS += -I $(EPICS_BASE)/../modules/archive/dbd

This is saved in CONFIG_COMMON.diff.Mar72012_1

  6377   Wed Mar 7 18:00:39 2012 Ryan FisherSummaryComputer Scripts / ProgramsAlterations to base epics install for installing aLIGO conlog:

Quote:

In order to install the necessary extensions to epics to make the aLIGO conlog work, I have edited one file in the base epics install that affects makefiles:

/cvs/cds/caltech/apps/linux64/epics/base/configure/CONFIG_COMMON

Jamie said he prefers diffs, so I regenerated the original file and did a diff against the current file:

megatron:configure>diff CONFIG_COMMON.orig.reconstructedMar72012 CONFIG_COMMON.bck.Mar72012
206,207c206,210
< USR_CPPFLAGS =
< USR_DBDFLAGS =
---
> USR_CPPFLAGS = -I $(EPICS_BASE)/include
> USR_CPPFLAGS += -I $(EPICS_BASE)/include/os/Linux/
> USR_CPPFLAGS += -I $(EPICS_BASE)/../modules/archive/lib/linux-x86_64/
> USR_DBDFLAGS = -I $(EPICS_BASE)/dbd
> USR_DBDFLAGS += -I $(EPICS_BASE)/../modules/archive/dbd

This is saved in CONFIG_COMMON.diff.Mar72012_1

 After following up with Patrick Thomas for a while trying to make the extensions to epics work within the currently installed epics, he decided that we should just start over with a fresh install of epics 3.14.10.

I am installing this in /ligo/apps/linux-x86_64/epics/base-3.14.10/

Prior to all of this, I had done a lot of installation and configuration of the packages needed to make LAMP work:

sudo apt-get install lamp-server^

sudo apt-get install phpmyadmin

I then continued to follow the instructions on Patrick's wiki:

https://awiki.ligo-wa.caltech.edu/aLIGO/Conlog#EDCU_library

I installed the c_string library into /ligo/apps/linux-x86_64/ according to his instructions.  (prior to my installs, there was no /ligo/ on this machine at all, so I made the needed parent directories with the correct permissions).

That should be everything up to installing epics (working on that now).

  6387   Thu Mar 8 17:18:19 2012 Ryan FisherSummaryComputer Scripts / ProgramsAlterations to base epics install for installing aLIGO conlog:
I have the aLIGO Conlog 'working' at http://192.168.113.209/conlog/conlog.php

The process is running inside a screen on megatron.

To start it running, you need to set your environment, and then run the startup.c1conlog script :

cd /cvs/cds/rtcds/caltech/c1/target/conlog/conlogepics

source conlog_environment.txt

./startup.c1conlog

This will leave you at an epics prompt, which means the code is running. (that's why I left it running in a screen for now).

To change the list of channels when the conlog is running, you need to edit the file (s):

/ligo/caltech/data/conlog/c1/add_channel_names
/ligo/caltech/data/conlog/c1/remove_channel_names

Then start up medm as follows:

cd ~/ryan/

medm -x -macro "IFO=C1" medm/CONLOG.adl

Then click the Add channel list button or Remove channel list button.

To change the channels before running the conlog from a blank database, you would edit:
/ligo/caltech/data/conlog/c1/use_channel_names (I believe this should be read whenever the conlog is restarted, but I'm only sure it does the first time you start conlog).




Documenting the rest of the installation:


Successful? Installation of Fresh EPICS and Extensions



Fresh Copy of EPICS 3.14.10


* We restarted (on Patrick's suggestion) with a fresh copy of EPICS 3.14.10 in:
/ligo/apps/linux-x86_64/epics
* I had to set a clean environment:
* Then I downloaded the tarball, unpacked it, and simply ran make within it, and it worked!
* Next, I followed Patrick's wiki instructions with only modifications to the configure/RELEASE files for the archive and ioc/conlog extensions.
* Then I realized that I had to rebuild conlog ioc after adding a directory:
/ligo/apps/linux-x86_64/epics/iocs/conlog/iocBoot/iocc1_conlog
* I had to copy the files from the h1 directory and modify them so that all reference to h1 now point to c1 in the new directory
* I then rebuilt the conlog ioc (I had to make sure setenv SCRIPTS was run again because I had logged out and back in, and I reset the whole environment properly again)

Rest of Install


* I was able to fairly trivially follow through the rest of the installation steps on Patrick's wiki, up to the "Design" section.
* Now, there is no obvious way to move forward (nothing is actually running I believe).

New Install Instructions:


"
You want to create the EPICS IOC boot directory by doing the following:

In the top level of the IOC (.../epics/iocs/conlog) with the appropriate
paths:

/epics/base/bin/linux-x86_64/makeBaseApp.pl -i -t ioc c1_conlog
What application should the IOC(s) boot?
The default uses the IOC's name, even if not listed above.
Application name? conlog

Then you will have to update the st.cmd it creates. You can compare the
st.cmd and st.cmd.backup files in the other directories to see what needs
to be changed.

I don't know if just copying the directory will work, it might.

You will also probably want to change the following line in
/epics/modules/archive/archiveApp/src/drvArchive.c:

queue = epicsMessageQueueCreate(100000000, sizeof(struct message_type));

and reduce 100000000 to something smaller depending on the amount of ram
available to the computer. I think sizeof(struct message_type) is
something like 112 bytes. Then recompile.

You basically put a file with the list of channels to use in the directory
path for "use channel list filepath" in the following command in st.cmd:
drvArchive_read_channel_list_filepaths <add channel list filepath>,<remove
channel list filepath>,<use channel list filepath>
I can send you the script that I currently use to generate that channel
list if you want, but it may need to be changed for your setup.

Once you start the ioc, open the medm which can be checked out from
subversion here: cds_user_apps/trunk/cds/common/medm/CONLOG.adl
with macro substitution for IFO: medm -x -macro "IFO=C1"
and click on the button for "Use channel list".
The number of monitored channels should increase to the number of channels
in the file.

-Patrick

...

The perl script and example file are attached, just redirect the output of
the perl script to a file. It scans autoBurt.req files in a particular
directory and its subdirectories for channel names that meet certain
criteria. All the file contains is a list of channel names, one on each
line. To start the IOC, go to the target directory and run
./startup.c1conlog.

"

{{rpfisher:scan_autoburt.pl.txt|scan_autoburt.pl.txt}}



{{rpfisher:use_channel_names.txt|use_channel_names.txt}}



My Notes Regarding These Instructions:


* Throughout the installation instructions, it probably should have been made clear that the ifo is lowercase: eg c1 (but in the end the installation mixed C1 and c1 in different places)
* Also throughout, one should be careful to replace lho with your site (eg caltech) wherever it appears
* After running the first perl script to generate the iocBoot/iocc1_conlog directory, the goal is to rebuild the whole conlog ioc by running make from epics/iocs/conlog, but before doing that:
* I needed to change the suggested line in the archive module to match the correct RAM size of the machine I am installing on (I actually gave it just less than half the free RAM), then:
* Remake the archive module
* Change into the ioc/conlog directory, remove the iocBoot I had made before for c1, rerun the perl script above, then run make from the ioc/conlog directory.
* Once that is done, you need to edit the file st.cmd to add lines for the reading and writing of channels, such as:
dbLoadRecords("db/conlog.db","IFO=C1")
drvArchive_mysql "C1_conlog","/data/mysql/C1_conlog_epics_user"
drvArchive_read_channel_list_filepaths "/ligo/caltech/data/conlog/c1/add_channel_names","/ligo/caltech/data/conlog/c1/remove_channel_names","/ligo/caltech/data/conlog/c1/use_channel_names"
drvArchive_write_channel_list_filepaths "/ligo/caltech/data/conlog/c1/channel_names","/ligo/caltech/data/conlog/c1/monitored_channel_names","/ligo/caltech/data/conlog/c1/unmonitored_channel_names"
* I also had to rerun this set of commands once that was all done:
controls: cd /opt/rtcds/<site>/<ifo>/target/conlog/conlogepics
controls: cp -r /ligo/apps/linux-x86_64/epics/iocs/conlog/db ./
controls: cp -r /ligo/apps/linux-x86_64/epics/iocs/conlog/dbd ./
controls: cp /ligo/apps/linux-x86_64/epics/iocs/conlog/iocBoot/ioc<ifo>_conlog/envPaths ./
controls: cp /ligo/apps/linux-x86_64/epics/iocs/conlog/iocBoot/ioc<ifo>_conlog/st.cmd ./
controls: echo ./bin/linux-x86_64/conlog st.cmd > startup.<ifo>conlog
controls: chmod a+x startup.<ifo>conlog
* This set of instructions seemed to be lacking, so I added this:

cp -r /ligo/apps/linux-x86_64/epics/iocs/conlog/bin/ ./

* Now the executable runs but doesn't work:
* Fixes needed:
* Need to use root permissions and make sure the files in /data/mysql have the right names for the users the code expects and also have the right passwords. (have to match capitalization appropriately for the <ifo> tag everywhere
* Might need to go into mysql and add a new user with the proper capitalization also
* Need to edit the ld_library_path to point to the new epics libraries (see the suggested environment below)
* Now, the code seems to work, but dumps me at an "epics> " prompt, I'm asking Patrick what to do next.
* I was impatient and loaded up the medm screen, and found out that the one channel I had picked was not readable (unmonitored)
* I ran a modified version of Patrick's perl script to search autoBert files for channels, and replaced my use_channel_names file with the output of the script
* Now, the medm screen shows lots of monitored channels, and the conlog is filling up! (can see it from phpmyadmin)
* Next step: I wanted to get the php pages working, so I edited the files inside /var/www/conlog:
megatron:~/ryan>diff -u /var/www/conlog/conlog.php /var/www/conlog/conlog.php.bck.Mar82012_1
--- /var/www/conlog/conlog.php  2012-03-08 15:31:53.152547771 -0800
+++ /var/www/conlog/conlog.php.bck.Mar82012_1   2012-03-08 15:28:23.062704171 -0800
@@ -19,7 +19,7 @@
        <form action="query.php" method="post">
                <h3><label for="database">Database:</label></h3>
                <select id="database" name="database">
-                       <option value="C1_conlog">C1</option>
+                       <option value="h2_conlog">h2</option>
                </select>
 
                <h3><label for="included_channels">Included channels:</label></h3>

megatron:~/ryan>diff -u /var/www/conlog/query.php /var/www/conlog/query.php.bck.Mar82012_1
--- /var/www/conlog/query.php   2012-03-08 15:33:45.122550303 -0800
+++ /var/www/conlog/query.php.bck.Mar82012_1    2012-03-08 15:32:31.772554679 -0800
@@ -168,8 +168,8 @@
        }
 
        $database_name = $_POST["database"];
-       if ($database_name == 'C1_conlog') {
-               $server = '192.168.113.209';
+       if ($database_name == 'h2_conlog') {
+               $server = 'cdsconlog';
        }
        else {
                die('Unknown database.');

* Finally, the mysql server was denying access from outside queries, so I fixed that:
megatron:~/ryan>diff -u /etc/mysql/my.cnf /etc/mysql/my.cnf.bck.Mar82012_1
--- /etc/mysql/my.cnf   2012-03-08 15:35:57.122548370 -0800
+++ /etc/mysql/my.cnf.bck.Mar82012_1    2012-03-08 15:35:10.652559315 -0800
@@ -49,7 +49,7 @@
 #
 # Instead of skip-networking the default is now to listen only on
 # localhost which is more compatible and is not less secure.
-#bind-address          = 127.0.0.1
+bind-address           = 127.0.0.1
 #
 # * Fine Tuning
 #
megatron:~/ryan>
* Now, I think everything is working * almost:
* It seems that when you first start the conlog up, it finds all the variables and inserts values of "Null" for everything, but after that it detects changes properly!


Conlog Environment


Need to source this to use the new environment:

megatron:~>cat ~/ryan/conlog_enviroment.txt
  6391   Fri Mar 9 11:02:56 2012 Ryan FisherSummaryComputer Scripts / ProgramsAlterations to base epics install for installing aLIGO conlog:

Too Many Fast Changing EPICS in New Conlog


I have been monitoring the new conlog, and it already has far too many rows.

I'm going through the list of channels to exclude in the update_channels script for the conlog that is currently running and removing them from
the monitored channels in the new conlog using the remove_channel_names file and the medm screen (we may want to just wipe out the tables
and start over after this is set properly, but for now I'm keeping them):
#-- Exclude a few uninteresting or obsolete categories
if ( $chan =~ m/^[BIJ]$/ ||
$chan =~ m/IOO-MC_PWR_IN/ ||
$chan eq "C1:PSL-FSS_SLOWDC" ||
$chan =~ m/PSL-STAT_.*_BITS/ ||
$chan =~ m/:IOO-PZTM[12]_(PIT|YAW)_BIAS$/ ||
$chan =~ m/DAQ.*_cycle/ ||
$chan =~ m/DAQ.*rtSeconds/ ||
$chan =~ m/C1:-.*/ ||
$chan =~ m/C1:SUP/ ||
$chan =~ m/C1:SP/ ||
$chan =~ m/C1:X/ ||
$chan =~ m/C1:TST/ ||
$chan =~ m/C1:RF/ ||
$chan =~ m/C1:UCT/ ||
$chan =~ m/C1:DU/ ||
$chan =~ m/C1:MCP/ ||
$chan =~ m/C1:MCS/ ||
$chan =~ m/C1:FEC/ ||
$chan =~ m/C1:PEM/ ||
$chan =~ m/C1:LSP/ ||
$chan =~ m/C1:NIO/ ||
$chan =~ m/C1:WFS/ ||
$chan =~ m/C1:ASC-WFS/ ||
$chan =~ m/C1:ASC-SP/ ||
$chan =~ m/C1:VG/ ||
$chan =~ m/C1:IOO-DOF/ ||
$chan =~ m/C1:IOO-EO/ ||
$chan =~ m/Name/ ||
$chan =~ m/DEFAULTNAME/ ||
$chan =~ m/:IOO-PZT.*OFFSET/ ||
$chan =~ m/PD_VAR$/ ||
$chan =~ m/_INMON$/ ||
$chan =~ m/_EXCMON$/ ||
$chan =~ m/_OUT16$/ ||
$chan =~ m/_OUTMON$/ ||
$chan =~ m/_OUTPUT$/ ||
$chan =~ m/_RSET$/ ||
$chan =~ m/_ALIVE$/ ||
$chan =~ m/VMon$/ ||
$chan =~ m/PDMon$/ ||
$chan =~ m/(BiasVMon|FE_PPOLL|MASTER_OVERFLOW|FSS_TIDALSET|CPU_LOAD|CDM
_STAT|State_Bits|INDCOFFSET)/ )

With these removals, only 15493 channels are being monitored now.
  6394   Fri Mar 9 15:48:56 2012 Ryan FisherSummaryComputer Scripts / ProgramsAlterations to base epics install for installing aLIGO conlog:

I decided to make a backup of the database and then delete it and make a new database:
 
cd ~/ryan/database_dumpMar92012
mysqldump -u root -p C1_conlog > C1_conlog.dump.Mar92012 
Note: it appears this failed the first time, thankfully this wasn't a production service yet... In the future, do not trust this backup method for important data!

Next, log into mysql as root, dump the database, remake it and grant privileges again.:
(This is saved in megatron:~/ryan/restore_database.txt
megatron:~/ryan>mysql -u root -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 174
Server version: 5.1.41-3ubuntu12.10 (Ubuntu)

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> list databases;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'list databases' at line 1
mysql> list users;      ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'list users' at line 1
mysql> use C1_conlog
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> list users;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'list users' at line 1
mysql> select User from mysql.user;                                             +------------------+
| User             |
+------------------+
| php              |
| C1_conlog_epics  |
| c1_conlog_epics  |
| root             |
| C1_conlog_epics  |
| c1_conlog_epics  |
| debian-sys-maint |
| root             |
| root             |
+------------------+
9 rows in set (0.00 sec)

mysql> show databases;                                                          +--------------------+
| Database           |
+--------------------+
| information_schema |
| C1_conlog          |
| mysql              |
+--------------------+
3 rows in set (0.00 sec)

mysql> drop database C1_conlog ;
Query OK, 2 rows affected (0.56 sec)

mysql> create database C1_conlog;
Query OK, 1 row affected (0.00 sec)

mysql> use C1_conlog ;
Database changed
mysql> SET SQL_MODE="NO_AUTO_VALUE_ON_ZERO";
Query OK, 0 rows affected (0.00 sec)

mysql>
mysql> CREATE TABLE `channels` (
    ->   `channel_id` mediumint(8) unsigned NOT NULL AUTO_INCREMENT,
    ->   `channel_name` varchar(60) NOT NULL,
    ->   PRIMARY KEY (`channel_id`),
    ->   UNIQUE KEY `channel_name` (`channel_name`)
    -> ) ENGINE=MyISAM  DEFAULT CHARSET=latin1;
Query OK, 0 rows affected (0.04 sec)

mysql>
mysql> CREATE TABLE `data` (
    ->   `acquire_time` decimal(26,6) NOT NULL,
    ->   `channel_id` mediumint(8) unsigned NOT NULL,
    ->   `value` varchar(40) DEFAULT NULL,
    ->   `status` tinyint(3) unsigned DEFAULT NULL,
    ->   `connected` tinyint(1) unsigned NOT NULL,
    ->   PRIMARY KEY (`channel_id`,`acquire_time`)
    -> ) ENGINE=MyISAM DEFAULT CHARSET=latin1;
Query OK, 0 rows affected (0.03 sec)

mysql> grant select, insert, update, execute on * to 'c1_conlog_epics'@'127.0.0.1';  Query OK, 0 rows affected (0.00 sec) 
mysql> grant select, insert, update, execute on * to 'C1_conlog_epics'@'127.0.0.1';   Query OK, 0 rows affected (0.00 sec) 
 mysql> grant select, insert, update, execute on * to 'c1_conlog_epics'@'localhost';  Query OK, 0 rows affected (0.00 sec) 
mysql> grant select, insert, update, execute on * to 'C1_conlog_epics'@'localhost';
Query OK, 0 rows affected (0.00 sec)

mysql> grant select on C1_conlog to 'php'@'%';
ERROR 1146 (42S02): Table 'C1_conlog.C1_conlog' doesn't exist
mysql> grant select on * to 'php'@'%';
Query OK, 0 rows affected (0.00 sec)

mysql> select * from mysql.users
    -> ;
ERROR 1146 (42S02): Table 'mysql.users' doesn't exist
mysql> select User from mysql.user;        
| C1_conlog_epics  |
| c1_conlog_epics  |
| root             |
| C1_conlog_epics  |
| c1_conlog_epics  |
| debian-sys-maint |
| root             |
| root             |
+------------------+
9 rows in set (0.00 sec)

mysql> Bye 



Next, I decided that I want to index on the acquire_time instead of the combination of channel_id and acquire_time (I think it makes a lot of sense for several query types, and especially debugging the conlog!):
mysql> create index acquire_time_index on data(acquire_time);
Query OK, 0 rows affected (0.04 sec)
Records: 0  Duplicates: 0  Warnings: 0

Next Fix:


The above worked well, but when I restarted the conlog, I had to re-execute the "remove_channels" from the medm, because initially all channels were being loaded (use_channel_names had all the channels still).
Additionally, there were a lot of channels with "*RMS*" in the name that were being recorded, and were changing relatively quickly, so I have added those to the remove_channel_names file.

I am going to: Backup the files in /ligo/caltech/data/conlog/c1
Edit use_channel_names to only have the good channels.
Dump the database again
Stop conlog.
Wipe the database again.
Remake the database again (with permissions and the new index).
Restart the conlog and hope!

The fix above seems to be in place and working. The database has the initial entries for the channels it monitors and is not growing without operators changing EPICs values.

  6396   Fri Mar 9 16:28:10 2012 Ryan FisherSummaryComputer Scripts / ProgramsAlterations to base epics install for installing aLIGO conlog:
I created a page on the wiki for the new EPICS log (conlog):
https://wiki-40m.ligo.caltech.edu/aLIGO%20EPICs%20log%20%28conlog%29

I also edited this with restart instructions:
https://wiki-40m.ligo.caltech.edu/Computer_Restart_Procedures#megatron
  7001   Mon Jul 23 07:39:55 2012 Ryan FisherSummaryComputer Scripts / ProgramsAlterations to base epics install for installing aLIGO conlog:
Note: The Conlog install instructions that I started from were located here:
https://awiki.ligo-wa.caltech.edu/aLIGO/Conlog
  8185   Wed Feb 27 14:59:01 2013 EvanUpdate Altered MC demodulation phase

 I took out a short (~12 cm) SMA cable from the "LO input" path into the MC demod board in an attempt to maximize the power in Q and minimize the power in I. The path might benefit from being shortened a little more, but it's hard to tell since I is noisy. (In the attached plots, channel 1 is Q and channel 2 is I.)

Should you find it necessary to restore the original path length, the cable I took out is in the "SMA ONLY" tupperware and has a printed label with "5" on it.

  14544   Mon Apr 15 22:39:10 2019 gautamUpdateFrequency noise measurementAlternate setup with PSL pickoff

[anjali, gautam]

just main points, anajli is going to fill out the details.

To rule out mode-matching as the reason for non-ideal output from the MZ, I suggested using the setup I have on the NW side of the PSL enclosure for the measurement. This uses two identical fiber collimators, and the distance between collimator and recombination BS is approximately the same, so the spatial modes should be pretty well matched. 

The spooled fiber we found was not suitable for use as it had a wide key connector and I couldn't find any wide-key FC/PC to narrow-key FC/APC adaptors. So we decided to give the fiber going to the Y end and back (~90m estimated length) a shot. We connected the two fibers at the EY table using a fiber mating sleeve (so the fiber usually bringing the IR pickoff from EY to the PSL table was disconnected from its collimator). 

In summary, we cannot explain why the contrast of the MZ is <5%. Spatial mode-overlap is definitely not to blame. Power asymmetry in the two arms of the MZ is one possible explanation, could also be unstable polarization, even though we think the entire fiber chain is PM. Anjali is investigating.

 


We saw today that the Thorlabs PM beam splitters (borrowed from Andrew until our AFW components arrive) do not treat the two special axes (fast and slow) of the fiber on equal footing. When we coupled light into the fast axis, we saw huge asymmetry between the two split arms of the beamsplitter (3:1 ratio in power instead of the expected 1:1 for a 50/50 BS). Looking at the patch cord with an IR viewer, we also saw light leaking through the core along it. Turns out this part is meant to be used with light coupled to the slow axis only.

ELOG V3.1.3-