daqd on fb0 seems to be broken. i can't get any data from the past, hours, days or weeks ago.
I always get:

Connecting to NDS Server fb0 (TCP port 8088)
Connecting.... done
read(); errno=0
LONG: DataRead = -1
No data found

even, when using slow epics channels.
But if i telnet into it and pull up the channel status everything is fine and lots of channels are written with the correct data rate.
So i don't know, fb1 is working fine. Getting a channel list from fb0 too.

The error message you get when starting the daqd on fb1 is not new, but Alex doesn't know what it means and it only happened a couple of times so far and everything is working.
ca "CA.Client.Exception" is our network problem in the PSL lab...

241 channels total
0 gds channels
0 gds channels aliases
240 trend channels
1Hz clock
chnum   slow    |name                           |rate   |trend  |group  |bps    |bytes  |offset |type   |active
0       0       C2:PEM-SENS1_TEMP                               16      1       0       4       64      0       4       1
0       0       C2:PEM-SENS1_HUMIDITY                           16      1       0       4       64      64      4       1
0       0       C2:PEM-SENS1_PRESSURE                           16      1       0       4       64      128     4       1
0       0       C2:PEM-SENS1_BATTERY                            16      1       0       4       64      192     4       1
0       0       C2:PEM-SENS1_UPDATE                             16      1       0       4       64      256     4       1
0       0       C2:PEM-SENS1_STRENGTH                           16      1       0       4       64      320     4       1
0       0       C2:PEM-SENS1_SUCCESS                            16      1       0       4       64      384     4       1
0       0       C2:PEM-SENS1_ACTIVE                             16      1       0       4       64      448     4       1
0       0       C3:PEM-SENS1_TEMP                               16      1       0       4       64      512     4       1
0       0       C3:PEM-SENS1_HUMIDITY                           16      1       0       4       64      576     4       1
0       0       C3:PEM-SENS1_PRESSURE                           16      1       0       4       64      640     4       1
0       0       C3:PEM-SENS1_BATTERY                            16      1       0       4       64      704     4       1
0       0       C3:PEM-SENS1_UPDATE                             16      1       0       4       64      768     4       1
0       0       C3:PEM-SENS1_STRENGTH                           16      1       0       4       64      832     4       1
0       0       C3:PEM-SENS1_SUCCESS                            16      1       0       4       64      896     4       1
0       0       C3:PEM-SENS1_ACTIVE                             16      1       0       4       64      960     4       1
0       0       C4:PEM-SENS1_TEMP                               16      1       0       4       64      1024    4       1
0       0       C4:PEM-SENS1_HUMIDITY                           16      1       0       4       64      1088    4       1
0       0       C4:PEM-SENS1_PRESSURE                           16      1       0       4       64      1152    4       1
0       0       C4:PEM-SENS1_BATTERY                            16      1       0       4       64      1216    4       1
0       0       C4:PEM-SENS1_UPDATE                             16      1       0       4       64      1280    4       1
0       0       C4:PEM-SENS1_STRENGTH                           16      1       0       4       64      1344    4       1
0       0       C4:PEM-SENS1_SUCCESS                            16      1       0       4       64      1408    4       1
0       0       C4:PEM-SENS1_ACTIVE                             16      1       0       4       64      1472    4       1
0       0       C5:PEM-SENS1_TEMP                               16      1       0       4       64      1536    4       1
0       0       C5:PEM-SENS1_HUMIDITY                           16      1       0       4       64      1600    4       1
0       0       C5:PEM-SENS1_PRESSURE                           16      1       0       4       64      1664    4       1
0       0       C5:PEM-SENS1_BATTERY                            16      1       0       4       64      1728    4       1
0       0       C5:PEM-SENS1_UPDATE                             16      1       0       4       64      1792    4       1
0       0       C5:PEM-SENS1_STRENGTH                           16      1       0       4       64      1856    4       1
0       0       C5:PEM-SENS1_SUCCESS                            16      1       0       4       64      1920    4       1
0       0       C5:PEM-SENS1_ACTIVE                             16      1       0       4       64      1984    4       1
0       0       C6:PEM-SENS1_TEMP                               16      1       0       4       64      2048    4       1
0       0       C6:PEM-SENS1_HUMIDITY                           16      1       0       4       64      2112    4       1
0       0       C6:PEM-SENS1_PRESSURE                           16      1       0       4       64      2176    4       1
0       0       C6:PEM-SENS1_BATTERY                            16      1       0       4       64      2240    4       1
0       0       C6:PEM-SENS1_UPDATE                             16      1       0       4       64      2304    4       1
0       0       C6:PEM-SENS1_STRENGTH                           16      1       0       4       64      2368    4       1
0       0       C6:PEM-SENS1_SUCCESS                            16      1       0       4       64      2432    4       1
0       0       C6:PEM-SENS1_ACTIVE                             16      1       0       4       64      2496    4       1
0       0       C2:PSL-AMP_MANUALMODE                           16      1       0       4       64      2560    4       1
0       0       C2:PSL-AMP_DBILOCK                              16      1       0       4       64      2624    4       1
0       0       C2:PSL-AMP_DCUROK                               16      1       0       4       64      2688    4       1
0       0       C2:PSL-AMP_ENABLETEC                            16      1       0       4       64      2752    4       1
0       0       C2:PSL-AMP_ERR                                  16      1       0       4       64      2816    4       1
0       0       C2:PSL-AMP_OFF                                  16      1       0       4       64      2880    4       1
0       0       C2:PSL-AMP_RESET                                16      1       0       4       64      2944    4       1
0       0       C2:PSL-AMP_DTEMPERR                             16      1       0       4       64      3008    4       1
0       0       C2:PSL-AMP_ENABLEPWRDOG                         16      1       0       4       64      3072    4       1
0       0       C2:PSL-AMP_SYSTEMOK                             16      1       0       4       64      3136    4       1
0       0       C2:PSL-AMP_LOWPWR                               16      1       0       4       64      3200    4       1
0       0       C2:PSL-AMP_CHILLERR                             16      1       0       4       64      3264    4       1
0       0       C2:PSL-AMP_DVOLT1                               16      1       0       4       64      3328    4       1
0       0       C2:PSL-AMP_DVOLT2                               16      1       0       4       64      3392    4       1
0       0       C2:PSL-AMP_PWR1                                 16      1       0       4       64      3456    4       1
0       0       C2:PSL-AMP_PWR2                                 16      1       0       4       64      3520    4       1
0       0       C2:PSL-AMP_PWR3                                 16      1       0       4       64      3584    4       1
0       0       C2:PSL-AMP_D1TEMP                               16      1       0       4       64      3648    4       1
0       0       C2:PSL-AMP_D2TEMP                               16      1       0       4       64      3712    4       1
0       0       C2:PSL-AMP_D3TEMP                               16      1       0       4       64      3776    4       1
0       0       C2:PSL-AMP_D4TEMP                               16      1       0       4       64      3840    4       1
0       0       C2:PSL-AMP_D1PWR                                16      1       0       4       64      3904    4       1
0       0       C2:PSL-AMP_D2PWR                                16      1       0       4       64      3968    4       1
0       0       C2:PSL-AMP_D3PWR                                16      1       0       4       64      4032    4       1
0       0       C2:PSL-AMP_D4PWR                                16      1       0       4       64      4096    4       1
0       0       C2:PSL-AMP_DCUR1                                16      1       0       4       64      4160    4       1
0       0       C2:PSL-AMP_DCUR2                                16      1       0       4       64      4224    4       1
0       0       C2:PSL-AMP_XTALTEMP                             16      1       0       4       64      4288    4       1
0       0       C2:PSL-AMP_SECS                                 16      1       0       4       64      4352    4       1
0       0       C2:PSL-AMP_MINS                                 16      1       0       4       64      4416    4       1
0       0       C2:PSL-AMP_HRS                                  16      1       0       4       64      4480    4       1
0       0       C2:PSL-AMP_DAYS                                 16      1       0       4       64      4544    4       1
0       0       C2:PSL-NPRO_D1PWR                               16      1       0       4       64      4608    4       1
0       0       C2:PSL-NPRO_D2PWR                               16      1       0       4       64      4672    4       1
0       0       C2:PSL-NPRO_D1TEMPSET                           16      1       0       4       64      4736    4       1
0       0       C2:PSL-NPRO_D2TEMPSET                           16      1       0       4       64      4800    4       1
0       0       C2:PSL-NPRO_D1TEMP                              16      1       0       4       64      4864    4       1
0       0       C2:PSL-NPRO_D2TEMP                              16      1       0       4       64      4928    4       1
0       0       C2:PSL-NPRO_D1TEMPERR                           16      1       0       4       64      4992    4       1
0       0       C2:PSL-NPRO_D2TEMPERR                           16      1       0       4       64      5056    4       1
0       0       C2:PSL-NPRO_TEMPGUARD1                          16      1       0       4       64      5120    4       1
0       0       C2:PSL-NPRO_TEMPGUARD2                          16      1       0       4       64      5184    4       1
0       0       C2:PSL-NPRO_XTALTEMPSET                         16      1       0       4       64      5248    4       1
0       0       C2:PSL-NPRO_XTALTEMP                            16      1       0       4       64      5312    4       1
0       0       C2:PSL-NPRO_XTALTEMPERR                         16      1       0       4       64      5376    4       1
0       0       C2:PSL-NPRO_CURSET                              16      1       0       4       64      5440    4       1
0       0       C2:PSL-NPRO_CUR                                 16      1       0       4       64      5504    4       1
0       0       C2:PSL-NPRO_NEMON                               16      1       0       4       64      5568    4       1
0       0       C3:PSL-GEN_DAQ14                                16      1       0       4       64      5632    4       1
0       0       C3:PSL-GEN_DAQ15                                16      1       0       4       64      5696    4       1
0       0       C3:PSL-GEN_DAQ16                                16      1       0       4       64      5760    4       1
0       0       C3:PSL-GEN_D2A3                                 16      1       0       4       64      5824    4       1
0       0       C3:PSL-GEN_D2A4                                 16      1       0       4       64      5888    4       1
0       0       C3:PSL-GEN_D2A5                                 16      1       0       4       64      5952    4       1
0       0       C3:PSL-GEN_D2A6                                 16      1       0       4       64      6016    4       1
0       0       C3:PSL-GEN_D2A7                                 16      1       0       4       64      6080    4       1
0       0       C3:PSL-GEN_D2A8                                 16      1       0       4       64      6144    4       1
0       0       C3:PSL-ACAV_SENS1                               16      1       0       4       64      6208    4       1
0       0       C3:PSL-ACAV_SENS2                               16      1       0       4       64      6272    4       1
0       0       C3:PSL-ACAV_SENS3                               16      1       0       4       64      6336    4       1
0       0       C3:PSL-ACAV_SENS4                               16      1       0       4       64      6400    4       1
0       0       C3:PSL-ACAV_OOL                                 16      1       0       4       64      6464    4       1
0       0       C3:PSL-ACAV_TEMP                                16      1       0       4       64      6528    4       1
0       0       C3:PSL-ACAV_TEMPAVG                             16      1       0       4       64      6592    4       1
0       0       C3:PSL-ACAV_HEATER                              16      1       0       4       64      6656    4       1
0       0       C3:PSL-ACAV_SETPT                               16      1       0       4       64      6720    4       1
0       0       C3:PSL-ACAV_KP                                  16      1       0       4       64      6784    4       1
0       0       C3:PSL-ACAV_KI                                  16      1       0       4       64      6848    4       1
0       0       C3:PSL-ACAV_KD                                  16      1       0       4       64      6912    4       1
0       0       C3:PSL-ACAV_ENABLE                              16      1       0       4       64      6976    4       1
0       0       C3:PSL-ACAV_TIMEOUT                             16      1       0       4       64      7040    4       1
0       0       C3:PSL-ACAV_SCALE                               16      1       0       4       64      7104    4       1
0       0       C3:PSL-ACAV_RFPDDC                              16      1       0       4       64      7168    4       1
0       0       C3:PSL-ACAV_RCTRANSPD                           16      1       0       4       64      7232    4       1
0       0       C3:PSL-ACAV_LOCKEDLEVEL                         16      1       0       4       64      7296    4       1
0       0       C3:PSL-RCAV_SENS1                               16      1       0       4       64      7360    4       1
0       0       C3:PSL-RCAV_SENS2                               16      1       0       4       64      7424    4       1
0       0       C3:PSL-RCAV_SENS3                               16      1       0       4       64      7488    4       1
0       0       C3:PSL-RCAV_SENS4                               16      1       0       4       64      7552    4       1
0       0       C3:PSL-RCAV_OOL                                 16      1       0       4       64      7616    4       1
0       0       C3:PSL-RCAV_TEMP                                16      1       0       4       64      7680    4       1
0       0       C3:PSL-RCAV_TEMPAVG                             16      1       0       4       64      7744    4       1
0       0       C3:PSL-FSS_HEATER                               16      1       0       4       64      7808    4       1
0       0       C3:PSL-RCAV_SETPT                               16      1       0       4       64      7872    4       1
0       0       C3:PSL-RCAV_KP                                  16      1       0       4       64      7936    4       1
0       0       C3:PSL-RCAV_KI                                  16      1       0       4       64      8000    4       1
0       0       C3:PSL-RCAV_KD                                  16      1       0       4       64      8064    4       1
0       0       C3:PSL-RCAV_ENABLE                              16      1       0       4       64      8128    4       1
0       0       C3:PSL-RCAV_TIMEOUT                             16      1       0       4       64      8192    4       1
0       0       C3:PSL-RCAV_SCALE                               16      1       0       4       64      8256    4       1
0       0       C3:PSL-RCAV_RFPDDC                              16      1       0       4       64      8320    4       1
0       0       C3:PSL-RCAV_RCTRANSPD                           16      1       0       4       64      8384    4       1
0       0       C3:PSL-RCAV_LOCKEDLEVEL                         16      1       0       4       64      8448    4       1
0       0       C3:PSL-FSS_RFPDDC                               16      1       0       4       64      8512    4       1
0       0       C3:PSL-FSS_LODET                                16      1       0       4       64      8576    4       1
0       0       C3:PSL-FSS_PCDET                                16      1       0       4       64      8640    4       1
0       0       C3:PSL-FSS_FAST                                 16      1       0       4       64      8704    4       1
0       0       C3:PSL-FSS_PCDRIVE                              16      1       0       4       64      8768    4       1
0       0       C3:PSL-FSS_RCTRANSPD                            16      1       0       4       64      8832    4       1
0       0       C3:PSL-FSS_RCTLL                                16      1       0       4       64      8896    4       1
0       0       C3:PSL-FSS_VCODET                               16      1       0       4       64      8960    4       1
0       0       C3:PSL-FSS_TIDALOUT                             16      1       0       4       64      9024    4       1
0       0       C3:PSL-FSS_MODET                                16      1       0       4       64      9088    4       1
0       0       C3:PSL-FSS_VCODETPWR                            16      1       0       4       64      9152    4       1
0       0       C3:PSL-FSS_MIXERM                               16      1       0       4       64      9216    4       1
0       0       C3:PSL-FSS_SLOWM                                16      1       0       4       64      9280    4       1
0       0       C3:PSL-FSS_VCOM                                 16      1       0       4       64      9344    4       1
0       0       C3:PSL-FSS_TIDALINPUT                           16      1       0       4       64      9408    4       1
0       0       C3:PSL-FSS_SW1                                  16      1       0       4       64      9472    4       1
0       0       C3:PSL-FSS_SW2                                  16      1       0       4       64      9536    4       1
0       0       C3:PSL-FSS_PHFLIP                               16      1       0       4       64      9600    4       1
0       0       C3:PSL-FSS_VCOTESTSW                            16      1       0       4       64      9664    4       1
0       0       C3:PSL-FSS_VCOWIDESW                            16      1       0       4       64      9728    4       1
0       0       C3:PSL-FSS_INOFFSET                             16      1       0       4       64      9792    4       1
0       0       C3:PSL-FSS_MGAIN                                16      1       0       4       64      9856    4       1
0       0       C3:PSL-FSS_FASTGAIN                             16      1       0       4       64      9920    4       1
0       0       C3:PSL-FSS_PHCON                                16      1       0       4       64      9984    4       1
0       0       C3:PSL-FSS_RFADJ                                16      1       0       4       64      10048   4       1
0       0       C3:PSL-FSS_SLOWDC                               16      1       0       4       64      10112   4       1
0       0       C3:PSL-FSS_VCOPWR                               16      1       0       4       64      10176   4       1
0       0       C3:PSL-FSS_VCOMODLEVEL                          16      1       0       4       64      10240   4       1
0       0       C3:PSL-FSS_TIDALSET                             16      1       0       4       64      10304   4       1
0       0       C3:PSL-FSS_LOCK                                 16      1       0       4       64      10368   4       1
0       0       C3:PSL-FSS_SLOWLOOP                             16      1       0       4       64      10432   4       1
0       0       C3:PSL-PMC_PMCTLL                               16      1       0       4       64      10496   4       1
0       0       C3:PSL-PMC_RFPDDC                               16      1       0       4       64      10560   4       1
0       0       C3:PSL-PMC_LODET                                16      1       0       4       64      10624   4       1
0       0       C3:PSL-PMC_PMCTRANSPD                           16      1       0       4       64      10688   4       1
0       0       C3:PSL-PMC_PCDRIVE                              16      1       0       4       64      10752   4       1
0       0       C3:PSL-PMC_PZT                                  16      1       0       4       64      10816   4       1
0       0       C3:PSL-PMC_MODET                                16      1       0       4       64      10880   4       1
0       0       C3:PSL-PMC_PMCERR                               16      1       0       4       64      10944   4       1
0       0       C3:PSL-PMC_SW1                                  16      1       0       4       64      11008   4       1
0       0       C3:PSL-PMC_SW2                                  16      1       0       4       64      11072   4       1
0       0       C3:PSL-PMC_PHFLIP                               16      1       0       4       64      11136   4       1
0       0       C3:PSL-PMC_BLANK                                16      1       0       4       64      11200   4       1
0       0       C3:PSL-PMC_GAIN                                 16      1       0       4       64      11264   4       1
0       0       C3:PSL-PMC_INOFFSET                             16      1       0       4       64      11328   4       1
0       0       C3:PSL-PMC_PHCON                                16      1       0       4       64      11392   4       1
0       0       C3:PSL-PMC_RFADJ                                16      1       0       4       64      11456   4       1
0       0       C3:PSL-PMC_RAMP                                 16      1       0       4       64      11520   4       1
0       0       C3:PSL-PMC_LOCK                                 16      1       0       4       64      11584   4       1
0       0       C3:PSL-PEM_RMTEMP                               16      1       0       4       64      11648   4       1
0       0       C3:PSL-PEM_BOXTEMP                              16      1       0       4       64      11712   4       1
0       0       C4:TCS-HWS_TEMP_SENSOR                          16      1       0       4       64      11776   4       1
0       0       C4:TCS-HWS_TEMP_DIGITIZER                       16      1       0       4       64      11840   4       1
0       0       C4:TCS-HWS_TAP1GAIN                             16      1       0       4       64      11904   4       1
0       0       C4:TCS-HWS_TAP2GAIN                             16      1       0       4       64      11968   4       1
0       0       C4:TCS-HWS_PRETRIGGER                           16      1       0       4       64      12032   4       1
0       0       C4:TCS-HWS_DATA_MODE                            16      1       0       4       64      12096   4       1
0       0       C4:TCS-HWS_BINNING_MODE                         16      1       0       4       64      12160   4       1
0       0       C4:TCS-HWS_GAIN_MODE                            16      1       0       4       64      12224   4       1
0       0       C4:TCS-HWS_OUTPUT_CONFIG                        16      1       0       4       64      12288   4       1
0       0       C4:TCS-HWS_EXPOSURE_MODE                        16      1       0       4       64      12352   4       1
0       0       C4:TCS-HWS_SYNC_FREQ                            16      1       0       4       64      12416   4       1
0       0       C4:TCS-HWS_EXPOSURE_TIME                        16      1       0       4       64      12480   4       1
0       0       C4:TCS-HWS_SERIAL_COMMS_STATUS                  16      1       0       4       64      12544   4       1
0       0       C4:TCS-HWS_DEFOCUS                              16      1       0       4       64      12608   4       1
0       0       C4:TCS-HWS_CENTROID_RMS                         16      1       0       4       64      12672   4       1
0       0       C4:TCS-HWS_SINGLE_FRAME_BKGD                    16      1       0       4       64      12736   4       1
0       0       C4:TCS-HWS_AVERAGE_INTENSITY                    16      1       0       4       64      12800   4       1
0       0       C4:TCS-HWS_N_CENTROID_CALC                      16      1       0       4       64      12864   4       1
0       0       C4:TCS-HWS_ZERNIKE_P_-1+1                       16      1       0       4       64      12928   4       1
0       0       C4:TCS-HWS_ZERNIKE_P_+1-1                       16      1       0       4       64      12992   4       1
0       0       C4:TCS-HWS_ZERNIKE_P_-2+2                       16      1       0       4       64      13056   4       1
0       0       C4:TCS-HWS_ZERNIKE_P_-0+2                       16      1       0       4       64      13120   4       1
0       0       C4:TCS-HWS_ZERNIKE_P_+2+2                       16      1       0       4       64      13184   4       1
0       0       C4:TCS-HWS_ZERNIKE_P_-3+3                       16      1       0       4       64      13248   4       1
0       0       C4:TCS-HWS_ZERNIKE_P_-1+3                       16      1       0       4       64      13312   4       1
0       0       C4:TCS-HWS_ZERNIKE_P_+1+3                       16      1       0       4       64      13376   4       1
0       0       C4:TCS-HWS_ZERNIKE_P_+3+3                       16      1       0       4       64      13440   4       1
0       0       C4:TCS-HWS_ZERNIKE_P_-4+4                       16      1       0       4       64      13504   4       1
0       0       C4:TCS-HWS_ZERNIKE_P_-2+4                       16      1       0       4       64      13568   4       1
0       0       C4:TCS-HWS_ZERNIKE_P_-0+4                       16      1       0       4       64      13632   4       1
0       0       C4:TCS-HWS_ZERNIKE_P_+2+4                       16      1       0       4       64      13696   4       1
0       0       C4:TCS-HWS_ZERNIKE_P_+4+4                       16      1       0       4       64      13760   4       1
0       0       C4:TCS-ATHENA_ADC0                              16      1       0       4       64      13824   4       1
0       0       C4:TCS-ATHENA_ADC1                              16      1       0       4       64      13888   4       1
0       0       C4:TCS-ATHENA_ADC2                              16      1       0       4       64      13952   4       1
0       0       C4:TCS-ATHENA_ADC3                              16      1       0       4       64      14016   4       1
0       0       C4:TCS-ATHENA_ADC4                              16      1       0       4       64      14080   4       1
0       0       C4:TCS-ATHENA_ADC5                              16      1       0       4       64      14144   4       1
0       0       C4:TCS-ATHENA_ADC6                              16      1       0       4       64      14208   4       1
0       0       C4:TCS-ATHENA_ADC7                              16      1       0       4       64      14272   4       1
0       0       C4:TCS-ATHENA_ADC8                              16      1       0       4       64      14336   4       1
0       0       C4:TCS-ATHENA_ADC9                              16      1       0       4       64      14400   4       1
0       0       C4:TCS-ATHENA_ADC10                             16      1       0       4       64      14464   4       1
0       0       C4:TCS-ATHENA_ADC11                             16      1       0       4       64      14528   4       1
0       0       C4:TCS-ATHENA_ADC12                             16      1       0       4       64      14592   4       1
0       0       C4:TCS-ATHENA_ADC13                             16      1       0       4       64      14656   4       1
0       0       C4:TCS-ATHENA_ADC14                             16      1       0       4       64      14720   4       1
0       0       C4:TCS-ATHENA_ADC15                             16      1       0       4       64      14784   4       1
0       0       C4:TCS-ATHENA_DAC0                              16      1       0       4       64      14848   4       1
0       0       C4:TCS-ATHENA_DAC1                              16      1       0       4       64      14912   4       1
0       0       C4:TCS-ATHENA_PD_VOLTAGE                        16      1       0       4       64      14976   4       1
0       0       C4:TCS-ATHENA_TEMP_SENS_V                       16      1       0       4       64      15040   4       1
0       0       C4:TCS-ATHENA_I_SET_VOLTS                       16      1       0       4       64      15104   4       1
0       0       C4:TCS-ATHENA_I_ACTUAL_VOLTS                    16      1       0       4       64      15168   4       1
0       0       C4:TCS-ATHENA_I_LIM_VOLTS                       16      1       0       4       64      15232   4       1
0       0       C4:TCS-ATHENA_PD_EXTERNAL_V                     16      1       0       4       64      15296   4       1
10001   0       C2:FB1-FB_DUMMY                                 16384   0       0       4       65536   15360   4       0

I can't get data. mDV and DV showed the same failed behavior. (They did nothing when you request data).

I looked in the daqd log files:

/cvs/cds/caltech/target/fb/daqd.log had:

[Tue Aug 31 22:01:37 2010] ->19: version
[Tue Aug 31 22:01:37 2010] ->19: revision
[Tue Aug 31 22:01:37 2010] no_average=1

Tue Aug 31 22:01:37 2010 [16601]:decimation flag=1
[Tue Aug 31 22:01:37 2010] ->19: start net-writer 967351844 20 {"C2:ATF-CTRL_ISS_1_OUT_DAQ" 256}
[Tue Aug 31 22:01:37 2010] UNIX connect(); errno=111

[Tue Aug 31 22:01:37 2010] connection closed on fd=20

The channel I requested is shown as being written in the C2ATF.ini file.

/cvs/cds/caltech/target/fb1/daqd.log had:

[Tue Aug 31 21:42:30 2010] ->8: revision
[Tue Aug 31 21:42:30 2010] ->8: status channels 3
[Tue Aug 31 21:42:30 2010] connection on fd 10 closed
[Tue Aug 31 21:42:31 2010] connection on port 38943 from 10.0.1.11; fd=17
[Tue Aug 31 21:42:32 2010] ->8: version
[Tue Aug 31 21:42:32 2010] ->8: revision
[Tue Aug 31 21:42:32 2010] ->8: gps
[Tue Aug 31 21:42:32 2010] connection on fd 17 closed
CA.Client.Exception...............................................
    Warning: "Virtual circuit unresponsive"
    Context: "10.0.0.3:5064"
    Source File: ../tcpiiu.cpp line 921
    Current Time: Tue Aug 31 2010 21:43:43.255113000
..................................................................
CA.Client.Exception...............................................
    Warning: "Virtual circuit unresponsive"
    Context: "10.0.0.2:5064"
    Source File: ../tcpiiu.cpp line 921
    Current Time: Tue Aug 31 2010 21:44:20.907151000
..................................................................
CA.Client.Exception...............................................
    Warning: "Virtual circuit unresponsive"
    Context: "10.0.0.3:5064"
    Source File: ../tcpiiu.cpp line 921
    Current Time: Tue Aug 31 2010 21:45:43.759006000
..................................................................
CA.Client.Exception...............................................
    Warning: "Virtual circuit unresponsive"
    Context: "10.0.0.2:5064"
    Source File: ../tcpiiu.cpp line 921
    Current Time: Tue Aug 31 2010 21:47:12.898705000
..................................................................
CA.Client.Exception...............................................
    Warning: "Virtual circuit unresponsive"
    Context: "10.0.0.3:5064"
    Source File: ../tcpiiu.cpp line 921
    Current Time: Tue Aug 31 2010 21:49:51.762671000
..................................................................
CA.Client.Exception...............................................
    Warning: "Virtual circuit unresponsive"
    Context: "10.0.0.2:5064"
    Source File: ../tcpiiu.cpp line 921
    Current Time: Tue Aug 31 2010 21:53:36.383864000
..................................................................
CA.Client.Exception...............................................
    Warning: "Virtual circuit unresponsive"
    Context: "10.0.0.3:5064"
    Source File: ../tcpiiu.cpp line 921
    Current Time: Tue Aug 31 2010 21:53:58.267296000
..................................................................
CA.Client.Exception...............................................
    Warning: "Virtual circuit unresponsive"
    Context: "10.0.0.2:5064"
    Source File: ../tcpiiu.cpp line 921
    Current Time: Tue Aug 31 2010 21:55:14.367285000
..................................................................
CA.Client.Exception...............................................
    Warning: "Virtual circuit unresponsive"
    Context: "10.0.0.2:5064"
    Source File: ../tcpiiu.cpp line 921
    Current Time: Tue Aug 31 2010 21:56:37.363180000
.................................................................

I then restarted the daq daemon on fb1 (sudo pkill daqd), and the logfile poopped this out:

A call to "assert (pE == &entry)" failed in ../../cds/project/daq/fb2/exServer.h line 346.
EPICS Release EPICS R3.14.10- $R3-14-10-RC2$ $2008/10/10 15:01:51$.
Current time Tue Aug 31 2010 22:02:42.341179000.
Please E-mail this message to the author or to tech-talk@aps.anl.gov
Calling epicsThreadSuspendSelf()
[Tue Aug 31 22:02:43 2010] Minute trender made GPS time correction; gps=967352577; gps%60=57\

Dataviewers error was:

Connecting to NDS Server fb (TCP port 8088)
Connecting.... done
read(); errno=0
LONG: DataRead = -1
No data found

read(); errno=9
read(); errno=9
T0=10-09-01-05-13-55; Length=10 (s)
No data output.
 

Is there a reason for this (on ssh into fb from ws2):

RSA host key for fb has changed and you have requested strict checking.
Host key verification failed.

I am running some stuff on FB1 (awg for a Cal line mainly) on which my data depends. Please restart/reboot/rebuild nothing on fb1 or fb(0). WS1 and 2 are meat as soon as I get everything set.

changed default shell on fb1 back to bash temporarily, until the problems with tcsh are solved. The change to tcsh also causes that no sftp-service is running on fb1 and one can't use putty/winscp or other sftp tools for file transfer from/to fb1. changing back to bash fixes all the problems, including that all tools are working now.

Once we fixed those problems we can change to tcsh permanently

or switch back to bash shell in the meantime. That's working too. Simply open a shell, type in bash and then run them from the command line.

All computers have problems finding stuff since rana changed the shell.. This includes fb1 where the path doesn't contain the right directories anymore and you can't run dataviewer, diaggui, foton, ezcaread, ezcawrite etc...
Also fb1 isn't running sftp services anymore and i can't copy anything via sftp on that machine...

probably my fault. It looks like the working DTT is on fb1 in the /apps directory, but not in the /opt/apps directory. Something has gotten pointed to the wrong DTT during my path/shell cleanup.

Just use the fb1 version of diaggui and foton for now. I'll figure out which binaries are bad and then delete them tomorrow.

The Foton button didn't work on WS1.

• The button does: sh -c 'cd /cvs/cds/caltech/chans; /opt/apps/Linux/gds/bin/foton'
• I looked in /opt/apps/Linux/gds/bin and foton was there.
• I tried to manually execute the command and got the error:   sh: line 1:  2793 Aborted                 /opt/apps/Linux/gds/bin/foton
• DTT is similarly broken, and says "abort" when I try to run it in a terminal.
• Same problem on ws2
• The DV button (/opt/apps/Linux/dataviewer/dataviewer) works.

Have we started changing things?

I also restarted WS1 - no change. Not sure if there's some reason to not restart the frontend (via startatf then reboot) - there were silly reasons not to do other things.

The lab has turned into a pit again. Please clean up your stuff. There cannot be tools lying around or drinks or water bottles or used booties or scraps of paper.

Empty the trashes, change the sticky mats, etc.

I also power cycled the router. This router was just a bad choice - I suggest we just get the 24-port router which the 40m is using. I'll ask Joe if it will work.

i've tested it again and it's still working. You can find a simple,  working example for mDV getting some trend data for two of my channels a couple of days ago attached here:

 you have to change the mdv_config file to point to 131.215.115.216:8088 instead of the 40m framebuilder

I got the PLL back up and running yesterday before I left. I spent a disproportionate amount of time not knowing that the loop was not closed because "external source" was not on, despite that FM modulation was on and the source was set to external. I'm surprised there is no "modulation source to modulation ON/OFF" or "Displayed instrument settings to actual instrument settings ON/OFF" toggle on top of this.

The gyro is now in a state where it auto-acquires lock in both directions (though it sometimes catches on the TEM10), and the beat frequency is well within the PLL's lock range, so we get a gyro signal automatically. This signal (external voltage on the VCO) is acquired at GENERIC_DOF6_OUT, while TRANS DC is picked up at GENERIC_DOF7_OUT. I think DOF8_OUT has the actuation signal to the AOM VCO, but I don't remember for sure.

I made changes to various settings along the way. Here are the current ones:

• CW PDH (AOM): G = 7, inv ON, boost ON, phi = 5. There is now a 20 dB attenuator at the output to the VCO. This seemed to help with keeping the gain at a reasonable value while not limiting our range on the VCO in lowering the deviation.
• CCW PDH: G = 7, inv OFF, boost ON, phi = 4
• AOM VCO: f_nom = 47.451 MHz, dev = 100 kHz
• PDH LO: f_mod = 18 MHz. I tried using 33 MHz but the signal out of the (low-bandwidth) CW PD was puny and this generated noise downstream. This frequency appears devoid of other-modely interference.
• PLL: f_nom = 94.895719 MHz, dev = 375 kHz, SR560 G = 1 with no filtering (i.e. doing nothing)

I was not able to take a spectrum with DTT as the FB was dead, but I dumped the gyro signal into the SR785 as I was monkeying in the parameter space, and I was able to get the gyro noise at and slightly below 10 Hz to be on par with or ~ a factor of 2 or 3 better than it was before. This is taking into account that the previous calibration was off by a factor of 2*pi (lambdaP/4A takes optical Hz to rad/s, not RPS), so the noise can be made roughly 10-20 times lower than it used to be.

That said, it bothers me that we have no way of knowing at present if we are increasing our SNR or just reducing both noise AND signal whenever we make a change to a setting. I could have "made the noise much lower" by simply reducing the gain in the CW path, but this is not what would really be happening.

My issues with the PLL and settings-tweaking, along with Pinkesh's defense, precluded me from taking the transfer functions I wanted to take. These are the first step in characterizing our loops and ultimately in improving our SNR. They must be done before anything else.

Still having issues, but now more stuff is not working. Yesterday evening before I left, I noticed that I could load realtime data in DV, but not trended playback (didn't try raw). Still the same issue from here in FL: I get "NO DATA FOUND". I have verified that my channels are active and acquired.

As far as mDV is concerned, I don't get an error anymore when trying to get C2 data with get_data, but instead it just freezes at "C...". Now ligoDV doesn't work, either; it just spits out errors.

I want to try DTT, but without any time series data I don't know of a good time to try taking a spectrum. Is anything working for anyone?

Seems to be working now for me. Maybe Zach can tell us when he gets to Fla if his issues are resolved post reboot.

 i used mDV to get temp data about an hour ago and it was working...

It was (unintentionally?) rebooted when Koji and I were working down here a couple months ago. mDV update incoming

 I think that's right - we won't need any better than 1 mTorr. As long as there is no huge leak, we should be fine. It would be handy to have a (EPICS trendable) gauge on the system so that we can know if its leaking.

The system's total volume will be ~20 liters. So we need the leak rate to be below ~1e-6 Torr-liters / hour. A suggestion from Mike Z is to use the usual flexi-hosing from Norcal instead of the rigid nipple type of tube

I was talking about before. flexi hose link ..... I think we can use the 2" ID, 24" long tubes and make up for the difference in the length in the corner chambers. The length of each side of the gyro should be 29.5" with a 100 MHz FSR.

Thanks for taking care David.

It shouldn't make them die as only the tools we use are located on fb1, nothing else. So i don't know. But they should cpme back as soon fb1 is back too.

I can't remember but the last time we rebooted fb1 was last year or so i think. I know it's not a windows computer but maybe it needs to be rebooted once a year

Do you still have problems using mDV?

fb1 appears to have crashed or been restarted. ws1 and ws2 are nonresponsive (as a result?).

fb1 wants to check sdb1, I will let it do so (check forced blah blah).

If I understand correctly, ws1 and 2 drop when fb1 drops because fb1 hosts the /cvs/cds filesystem. I don't understand why this makes them die.

can someone plz check or reboot fb1. It doesn't respond anymore but the router is working fine as i can log into it...

if it doesn't come up plz check the main monitor on top of the blck rack what it says...

1) : the CVS-directory is a GLOBAL one which contains EVERYTHING for ANY computer in the lab area !.
So the basic answer is : the directory for fb is /cvs/cds/caltech/target/fb/ and the one for fb1 is /cvs/cds/caltech/target/fb1/.
This is important as the versions are different !!!

DON'T USE THE WRONG ONE ON THE WRONG COMPUTER !!

2): everything is working if you do it right on your computer. i tried it yesterday and today, using mDV on my personal computer from my hotel room.
So if anything isn't working it's probably a local problem!

3) the nds-proxy isn't working yet, so don't try to start it.

We're not looking for super high-vacuum though are we?  Maybe we can get away with borrowing the small turbo pumping station from the suspensions lab to pump it down and then we can just valve it off.

Also, we have the cylinder head for a tank of helium now so we should order in a tank to try that (the tanks get delivered really fast so that shouldn't be a problem).  Of course we'll at least need windows before doing that.

I've asked Gina to check on the CVI W2 window order.  The order went in on 22nd July and CVI said that they had them in stock.

 I put the gyro back into the normal configuration today. It is now locking in both directions again. The bandwidth seems slightly improved, as I didn't have to be quite as careful when working on the table to keep it from dropping, but it is far from good, and the 200kHz+ oscillation still shows up with enough gain. I had set up the PLL but was having problems locking it when I had to leave.

A couple changes:

• I kept the high-speed PDA10A that I used for the linear cavity in place as the CCW REFL PD, instead of the old PDA36A. We eventually want to have the same two PDs for both REFLs, but this doesn't really matter at the moment, and we might as well maximize our optical gain in the main locking loop while using the design modulation frequency of 33 MHz. The CW loop is still a 17-MHz diode, but this loop seems to like the lowest gain setting anyway, and we may end up wanting to reduce lower gain limit of the CW PDH box.
• The BS1 in the TRANS readout was a 45P, but we are using S, so I changed it to a 45S.

I took some time to label nearly all of the cables we work with at the controls interface. The REFL PD signals, the CW & CCW error signals, the PZT and AOM VCO control signals, the TRANS DC signal, the SWEEP drive from the function generator, and two cables going to dedicated DAQ channels have all been labeled. This way we don't have to tug on cables or follow them across the table to figure out what is what. We should be prudent that we don't change them around without relabeling them appropriately

Tomorrow morning I'm going to figure out what's wrong with the PLL, then lock it and take a spectrum to see if things have gotten better with the new PD, new modulation frequency, and fresh realignment. Then I plan to take a bunch of transfer functions to try and diagnose our loop problem. Specifically:

1. The closed-loop TF of each direction
2. The OLTF from the SWEEP input of each PDH box to its output

(2) is necessary to obtain the OLTF of the loops from (1), as we will be using the SWEEP input for the excitation, and the TF from SWEEP to OUT is not the same as that from IN to OUT.

For ease of use, isn't possible to just load the calibrations into EPICS so that the _OUT channels of these filter banks are calibrated into rads?

i.e. write a matlab or shell script that calculates the peak to peak values and then uses 'ezcawrite' to put the correct value into the GAIN field of that filter module.

Then as long as things don't drift at the 20% level, the data written to disk will be calibrated. There can even be a DAQ channel called PHASE_532 and PHASE_1064.

Then there could be a mDV script that just prints out the calibrated spectrum with noise budget automatically.

On the DAQ side, I was able to get the channel list with mDV by just pointing the mdv_config.m file at ganymede.ligo.caltech.edu (which is fb1 from the outside). However,

I didn't get any data. I looked on fb1 and for some reason, its running 2 copies of the daqd and nds jobs. fb (aka fb0) is just running 1 copy, as is normal. I suggest that

fb1 should only run daqd/nds from its own fb1 directories and that the fb processes on fb1 should be killed.

The NDSPROXY is a TCL script which we should run in the future on our future gateway machine, but not yet.

MZ Sum and Difference spectra as acquired by the DAQ: Calibrations on plot.

I was unable to get the channel list via get_data in matlab with a presumably properly configured mdv_config file. Zach says he is epxeriencing the same woes. mDV doesn't work, ligoDV does, dataviewer does. I looked at the nds log file (in /cvs/cds/caltech/target/fb/nds.log) and it seems to have stopped responding.

I heard scuttlebutt that this happened last week, and people found there was really poor/wrong documentatation on how to fix it. I found no record of these tribulations on the elog or the wiki. Maybe Frank or Alastair will read this and, in pity of us, post what happened last week, and maybe even what fixed the problem.

Confusion: There is both /cvs/cds/caltech/target/fb/ and /cvs/cds/caltech/target/fb1. When I looked at the log files, /cvs/cds/caltech/fb/nds.log seemed to be the one we were using recently (It had entries from August 9th - last night, when a DTT query for a test point from a DAQ channel seems to have crashed it). However, in /etc/inittab, it appears /cvs/cds/caltech/target/fb1/start_nds.inittab is the file executed, which does not correspond to the nds log file which reflected our recent queries.

Contents of each:

• cat /cvs/cds/caltech/target/fb/start_nds.inittab
  #!/bin/sh
  exec env LD_LIBRARY_PATH=/opt/epics-3.14.7-i386/base/lib/linux-x86/:/usr/local-2.95.3/lib /cvs/cds/caltech/target/fb/nds /cvs/cds/caltech/target/fb/pipe >& /cvs/cds/caltech/target/fb/nds.log
   
• cat /cvs/cds/caltech/target/fb1/start_nds.inittab
  #!/bin/sh
  exec su controls -c 'env /cvs/cds/caltech/target/fb1/nds /cvs/cds/caltech/target/fb1/pipe' >& /cvs/cds/caltech/target/fb1/nds.log
   
• cat /cvs/cds/caltech/target/fb1/ndsproxy/start_ndsproxy
  #!/bin/csh
  
  # Startup script for NDS proxy
  # - should make this run in a cron or init tab
  #
  #
  
  # This is because sometimes the old proxy is running
  pkill tclshexe
  
  # This is the home directory for NDS Proxy
  #set dir = '/cvs/cds/caltech/target/ndsproxy'
  set dir = '/cvs/cds/caltech/target/fb1/ndsproxy'
  
  # The command string - the log file gets the PID in the file name
  echo "Starting NDSProxy"
  $dir/ndsproxy -l 8088 -s fb -p 8088  >>&! ${dir}/log/ndsproxy_`date +%e%b%y`_$$.log &
• There is no /cvs/cds/caltech/target/fb/ndsproxy folder.
• last line of cat /etc/inittab:
  nds:35:respawn:/cvs/cds/caltech/target/fb1/start_nds.inittab
•  

WHICH OF THESE THREE IS THE RIGHT ONE TO BE RUNNING? It would be wonderful to get this documented.

One possible vacuum chamber solution for the Gyro is to use long tubes for the Gyro arms and then to have a small chamber with ports at each corner.

I looked a little at using stock MDC vacuum parts for this; its not out of the question.

For the tubes, we could use something like their NW50 Kwik-Flange nipple. It has an OD of 2" and a length of 6.5". Its $63.

For the corners, we can use one of their '5-way crosses' like the 406002. Its basically 5 flanges welded onto a shell. Depending on the size its ~$250 ea.

I would prefer to get one long tube for each arm, rather than stick a bunch of short ones together, so I'll get a quote from MDC on a custom job.

Uncoated quartz viewports are ~$250 ea. I expect that we will want AR coated and angled viewports. Maybe $400 ea then.

So the total cost, without pumps would be ~5k$.

 EDIT: The calibration has been corrected to include the right NPRO temp control coefficient as measured by Rana at the 40m. This makes it of roughly the same level as the incoherent noise in the MZ measurement, but still different in shape. I think the coherent noise dominates in this measurement because of the shorter path length, so it may be that the MZ noise limits us more in the real gyro (this will likely change after we put in the windows).

Here is a spectrum of the slow control signal to the NPRO calibrated to meters. The calibration is:

6.103 x 10^-4 V/ct * 1.1 GHz/V * (c / 1064 nm) * 0.75 m = 1.78 x 10^-9 m/ct

Unlike yesterday's measurement, this does not have the right shape to be our current limiting noise source. It does, however, pose an even bigger threat than the noise measured yesterday given our current understanding of how noise couples in. That is, if in fact the differential-mode noise is limiting us now because of our week feedback loops, then once we take care of that we will have to deal with this 0.1-um level noise and its FSR coupling, which raises the "displacement noise altering FSR" curve on the NB by about an order of magnitude.

The fact that this noise is directionally preferential---it shows up along one leg but not in the difference between two like paths, as yesterday---leads me to believe that it is not air noise. The calibration may be off in one of the two (or both) measurements but the data are qualitatively different, and I don't think we would see this if air noise were dominant. 

Let me clarify what I think is happening:

There are two types of noise we are considering here

1. Incoherent noise from the shaking of the mounts and from air currents. This shows up in the MZ-type measurement and in the linear cavity measurement.
2. Coherent noise from the stretching of the table. This shows up only in the linear cavity measurement

Now,

• If our locking loops have infinite gain and bandwidth, then we only see either source as it affects the area of the cavity. That is, we don't see direct length-to-frequency coupling from cavity noise, but rather only the modulation of the 1-FSR offset due to a CM change in cavity size. In this case, we will be concerned with the loudest of the above sources of noise, which I believe (given these past two measurements) is (2).
• Since our locking loops are far from perfect, we are seeing the noise from (1) couple in much more directly than it will in the final gyro. I do not propose a mechanism, but, for example, it could be that we are seeing noise from the (sometimes shakier) non-cavity optics that should be suppressed by the loops.

If I am right, then we may seriously start to consider the idea of locking the laser to a stable reference, then locking the CCW length of the gyro cavity to the laser via PZT.

Temperature noise may be a real problem. Probably Frank has a spectrum of the room's temperature noise handy and you can multiply by the CTE of the table top.

I guess the MZ noise and the gyro noise ought to be measured with the air on/off. I think it will point at the HVAC.

Maybe the word "mode" isn't the right choice. I'm not talking about a vibrational mode, but rather the low-frequency thermal expansion of the table. I find it hard to believe that the perimeter of the table won't change by more than 10^-10 m or so over 1-10 mHz timescales.

Regarding the AC on/off spectrum, do you mean a spectrum of the gyro signal, or some other configuration?

I have a feeling that Alastair will know better what to do about the windows, but I can try and think of something in his absence.

About the word wrapping, I don't know; I've never had an issue in Safari (OS X) or Firefox (Linux).

 I don't think there's any breathing mode being excited. At low frequencies, we are most likely measuring the air noise. Since the table has its first eigenfrequency above 100 Hz, the flexing at 100 mHz is suppressed by a factor of lots.

Most likely, we just need to plug the giant holes on that box. Where are those windows??? Also, take a spectrum with the AC turned off and make a comparison with it on/off.

P.S. Why do the entries in this elog not word-wrap in some browsers, while the ones in the TCS lab tab do word wrap?

New calibration - 10.4 and 15.2 Vpp for green and IR, respectively.

 After yesterday's measurement, it dawned on me that the noise measured in the free-running MZ setup is not the only kind of displacement noise we are worried about. It tells us something about the high-frequency shaking of the optics but does not tell us anything about the lower-frequency "breathing mode" type behavior as this is common to both MZ legs. While the gyro is still in pieces (sorry Alastair), I thought I might run a single-arm test, gyro style.

Simply put, I made a linear cavity out of one leg of the gyro, using the big input mirror and the southwestern curved mirror (the one by the laser). This is easy to do because the gyro modematching solution also works for this setup. The idea here is to let it run for some time and measure the low-frequency length drift. Of course, the assumption is that the expansion of the table is isotropic, so that this length drift is an indication of how the area of the cavity will change over long timescales. I don't see why this should not be the case.

Below is a photo diagram of the setup. I left the CW path blocked upstream of its faraday isolator, then installed a PBS/QWP isolator before the last steering mirror into the cavity on the CCW path. The QWP was stolen from the AOM double-pass setup temporarily. I was able to set up the REFL optics in such a way that the normal gyro optics are unaffected; with any luck, we will need no additional realignment as a result of this test.

The REFL PD is a PDA10A, and the modulation frequency is 33 MHz. I have the slow and fast controls signals acquired, as well as the transmission, so that I can monitor it overnight. I can force the thing to re-lock by messing with the offset to the slow control. This should not be necessary, though, as for some reason we have a MUCH stabler lock with the linear cavity. I have not taken any transfer functions, but it seems clear that we have much higher bandwidth, as I can knock on the table--hard--with no problems. Of course, the cavity is now highly overcoupled, which I believe gives us more optical gain, but in this configuration I can turn the PDH box gain way up before seeing any resonances. The fact that I'm using a 150-MHz REFL PD probably doesn't hurt. This might be reason to believe that our crappy lock issue is PD-related.

 I also added 2 new channels and dropped 2 old ones. DAQ+TP rate is 1703.

  I used the GUI to add 6 channels. I logged into fb0 and did "sudo pkill daqd"

• CTRL_ISS_1_OUT = IR PD diff
• CTRL_ISS_2_OUT = Green diff
• CTRL_SIG_1_OUT = IR sum
• CTRL_SIG_2_OUT = Green sum
• GENERIC_GEN3_OUT = NPRO power
• GENERIC_GEN4_OUT = PMC Trans Power

I added all the above at 256 Hz (manually adding a 4th order butterworth to each channel at 128 Hz because of the decimation the DAQ uses)

 I had to manually click the "DAQ reload" button on the GDS_TP medm screen to make a funny desynch error dissapear.

The channels seem fine when compared to their fast testpoint counterparts.

Burtrestore to Aug 23 00:25

BURT seems to have not been done properly at some point in the last few days (all epics values blank in C2ATF) - I opened up the burtgooey and restored to just before the last backup of the .ini file. This restored everything. Things added since then will have to be toggled back by hand.

Do we mean to be taking snapshots every hour?

That weird little curve at the low frequency end of the graph is an artificial product of the DTT-- just check the "remove mean" box and it will fix it.

Someone burned it.

 While I was using the ThorLabs power meter today, I noticed that the slide-out filter appears to have a small bump on it near the center. I don't think I dinged it on anything, and it doesn't even really look like the result of a ding. It's tough to tell from the picture, but the bump is raised up, and not dented in. Has anyone seen this before (on our unit or elsewhere)?

 Today, I turned the input and output mirrors round 90 degrees to take a displacement noise spectrum on our table. There were several steps:

1. Turn the mirrors
2. Block one input beam path (I chose to use the simpler CCW beam so that I didn't have to have the AOM running, etc., so I blocked the CW path)
3. Rearrange the setup at the output (in as minimally invasive a way as possible) so that I was looking directly at the power emerging from the two directions. I set up two PDA100A's since they are wide-area and we don't need anything too fast for this.
4. Ideally, this measurement is done with 50/50 splitters at the input and output, so that the contrast is 100% when the beams are overlapped. Our mirrors are very reflective, so I rotated the input polarization from S to P to take advantage of the fact that the mirrors' R-to-T ratios are not quite as high for P light. I also had the EOM off to avoid adding extra AM and other junk.

I routed the outputs of the PDs to the DAQ, and acquired:

1. The PD signal with the best contrast. (One output beam has a twice reflected beam plus a twice transmitted beam, while the other has two beams that have each been reflected and transmitted once. If the input and output mirrors had the same R's and T's, then this emerging beam would have 100% contrast. They don't, so it doesn't, but it's better than the other.)
2. The sum, so that I could normalize the signal in (1) to reject AM.

I then fine-tuned the alignment so that the output was sitting roughly halfway between a bright fringe and a dark one (so that the power was roughly linear with displacement) and took data.

I wanted to normalize the data to the sum as mentioned above, but I don't know of a good way to do it with DTT or the like short of using Simulink to build a divider block and acquire the processed data. In any case, I ligoDV'd the data into Matlab and normalized it there, and the difference is almost unnoticeable. I can include this later if we really think it's necessary. Here is the spectrum:

It is similar in shape and of roughly the same order of magnitude as the MZ noise that Rana measured before (see the SVN NB). It is a bit lower, especially at low frequency, but it's worth noting that the low-frequency noise goes up quite a bit when the lid of our box is open (good work, Alastair).

The calibration here is a bit iffy. I have

dP/dx|_phi=0 ~ 4 * pi * r * t * sqrt(P1 * P2) / lambda,

where r, t are the amplitude reflectivity and transmissivity, and P1&2 are the powers in the two beams reaching the output mirror from each side. There is a 50/50 splitter between the output mirror and the PD (I didn't notice this until after), so this is reduced overall by a factor of two.

The PDA100A was on its 0 dB setting, giving G_PD = 0.7 A/W * 1510 V/A, and I took the ADC gain to be 6.103 x 10^-4 V/ct.

Now the iffy part. Using the powers I measured before the recombination and the r's and t's I measured some weeks ago (yes, for P), I get that the calibration is 2.06 x 10^-10 m/ct. However, if I use the measured maximum and minimum values on the PD over full-wavelength swings of the output mirror to calculate dP/dx directly (using the cross term in the equation P_PD = |[rE2exp(i*phi) + itE1]|² ), I get 3.45 x 10^-11 m/ct. It is this second calibration that appears above, but it remains to be understood why there is a discrepancy. I could understand it if the contrast got worse due to imperfect overlap, which there surely was, but it appears to be better than calculated. Does not compute.

This would appear to exonerate displacement noise as our limiting noise source at the moment (we've plugged similar numbers into the budget and found that we should be orders and orders of magnitude more sensitive), but it would be either foolish or irresponsible to deny the obvious likeness of the spectrum above and that of our gyro signal. Their characteristics are nearly identical, save for some suppression from the reasonable gain of our feedback loops at lower frequencies. I think we may have to consider the possibility that displacement noise is coupling in through some other mechanism that we haven't yet considered.

 I had to acquire a couple of new channels to do the displacement noise measurement today. I added:

• C2:ATF-GENERIC_DOF7_OUT
• C2:ATF-GENERIC_DOF8_OUT

I think I inadvertently deactivated C2:ATF-GENERIC_GEN8_OUT, but I don't think we need that one right now anyway.

I used the GUI, and more than one backup .ini file was saved as I did not add the right channels at first. All seems to be working well.

 I created a page called "Gyro Equipment" on the Gyro Wiki. This page has links to subpages called "Laser", "Mirrors", "Photodiodes", "PDH Boxes", and "Misc Electronics". I imagined that we can use this to keep track of what equipment we are using when, in case we ever forget. The only living page is "PDH Boxes", which contains information (transfer functions, updated schematics, and--soon--pictures) about each of our boxes as well as a link to the original schematic on the DCC. We should populate the other pages ASAP.

 1) User directories have been moved into /users. There is now the awesome /users/abrooks/aidan/Aidan/ directory.

Pure awesomeness.

 1) User directories have been moved into /users. There is now the awesome /users/abrooks/aidan/Aidan/ directory.

2) The default shell on fb1, ws1, and ws2 is now tcsh. I copied over the .cshrc from the 40m, modified it, and put it into target/.

   The .cshrc in the controls/ directories now is a symlink pointing to target/cshrc.atf. 
3) LIGOTOOLS installed in caltech/apps/linux/. There is now also a apps/linux32/ for all the 32bit apps.

4) MEDM is now setup to use scalable fonts. To launch the usual sitemap, now type 'medm_norm' at the terminal.
   You can do your screen editing using scalable fonts and then everything will look fine with both scalable
   and fixed fonts.

 you can bring your crew but.. nevermind.

Alastair and I tried out the new-and-improved box #2215 on the gyro today, to see if the noise was improved at all. The short answer is "not really". There is some slight improvement at higher frequencies but at lower frequency the noise is the same or worse, depending on the gain settings we chose. 

I think there is something tricky about how we distribute the gain in the AOM loop (i.e. between the PDH box and the 'deviation' of the VCO), as playing with these

settings produced substantially different results in that loop, as well as in the gyro signal when it was locked.

It bothers me that the CCW (non-AOM) loop is so finicky; it would be nice to be able to achieve the many-kHz bandwidth we desire, but we are clearly limited by some resonance somewhere.

I think this should be our main priority right now, as it not only limits our sensitivity but also proves to be a huge pain in the ass when doing anything that involves touching the table.

Before we take our next breath, we should rotate the input and output mirrors 90 degrees and get a displacement noise measurement (so we can stop living in the past with the ol' 40m MZ).

After that, I would like to fully realign the gyro cavity, as well as get a reasonable modematching solution done for the CW path like there is for CCW. Then we will have decent coupling from both directions and we can tackle the servo stabilization issue.

 I finished making the modifications to box #2215 (aka "box 2", "the CW box", "the AOM path box", "Killer" etc etc) so that it is identical to #1437 ("box 1"), which Rana modified a few months ago. Attached is the transfer function, which you can compare with that of the other.

I plan to update the schematics and put them---along with these transfer functions and pictures of the boxes---on the Gyro wiki.

• File saved under /users/dmass/rawdata/MZSpecs/scrn000(2/3).txt
• Calibration: 7.04Vpp = Green Diff Span (balanced)
• 14.2Vpp = IR Diff Span (balanced)

 Big things not done:

• Phase matching temperature found
• Box not weighed down / screwed down

Would like to do

• Quick calibration of MZ PZT control signal to get this trended as well, so I can simultaneously measure ERR and CONTROL of IR along side the  green spectrum

Some CDS notes:

1) I suggest that we abandon the 'Router as Port Forward' scheme and just adopt what is running at the 40m and the sites:     A gateway machine sits on the LIGO GC network and allows access to the inside network. A NAT router manages the traffic     from the inside network to the outside. I'm not arguing the merits of one or the other, just that its simpler to maintain    if we are all the same. 
2) The Linux CDS apps we are running are all in /apps/Linux/. I suggest that we move to the same apps structure as is being    adopted for aLIGO. The 40m currently has /cvs/cds/caltech/apps/linux, apps/linux64/, and apps/solaris/. We can pick something    similar and then just move all of the binaries over. Its possible, but unlikely, that we will want some solaris. There's also    this /opt/apps/ area which is partially redundant with the /apps/ stuff. Very likely, this is also causing us problems. 
3) We should also switch over all machines to use tcsh instead of bash. All of the sites have tcsh as the standard shell on    all workstations and all accounts, so it will make the setup of the standard environment using the stdenv.csh scripts easier.     We could argue the merits of bash over tcsh until the cows come home, but its all already been hashed out on internet newsgroups.

4) For some reason, we have bad user directory stuff going on like /cvs/users/abrooks and /cvs/users/users/aidan. Please don't    make your user directories in the ~/ home directory but instead everyone just use e.g. /cvs/users/homer/. /cvs is the shared    disk and is visible on everything. This is the one that we will set up for the daily cron rsync backup to CACR like the 40m does.

5) For BURT, today I made a /cvs/cds/caltech/burt for us along with a autoburt/ dir and a snapshots/ dir in there. This sets up
   the directory structure until 2016. I also copied over the burt.cron and the autoburt.pl. The autoburt.pl has a bunch of hacky
   stuff for finding out what site we're at and bypassing the Y2010 problem, but I'm hacking the hack.
   
6) For BURT, I set the hourly cron on fb1 by using 'crontab -e'. At 18 minutes after the hour, each hour, it will save a snapshot
   of whatever in the target/ dir has an autoBurt.req file. Log files are written to autoburt/logs/ and the snaps are saved in
   autoburt/snapshots/. Its failing on the pslepics right now since Frank just turned it off, but its a good test to show that
   a dead FE doesn't crash the autoburt.

   To do restores after a reboot, you can use the 'burtgooey' tool. In principle the 'startatf' will also do an automatic restore
   but that relies on the system being in a good state the last time the 'killatf' was run.

7) Next, we should set up the CONLOG infrastructure so that we can easily find diffs in control settings.

8) We also want to record all of the SLOW channels from our MEDM screens as DAQ channels (e.g. INMON, EXCMON, OUTMON, OUT16, & OUTPUT).
   For this we need to copy over the script Rob wrote to automatically produce an .ini file from the .db file in the corresponding
   target directory.

That is all.

i've killed the PSL RT frontend code. Everything else should be ok, so plz check if everything is working.
If not plz let me know and i will fix it.

Rana and I re-made the front end code today and restarted the front end at around 8pm.  We didn't change any of the model - this was mainly just an exercise to try to document the process so that it can be put up on the wiki.

Rana also discovered that BURT is working, and we can use this to log and restore the values in our MEDM screens.  We just need to manually write and restore the BURT snapshot files just now.  The auto-BURT isn't working for the moment.

After restarting we now have a new C2ATF.ini file, so if anyone was recording frames in C2 then they will need to use the GUI to restart the frame logging for that channel (use daqconfig to get the GUI up).  Since this GUI is working we should all start using it exclusively when we add new channels.