The FE web view seems not up-to-date, does it? ( maybe for a year)

https://nodus.ligo.caltech.edu:30889/FE/c1mcs_slwebview_files/index.html

Summary:

c1rfm / c1lsc / c1ass / c1sus were modified. They were recomplied and installed. They are running fine
and confirmed PRMI locking (attempt), arm locking, and Yarm ass with the new codes.

Motivation:

1a. SQRTing switching for POP110 was wrong. 0 enabled sqrting, 1 disabled sqrting. I wanted to fix this.
1b. Sqrting for POP22 was not implemented.

2. Preparation for the shadow sensor control with POPDC.

3. ASS had only an input. I want to run two ASS for the X and Y arms.

SQRTing for POP110/22:

- Flipped the input of the bypass switch. Correspoding MEDM indicators are fixed on the power normalization screen.
- Copied the sqrting structure from POP110 to POP22. Correspoding MEDM buttom was made on the power normalization screen.

- The function of the sqrting buttons were confirmed.

Additional ASS output:

- The output path "NPRO" was removed. Corresponding RFM channels have also removed.
- The previous NPRO path was turned to the "ASS1" path. The previous "ASS" path was turned to "ASS2".
- Corresponding shared memory channel are created/renamed.
- c1ass was modified to receive the new ASS shared memory channels. ASS1 is assigned to the X arm. ASS2 is assigned to the Y arm
- The output matrix screen and the lockin screen were modified accordingly.
- Only script/ASS/Arm_ASS_Setup.py was affected. The corespoding lines (matrix assignment) was fixed.

- The function of Den's version of  ASS was confirmed.

LSC->PRM ASC path

- We want to connect POPDC to PRM ASC. POPDC is acquired on c1lsc.
- So, for now we use the LSC input matrix to assign POPDC to one of the servo bank.
- The last row of the LSC output matrix was assigned to the PCIE connection to c1sus.
- This PCIE connection was connected to the PRM ASC YAW input.

- The connection between LSC and SUS was confirmed.

- During this process I found that there are bunch of channels transferred from LSC to SUS via RFM.
  These channels are transferred via PCIE(dolphin) and then via RFM. But LSC and SUS are connected
  with dolphin. So this just adds additional sampling delay while there is no benefit. I think we should remove the RFM part.
  Note that we need to use RFM for the end mirrors but this also should use only the RFM connection.

Rebuilding the codes

- Prior to the tests of the new functionalities, the codes were rebuild/installed as usual.
- The suspension were shutdown with the watch dogs before the restart of the realtime codes.
- Once the realtime codes were restarted successfully, the watch dogs were reloaded.
- As we removed/added the channels, fb was restarted.
- c1rfm / c1lsc / c1ass / c1sus codes were checked-in to svn
 

Someone for some reason added full-rate DAQ specification to some ADC3 channels in the c1sus IOP model (c1x02):

#DAQ Channels

TP_CH15 65536
TP_CH16 65536
TP_CH17 65536
TP_CH18 65536
TP_CH19 65536
TP_CH20 65536
TP_CH21 65536

These appear to be associated with c1pem, so I'm guessing it was Den (particularly since he's the worst about making modifications to models and not telling anyone or logging or svn committing).

I'm removing them.

Previously, for some reason, many IPC connections were routed through the c1rfm model, even if a direct IPC connection was possible.  It's unclear why this was done.  I spoke to Joe B. about it and he couldn't remember either.  Best guess is that it was just for book keeping purposes.  Or maybe some old timing issue that has been fixed by DMA fixes in the RTS.  So the point is that it's no longer needed, and we can reduce delays by making direct connections.

I made direct IPC connections from c1lsc to both c1sus and c1mcs, bypassing the c1rfm, through which they had previously been routed.  All models were rebuilt/installed/restarted and everything seems to be working fine.

As with the previous post, I eliminated and unnecessary hop through c1rfm for the c1als --> c1mcs connection for the ALS output to MC2 POS.

As a side note, we might considering piping the ALS signals into the LSC input matrix, elevating them to actual LSC error signals, which in some since they are.  It's just that right now we're routing them directly to the actuators without going through the full LSC control.

As with the last two posts, I eliminated more unnecessary passing through c1rfm for IPC connections between c1mcs and c1ioo.

All models were rebuilt/installed/restarted and svn committed.  Everything is working and we have eliminated almost all IPC errors and significantly simplified things.

We need ADC and DAC channels for a couple of things:

• POP QPD: 3x ADC
• ALS PZTs: 3x 2x 2x DAC (three pairs of PZTs, at ends and vertex, each with two channels for pitch and yaw)
• Fibox: 1x DAC

What's being used:

• c1iscex/c1iscey:
  
  • DAC_0:   7/16 = 9 free
  • ADC_0: 17/32 = 15 free
• c1sus:
  
  • DAC: ?
  • ADC: ?
• c1ioo
  
  • DAC_0:   0/16 = 16 free ?? This one is weird. DAC in IO chassis, half it's channels connected to cross connect (going ???), but no model links to it
  • ADC_0: 23/32 = 9 free
  • ADC_1:  8/32 = 24 free
• c1lsc
  
  • DAC_0: 16/26 = 0 free
  • ADC_0: 32/32 = 0 free

What this means:

• We definitely have enough DACs for the ALS PZTs.  The free channels are also in the right places: at the end stations and in the c1ioo FE, which is close to the PSL and hosts the c1als controller.
• We appear to have enough ADCs for the QPD in c1ioo.
• We don't have any available DAC outputs in c1lsc for the Fibox.  If we can move the Fibox to the IOO racks (1X1, 1X2) then we could send LSC channels to c1ioo and use c1ioo's extra DAC channels.

Of course we'll have to investigate the AA/AI situation as well.  I'll try to asses that in a follow up post.

PS: this helps to identify used ADC channels in models:

grep adc_ sus/c1/models/c1scx.mdl | grep Name | awk '{print $2}' | sort | uniq

It looks like we have spare channels in the AA chassis for the existing c1ioo ADC inputs to accommodate the POP QPD. 

We need AI interfaces for the ALS PZTs.  What we ideally need is 3x D000186, which are the eurocard AI boards that have the flat IDC input connects that can come straight from the DAC break-out interfaces.  I'm not finding any in the spares in the spare electronics shelves, though.   If we can't find any we'll have to make our own AI interfaces.

I noticed a couple potential issues in some of the models while I was investigating the ADC/DAC situation:

c1ioo links to ADC1, but there are broken links to the bus selector that is supposed to be pulling out channels to go into the PSL block.  They're pulling channels from ADC0, which it's not connected to, which means these connections are broken.  I don't know if this means the current situation is broken, or if the model was changed but not recompiled, or what.  But it needs to be fixed.

c1scy connects ADC_0_11, label "ALS_PZT", to an EpicsOutput called "ALS_LASER_TEMP", which means the exposed channel is called "C1:SCY-ALS_LASER_TEMP".  This is almost certainly not what we want.  I don't know why it was done this way, but it probably needs to be fixed.  If we need and EPICS record for this channel it should come from the ALS library part, so it gets the correct name and is available from both ends.

1. What are we using 16 DAC channels for in the LSC?
2. What are the functions of those IOO DAC channels which go to cross-connects? If they're not properly sending, then we may have malfunctioning MC or MCWFS.
3. Can we just use the SLOW DAC (4116) for the ALS PZTs? We used this for a long time for the input steering and it was OK (but not perfect).

For the new input and output tip-tilts.  Two input, two output, each requires four channels.

I have no idea.  I don't know what the hardware is, or is supposed to be, connected to.  DAC for WFS??  Was there at some point supposed to be fast output channels in the PSL?

 Probably. I'm not as familiar with that system.  I don't know what the availability of hardware channels is there.  I'll investigate tomorrow.

While looking at the DAC anti-imaging filters, Koji noticed an odd feature of the DAC output:

What you see here is 16kHz double data from a model right before the DAC part ('C1:SUS-PRM_ULCOIL_OUT', blue), and the full 64kHz int data sent to the physical DAC as reported by the IOP ('C1:X02-MDAC0_TP_CH0', green).  The balls are the actual sample values (as expected there are four green balls for every blue).  The output data is being ramped continuously between 0 and 1.

As the output data crosses the half-count level, the integer DAC output oscillates continuously at every 64kHz sample between the bounding integer values (in this case 0 and 1).

Here's the data as we hold the output continuously at the half-count level; the integer DAC output just oscillates continuously:

After some probing I found that the oscillation happens between [-0.003 +0.004] of the half-count level.

The result of this is a fairly strong 32 kHz line in the DAC analog output.

We looked in the controller.c and couldn't identify anything that would be doing this.  This is the output procedure as I can see it (controller.c lines): 

1. The double from the model is passed to the IOP
2. The IOP applies a sample-and-hold or zero-pad if the model is running at a slower speed than the IOP (1799)
3. The data is then anti-image filtered (1801)
4. A half-integer is added/subtracted before casting such that the cast is a round instead of a floor (1817)
5. The data double is cast to an int (1819)
6. The data is written to the DAC (1873)

There's nothing there that would indicate this sort of bit flipping.

Let me make a complimentary comment on this effect.

Because of this oscillation feature, we have a 32kHz peak in the DAC spectrum rather than a 64kHz peak.

For advanced LIGO, the universal AI (D070081) was made to have 3rd-order 10kHz LPF with 64kHz notch.
If we have a higher peak at 32kHz (where the rejection is not enough) than at 64kHz, the filter does not provide
enough filtering of the DAC artifacts.

For the 40m, the original filter had the cut off of 7kHz as the sampling rate was 16kHz.
If we want to extend the frequency range by 4times, the correspoding cut off should be 28kHz.
The rejection is again not enough at 32kHz.

If this peak is an avoidable feature by using simple sample&hold the peak freq is pushed up to
64kHz and we can use the AI filters as planned.

After querying CDS folks about this issue, I got some responses that indicated the problems was likely limit-cycle oscillations due to zero-padding of the data when upsampling.  Tobin ran some Matlab tests to confirm this issue.

Starting in RCG 2.5 there is a new "no_zero_pad=1" cdsParameters option turns zero padding OFF.  I tried enabling this option c1scy to see how the behavior changed.  Sure enough, the 32 kHz oscillations mostly went away.  There are no oscillations for outputs held at the half-count value, and the oscillations around the half-count transitions went away as well.

The only thing I could see is a bit of oscillation when converging on a constant half-count value that went away after a couple of milliseconds:

So we might consider adding the no_zero_pad=1 option to all of our coil driver outputs, which might eliminate the need to add notches at the Nyquist in the analog anti-image filters

Is this limit cycle caused by the residual of the digital AI filtering at the half sampling freq and that his the threshold?
Or is this some nonlinear effect? If this is a linear effect associated with the zero-padding, the absolute
value of the DC may affect the amplitude of the oscillation. (Or equivalently the range of the DC where we get this oscillation.)

Anyway, it seems that we should use no-zero-padding.

You pointed out the ringdown of the digital AI filter in the sample-hold case (i.e. no-zero-padding case).
How does it look like in the conventional zero-padding case?

This is a good question.  We may be able to test if it's a linear effect if we have enough DAC range to get the oscillation to be more than a single sample.

 In the zero-pad case the oscillation just continues indefinitely at the half-count value, so it never dies out (at least as far as I can tell).

It seems that the effect is from the (linear) residual fluctuation of the digital AI filter for the zero-padded signal.

Namely, if we give the larger constant number, we get more oscillation.

c1scy, the controller model at the Y END, is still running very long, typically at 55/60 microseconds, or ~92% of it's cycle.  It's currently showing a recorded max cycle time (since last restart or reset) of 60, which means that it has actually hit it's limit sometime in the very recent past.  This is obviously not good, since it's going to inject big glitches into ETMY.

c1scy is actually running a lot less code than c1scx, but c1scx caps out it's load at about 46 us.  This indicates to me that it must be some hardware configuration setting in the c1iscey computer.

I'll try to look into this more as soon as I can.

I have attempted to reconcile all of the ADC connections to c1ioo.  Upon close inspection, it appears that there was a lot of legacy stuff hanging around.  Either that or things have not been properly connected.

The c1ioo front end machine has two ADC cards, ADC0 and ADC1, which are used by two models, c1ioo and c1als.  The CURRENT ADC connections are listed in the table below.  The yellow cells indicate connections that were moved.  The red cells indicate connections that were removed/unplugged:

The following changes were made:

• "MC L" had been connected to ADC_0_0, moved to ADC_0_6
• "MC F" had been connected to ADC_0_6, moved to ADC_0_7

The c1ioo model was rebuilt/restarted to reflect this change.

The PSL-FSS_MIXER and PSL-FSS_NPRO connections were broken in the c1ioo so I fixed them when I moved the MC channels.

All the removed connections from ADC1 were not used by any of the front end models, which is why I unplugged them.  Except for the MC DAQ interface J1 and MC servo DAQ connections, I left all other cables plugged in to wherever they were coming from.  The MC cables I did fully remove.

I don't know what these connections were meant for.  Presumably they expose they expose some useful DAQ channels that we're now getting elsewhere, but I'm not sure.  We don't currently have an ISS, which is presumably why the cables labelled "ISS" are not going anywhere.

TODO

I would like to see some more 4-pin lemo --> double BNC cables made.  That would allow us to more easily use the ADC1 generic interface board:

• Moved ALS TRX/Y to ADC1, so that we can keep all the ALS connections together in ADC1.
• POP QPD X/Y/SUM

We should also figure out if we're sub-optimally using the various "DAQ" connections to the DAQ cable connectiosn to the eurocrate DAQ interface cards and servo boards.

The c1als model was pulling out some ADC0 connections that were no longer used for anything:

• ADC_0_1 --> sfm "FD" --> IPC "C1:ALS-SCX_FD"
• ADC_0_5 --> sfm "OCX" --> term
• ADC_0_6 --> sfm "ADC" --> term

The channels would have shown up as C1:ALS-FD, C1:ALS-OCX, C1:ALS-ADC.  The IPC connection that presumably was meant to go to c1scx is not connected on the other end.

I removed all this stuff from the model and rebuilt/restarted.

Following Jamie's table in elog 8654, I have connected up the channels 0, 1 and 2 from ADC0 on the IOO computer to rfm send blocks, which send the signals over to the rfm model, and then I use dolphin send blocks to get over to the ass model on the lsc machine. 

I'm using the 1st 3 channels on the Pentek Generic interface board, which is why I'm using channels 0,1,2. 

I compiled all 3 models (ioo, rfm, ass), and restarted them.  I also restarted the daqd on the fb, since I put in a temporary set of filter banks in the ass model, to use as sinks for the signal (since I haven't done anything else to the ASS model yet).

All 3 models were checked in to the svn.

I have implemented a proto-ASC in the ASS model.  

In an ASC block within the ASS model, I take in the POP QPD yaw, pit, and sum signals.  I ground the sum, since I don't have normalization yet (also, the QPD that we're using normalizes in the readout box already).  The pit and yaw signals each go through a filter bank, and then leave the sub-block so I can send the signals over to the SUS model, to push on PRM ASCPIT and ASCYAW. 

In doing this, I have removed the temporary PRM ASCYAW connection that Koji had made from the secret 11'th row of the LSC output matrix (see Koji's elog 8562 for details from when he implemented this stuff).

LSC, SUS and ASS were recompiled, and restarted.  I also restarted the daqd on the fb.

I am working on making the Proto-ASC less "proto".  I have put IPC senders in the LSC model to send the cavity trigger signals over to the ASS model, for ASC use.  I'm partially done working on the ASC end of things to implement triggering.

LSC should be compile-able right now, ASS is definitely not.  But, I expect that no one should need to compile either before I get back in a few hours.  If you do - call me and we'll figure out a plan.

I have finished my work on the LSC and ASS models for now. The triggering is all implemented, and should be ready to go.  There are no screens yet.

I have *not* compiled either the LSC or the ASS, since Rana and Manasa still have the IFO.

The bank marked channel 9-16 is free, but the connector is a 40 pin IDC and I need to know the exact pin-out configuration before I can set about making the custom ribbon cable that will send the control signals from the DAC card to the PZT driver board. 

The DAC interface board on rack 1Y4 seems to be one of the first versions of this board, and has no DCC number anywhere on it. Identical modules on other racks have the DCC number D080303, but this document does not exist and there does not seem to be any additional documentation anywhere. The best thing I could find was the circuit diagram for the ADL General Standards 16-bit DAC Adapter Board, which has what looks like the pin-out for the 68 pin SCSI connector on the DAC Interface board. Koji gave me an unused board with the same part number (D080303) and I used a multimeter and continuity checking to make a map between DAC channels, and the 40 pin IDC connector on the board, but this needs to be verified (I don't even know if what is sitting inside the box on 1Y4 is the same D080303 board).

Jenne suggested making a break-out cable to verify the pin-outs, which I did with a 40-pin IDC connector and a bit of ribbon wire. The other end of the ribbon wire has been stripped so that we can use some clip-on probes and an oscilloscope to verify the pin-outs by sending a signal to DAC channels 9 through 16 one at a time. On the software side, Jenne did the following:

• Restarted the mx_stream on c1iscey  (unrelated to this work)
• 8 Excitation points added in the simulink model on c1scy 
• Model compiled and installed

We have not restarted c1scy yet as Annalisa is working on some Y-arm stuff right now. We will restart c1scy and use awggui to perform the test once she is done.

 Pink edits by JCD

 Jenne just rebooted c1scy and daqd on the framebuilder. We will do the actual test after lunch.

  Summary:

The pin-outs for the DAC interface board have been determined.

Details:

• I used a temporary break-out cable (pic attached) and connected the 40pin IDC connector on this to the DAC interface board at 1Y4.
• I had a hypothetical pin-out map which was to be verified. So I connected pairs of ribbon wire to an oscilloscope in the configuration which I believed to be correct, and then used awggui to send a 3Hz, 10000 count sine-wave to the corresponding channel via the excitation points set up earlier.
• I verified that the correct waveform showed up on the scope screen. I then tried sending the same signal to another DAC channel and verified that there were no accidental shorts/bad connections. The signal was fairly noisy, but this was probably because of the makeshift connections.
• Repeated the above for all 8 channels in the bank marked 9-16 on the DAC interface board.

Turns out that my deductions using the D0902496 wiring diagram, a spare D080303 DAC to IDC adaptor and a multimeter were correct! The pin outs as determined by this test are sketched in the graphic below.

To Do:

•  Now that the pin-outs have been determined, I need to go about making the custom ribbon that will connect the 40pin IDC on the DAC interface board to the 10-pin IDC on the PZT driver board. Because there is a pair of wires that will have to be 'skipped' while going from the 40-pin to the 10 pin IDC (corresponding to the pair not-connected between two DAC channels on the 40-pin IDC), this may be tricky.

Misc:

The excitation points added to the simulink model are still there, I plan on keeping it as such till I finish installation of the boards as they will be useful for testing purposes.

Pin-Outs of the DAC to IDC Adaptor (D080303) inside the "DAC Interface Box at 1Y4":

Makeshift break-out ribbon cable:

c1sus, c1ioo, c1iscex and c1iscey were down. Why? I was trying to lock the arm and I found that around this time, several computers stopped working mysteriously. Who was working near the computer racks at this time???

I did an ssh into each of these machines and rebooted them sudo shutdown -r now

But then I forgot / neglected/ didn't know to bring back some of the SLOW Controls computers because I am new here and these computers are OLD. Then Rana told me to bring them back and then I ignored him to my great regret. As it turns out he is very wise indeed as the legends say.

So after awhile I did Burtgooey restore of c1susaux (one of the OLD & SLOW ones) from 03:07 this morning. This brought back the IMC pointing and it locked right away as Rana foretold.

Then, I again ignored his wise and very precious advice much to my future regret and the dismay of us all and the detriment of the scientific enterprise overall.

Later however, I was forced to return to the burtgooey / SLOW controls adventure. But what to restore? Where are these procedures? If only we had some kind of electronics record keeping system or software. Sort of like a book. Made of electronics. Where we could LOG things....

But then, again, Rana came to my rescue by pointing out this wonderful ELOG thing. Huzzah! We all danced around in happiness at this awesome discovery!!

But wait, there was more....not only do we have an ELOG. But we also have a thing called WIKI. It has been copied from the 40m and developed into a thing called Wikipedia for the public domain. Apparently a company called Google is also thinking about copying the ELOG's 'find' button.

When we went to the Wiki, we found a "Computer Restart Procedures" place which was full of all kinds of wonderous advice, but unfortunately none of it helped me in my SLOW Controls quest.

Then I went to the /cvs/cds/caltech/target/ area and started to (one by one) inspect all of the targets to see if they were alive. And then I burtgooey'd some of them (c1susaux) ?? And then I thought that I should update our 'Computer Restart Procedures' wiki page and so I am going to do so right now ??

And then I wrote this elog.

The proto-ASC now includes triggering.  I have updated the hacky temp ASC screen to show the DoF triggering.  I have to go, but when I get back, I'll also expose the filter module triggering.  So, for now we may still need the up/down scripts, but at least the ASC will turn itself off if there is a lockloss.

I found CDS rt processes in red. I did 'mxstreamrestart' from the medm. It did not help. Also ssh'd into c1iscex and tried 'mxstreamrestart' from the command line. It did not work either.

I thought restarting frame builder would help. I ssh'd to fb. But when I try to restart fb I get the following error:

controls@fb ~ 0$ telnet fb 8088
Trying 192.168.113.202...
telnet: connect to address 192.168.113.202: Connection refused

daqd was restarted.

- tried telnet fb 8088 on rossa => same error as manasa had

- tried telnet fb 8087 on rossa => same result

- sshed into fb ssh fb

- tried to find daqpd by ps -def | grep daqd => not found

- looked at wiki https://wiki-40m.ligo.caltech.edu/New_Computer_Restart_Procedures?highlight=%28daqd%29

- the wiki page suggested the following command to run daqd /opt/rtcds/caltech/c1/target/fb/daqd -c ./daqdrc &

- ran ps -def | grep nds => already exist. Left untouched.

- Left fb.

- tried telnet fb 8087 on rossa => now it works

FE Web view was broken for a long time. It was fixed now.

The problem was that path names were not fixed when we moved the models from the old local place to the SVN structure.

The auto updating script (/cvs/cds/rtcds/caltech/c1/scripts/AutoUpdate/update_webview.cron) is running on Mafalda.

Link to the web view: https://nodus.ligo.caltech.edu:30889/FE/

 In light of recent events and the decision to test the piezo tip-tilts for green beam steering on the X-end table, I have set up 8 excitation points to channels 8 through 15 of the DAC on c1scx (as was done earlier for the DAC at 1Y4 with Jenne's help) in order to verify that the pin-outs of the DAC interface board. I have not yet compiled the model or restarted the computer, and will do these tomorrow, after which I will do the test. The channels are named YYY_CHAN9 etc. 

 I just compiled and installed the model with the excitation points on c1scx and then restarted framebuilder. The channels I set up are now showing up in the awggui dropdown menu. I will do the tests on the DAC channels shortly.

Just to keep things on record, these are the steps I followed:

• opened the model c1scx (path: /opt/rtcds/userapps/release/sus/c1/models) with MATLAB
• Added 8 excitation points and saved the model. A copy has been saved as c1scx.mdl.r2010b because of the recent upgrade to r2013a. 
• ssh to c1iscex (computer running the model c1scx). 
• Entered the following sequence of commands in terminal: rtcds make c1scx ,  rtcds install c1scx , rtcds start c1scx 
• ssh to framebuilder, and restarted the framebuilder by entering telnet fb 8088   and then   shutdown.

 I just finished carrying out the same checks for the DAC at 1X9 (with channels 9 through 16 that are unused as of now) as those I had done for the DAC at 1Y4, as the hardware prep up till now was done with the characterisation of the DAC at 1Y4. Conclusions:

• The accessible range of output voltage are -10 V to +10V w.r.t ground --> No change needs to be made to the gain of the HV amplifier stage on the PZT Driver Board
• The pin-outs of the DAC Adaptor Board at 1X9 is identical to that at 1Y4 --> Custom ribbons do not need to be modified.
• The PSD of the DAC output has a peak at 64 kHz --> Notches on AI Board do not need to be moved again.

I will now proceed to install various pieces of hardware (AI Board, PZT driver board, HV Power Supply and cabling) at 1X9, while not making the connection to the PZTs till I receive the go ahead. 

I have made a new model for the endtable PZT servo, and have put it in c1iscex. Model name is c1asx. Yesterday, Koji helped me start the model up. The model seems to be running fine now (there were some problems initially, I will post a more detailed elog about this in a bit) but some channels, which are computer generated, don't seem to exist (they show up as white blocks on the MEDM GDS_TP screen). I am attaching a screenshot of the said screen and the names of the channels. More detailed elog about what was done in making the model to follow.

Channel Names:

C1:DAQ-DC0_C1ASX_STATUS (this is the channel name for the two leftmost white blocks)

C1:DAQ_DC0_C1ASX_CRC_CPS

C1:DAQ-DC0_C1ASX_CRC_SUM

 These are roughly the steps I followed in setting up the new model for the endtable PZT servo - C1ASX.

Simulink model:

I made a SIMULINK model of the servo, using MATLAB R2013a. The path to the model is /opt/rtcds/caltech/c1/userapps/release/isc/c1/models/c1asx.mdl. I am listing the parameters set on the CDS_PARAMETERS block:

• host = c1iscex
• site = c1
• rate = 16k
• dcuid = 44 (which I chose after making sure that this dcuid was not used on this list which was last updated end Feb 2013)
• specific_cpu = 5 (again chosen after checking the available CPUs in the above list).
• adc_Slave = 1
• shmem_daq = 1
• no_rfm_dma = 1
• biquad = 1

Making, Compiling and Installing the Model:

After saving the model, I ssh-ed into c1iscex and ran the following commands:

rtcds make c1asx - this gave me a whole bunch of errors initially, which I tracked down to a naming problem in some of the from and goto flags: there should not be any spaces.

rtcds install c1asx 

rtcds start c1asx - this gave me an error which said something like 'can't start/stop model.' Koji pointed out that given that a new model is being started, there is an additional step involved, which is to add the model name to the rtsystab file (this is located at /diskless/root/etc/rtsystab on framebuilder, and is mirrored in the various computers. It would be advisable to make sure that the changes are mirrored in the corresponding file on the computer in which the new model is being installed). 

After adding the model name to the rtsystab file, I tried running rtcds start c1asx again. This time, no errors were output, but the model was not up and running as verified by looking at the C1:ASX_GDS_TP medm screen.

 Debugging 

Koji suggested making a simple model (1 CDS parameters block, 1 ADC block and 2 filter modules, appropriately terminated) and see if that starts up, which it did. I then tried adding my servo minus the DAC block and recompiled and restarted the model. This too worked fine. I figured that the next logical step would be to add the DAC block to the model, and restart the model. But when I tried this, c1iscex crashed .

Jenne helped in restoring things to a working state (we reverted the c1asx model to just 2 filter modules, and went to the X-end and restarted the computer. This did not work the first time so I went back in and restarted it again, at which point we were able to ssh into c1iscex again and restart the four models running on it).

Since Manasa and Koji were working on getting things set up for the pumpdown,I did not try anything again till later in the evening, when Koji helped in debugging the problem further. In the meantime, at Jenne' suggestion, I made the model once again in MATLAB R2010b. In the evening, when I tried restarting the model, Koji suggested that the DAC channels in c1asx may be used by other models, at which point I realised I had set up excitation points on channels 8 through 15 of the DAC in c1scx (detailed here) in order to test the hardware at 1X9. I removed the excitation points from channels 8-11 of the DAC block in c1scx (these are the ones used in c1asx), and recompiled and restarted c1asx (using the above sequence of commands). I then tried recompiling and starting c1asx once more, and this time, it worked . At least, the GDS_TP screen suggests that the model is running alright, except for the fact that some computer generated channels seem to be missing. This problem is unresolved for now, and probably has something to do with the fact that C1ASX channels do not appear in Dataviewer.

I do not think this has to do with restarting framebuilder (I did the usual telnel fb 8088 followed by shutdown). In any case, I have added the new model to the CDS_FE_STATUS screen, and will continue to debug the same. I have also got a template medm screen (work in progress) which I will elog about soon as I get it done.

Note to self: There are 4 more excitation channels still hooked up to the DAC (channels 12-15) in the c1scx model. I plan to remove these and put them in c1asx.

I don't know what's going on here (why the channels are white), and I don't yet have a suggestion of where to look to fix it but...

Is there a reason that you're making a new model for this?  You could just use and existing model at c1iscex, like the c1scx, and put your stuff in a top-names block.  Then you wouldn't have to worry about all of the issues with adding and integrating a new model.

Koji just fixed this.

It seems that the new model's channels were not automatically added to the master file in the framebuilder (/opt/rtcds/caltech/c1/target/master). Adding the following two lines to the master file fixed the problem;

/opt/rtcds/caltech/c1/chans/daq/C1ASX.ini

/opt/rtcds/caltech/c1/target/gds/param/tpchn_c1asx.par

The box is now green. It looks like C1ASX.ini is created automatically in /opt/rtcds/caltech/c1/chans but the master file needs to be manually edited. The channels are now showing up on dataviewer etc. I have updated the information on the wiki page.

I have made some minor changes to the model, made all the MEDM screens, and linked monitors on these to the appropriate channels. I have borrowed heavily from the C1ASS MEDM screens (particularly for the small filter modules-it was convenient to just copy and paste an existing module, and edit the channel names using EMACS/GEDIT), and have edited these to suit the needs of this servo. Some features:

• The feedback signal (only the output of the servo to the PZTs, plus any contribution from the on-screen sliders, and not including the LO output) is monitored with both a slow (using CDS_EPICS_OUTPUT block from the CDS_PARTS library) and fast channel (using Test Point from the same library). The idea is that it would be useful to know the output to the PZTs such that if coarse adjustment ever needs to be done at the endtable, the PZTs can be restored to the middle of its operating range by means of the sliders.
• Sliders are incorporated into the master screen for adjusting the output to the PZTs. There are text-input fields below the sliders as well, which control the same channel.
• I have removed the 4 remaining excitation points to the DAC set up in C1SCX, and have relocated them to channels 12-15 of the DAC in C1ASX.

I think I am now ready to take some measurements and try and optimize this servo. There is no green transmission at the PSL table at the moment, so not much can be done, though the first step would be to take the power spectrum of the error signal, which would help me decide the appropriate frequencies for the LOs. I would then have to add the appropriate filters to the model. The last, and most difficult step, would be the measurement of the output matrix, though Koji has given me some ideas about how this measurement can be done. I also have a template script ready, though I will only finalise this after optimising the servo and running it a couple of times manually.

Attached are screenshots of the MEDM screens.

The following slow channels have been added and are now being recorded by FB.

C1:ALS-X_OVEN_TEMP

C1:ALS-Y_OVEN_TEMP

C1:ALS-BEATX_FREQ

C1:ALS-BEATY_FREQ

Details:

In order to integrate the data collected by the Raspberry-Pi from the Y-end doubling oven temperature controller and also the data from the frequency counter which will be hooked up to monitor the beat frequency, Koji helped me set up some slow EPICS record channels (in ALS as we felt this was most appropriate). The procedure for setting up slow channels was as follows (virtually identical to what is detailed in this elog:

1. Add the channel names to the file C0EDCU.ini (path = /opt/rtcds/caltech/c1/chans/daq/C0EDCU.ini).
2. Make a database (.db) file so that these channels are actually recorded (path = /cvs/cds/caltech/target/c1aux/als.db).
3. Restart framebuilder. 
4. Verify that the channels indeed exist and can be read and written to using ezcaread and ezcawrite.

I will now integrate these channels into my scripts, and make some simple MEDM screens.

In order to decide what frequencies to dither the 4 degrees of freedom (M1-pitch&yaw, M2-pitch&yaw) at, I took the power spectrum of the X and Y-arm green transmission (C1:ALS-TRX_OUT, C1:ALS-TRY_OUT). Plots showing the power spectra are attached. Looking at the power spectra, I would think that for the X-arm, it would be okay to dither at 40, 50, 60 and 70 Hz. In order to check if the piezos could respond to these frequencies, I used my QPD setup and shook the PZTs with a 100Hz, 1Vpp sinusoid, and saw that the spot moved smoothly on the QPD.

 As for choosing the modulation amplitude, I did a simplistic approximation assuming that the misalignment only rotates the beam axis relative to the cavity axis, and determined what angle coupled 10% of the power into the next eigenmode. Assuming that this is small enough such that if we are already locked to TEM00, the dither won't kick it up to some higher-order mode, the LO amplitude should be in the range of 30-60 digital counts (determined using the PZT calibration constants determined here. This corresponds to a sine-wave of ~50mV amplitude reaching the PZTs (after HV amplification). I am not sure if this is too small, but according to the PZT datasheet, these platforms are supposed to have a resolution of 0.02 urad, which would correspond to the input signal changing by ~0.1 mV, so this signal should be capable of dithering the tip-tilt. 

 I have already added band-pass filters centered at these frequencies to the model (with a passband of 5Hz, 2Hz on either side), and low-pass filters to pull out the DC component of the output of the lock-in amplifiers. It remains to tune the gains of the filter stages. These parameters (frequency, amplitude of the LOs) may also have to be changed after tests). Hopefully the PZTs can be plugged in tomorrow, and I can try and make a measurement of the output matrix. 

Koji also suggested that it may be good to have a path in the model that feeds back to the PZTs by dithering the cavity mirrors as opposed to the PZT mounted mirrors. I will work on incorporating this into the SIMULINK model (c1asx.mdl) and also into the master medm screen.

Notes:

1. The spot size of the X-arm green transmission on the PD was larger than the active surface. I moved the GTRX PD a little back and put in a lens (KPX085, 62.9mm FL, AR.14) in front of the PD, such that the spot is now occupying about 1/4th of the active surface area. The lens was mounted in a Thorlabs LMR1mount, and has been labelled.
2. I made a slight change to the SIMULINK model, so as to calibrate the PZT sliders to (approximately) volts (I added a multiplier block that multiplies the slider value by constant value 3267.8). The idea is that we can approximately relate the slider value to tilt, knowing the calibration constant in mrad/V for the PZTs.

Power Spectra of Arm Green Transmission:

- I suppose the green transmission paths were thoroughly inspected and aligned in prior to the measurement

- Of course it is a BAD idea to use 60Hz as the LO frequency.

- Power spectra should be plotted as "RIN (relative intensity noise)" as the DC of 1 and 100 gives you 100 times different power spectra for the same beam.
  Don't forget to subtract the offset from your DC values.

I redid the power spectrum measurement for the X-arm green transmission after aligning the arm to green using the ITMX/ETMX Pitch and Yaw sliders on IFOalign.

The Y-axis now reflects the relative intensity noise (RIN), which I obtained by taking the average value of the X-arm green transmission using tdsavg. Based on this measurement, I have now picked four new frequencies at which to try and modulate the PZT mirrors: 10, 19, 34 and 39 Hz. Bandpass filters in the LIA stage have been appropriately modified. 

Power Spectrum:

 I have done some preliminary testing of the X-End Green ASS Servo. I will write a more detailed elog about this soon, but I thought I'd note down the important stuff here.

Yesterday, while we were venting, I aligned the X-arm to the green using the sliders on IFOalign, maximizing the transmission. Then I retook a power spectrum so as to determine the LO frequencies. Jenne pointed out that LO frequencies should not be integers (it usually suffices to append a .098725 or something to the frequency) so I made the necessary changes.

I did a first run of the servo yesterday, and more runs today. Notable points:

1. I was able to lock to 00 from a 08 or 09 mode using the PZT sliders
2. The green transmission having locked to 00 was ~0.2. I then ran the servo and got it up to ~0.4 and then 0.6 (see time series plot attached). The servo was able to recover this level of transmission after misaligning the steering mirrors using the PZT sliders.
3. This was not the optimal transmission level as when Koji moved ETMX a little, the transmission improved.
4. The actuators are degenerate. Most of the time, only two of the four servos are doing anything significant. This is probably because of the fact that the two steering mirrors are so close to each other, that moving one or the other produces virtually the same effect. I do however have some cool videos of mode-hopping :)
5. The range of actuation of the PZTs is probably not enough to maximize the green transmission from an arbitrary state because of point 4 (i.e. we need to move one mirror in some direction a lot, and move the other a lot to compensate for the change, and the overall gain in input pointing/alignment is marginal). It may be that things will be slightly better at the Y-end. It would also be interesting to see if there is any improvement in the servo performance by dithering the cavity mirrors as opposed to the PZT mirrors.
6. To this end, I tried modifying the c1asx model to incorporate an option to dither the cavity mirrors. The plan was to make a second set of LOs in the model that output to ITMX and ETMX suspensions. However, for some reason, when I recompiled the model and restarted it, c1iscex crashed. Parity has now been restored. Note that in order to accommodate the new LOs, I had to make changes to C1SUS, C1RFM and C1SCX as well. I have since removed all my additions, saved, built and installed these models, but have not restarted them (with the exception of C1SCX which restarted when I manually restarted c1iscex). 
7. The plan tomorrow is to try incorporating cavity dither into the model again. This time, I'll try grabbing the LO-related signals from c1ass directly, as I am not clear why my approach did not work.

More details to follow.

Over the last three days, I've had the interferometer to test and optimize the ASX Servo. Based on what I have seen, I think the conclusion is that with the current parameters, the servo does its job provided the input pointing set up at the endtable with the coarse adjustment knobs is reasonably good. Once the cavity is aligned and IR transmission maximized using ASS, I have been able to get the green transmission up to 0.8 which is close to the best we had pre-vent. I have not been elogging regularly over the last few days, so this one is going to be a longish one.

Major changes made:

1. The SIMULINK model has been modified to accommodate an option to dither the cavity mirrors and not the PZT mirrors. Details are as follows:
   • I have sent the LO signals (CLK,SIN and COS) from the ASS model to the ASX model via the RFM model. Appropriate changes were made to all these three models, and recompiling and restarting the models was done without issue. The SIN and COS signals are used to demodulate green transmission at the dither frequencies. ***The CLK signal is not required to be sent between models as it is not being used by ASX (I turn the dither ON using the channels already set up for ASS). I realised this a little late, and at present the ASS and RFM models are compiled such that the CLK signal is also sent from ASS to RFM. This can be removed, thus freeing up 4 unnecessary inter-process communication channels. Also, I am not too sure if this is relevant, but the maximum computation time of both the RFM and ASX models seem to have gone up after I added these inter-process communication links.***
   • The rest of this part of the servo is a replica of the part where PZT mirrors are dithered. At present the servo output is the sum of its two branches (PZT mirror dither branch and cavity mirror dither branch) which works fine under the assumption that at any one time, only one arm will run. Ideally, the summing block should be replaced by a switch. However, when I tried (in an earlier attempt to include the cavity dither) to do this and restart the model, c1iscex crashed, and so I decided against using the switch block for this trial.
   • The control signal generated using green transmission demodulated at the ETM dither frequencies are used to actuate on M1 while the ITM ones are used to actuate on M2. Of course, by setting the appropriate off-diagonal elements in the output matrix, this can be modified as desired.
2. The main MEDM screen has been updated to reflect the new additions to the SIMULINK model. Screenshot is attached. The picture isn't entirely accurate as the monitor channels in the upper row actually show the servo output + slider output. This needs to be changed in the model, and a new set of monitors need to be added to the MEDM screen. In the end, we require four sets of monitor-points in the model: PZT dither servo output, cavity dither servo output, sum of these with any offset from the PZT sliders, and the sum of the latter with the dither signal (this is what eventually goes to the PZT mirrors while the dither is on).
3. I added scripts to the MEDM screen that turn the PZT mirror dither servo on and off. Note that when you want to run a new script on an MEDM screen using medmrun, you need to change the permissions of the file by going to the path where your script is located and running chmod 755 <name of script>. Manasa has updated the same on the wiki.

 Details of tests runs:

For the most part, I have been trying to optimize the PZT mirror dither servo. To this end, I did the following:

• Went to the X-end and fixed the input pointing which was not optimal. Manasa first aligned the arm and ran ASS to maximize the IR transmission. I then used the coarse adjustment knobs on the mirror mounts to get the green transmission up to ~0.6.
• I then set the following parameters in the servo (these are all in the script, path to which is /opt/rtcds/caltech/c1/scripts/ASX):
  1. LO frequencies of 10, 19, 34 and 39 Hz respectively for M1 PIT, M1 YAW, M2 PIT and M2 YAW.
  2. LO amplitudes of 75 for all the four degrees of freedom (determined by using PZT calibration to see what amplitude would couple 10% of power into the first higher-order-mode assuming a perfectly aligned beam to start with.
  3. LIA BP filters centered at the above frequencies with 2Hz passband on either side.
  4. LIA LP filters with corner frequency 0.5 Hz.
  5. LIA Signal filter bank gain set to 100 for all degrees of freedom.
  6. LIA Demod I phase filter bank gain set to 5 for all degrees of freedom.
  7. Control filter gains to 1 for all degrees of freedom (control filters are all integrators).
  8. Demod phase set to 0 for all degrees of freedom. I did not really try to optimize this but the servo seems to be doing reasonably well even with this setting.
  9. Overall servo gain to 1 (the servo worked well when I increased this to 5 as well, but became unstable when I increased it further).
• I ran the servo. Observations were as follows:
  
  • Having fixed the input pointing to get green transmission up to ~0.6, the servo was able to improve it to ~0.8, which is the best we had after hours spent at the X-end prior to the vent.
  • Given a good input pointing, we can use the PZT mirrors to lock to 00 mode from some misaligned state using either the sliders, or by leaving the servo on, and helping it out at the points where it gets 'stuck' in some higher mode using either the sliders or by toggling the shutter.
  • In order to recover green transmission of ~0.8, it was often necessary to first run ASS to optimize the IR transmission. Otherwise, green-transmission saturates at ~0.6 or 0.4 depending on the misalignment of the arm cavity mirrors. The servo was unable to change the input pointing enough to deal with overly misaligned cavity mirrors. 
  • The servo is sometimes capable of bringing about mode-hopping from a higher order mode to a lower one, though this is not always the case as the PDH lock is sometimes too strong, in which case I toggled the shutter after which the servo kicked in.
  • I tested the servo under as many different conditions as I could. For instance, having left the green shutter open overnight, I saw that the transmission had fallen from 0.8 (which was what we saw on Thursday night) to ~0.4 on Friday morning. Running the servo got the transmission up to 0.6. I then asked Manasa to run ASS, (while leaving the ASX servo on), after which point the green transmission went up to 0.8. Sometimes, the servo locks to a 'bad' 00 mdoe, where the transmission saturates at ~0.2, but toggling the shutter fixes this most of the time.

Attempt to measure transfer function:

One of the things that came up during my presentation was how fast the loop was capable of responding. I was able to get a quantitative idea of this by playing around with the overall servo gain. Initially, it took ~30 seconds for the servo to get the transmission up to its peak value, with a servo gain of 1. When I ramped this up to 5, the response was much faster, with the peak transmission being reached in ~5seconds. 

I wanted to get a more quantitative picture, and hence tried to measure the transfer function by first injecting an excitation into the 'SIG' filter-bank in the demodulation stage. However, coherence between the IN1 and IN2 signals was very poor for all the amplitude configurations I tried. At Jenne's suggestion, I tried injecting the excitation at the control-filters stage, but found no improvement. Perhaps the amplitude envelope was wrong or the measurement technique has to be rethought. 

 Misc remarks:

1. M1 is the first steering mirror and M2 is the second one (right before the beam enters the arm cavity).
2. Though I have added the cavity dither feature to the model, I was not able to optimize this servo. Some calculations need to be done to get an estimate of the output matrix, after which the filter gains etc can be optimized.
3. Today, I cleaned up my temporary setup at the SP table to calibrate the PZTs. Most of the hardware for the Y-end is now in the tupperware box. The QPD and laser have been restored to the optical bench next to MC2 where I found them. The second KEPCO HV supply which I had set up has now been installed at 1Y4 in anticipation of the PZT mirrors at the Y-endtables. It is currently powered OFF.
4. Performance plots to follow as I have not pulled the data out yet.
5. I had bought a cake from chandler today in an effort to clear my meal plan, but in the rush in the afternoon, completely forgot about it. It is in the fridge, and is strawberry tart, hope it tastes good.

 New MEDM screen:

 Getting rid of the LO transmission will certainly help / be good.  After adding these channels, the RFM model is regularly hitting 62usec (out of a max acceptable of 60).

I'm not really sure why the ASS was involved in this.  I feel like it might have been simpler to just do everything in the ASX model, to keep things cleaner.  Also, the IPC blocks for this stuff (in both ASS and ASX) are not on the top level of the model.  I had thought that this was expressly forbidden (although I'm not sure why).  I'm emailing Jamie, to see if he remembers what, if anything, is breakable if the IPC blocks are down a level.

I'm not sure if it's forbidden by the RCG, but you should definitely NOT do it.  All IO, whether it be between ADC/DACs or IPCs, should always be at the model top level.  That's what keeps things portable, and makes it easier to keep track of where are signals are going/coming from.

I started to modify the c1asx model to reduce the RFM model from hitting its max time.
Instead of bringing in ASS, I have modified ASX to do everything and only the clock signals to ITMX pitch and yaw are now going through RFM. RFM is still hitting 62usec and I suppose that is because of the problems with c1iscex.

c1iscex not happy

Cause and symptoms
While restarting the models, c1iscex crashed a couple of times because of some errors and had to be powercycled. The models were modified and they seem to start ok.
But it looks like there is something wrong with c1iscex since the models were started. The GPS time is off and C1:DAQ-DC0_C1X01_CRC_SUM keeps building up even for c1x01 which was left untouched.

Trial treatments
1. Since c1x01 ans c1spx were not touched,c1scx and c1asx were killed and we tried to start the other models. This did not help.
2. Koji did a manual daqd restart which did not help either.

We are leaving c1iscex as is for the time being and calling Jamie for help.

P.S. While making the models, I had created IPCx_PCIE blocks in c1iscex which do not exist. I changed them to RFM and SHMEM blocks. This did not allow me to compile the model and was only spitting errors of IPCx mismatch. After some struggle and elog search I figured out from an old elog that eventhough the IPCx blocks are changed in the model, the old junk exists in the ipc file in chans directory. I deleted all junk channels related to the ASX model. The model compiled right away.

 Sorrensen ps ouput of +15V at rack 1X9 was current limited to 10.3V @ 2A

Increased threshold to 2.1A  and the voltage is up to 14.7V