[Ian, Raj, Tega]

Here is the comparison between the results of Raj's python model and the transfer function measurement done on the plant model by Tega and me.

As You can see in the graphs there are a few small spots of disagreement but it doesn't look too serious. Next we will measure the signals flowing through the entire plant and controller.

For a nicer (and printable) version of these plots look in the zipped folder under Plots/Plant_TF_Individuals.pdf

[Tega, Ian]

TODO

1. Investigate cross-coupling btw the various degrees of freedom (dof) - turn on noise for each dof in the plant model and measure the transfer function of the other dofs.

2. Get a closed-loop transfer function using noise injection and give a detailed outline of the procedure in elog - IN1/IN2 for each TM_RESP filter while the others are turned off.

3. Derive analytic model of the closed-loop transfer functions for comparison.

4. Adapt control filters to fit optimized analytical solutions.

[Ian, Tega]

Connected the New SUS screens to the controller for the simplant model. Because of hard-coded links in the medm screen links, it was necessary to create the following path in the c1sim computer, where the new medm screen files are located:

We noticed a few problems:

1. Some of the medm files still had C1 hard coded, so we need to replace them with $IFO instead, in order for the custom damping filter screen to be useful.

2. The "Load coefficient" button was initially blank on the new sus screen, but we were able to figure out that the problem came from setting the top-level DCU_ID to 63.

[TODO]

Get the data showing the controller damping the pendulum. This will involve tweaking some gains and such to fine-tune the settings in the controller medm screen. Then we will be able to post some data of the working controller.

[Useful aside]

We should have a single place with all the instructions that are currently spread over multiple elogs so that we can better navigate the simplant computer.

[Ian, Paco, Tega]

Last night we set up the four main matrices that handle the conversion between the degrees of freedom bases and the sensor bases. We also wrote a bash script to automatically set up the system. The script sets the four change of bases matrices and activates the filters that control the plant. this script should fully set up the plant to its most basic form. The script also turns off all of the built-in noise generators. 

After this, we tried damping the optic. The easiest part of the system to damp is the side or y motion of the optic because it is separate from the other degrees of freedom in both of the bases. We were able to damp that easily. in attachment 1 you can see that the last graph in the ndscope screen the side motion of the optic is damped. Today we decided to revisit the problem.

Anyways, looking at the problem with fresh eyes today, I noticed the in pit2pit coupling has the largest swing of all the plant filters and thought this might be the reason why the inputs (UL,UR,LR,LL) to the controller was hitting the rails for pit DoF. I reduce the gain of the pit2pit filter then slowly increased it back to one. I also reduced the gain in the OSEM input filter from 1 to 1/100. The attached image (Attachment2) is the output from this trial. This did not solve the problem. The output when all OSEM input filter gain set to one is shown in Attachment2.

We will try to continue to tweak the coefficients. We are probably going to ask Anchal and Paco to sit down with us and really hone in on the right coefficients. They have more experience and should be able to really get the right values.

Finally got the SIMPLANT damping to work following Rana's suggestion to try damping one DoF at a time, woo-hoo!

At first, things didn't look good even when we only focus on the POS DoF. I then noticed that the input value (X1:SUS-OPT_PLANT_TM_RESP_1_1_IN1) to the plant was always zero. This was odd bcos it meant the control signal was not making its way to the plant. So I decided to look at the sensor data

(X1:SUS-OPT_PLANT_COIL_IN_UL_OUTPUT, X1:SUS-OPT_PLANT_COIL_IN_UR_OUTPUT, X1:SUS-OPT_PLANT_COIL_IN_LR_OUTPUT, X1:SUS-OPT_PLANT_COIL_IN_LL_OUTPUT)

that adds up via the C2DOF matrix to give the POS DoF and I noticed that these interior nodes can take on large values but always sum up to zero because the pair (UL, LL) was always the negative of (UR,LR). These things should have the same sign, at least in our case where only the POS DoF is excited, so I tracked the issue back to the alternating (-,+,-,+,-) convention for the gains

(X1:SUS-OPT_CTRL_ULCOIL_GAIN, X1:SUS-OPT_CTRL_URCOIL_GAIN, X1:SUS-OPT_CTRL_LRCOIL_GAIN, X1:SUS-OPT_CTRL_LLCOIL_GAIN, X1:SUS-OPT_CTRL_SDCOIL_GAIN)

of the Coil Output filters used in the real system, which we adopted in the hopes that all was well. Anyways, I changed them all back to +1. This also means that we need to change the sign of the gain for the SIDE filter, which I have done also (and check that it damps OK). I decided to reduce the magnitude of the SIDE damping from 1 to 0.1 so that we can see the residuals since the value of -1 quickly sends the error to zero.  I also increased the gain magnitude for the other DoF to 4.

When looking at the plot remember that the values actually represent counts with a scaling of 2^15 (or 32768) from the ADC. I switched back to the original filters on FM1 (e.g. pit_pit ) without damping coefficients present in the FM2 filter (e.g. pit_pit_damp).

FYI, Rana used the ETMY suspension MEDM screen to illustrate the working of the single suspension to me and changed maybe POS and PITCH gains while doing so.

Also, the Medify purifier 'replace filter' indicator issue occurred bcos the moonlight button should have been pressed for 3 seconds to reset the 'replace filter' indicator after filter replacement.

Restarted the C1sim machine at about 12:30 to help diagnose a network problem. Everything is back up and running

Trying to vectfit to the data taken from the DFD previously but failing horribly. I will update this post as soon as I get anything semi-decent. For now here is this fit.

I think the fit fails as the measurement quality is not good enough.

We had several problems with our NDS2 server configuration. It runs on megatron, but I think it may have had issues since perhaps not everyone was aware of it running there.

1. channel lists were supposed to updated regularly, but the nds2_nightly script did not exist in the specified directory. I have moved it from Joe Areeda's personal directory (/home/nds2mgr/joework/server/src/utils/) to nds2mgr/channel-tracker/.
2. The channel history files (/home/nds2mgr/channel-tracker/channel_history/) are stored on the local megatron disk. These files had grown up to ~50 GB over tha past several years. I backed these up to /users/rana/, and then wiped them out so that the NDS could regen them. Now that the megatron local disk is not full, it seems to work in giving raw data.
3. Need to confirm that this serves up trend data (second and minute)
4. I think there is a nds2-server package for Debian, so we should update megatrons OS to the preferred flavour of DebIan and use that. Who to get to help in this install?

Since Megatron is currently running the "Shanghai" Quad-core Opteron processor from ~2009,  its ~time to replace it with a more up to date thing. I'll check with Neo to see if he has any old LDAS leftovers that are better.

I was trying to debug why the NPRO PZT is all over the place, and it turns out that the new FSS SLOW script is not actually running.

The BLINKY is blinking, but the script is not running. I wasn't able to figure out how to kill the broken Docker thing, but if the code reports that its running but actually does not, we should probably just put back the old perl or python script that ran before. I don't know how to debug this current issue, but the IMC locks will be limited in length due to this servo being broken. Whoever knows about this, please stop that Docker PID and we can just run the old python script on megatron.

I also tried to post a trend plot, but the minute trends don't yet reach the current date (!!!). They seem to have stopped recording a few days ago, so I guess the Framebuilder still needs some help or its tough to figure out things like when exactly the new SLOW servo stopped working.

The problem with trends was due to the epics data collection process (standalone_edc) that runs on c1sus. When all the FEs were rebooted earlier this week, this process was started automatically, but for some reason it hasn’t been doing its thing and sending epics data to the framebuilder. I restarted it just now, and it’s working again. Until this problem is sorted out, we need to remember to check on this process after rebooting c1sus.

I stopped the Docker PID script and started the old python script on megatron. Instructions on how to do this are here.

On optimus I ran:

On megatron I ran:

However, the daemon service keeps failing and restarting. So currently the FSSSlow is not running. I do not know how to debug this script.

On a side note, I tested the docker service by restarting it, and it was working. From the logs, it seems like it got stuck because it could not find C1:IOO-MC_LOCK channel which occurs when c1psl epics servers fail or get stuck. The blinker on this script runs when the script is running but it does not stop if the script gets stuck somewhere. If someone decides to use this script in the future, they would need to correct error catching so that no reply from caget looks like an error and the script restarts rather than keep trying to get the channel value. Or the blinker implementation should change in the script so that it displays a stuck state.

I've moved the FSS Slow PID script running to megatron through systemd daemons. The script is working as expected right now. I've updated megatron motd and the always running scripts page here.

I have pushed changes made to the toggleWFSoffsets.py script to the git.

This file may be found in: "/opt/rtcds/caltech/c1/Git/40m/scripts/MC/WFS/"

The changes made are:

Updated the script to allow for toggling step responses on either optics or sensors (default = optics), chosen by user

The script orignally asked user to make any last changes to the offsets before hitting enter to run without displaying the new changes.

Now the script checks for changes made by the user to the offsets and displays them if detected. If no changes are made, the code starts running the steps.

If the summary pages go down, it could be from a break in the script or some small error. The first remedy for fixing this can be to remove the cron jobs in the que and restart the "gw_daily_summary.sub" and "gw_daily_summary_rerun.sub" scripts. 

For more information on how to do this, follow instructions found in the wiki.

# On Linux, "ezcastep" will interpret negative steps as command line arguments,
# because the GNU library interprets anything starting with a dash as a flag.
# There are two ways around this.  One is to set the environment variable
# POSIXLY_CORRECT and the other is to inject "--" as a command line argument
# before any dashed arguments you don't want interpreted as a flag.  The former
# is easiest to use here:

if (`uname` =~ m/Linux/) {
    # Add an environment variable for child processes
    $ENV{'POSIXLY_CORRECT'} = 1;
}

pushd /cvs/cds/caltech/users/tf/build/labca_2_1/bin/linux-x86
setenv PATH ${PATH}:`pwd`
cd /cvs/cds/caltech/users/tf/build/labca_2_1/lib/linux-x86
setenv LD_LIBRARY_PATH ${LD_LIBRARY_PATH}:`pwd`
popd

addpath /cvs/cds/caltech/users/tf/build/labca_2_1/bin/linux-x86/labca
help labca
foo = lcaGet('C1:PSL-FSS_RCTRANSPD')