This issue is very weird and still unresolved. Without WFS loops, we'll have to realign IMC often and we might loose IMC alignment completely during weekends or long weekends.
I tried following things today but nothign worked:
- Blocked WFS PDs and reset DC offsets (sitemap>C1IOO>C1IOO_WFS_MASTER>! Actions>Correct WFS DC Offsets).
- Switched off MC chamber lights.
I felt that they might be on, but later I feel that wasn't the case. Anyways, this didn't help.
- Algined IMC manually using cavAlign tool with MC2-MC3 and then tweaking MC1 and MC3 a little bit. Reach 13.6k in C1:IOO-MC_TRANS_SUM. Then I unlocked IMC with autolocker off, centered beam on WFSs (they were pretty off even though we have been centering them this week), and then reset RF offsets (sitemap>C1IOO>C1IOO_WFS_MASTER>! Actions>Correct WFS DC Offsets). This did not help either.
- The fact that IMC started misbehaving since Thursday onwards was bugging me that maybe the FE models did not come online properly, that maybe some RTS link is broken in IOO model which is causing the feedback loop to not work. So I went ahead and restarted all models, that didn't help either.
- Now we have a restartAllModels.sh script which restarts all cds system and restores state to just before restarting. It also makes sure that watchdogs are engaged safely particularly for new suspesnions where alignment offsets are ramped.
We need to investigate this as first priority. Maybe some cable is loose, some PD power supply not working etc. Until we fix this, people should align IMC to > 12000 transmission counts whenever they have a spare 5 min. We need to work in place of WFS for sometime.