I had aligned the North Cavity before the weekend and was about to align south one today when I saw that the modematching on the North Cavity has fallen to 20%. This is a tell-tale sign of vacuum can temperature changing too much. When I checked, indeed the temperature sensors were railing to their lower most value of 26.599 Celcius. Same for both in-loop and out of loop sensors. While the table top temperature sensor was giving a meaningful value.
Investigation begins
I first checked point by point the temperature sensor board LIGO-D1800304. From ADC all the way back to the AD590s.The two sensors were indeed giving a voltage of 4.8 V through a transimpeadance of 16.25k which meant 295.385 uA of current corresponding to 22.23 Celcius. So indeed the sensors were telling me that the can had cooled down to almost room temperature over the past 2 days.
Framebuilder logs
I checked the framebuilder logs to figure out what has happened. The temperature was being stabalized at 34.38 Celcius by the PID script. At around 15;20:25 Aug 16, 2019 PDT, the temperature starts decaying. Infact, I should use this data in future to calculate current time constant of temperature decay of the can through the insulation. The table temperature was around 19 Celcius all this time. I found out that FB4 is not recording the channels that I have added later. I need to look into this as well.
Attached is the decay plot of the temperature
Possible points of problem
After a deep search through elog, CTN:2043 and CTN:2045 are the most relevant latest post. Kira and Kevin worked on this heater drier circuit and I'm doubting that something blew up in this circuit CTN:1903. I guess this would be my first point of attack.
Another possibility is that the heater load got disconnected. Well, I just checked the resistances and I get values of 38, 70, 70 and 38 Ohms. These are the right values according to CTN:1750. This is not the issue.
Lastly, the worst possible issue would be that the temperature sensors are reporting wrong temperature. But looking at the the properexponential decay of temperature reported by them, I would first assume that they are working fine.
Code and Data |