I made a model for our seismometer can using actual data so that we know approximately what the time constant should be when we test it out. I used the appendix in Megan Kelley's report to make a relation for the temperature in terms of time.
In our case, we will heat the can to a certain temerature and wait for it to cool on its own so
We know that where k is the k-factor of the insulation we are using, A is the area of the surface through which heat is flowing, is the change in temperature, d is the thickness of the insulation.
We can take the derivative of this to get
We can guess the solution to be
where tau is the time constant, which we would like to find.
The boundary conditions are and . I assumed we would heat up the can to 40 celcius while the room temp is about 24. Plugging this into our equations,
We can plug everything back into the derivative T'(t)
Equating the exponential terms on both sides, we can solve for tau
Plugging in the values that we have, m = 12.2 kg, c = 500 J/kg*k (stainless steel), d = 0.1 m, k = 0.26 W/(m^2*K), A = 2 m^2, we get that the time constant is 0.326hr. I have attached the plot that I made using these values. I would expect to see something similar to this when I actually do the test.
To set up the experiment, I removed the can (with Steve's help) and will place a few heating pads on the outside and wrap the whole thing in a few layers of insulation to make the total thickness 0.1m. Then, we will attach the heaters to a DC source and heat the can up to 40 celcius. We will wait for it to cool on its own and monitor the temperature to create a plot and find the experimental time constant. Later, we can use the heatng circuit we used for the PSL lab and modify the parts as needed to drive a few amps through the circuit. I calculated that we'd need about 6A to get the can to 50 celcius using the setup we used previously, but we could drive a smaller current by using a higher heater resistance.
I performed a test with the can last week with one layer of insulation to see how well it worked. First, I soldered two heaters together in series so that the total resistance was 48.6 ohms. I placed the heaters on the sides of the can and secured them. Then I wrapped the sides and top of the can in insulation and sealed the edges with tape, only leavng the handles open. I didn't insulate the bottom. I connected the two ends of the heater directly into the DC source and drove the current as high as possible (around 0.6A). I let the can heat up to a final value of 37.5C, turned off the current and manually measured the temperature, recoding the time every half degree. I then plotted the results, along with a fit. The intersection of the red line with the data marks the time constant and the temperature at which we get the time constant. This came out to be about 1.6 hours, much longer than expected considering that onle one layer instead of four was used. With only one layer, we would expect the time constant to be about 13 min, while for 4 layers it should be 53 min (the area A is 0.74 m^2 and not 2 m^2).
Updated some values, most importantly, the k-factor. I had assumed that it was in the correct units already, but when converting it to 0.046 W/(m^2*K) from 0.26 BTU/(h*ft^2*F), I got the following plot. The time constant is still a bit larger than what we'd expect, but it's much better with these adjustments.
For our next steps, I will measure the time constant of the heater without any insulation and then decide how many layers of it we will need. I'll need to construct and calibrate a temperature sensor like the ones I've made before and use it to record the values more accurately.
For the insulation, I have decided to use this one (Buna-N/PVC Foam Insulation Sheets). We will need 3 of the 1 inch plain backing ones (9349K4) to wrap a few layers around it. I'll try two layers for now, since the insulation seems to be doing quite well according to initial testing.
I've ordered 4 of these from McMaster. Should be delivered to the 40m by noon tomorrow.
Kira and I also discussed the issiue. It would be good if someone can hunt aroun on the web and get some free samples of non-shedding foam with R~4.
Yesterday, while we were bringing the CDS system back online, we noticed that the control room wall StripTool traces for the seismic BLRMS signals did not come back to the levels we are used to seeing even after restarting the PEM model. There are no red lights on the CDS overview screen indicative of DAQ problems. Trending the DQ-ed seismometer signals (these are the calibrated (?) seismometer signals, not the BLRMS) over the last 30 days, it looks like
I poked around at the electronics rack (1X5/1X6) which houses the 1U interface box for these signals - on its front panel, there is a switch that has selectable positions "UVW" and "XYZ". It is currently set to the latter. I am assuming the former refers to velocities in the xyz directions, and the latter is displacement in these directions. Is this the nominal state? I didn't spend too much time debugging the signal further for now.
UVW refers to the 3 internal, orthogonal velocity sensors which are not aligned with the vertical or horizontal directions. XYZ refers to the linear combinations of UVW which correspond to north, east, and up.
Five mechcanical traps set inside of boxes. Red-white warning tape on top of each.
Last jump at rack Y2.
Gautam and I set up the insulated seismometer can in the lab today. I had previously wired up the two heaters I placed onto the sides of the can in parallel to get a total resistance of 12.5 ohms and then I wrapped the whole can in 3 layers of insulation (k-factor 0.25). We placed it on a large sheet of insulation as to not crush the wires leading out the bottom of the can. I stuck on one of my AD590 sensors to the inside of the can onto the copper lining using duct tape, though this is only a temporary solution. In the future, it would be nice to have some sort of thermal clamp to secure the sensor to the can. To provide power to the heater circuit board and the temperature sensor board, we got a powerstrip and plugged in two power supplies and a function generator into it. The heater circuit (attachment 3) is powered by one of the power supplies and the function generator, while the temperature sensor (attachment 5) is stuck to the side of the can and is powered by the second power supply. The heater circuit's MOSFET (IRF640, attachment 4) is placed on a metal block and sandwiched between two more to make sure it doesn't move around. The temperature sensor is connected by a long BNC cable to the channels in attachment 6.
gautam: we plugged the BNC output of Kira's temperature sensor circuit to J7 on the AA input chassis in 1X2 - this corresponds to ADC1 input 12 in c1ioo. I then made a "PEM" namespace block inside the c1als model, and placed a single CDS filter module inside it (this can be used for calibration purposes). The filter module is named "C1:PEM-SEIS_EX_TEMP", and has the usual CDSfilt channels available. I DQ'ed the output of the filter module (@256 Hz, probably too high, but I'm holding off on a recompile for now). Recompilation and model restart of c1als went smoothly.
2 bench power supplies are being used for this test, we can think of a more permanent solution later.
**25 Jan noon: Added another filter module, "C1:PEM-SEIS_EX_TEMP", to which Kira is hooking up a second temperature sensor, which will serve as a monitor of the "Ambient" lab temperature. Added DQ channel for the output of this filter module, fixed sampling to 32Hz. Compile and restart went smooth.
M4 local earthquake at 10:10 UTC There is no sign of damage.
....here is an other one.........M5.8 Ferndale, CA at 16:40 UTC
We started the actual heating test today and it seems to be working so far. Hoping to heat it to about 40C. We also set up another temperature sensor to measure the lab temperature and connected it to J7, bottom.
After almost 3 hours the temperature rose by about 3.5C. Seems a bit slow, but we can drive it more if necssary. The heating curve itself is exponentiial, which is a good sign.
The final temperature reached in about 4.5 hours is 30.5C, while the starting temperature is about 24C. I can't seem to screenshot the data for some reason.
Also, I will calibrate the lab temperature sensor to Celcius in the near future so that we would have a working sensor inside the lab.
After taking the measurements, calibrating them (approximately), and filterting them, I created the following plot. The exponential fit is quite good, as the error is not more than 0.03 C. I used the python function curve_fit in order to get this, and it gave me the time constant as well, which came out to 0.357 hr. From my previous calculations here, I plugged in the values we have (m = 12.2 kg, c = 500 J/kg*k, d = 0.0762 m, k = 0.26 W/(m^2*K), A = 1 m^2), and got that
This is a bit off, but it's probably due to the parameters not being exactly what I supposed them to be, and heat losses through the bottom of the can.
Attached the program I used to create the plot
from scipy.optimize import curve_fit
from scipy.signal import decimate
import cdsutils as cds
import numpy as np
import matplotlib.pyplot as plt
channel = ['C1:PEM-SEIS_EX_TEMP_OUT_DQ']
tstart = 1200962230
tend = 1201041084
I decided to plot the temperatures measured over two days for the sensor inside the can and inside the lab just to see if there was any significant difference between the two, and obtained the following plot. This shows that there is a difference in measurements of a few 0.01 C. The insulated seismometer can didn't change temperature as much as the lab did, which is as expected. I'll work on properly calibrating the sensors sometime in the future so that we can use the sensor that's just in the lab as an accurate thermometer.
I subtracted out the lab temperature change during the period of cooling to see if it would have a significant effect on the time constant, but when I fit the new data, the time constant came out to 0.355 hr, which is not a significant change from the value of 0.357 that I got earlier.
We did a survey of the lab today to figure out some of the logistics for the PID control test for the seismometer can. Kira will upload sketches/photos from our survey. Kira tells me we need
There are no DAC channels available in the c1ioo rack. In fact, there is a misleading SCSI cable labelled "c1ioo DAC0" that comes into the rack 1X3 - tracing it back to its other end, it goes into the c1ioo expansion chassis - but there are no DAC cards in there, and so this cable is not actually transporting any signals!
So I recommend moving the whole setup to the X end (which is the can's real home anyways). We plan to set it up without the seismometer inside for a start, to make sure we don't accidentally fry it. We have sufficient ADC and DAC channels available there (see Attachments #1 and #2, we also checked hardware), and also Sorensens to power the heater circuit / temperature sensing circuit. Do we want to hook up the Heater part of this setup to the Sorensens, which also power everything else in the rack? Or do we want to use the old RefCav heater power supply instead, to keep this high-current draw path isolated from the rest of our electronics?
I have attached the sketch of the whole system (attachment 3) with all the connections and inputs that we will need. Attachment 4 is the rack with the ADC and DAC channels labeled. Attachment 5 is the space where we could set up the can and have the wires go over the top and to the rack.
Some points before we can set up the can:
Also, I need to eventually remake the connections on my circuit board because they are all currently test points. I also need to find a box for the heater circuit and figure out what to do with the MOSFET and heat sink for it. This can either be done before setting everything up, or we can just change it later once we have the final setup for the can ready.
If all of this looks good then we can begin the setup.
We set up a new rail for the Sorensens (attachment 1) and placed one of them down on this new rail (attachment 2). Unfortunately the older rail that had been used to support the other Sorensens (the top one in attachment 1) is thick and does not allow another one of the Sorensens to slide in between the current ones. So we will have to support all the ones on top with a temporary support, take out the old rail, and then insert the new ones before letting the new bottom rail carry the weight of all of the Sorensens. We will do that tomorrow.
In addition, we have to figure out how to lead all the cables to the can, but there are no holders on the side of the lab to do so. So, we decided that we would have a new one installed on the side shown in attachment 3 so that we wouldn't have to place the wires along the floor.
Also, there has been some space made for the can along with the new insulation. The stuff mounted on the wall was removed and will be reattached tomorrow so that it doesn't get in the way of the can anymore.
We installed and labeled the Sorensens today.
I checked channels 6 and 7 on the ADC and they have long wires leading to BNC ends and are currently not being used, so we could probably just attach the temperature sensors to those channels.
Rewired the temperature sensor inputs to Molex connectors so that we can now attach them to the +/- 15V Sorensens for input instead of using a power supply.
We began the setup for the lab temperature sensor today. First, we needed to add in a DIN fuse for both temperature sensors, which required us to shut down everything else first. To avoid having to do that next time, we made three instead of two spaces where we have + and - 15V. Attachment 1 shows the new fuses we installed, along with the fuses they connect to. Attachment 2 shows the wiring that we used to connect all the fuses. Attachment 3 shows the labeled long wires that are attached to the lab temperature sensor. The other end is labeled as well. I measured the voltage at the other end of the long cables, and while the -15V one looks good, the +15V one shows only about 13.5V.
edit (Tuesday) - I set up the other set of cables that will eventually lead to the sensor in the can, but neither of them are showing any voltage on the other end. I'll work on this issue tomorrow.
gautam: some additional remarks about the procedure followed:
I switched out the DIN fuses for the long cables and it fixed the issue of them not showing any votage on the other end. At first, the +15V cable worked and the -15V didn't, but when I switched the fuse for the -15V it began working, but the +15V stopped working. I then switched out the fuse for +15V and both cables began showing voltage. But for both the long cables and the shorter ones, they show +13.4V instead of +15V. Not sure what's going on there.
Since we decided to use the Acromag for readback of the temperature sensor for Kira's seismometer temperature control, I enabled logging of the channel Johannes had reserved for this purpose last week. Kira has made the physical connection of a temperature sensor to the BNC input for this channel - it reads back -2.92 V right now, which is around what I remember it being when Kira was doing her benchtop tests. I edited C0EDCU.ini to enable logging of this channel at 16 Hz. Presumably, a study of the ADC noise of the Acromag at low frequencies has to be made to ensure appropriate whitening (if any) can be added. Channel name is C1:PEM-SEIS_EX_TEMP_MON. Similarly, there is C1:PEM_SEIS_EX_TEMP_CTRL which is meant to be the control channel for the servoing. Calibration of the temperature sensor readback into temperature units remains. It also remains to be verified if we can have these slow EPICS channels integrated with a fast control model, or if the PID temperature control will be purely custom-script based as we have for the FSS slow loop.
I removed the fast channels I had setup temporarily in c1als. Recompilation and restart of the model went smoothly.
I ceated a simple circuit that takes in 15V and outputs precisely 5V by using a 12V voltage regulator LM7812 and an AD586 that takes the output of the voltage regulator and outputs 5V (attachment 1). We plugged this into the slow channel and will leave it running for a few hours to see if we still have the fluctuations we observed earlier and also fit the noise curve. We'll also test the fast channel later as well. Attachment 2 shows the setup we have in the lab, with the red and white cable plugged into the +15V power supply and the red and black cable connected to the slow channel.
I have attached the setup I completed today. The metal box contains the heater circuit and the board for the temperature sensor is right above it. This is basically the same setup as before, but I've just packaged everything up neater. I expect to be able to perform the test tomorrow and begin implementing PID control. I still need a DAC input for the heater circuit and the temperature sensor is having some issues as well.
The MOSFET was getting pretty hot, so I switched it out to a larger heat sink and it's fine now. I then used a function generator in place of the DAC to provide ~3.5V. I got the current in the circuit to 1.7A, which is as expected, since we have 24V input, the heater resistance is 12.5ohm and the resistor we are using is 1ohm, so 24V/(12.5+1)ohm = 1.7A. The temperature inside the can rose about 5 degrees in half an hour. The only issue now is the voltage regulators and OP amp inside the box get hot, though it doesn't seem to be dangerous. I switched the function generator input to a DAC and Gautam set it to 1.5V. If it works, then we'll leave this on overnight and work on the PID control tomorrow. I've attached images of the current heater circuit box when it is open and the new heat sink for the MOSFET.
gautam: we also tried incorporating the EPICS channels from the Acromag into the RTCDS so that we can implement PID control by using Foton. I tried doing this using the "EpicsIn" and "EpicsOut" blocks from CDS_PARTS. While the model recompiled smoothly, I saw no signals in the filter module i had connected in series with the EpicsIn block. So I just reverted c1pem to its original state and recompiled the model. Guess we will stick to python script PID reading EPICS channels to implement the PID servo.
according to the temp sensor readout, which was ~-3.35V which corresponds to ~335K, the temperature of the can is now 60 deg C. This is a bit warm for my liking so i'm turning the heater current down to 0 now by writing 0 to C1:PEM-SEIS_EX_TEMP_CTRL
we don't ever want to use our 16 kHz real time system for such low frequency action; its main purpose is for real-time controls, whereas we are OK with multiple seconds of delay in a thermal loop. The Python PID script is sufficient and highly reliable (after years of testing).
I fit the data that we got from the test. The time constant for the cooling came out to be about 4.5 hours. The error is quite large and we should add a low pass filter to the temperature sensor eventually in order to minimize the noise of the measurements.
I made sketches of the final setup. There will be a box in the rack that contains both the heater circuit and the temperature sensor boards. One of them is in the loop while the other isn't. Instead of having many cables leading to the can, there will only be these three, though they can be made into a single wire. It will be connected to the can through a D-9 connector. The second attachment is what will be inside of the box, with all the major wires and components labeled.
Edit: I've canged the layout to (hopefully) make the labels easier to read. I've also added in a cable to the ADC that reads out the voltage across the 1 ohm resistor. I also attached the circuit diagrams for the heater circuit and the temperature sensors. The one for the heater circuit was made by Kevin and I used the same design, except I have LM7818 and LM7918, since the 15V ones were not available at the time I made the circuit.
In addition, all the wires leading to the can will all be part of one bundle of wires (I didn't clearly indicate it as such). There will be a total of 6 wires: two are needed for the wire to supply power to the heater and will have a LEMO connector on the rack end and two are needed for each temperature sensor, which will be attached to the board directly on the rack end.
Also, we don't need two voltage regulators for each temperature circuit. We can just have one of each of LM7815 and LM7915 to supply +/- 15V to the boards.
We setup the channels for PID control of the seismometer can. First, we ssh into c1auxex and went to /cvs/cds/caltech/target/c1auxex2 and found ETMXaux.db. We then added in new soft channels that we named C1:PEM-SEIS_EX_TEMP_SLOWKP, C1:PEM-SEIS_EX_TEMP_SLOWKI, C1:PEM-SEIS_EX_TEMP_SLOWKD that will control the proportional, integral and differential gain respectively. These channels are used in the script FSSSlow.py for PID control. We then had to restart the system, but first we turned off the LSC mode and then shut down the watchdog on the X end. After doing the restart, we disabled the OPLEV as well before restarting the watchdog. Then, we enabled the LSC mode again. This is done to not damage any of the optics during the restart. The restart is done by using sudo systemctl restart modbusIOC.service and restarted with sudo systemctl status modbusIOC.service. Then, we made sure that the channels existed and could be read and writtten to, so we tried z read [channel name] and it read 0.0. We then did z write [channel name] 5, and it wrote 5 to that channel. Now that the channels work, we can implement the PID script and check to make sure that it works as well.
We closed the loop today and implemented the PID script. I have attached the StripTool graph for an integral value of 0.5 and proportional value of 20. We had some issues getting it to work properly and it would oscillate between some low values of the control voltage. The set point here was -3.20, which corresponds to about a 20 degree increase in temperature. The next step would be to find which values of Kp, Ki, and Kd would work in this case and low pass filter the signal from the temperature sensor, and also create an MEDM screen for easier PID control.
Can't really figure out what this plot means. We need to see the sensor (in units of deg C) and the control signal (in heating power (W)). The plot should show a few step responses with the PID loop on, so that we can see the loop response time. Please zoom in on the axes so that we can see what's happening.
I created two new channels today, C1:PEM-SEIS_EX_TEMP_MON_CELCIUS, which turns the output voltage signal into degrees C, and C1:PEM-SEIS_EX_TEMP_CTRL_WATTS, which takes the input voltage from the DAC and turns it into a value of watts. I'm trying to stabilize the temperature at 35 degrees, but it's taking a lot longer than expected. Perhaps we'll need to use different values for P and I and decrease the noise in the sensor, since right now there's about a 10 degree variation between the highest and lowest values.
I did a step response for the loop from 35 degrees to 40 degrees. The PID is not properly tuned, so the signal oscillates. In the graph, the blue curve is the temperature of the can in celcius and the green curve is the heating power in watts. The x-axis is in minutes. Before, the signal was too noisy to do a proper step response, so I placed a 3.3 microF capacitor in parallel with the resistor in my temperature sensor circuit (I'll draw and attach this updated version). This created a 5 Hz low pass filter and the signal is now pretty clean.
I also added in new Epics channels so that we could log the data using Data Viewer. The channels I added were C1:PEM-SEIS_EX_TEMP_MON_CELCIUS and C1:PEM-SEIS_EX_TEMP_CTRL_WATTS. I used 13023 as a guide on how to do this.
Update: the channels work and show data in Data Viewer
Edit: I've attached a photo of the circuit with the capacitor indicated. It is in parallel with the resistor below it. I've attached an updated circuit diagram as well.
I have been trying to tune the PID and have managed to descrease the oscillations without saturating the actuator. I'm going to model the system to calculate the exact values of P, I and D in order to get rid of the oscillations altogether. I was going to record the data using Data Viewer, but there seems to be some issue with that, so I'm using StripTool for now.
Made some changes:
There is also now a StripTool file in the scripts/PEM directory which has appropriate channel names and scales for PID loop tuning. Use this file!
I'm leaving it running over the weekend with K_I = -0.003. There is a StripTool on rossa which you can watch. The code itself is running on a tmux session on megatron. Let's ONLY run this code there until we're satisfied that things are good.
Update Sun Apr 8 00:40:11 2018: Lowered gain by factors of 3 down to -0.0001 Saturday afternoon. Seems like still oscillating a bit, now with a ~4 hour period. Setting it to -3e-5 now. Usually we have a linear feedback loop, but our actuation voltage actually gets squared (P = I^2 R) before being integrated to produce temperature. Wonder if we should think of linearizing the feedback control signal to make the loop act nicer.
Update Sun Apr 8 21:09:48 2018: Set K_I = -1e-5 earlier today. Seems to have stabilized nearly, but temperature swings are still +/- 1 K. Will need to add some proportional feedback (K_P) to increase the loop bandwidth, but system is at least sort of stable now. Probably should start construction of EY,BS systems now.
Earth quake M5.3 2018-04-05 19:29:16UTC Santa Cruz Island, CA
I created an MEDM screen for the PID control. In addition, I added a new EPICS channel for the setpoint so that it could be adjusted using the MEDM screen.
Edit: forgot to mention the channel name is C1:PEM-SEIS_EX_TEMP_SETPOINT
Edit #2: the path for the MEDM is /opt/rtcds/caltech/c1/medm/c1pem/C1PEM_SEIS_EX_TCTRL.adl
An update to the screen. I changed the min/max values for some of the parameters, as well as changing the script so that I could specify the integral gain in terms of 1e-5. I've also added this screen to the PEM tab in the sitemap.
Another update. I've changed the on/off button so that it's visible which state it's in. I did that by changing the type of C1:PEM-SEIS-EX_TEMP_SLOWLOOP from ai to bi (I checked the FSS script and copied the entry for the slowloop). Previously, MEDM was giving me an error that it wasn't an ENUM value when I wanted to use a choice button to indicate the value of slowloop, and this solved the issue. I've also added a StripTool button.
changed the setpoint of the EX Seismomter T ctrl servo from 35 to 39 C to see if this helps the stability by decreasing the cooldown time constant.
I've updated the sketches and added in front panels for the seismometer block and the 1U panel (attachments 3 and 4). There was an issue when it came to the panel on the block because the hole is only big enough for the cable that already exists there and there is no space to add in the D-9 connector. Not quite sure how to resolve this issue. Attachment 7 is the current panel on the seismometer block. Attachments 5 and 6 are the updated temperature circuit and the heater circuit.
The boxes will be located in the short racks at EX and EY to minimize cable length.
Can you please add dimensions to the drawing, so we can see if things fit and what the cable lenghts need to be?
For the panel on the granite slab, we should use a thinner piece of metal and mount it with an offset so that the D-sub cable can be fished through the hole in the slab. The hole is wide enough for 2 cables, but not 2 connectors.
Attached is a 8-day minute trend of the heater control signals, as well as the in-loop temperature sensor (which underestimates the true fluctuations; we really need an out-of-loop sensor attached to the can or seismometer).
You can see that since the last tuning (on the 13th), its been stable at the set point of 39 C with 8.5 - 10 W of heating power. Need to add the PID loop settings (all the sliders on the MEDM screen) to the frames so that we can help in diagnosing. Also, fix the spelling of "Celcisususs".
Yesterday, I changed the P gain of the PID loop from zero to +0.1. Seems good so far; will monitor for a couple days to see if we're in the right ballpark. Main issue in the stability may now be that the quantization noise is too big for the temperature sensor. If so, we should consider subtracting off the DC value (with a V ref) and then amplifying before ADC.