In the past 2 weeks I have finalized the pipeline for hyperparameter testing and speeding up the training dramatically. Here is a rundown of the most recent developments:
Moving on, I am ready to start making the environment more complicated: I now need to add things like varying T_env, measurement noise, heating delays, etc. Luckily, these changes can be done independently in PuckEnv.py, allowing us to just use the same code for training once the environment is up and bug-free. For testing purposes of the environments, I also have the env.ipynb, although it is not as streamlined as the rest of the code.
Aside on avg_return if you wonder what this number means: basically, at each step of the environment (the code that simulates the temperature of the puck) I am takig the error to be . Then, for an entire episode, the reward is given by the average of these errors (so it gives you average error for all steps in one episode). Then, the reported avg_return is calculated by taking the mean of average errors for 5 (as of now) episodes. Basically, sqrt(|avg_return|) gives you an average deviation away from T_ref: so if avg_return=-25, then you should expect the model to be consistently around 5 degrees away from T_ref.
Dismantled the seismometer circuit
At Rana's request, Deven and I have pulled all the wires out of the seismometer at the X end that Adviat set up the heater and sensors for. Right now, the seismometer is uncovered (no can) and the can I had to move next to the server rack that is close to the X end. I am attaching pictures of both the seismometer and the can.
I now realize that I have not turned off the power supply for the heater. If someone can please turn off the power supply that is shown in Advait's elog, please turn it off and reply to this elog. If you are unsure which power supply to turn off, read the first line of the elog that I replied to. Thanks!
Advait's circuits have been removed from the seismometer and stored in a box underneath the desk at the end of the control room (see picture). There is even a label on the box which reads "ANDREI CIRCUITS". At the seismometer there are still a few BNC cables left there: these are the cables that were used to interact with Advait's circuits. They have labels on them so that I can put back the circuits when I come back/when the seismometer can is available again
I've turned off the power supply. I've attached a picture of the rack with the switch that I flipped circled
1.5 days of happy psl-ioo with litle bumps in C1:PSL-126MOPA_HTEMP
Mode Power (V)
BE 0.36 **Bull's Eye mode is TEM02 + TEM20. This can be fixed by lens adjustment.