40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Wed Mar 4 20:42:53 2020, gautam, Update, Electronics, C1PSL acromag crate is sitting on the floor 
    Reply  Thu Mar 5 11:27:48 2020, gautam, Update, Electronics, C1PSL acromag crate is no longer sitting on the floor 
       Reply  Thu Mar 5 19:51:14 2020, gautam, Update, Electronics, IMC Servo board being tested 
          Reply  Fri Mar 6 01:12:10 2020, gautam, Update, Electronics, IMC Servo IN2 path looks just fine AO_inputTFs_5Mar.pdfsliderCal_5Mar.pdf
Message ID: 15254     Entry time: Thu Mar 5 11:27:48 2020     In reply to: 15251     Reply to this: 15257
Author: gautam 
Type: Update 
Category: Electronics 
Subject: C1PSL acromag crate is no longer sitting on the floor 

[jordan, gautam]

The C1PSL crate has now been installed in a more permanent way in the rack.

  • Top and bottom covers were re-attached after work yesterday.
  • +/- 24 V DC and +15 V DC power connectors were screwed on for better robustness (I had removed the fuse for the -24V supply as part of debugging yesterday, this was reconnected).
  • PSL diagnostics DB 25 cable was re-run appropriately over the cable tray and connected to the unit.
  • The chassis sits on some rails - these rails are mounted to the rack using rack nuts but that means the ears on the Acromag chassis no longer line up with any rack nut slots, and so the chassis is not bolted on to the rack.
  • We also took this opportunity to remove the c1iool0 VME chassis from 1X2 - given that the DAC and BIO cards of c1ioo (rtcds system) are unused, I felt comfortable disconnecting them and that made the removal relatively easy. The CDS overview MEDM screen reports no errors after this work.

After this work, I disabled logging and restarted the modbus service (and copied the current version of the systemd service file to the target directory for backup). The PMC and IMC lock alright. The system is now ready to be tested in-situ. I will separately continue my IMC Servo board tests in the evening.

One thought about how to protect against this kind of silent failure - how about we always run the modbus service with logging enabled, and then send out a warning email and stop the service if the logfile size suddenly blows up (which is characteristic of when the communications process dies)? This should be done in addition to the ping-ing of the individual IPs.

Regarding the burt-restore step that the systemd service runs after starting up the IOC - this is not even that useful, at least in the way it is currently setup (restore the "latest" burt snapshot file). If the maintenance takes >1hour as it often does, the "latest" snapshot for the system under maintenance is just garbage. So either the burt-restore should be for a "known good time" (dangerous because this will require frequent updates of the systemd service every time we find a new safe state) or we should just do it manually (my preference). Then there is no need to install custom packages on the server machine. Anyway, for now, I have not commented this step out.

Jordan is going to take pictures of all the electronics racks and update the relevant wiki pages.

Quote:

Jon is going to write up the details of todays adventures. But the C1PSL Acromag chassis is sitting on the floor between the IMC beamtube and the 1X1 electronics rack, and is very much a trip hazard. Be careful if youre in that area.

ELOG V3.1.3-