[Alex, Joe, Kiwamu]

Eventually all the front end computers came back !! 

There were two problems.

(1): C0DCU1 didn't want to come back to the network. After we did several things it turned the ADC board for C0DCU1 didn't work correctly.

(2): C1PEM1 and C0DAQAWG were cross-talking via the back panel of the crate.

(what we did)

* installed a VME crate with single back panel to 1Y6 and mounted C1PEM1 and C0DAQAWG on it. However it turned out this configuration was bad because the two CPUs could cross-talk via the back panel.

* removed the VME crate and then installed another VME crate which has two back panels so that we can electrically separate C1PEM1 and C0DAQAWG.  After this work, C0DAQAWG started working successfully.

 * rebooted all the front ends, fb40m and c1dcuepics.

 * reset the RFM bypath. But these things didn't bring C0DCU1 back.

 * telnet to C0DCU1 and ran "./startup.cmd" manually. In fact "./startup.cmd" should automatically be called when it boots.

 * saw the error messages from "./startup.cmd" and found it failed when initialization of the ADC board. It saids "Init Failure !! could not find ICS"

*  went to 1Y7 rack and checked the ADC. We found C0DCU1 had two ADC boards, one of two was not in used.

* disconnected all two ADCs and put back one which had not been in used. At the same time we changed the switching address of this ADC to have the same address as the other ADC. 

* powered off/on 1Y7 rack. Finally C0DCU1 got back.

* burtrestored the epics to the last Friday, May 21st 6:07am