40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Message ID: 4790     Entry time: Mon Jun 6 18:29:01 2011
Author: Jamie, Joe 
Type: Update 
Category: CDS 
Subject: COMPLETE FRONT-END REBUILD (WITH PROBLEMS (fixed)) 

Today Joe and I undertook a FULL rebuild of all front end systems with the head of the 2.1 branch of the RCG.  Here is the full report of what we did:

  1. checked out advLigoRTS/branches/branch-2.1, r2457 into core/branches/branch-2.1
  2. linked core/release to branches/branch-2.1
  3. linked in models to core/release/src/epics/simLink using Joe's new script (userapps/release/cds/c1/scripts/link_userapps)
  4. remove unused/non-up-to-date models:
  5. c1dafi.md
    c1lsp.mdl
    c1gpv.mdl
    c1sup_vertex_plant_shmem.mdl
  6. modified core/release/Makefile so that it can find models:
  7. --- Makefile	(revision 2451)
    +++ Makefile (working copy)
    @@ -346,7 +346,7 @@
    #MDL_MODELS = x1cdst1 x1isiham x1isiitmx x1iss x1lsc x1omc1 x1psl x1susetmx x1susetmy x1susitmx x1susitmy x1susquad1 x1susquad2 x1susquad3 x1susquad4 x1x12 x1x13 x1x14 x1x15 x1x16 x1x20 x1x21 x1x22 x1x23

    #MDL_MODELS = $(wildcard src/epics/simLink/l1*.mdl)
    -MDL_MODELS = $(shell cd src/epics/simLink; ls m1*.mdl | sed 's/.mdl//')
    +MDL_MODELS = $(shell cd src/epics/simLink; ls c1*.mdl | sed 's/.mdl//')

    World: $(MDL_MODELS)
    showWorld:
  8. removed channel files for models that we know will be renumbered
    • For this rebuild, we are also building modified sus models, that are now using libraries, so the channel numbering is changing.
  9. make World
    • this makes all the models
  10. make installWorld
    • this installs all the models
  11. Run activateDQ.py script to activate all the relevant channels
    • this script was modified to handle the new "_DQ" channels
  12. make/install new awgtpman:
  13. cd src/gds
    make
    cp awgtpman /opt/rtcds/caltech/c1/target/gds/bin
  14. turn off all watchdogs
  15. test restart one front end: c1iscex
  16. BIG PROBLEM

    The c1iscex models (c1x01 and c1scx) did not come back up.  c1x01 was running long on every cycle, until the model crashed and brought down the computer.  After many hours, and with Alex's help, we managed to track down the issue to a patch from Rolf at r2361.  The code included in that patch should have been wrapped in an "#ifndef RFM_DIRECT_READ".  This was fixed and committed to branches/branch-2.1 at r2460 and to trunk at r2461.

  17. update to core/branches/branch-2.1 to r2460
  18. make World && make installWorld with the new fixed code
  19. restarted all computers
  20. restart frame builder
  21. burt restored to 8am this morning
  22. turned on all watchdogs

Everything is now green, and things seem to be working.  Mode cleaner is locked.  X arm locked.

 

ELOG V3.1.3-