Apparently its possible to have working models get into a bad state in regards to shared memory, which prevents the model from working after killing it and restarting it. We found that by shutting all the models down, and then killing and restarting the setup_shmem process, it allowed models to function properly again.
The symptom was getting stuck at the burt restore step, according the log files (/opt/rtcds/caltech/c1/target/c1SYS/logs/log.txt). |