I tried a manual test of the new user space model. Since this is a user space process running it should have no affect on the rest of the front end system (which it didn't):
$ (cd /opt/rtcds/caltech/c1/target/c1dnn/c1dnnepics && ./startupC1)
$ taskset -c 6 /opt/rtcds/caltech/c1/target/c1dnn/bin/c1dnn -m c1dnn
Unfortunately, the process died with an "ADC TIMEOUT" error. I'm investigating why.
Once we confirm the model runs, we'll add the appropriate SHMEM IPC connections to connect it to the c1lsc model.
I tried moving the model to c1ioo, where there are plenty of free cores sitting idle, and the model seems runs fine. I think the problem was just CPU contention on the c1lsc machine, where there were only two free cores and the kernel was using both for all the rest of the normal user space processes.
So there are two options:
I'm going to start testing cpuset offline to figure out exactly what would need to be done.