40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Wed Sep 29 17:10:09 2021, Anchal, Summary, CDS, c1teststand problems summary c1teststand_issues_summary.pdf
    Reply  Thu Sep 30 14:09:37 2021, Anchal, Summary, CDS, New way to ssh into c1teststand 
    Reply  Mon Oct 4 11:05:44 2021, Anchal, Summary, CDS, c1teststand problems summary 
       Reply  Mon Oct 4 18:00:16 2021, Koji, Summary, CDS, c1teststand problems summary 
          Reply  Tue Oct 5 17:58:52 2021, Anchal, Summary, CDS, c1teststand problems summary 
       Reply  Tue Oct 5 18:00:53 2021, Anchal, Summary, CDS, c1teststand time synchronization working now 
          Reply  Mon Oct 11 17:31:25 2021, Anchal, Summary, CDS, Fixed mounting of mx devices in fb. daqd_dc is running now. 
             Reply  Mon Oct 11 18:29:35 2021, Anchal, Summary, CDS, Moving forward? 
                Reply  Tue Oct 12 17:20:12 2021, Anchal, Summary, CDS, Connected c1sus2 to martian network 
                   Reply  Tue Oct 12 23:42:56 2021, Koji, Summary, CDS, Connected c1sus2 to martian network 
                      Reply  Wed Oct 13 11:25:14 2021, Anchal, Summary, CDS, Ran c1sus2 models in martian CDS. All good! CDS_screens_running.png
                         Reply  Tue Oct 19 18:20:33 2021, Ian MacMillan, Summary, CDS, c1sus2 DAC to ADC test data3_Plots.pdfdata2_Plots.pdf
                            Reply  Tue Oct 19 23:43:09 2021, Koji, Summary, CDS, c1sus2 DAC to ADC test P_20211019_224433.jpgP_20211019_224122.jpgP_20211019_224400.jpgP_20211019_224411.jpg
                               Reply  Wed Oct 20 11:48:27 2021, Anchal, Summary, CDS, Power supple configured correctly. 
                               Reply  Tue Oct 26 18:24:00 2021, Ian MacMillan, Summary, CDS, c1sus2 DAC to ADC test data2_Plots.pdfdata3_Plots.pdf
             Reply  Tue Oct 12 17:10:56 2021, Anchal, Summary, CDS, Some more information 
Message ID: 16395     Entry time: Tue Oct 12 17:10:56 2021     In reply to: 16391
Author: Anchal 
Type: Summary 
Category: CDS 
Subject: Some more information 

Chris pointed out some information displaying scripts, that show if the DAQ network is working or not. I thought it would be nice to log this information here as well.

controls@fb1:/opt/mx/bin 0$ ./mx_info
MX Version: 1.2.16
MX Build: controls@fb1:/opt/src/mx-1.2.16 Mon Aug 14 11:06:09 PDT 2017
1 Myrinet board installed.
The MX driver is configured to support a maximum of:
    8 endpoints per NIC, 1024 NICs on the network, 32 NICs per host
===================================================================
Instance #0:  364.4 MHz LANai, PCI-E x8, 2 MB SRAM, on NUMA node 0
    Status:        Running, P0: Link Up
    Network:    Ethernet 10G

    MAC Address:    00:60:dd:45:37:86
    Product code:    10G-PCIE-8B-S
    Part number:    09-04228
    Serial number:    423340
    Mapper:        00:60:dd:45:37:86, version = 0x00000000, configured
    Mapped hosts:    3

                                                        ROUTE COUNT
INDEX    MAC ADDRESS     HOST NAME                        P0
-----    -----------     ---------                        ---
   0) 00:60:dd:45:37:86 fb1:0                             1,0
   1) 00:25:90:05:ab:47 c1bhd:0                           1,0
   2) 00:25:90:06:69:c3 c1sus2:0                          1,0

 

controls@c1bhd:~ 1$ /opt/open-mx/bin/omx_info
Open-MX version 1.5.4
 build: root@fb1:/opt/src/open-mx-1.5.4 Tue Aug 15 23:48:03 UTC 2017

Found 1 boards (32 max) supporting 32 endpoints each:
 c1bhd:0 (board #0 name eth1 addr 00:25:90:05:ab:47)
   managed by driver 'igb'

Peer table is ready, mapper is 00:60:dd:45:37:86
================================================
  0) 00:25:90:05:ab:47 c1bhd:0
  1) 00:60:dd:45:37:86 fb1:0
  2) 00:25:90:06:69:c3 c1sus2:0

 

controls@c1sus2:~ 0$ /opt/open-mx/bin/omx_info
Open-MX version 1.5.4
 build: root@fb1:/opt/src/open-mx-1.5.4 Tue Aug 15 23:48:03 UTC 2017

Found 1 boards (32 max) supporting 32 endpoints each:
 c1sus2:0 (board #0 name eth1 addr 00:25:90:06:69:c3)
   managed by driver 'igb'

Peer table is ready, mapper is 00:60:dd:45:37:86
================================================
  0) 00:25:90:06:69:c3 c1sus2:0
  1) 00:60:dd:45:37:86 fb1:0
  2) 00:25:90:05:ab:47 c1bhd:0

These outputs prove that the framebuilder and the FEs are able to see each other in teh DAQ network.


Further, the error that we see when IOP model is started which crashes the mx_stream service on the FE machines (see 40m/16391) :

isendxxx failed with status Remote Endpoint Unreachable

This has been seen earlier when Jamie was troubleshooting the current fb1 in martian network in 40m/11655 in Oct, 2015. Unfortunately, I could not find what Jamie did over a year to fix this issue.

ELOG V3.1.3-