EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  <20232024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  <20232024 
<== Date ==> <== Thread ==>

Subject: RE: linux kernel module for mrfioc2 on MTCA
From: 문건영(선형가속기팀) via Tech-talk <tech-talk at aps.anl.gov>
To: Han Lee <jeonglee at lbl.gov>
Cc: 박소희 <sohee5746 at postech.ac.kr>, "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov>
Date: Fri, 6 Jan 2023 07:15:30 +0000

Dear All,

 

With the help of Dr. Han, I was able to run mrfioc2.

However, I have one question.

 

I have one EVG and two EVRs as shown below.

(mTCA-EVM-300, mTCA-EVR-300, mTCA-EVR-300RF)

 

nat> show_fru

 

FRU Information:

----------------

FRU  Device   State  Name

==========================================

  0   MCH       M4    NAT-MCH-CM

  3   mcmc1     M4    NAT-MCH-MCMC

  5   AMC1      M4    CCT AM G64/471

  6   AMC2      M4    mTCA-EVM-300

  7   AMC3      M4    mTCA-EVR-300

  8   AMC4      M4    mTCA-EVR-300RF

 40   CU1       M4    Schroff uTCA CU

50   PM1       M4    NAT-PM-AC600D

60   Clock1    M4    MCH-Clock

61   HubMod1   M4    MCH-PCIe

92   AMC3-RTM  M4    mTCA-EVRTM-300

==========================================

 

I checked it by referring to lspci of mrfioc2 to distinguish 3 cards.

 

# dmesg | grep mrf

mrf: loading out-of-tree module taints kernel.

mrf-pci 0000:06:00.0: Attaching MRF device w/o PLX bridge (00007011)

mrf-pci 0000:06:00.0: MRF Setup complete

mrf-pci 0000:09:00.0: Attaching MRF device w/o PLX bridge (00007011)

mrf-pci 0000:09:00.0: MRF Setup complete

mrf-pci 0000:0a:00.0: Attaching MRF device w/o PLX bridge (00007011)

mrf-pci 0000:0a:00.0: MRF Setup complete

#

 

# lspci -v -s 6:00.0

06:00.0 Signal processing controller: Xilinx Corporation Device 7011

           Subsystem: Device 1a3e:132c

           Flags: bus master, fast devsel, latency 0, IRQ 16

           Memory at df200000 (32-bit, non-prefetchable) [size=256K]

           Capabilities: [40] Power Management version 3

           Capabilities: [48] MSI: Enable- Count=1/1 Maskable- 64bit+

           Capabilities: [60] Express Endpoint, MSI 00

           Capabilities: [100] Device Serial Number 00-00-00-00-00-00-00-00

           Kernel driver in use: mrf-pci

           Kernel modules: mrf

 

# lspci -v -s 9:00.0

09:00.0 Signal processing controller: Xilinx Corporation Device 7011

           Subsystem: Device 1a3e:132c

           Flags: bus master, fast devsel, latency 0, IRQ 18

           Memory at df100000 (32-bit, non-prefetchable) [size=256K]

           Capabilities: [40] Power Management version 3

           Capabilities: [48] MSI: Enable- Count=1/1 Maskable- 64bit+

           Capabilities: [60] Express Endpoint, MSI 00

           Capabilities: [100] Device Serial Number 00-00-00-00-00-00-00-00

           Kernel driver in use: mrf-pci

           Kernel modules: mrf

 

# lspci -v -s a:00.0

0a:00.0 Signal processing controller: Xilinx Corporation Device 7011

           Subsystem: Device 1a3e:232c

           Flags: bus master, fast devsel, latency 0, IRQ 16

           Memory at df000000 (32-bit, non-prefetchable) [size=512K]

           Capabilities: [40] Power Management version 3

           Capabilities: [48] MSI: Enable- Count=1/1 Maskable- 64bit+

           Capabilities: [60] Express Endpoint, MSI 00

           Capabilities: [100] Device Serial Number 00-00-00-00-00-00-00-00

           Kernel driver in use: mrf-pci

           Kernel modules: mrf

 

https://github.com/epics-modules/mrfioc2/blob/master/documentation/lspci/mtca-evm-300.txt

In the document above, the device name is specified in the subsystem item as shown below, but I do not.

 

06:00.0 Signal processing controller: Xilinx Corporation XILINX PCI DEVICE

        Subsystem: Micro-Research Finland Oy MTCA Event Master 300

        Physical Slot: 4

 

I couldn't distinguish between EVG and EVR, so I had to run them one by one.

 

epics> mrmEvgSetupPCI("EVG","6:00.0")

Notice: devPCIFindSpec() expect B:D.F in hex

PCI Device not found

epics>

epics> mrmEvgSetupPCI("EVG","9:00.0")

PCI Device not found

epics>                      

epics>

epics> mrmEvgSetupPCI("EVG","a:00.0")

Device EVG  a:0.0

Using IRQ 16

FPGA version: 280e0207

mTCA-EVM-300 #Inputs FP:3 UV:16 RB:0

EVM automatically creating 'EVG:FCT', 'EVG:EVRD', and 'EVG:EVRU'

Found SFP EEPROM

Found SFP EEPROM

Found SFP EEPROM

Found SFP EEPROM

Found SFP EEPROM

Found SFP EEPROM

Found SFP EEPROM

Found SFP EEPROM

Warning: Recommended minimum firmware 2 version is 207.6, found 207.0

Found SFP Strangeness 00000000

Sequencer capability detected

VME 64: Out FP:8 FPUNIV:0 RB:0 IFP:8 GPIO:0

EVR FIFO task start

Warning: Recommended minimum firmware 2 version is 207.6, found 207.0

Found SFP Strangeness 00000000

Sequencer capability detected

VME 64: Out FP:8 FPUNIV:0 RB:0 IFP:8 GPIO:0

EVR FIFO task start

PCI interrupt connected!

epics>

epics> mrmEvrSetup'

Unbalanced quote.

epics> mrmEvrSetupPCI("EVR1","slot=4")

PCI Device not found on slot=4

epics> mrmEvrSetupPCI("EVR1","6:00.0")

Device EVR1  6:0.0 slot=(null)

Using IRQ 16

FWVersion 0x18120207

Found version 519

Found SFP EEPROM

Sequencer capability detected

mTCA: Out FP:4 FPUNIV:0 RB:16 IFP:2 GPIO:0

EVR FIFO task start

Enabling interrupts

epics> mrmEvrSetupPCI("EVR2","9:00.0")

Device EVR2  9:0.0 slot=(null)

Using IRQ 18

FWVersion 0x18100207

Found version 519

Found SFP EEPROM

Sequencer capability detected

mTCA: Out FP:4 FPUNIV:0 RB:16 IFP:2 GPIO:0

EVR FIFO task start

Enabling interrupts

epics>

 

Subsystem: Device 1a3e:232c

Subsystem: Device 1a3e:132c

Is there no problem if I separate it into the above information?

1a3e:232c is mTCA-EVM-300.

1a3e:132c is mTCA-EVR-300RF or mTCA-EVR-300.

I can distinguish the two EVRs after the IOC runs.

Is there any way to know before the IOC is implemented?

 

Best regards,

Geonyeong Mun

_________________________________________

Geonyeong Mun

Staff Engineer, Accelerator Division

 

Pohang Accelerator Laboratory (PAL)
37673 80 Jigokro-127-beongil, Nam-gu,
Pohang, Gyeongbuk, Korea

 

phone: +82 10 4164 - 2260

office: +82 54 279 - 1098

email: gymun at postech.ac.kr

web: pal.postech.ac.kr

 

From: Han Lee <jeonglee at lbl.gov>
Sent: Thursday, January 5, 2023 3:00 PM
To:
문건영(선형가속기팀) <gymun at postech.ac.kr>
Cc: tech-talk at aps.anl.gov
Subject: Re: linux kernel module for mrfioc2 on MTCA

 

The actual kernel driver is located in

 

There are several methods you may try, here is also very old repo, but you can find "dkms" information there.

 

 

Best,

Han

 

On Wed, Jan 4, 2023 at 9:57 PM Han Lee <jeonglee at lbl.gov> wrote:

Hi Geonyeong,

 

Please use the kernel driver for mrfiioc2 instead of what you used.

 

 

PCIID is not up-to-date, if you want to see through lspci, please check my own archived repo.

 

 

HTH,

Han

 

 

On Wed, Jan 4, 2023 at 9:41 PM 문건영(선형가속기팀) via Tech-talk <tech-talk at aps.anl.gov> wrote:

Dear All,

I am trying to MTCA-EVG-300 and MTCA-EVR-300U are working on Linux using mrfioc2.
But, the following error is displayed:

epics> mrmEvgSetupPCI("EVG",08,00,0)
Can't open /sys/module/mrf/parameters/interfaceversion in order to read kernel module interface version. Kernel module not loaded or too old.

I used the latest version driver provided by MRF. https://github.com/jpietari/mrf-linux-driver
Do I have to use a driver other than this one?

The mrf directory does not exist In the /sys/module directory.
Instead, pci_mrfevg, pci_mrfevr exists.

# ls /sys/module/pci_mrfev*
/sys/module/pci_mrfevg:
coresize   holders    initstate  refcnt     taint
drivers    initsize   notes      sections   uevent

/sys/module/pci_mrfevr:
coresize   holders    initstate  refcnt     taint
drivers    initsize   notes      sections   uevent

# lsmod
Module                  Size  Used by    Tainted: G 
pci_mrfevr             57344  0
pci_mrfevg             53248  0
nfsv3                  28672  1
nfs                   135168  2 nfsv3
lockd                  77824  2 nfsv3,nfs
grace                  16384  1 lockd
sunrpc                196608  6 nfsv3,nfs,lockd
x86_pkg_temp_thermal    16384  0
igb                   184320  0

# ls /dev/e*
/dev/ega0  /dev/ega2  /dev/era0  /dev/era2  /dev/erb0  /dev/erb2
/dev/ega1  /dev/ega3  /dev/era1  /dev/era3  /dev/erb1  /dev/erb3

# dmesg | tail
Event Generator PCI/PCIe driver exiting.
Event Receiver PCI/PCIe driver exiting.
Event Generator PCI/PCIe driver init.
mrfevg:BAR0 start df000000 end df07ffff mmaped 41180000
udevd[165]: specified group 'mrf' unknown
Event Receiver PCI/PCIe driver init.
mrfevr:BAR0 start df200000 end df23ffff, mmap 40780000
mrfevr: Device Type: MTCAEVR300
mrfevr:BAR0 start df100000 end df13ffff, mmap 41040000
mrfevr: Device Type: MTCAEVR300

Best regards,
Geonyeong Mun



--

Jeong Han Lee, Dr.rer.nat

Staff Scientist and Engineer

Lawrence Berkeley National Laboratory

1 Cyclotron Road Mailstop 46R0161

Berkeley, CA 94720, United States

Tel :+1-510-486-6163

Cell:+1-510-384-3868



--

Jeong Han Lee, Dr.rer.nat

Staff Scientist and Engineer

Lawrence Berkeley National Laboratory

1 Cyclotron Road Mailstop 46R0161

Berkeley, CA 94720, United States

Tel :+1-510-486-6163

Cell:+1-510-384-3868


Replies:
Re: linux kernel module for mrfioc2 on MTCA Michael Davidsaver via Tech-talk
Re: linux kernel module for mrfioc2 on MTCA Han Lee via Tech-talk
References:
linux kernel module for mrfioc2 on MTCA 문건영(선형가속기팀) via Tech-talk
Re: linux kernel module for mrfioc2 on MTCA Han Lee via Tech-talk
Re: linux kernel module for mrfioc2 on MTCA Han Lee via Tech-talk

Navigate by Date:
Prev: Re: linux kernel module for mrfioc2 on MTCA Han Lee via Tech-talk
Next: Scan rate '.01 second' not achievable Barrett (US), Patrick E via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  <20232024 
Navigate by Thread:
Prev: Re: linux kernel module for mrfioc2 on MTCA Han Lee via Tech-talk
Next: Re: linux kernel module for mrfioc2 on MTCA Michael Davidsaver via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  <20232024 
ANJ, 10 Feb 2023 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·