EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024 
<== Date ==> <== Thread ==>

Subject: fatal code: 30 (INTERNAL_ERROR_BAD_THREAD_DISPATCH_DISABLE_LEVEL)
From: Francisco Ramos via Tech-talk <tech-talk at aps.anl.gov>
To: EPICS tech-talk <tech-talk at aps.anl.gov>
Date: Wed, 12 Jan 2022 11:24:56 -0300
Dear community, 

    I'm writing to see if you can help me to solve a problem I'm doing a migrating the "PMAC-VME Device Driver" to EPCIS version 7.0.6.1. with rtems5. The driver is working for EPCIS version 3.14.8 and RTEMS 4.10.
    The code compiles perfectly for Epics 7 and rtems5, but when I deploy it on the MVM2700 LCU, I get the following error: Error code 30 (INTERNAL_ERROR_BAD_THREAD_DISPATCH_DISABLE_LEVEL). The RTEMS documentation describes the error as follows:
        INTERNAL_ERROR_BAD_THREAD_DISPATCH_DISABLE_LEVEL (30)
             It is illegal to call blocking operating system services with thread dispatching disabled, for example in interrupt context. 

    I attach the Stack Trace at the end of the sms. 

    I have run addr2line to get more information and you can give me suggestions on how I might fix the problem. Thanks in advance for your help. 

    In the following table, I have put the first group of "Stack Trace" for easier reading.  I copy below the code fragment of the "PMAC-VME Device Driver" referring to the line marked in bold in the table.  

static int mbxRcptISRcnt = 0;
PMAC_LOCAL void pmacMbxReceiptISR(void *p)
{
   PMAC_CTLR   *pPmacCtlr = (PMAC_CTLR *)p;

   cntRxISR++;
   epicsEventSignal (pPmacCtlr->ioMbxReceiptSem);
   mbxRcptISRcnt++;    // line 130 of the drvPmacVme.c file. 
   return;
}

    After the table, I attach the Stack Trace output and the ADDR2LINE output.
        
ADDRfunctionFile And Line
0x00254f84bsp_reset/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/lib/libbsp/powerpc/motorola_powerpc/../../../../../../../../rtems/c/src/lib/libbsp/powerpc/motorola_powerpc/../../../../../../bsps/powerpc/motorola_powerpc/start/bspreset.c:16
0x00254ec4bsp_fatal_extension/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/lib/libbsp/powerpc/motorola_powerpc/../../../../../../../../rtems/c/src/lib/libbsp/powerpc/motorola_powerpc/../../../../../../bsps/shared/start/bspfatal-default.c:99
0x00135648_User_extensions_Iterate/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/cpukit/../../../../../rtems/c/src/../../cpukit/score/src/userextiterate.c:172
0x00135854_Terminate/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/cpukit/../../../../../rtems/c/src/../../cpukit/score/src/interr.c:38
0x001358ac_Workspace_Handler_initialization/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/cpukit/../../../../../rtems/c/src/../../cpukit/score/src/wkspace.c:54
0x001321f4_Thread_Dispatch_enable/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/cpukit/../../../../../rtems/c/src/../../cpukit/score/src/threaddispatch.c:364
0x0012aa4c_POSIX_Get_error/gem_base/targetOS/RTEMS/rtems/kernel/rtems/cpukit/include/rtems/posix/posixapi.h:70
0x0010b08cepicsEventWait/builddir/build/BUILD/epics-base-git-13.a417dec/modules/libcom/src/O.RTEMS-mvme2700/../osi/os/posix/osdEvent.c:105
0x001039e8epicsEventMustWait/builddir/build/BUILD/epics-base-git-13.a417dec/modules/libcom/src/O.RTEMS-mvme2700/../osi/epicsEvent.cpp:127
0x000f502cerrlogFlush/builddir/build/BUILD/epics-base-git-13.a417dec/modules/libcom/src/O.RTEMS-mvme2700/../error/errlog.c:551
0x000fd6cccantProceed/builddir/build/BUILD/epics-base-git-13.a417dec/modules/libcom/src/O.RTEMS-mvme2700/../misc/cantProceed.c:69
0x0001360cpmacMbxReceiptISR/home/framos/work/pmaclib.git/pmaclibApp/src/O.RTEMS-mvme2700/../drvPmacVme.c:130
0x0025f628universeVMEISR/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/lib/libbsp/powerpc/motorola_powerpc/../../../../../../../../rtems/c/src/lib/libbsp/powerpc/motorola_powerpc/../../../../../../bsps/powerpc/shared/vme/vmeUniverse.c:1978
0x00262f64bsp_interrupt_handler_dispatch/gem_base/targetOS/RTEMS/rtems/kernel/rtems/bsps/include/bsp/irq-generic.h:274
0x0025dd08wrap_change_msr_done_std/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/lib/libbsp/powerpc/motorola_powerpc/../../../../../../../../rtems/c/src/lib/libbsp/powerpc/motorola_powerpc/../../../../../../bsps/powerpc/shared/exceptions/ppc_exc.S:129
        

Stack Trace
fatal source: 0 (INTERNAL_ERROR_CORE)
fatal code: 30 (INTERNAL_ERROR_BAD_THREAD_DISPATCH_DISABLE_LEVEL)
RTEMS version: 5.0.0.645dbc5fccd8b73e12ca17ab35962fe751854913
RTEMS tools: 7.5.0 20191114 (RTEMS 5, RSB 5 (803d42cda7b3), Newlib 7947581)
executing thread ID: 0x08b010010
executing thread name:
Printing a stack trace for your convenience :-)

*** FATAL ***
fatal source: 9 (RTEMS_FATAL_SOURCE_EXCEPTION)
exception vector 3 (0x3)
  next PC or address of fault = 0x0025cb8c
  saved MSR = 0x00001032
  context = interrupt, ISR nest level = 1
  thread dispatch disable level = 2
  R0  = 0x00254f84 R1  = 0x00494758 R2  = 0x00000000 R3  = 0x00000030
  R4  = 0x00000000 R5  = 0x002bca1c R6  = 0x004947b0 R7  = 0x0025dd08
  R8  = 0x004947a0 R9  = 0xa5a5a5a5 R10 = 0x004947a0 R11 = 0x00000000
  R12 = 0x40842402 R13 = 0x003006c0 R14 = 0x00ef1640 R15 = 0x00000000
  R16 = 0x00000000 R17 = 0x00000000 R18 = 0x00000000 R19 = 0x00000000
  R20 = 0x00000000 R21 = 0x00000000 R22 = 0x00000000 R23 = 0x00000000
  R24 = 0x00336560 R25 = 0x00000000 R26 = 0x002a41b0 R27 = 0x00135580
  R28 = 0x004948a8 R29 = 0x00494760 R30 = 0x0000001e R31 = 0x003174e8
  CR  = 0x40842408
  CTR = 0x00000017
  XER = 0x20000000
  LR  = 0x00254f88
  DAR = 0xa5a5a5a9
  executing thread ID = 0x0b010010, name =
Stack Trace:  
  IP: 0x0025cb8c, LR: 0x00254f88
--^ 0x00254f84--^ 0x00254ec4--^ 0x00135648--^ 0x00135854--^ 0x001358ac
--^ 0x001321f4--^ 0x0012aa4c--^ 0x0010b08c--^ 0x001039e8--^ 0x000f502c
--^ 0x000fd6cc--^ 0x0001360c--^ 0x0025f628--^ 0x00262f64--^ 0x0025dd08
   
*** FATAL ***
fatal source: 9 (RTEMS_FATAL_SOURCE_EXCEPTION)
exception vector 3 (0x3)
  next PC or address of fault = 0x0025d240
  saved MSR = 0x00001032
  context = interrupt, ISR nest level = 1
  thread dispatch disable level = 2
  R0  = 0x0025d270 R1  = 0x004945e8 R2  = 0x00000000 R3  = 0x002bdd8c
  R4  = 0x00000000 R5  = 0x002784e8 R6  = 0x00494578 R7  = 0x002784ea
  R8  = 0x002784e9 R9  = 0xa5a5a5a5 R10 = 0x800003fd R11 = 0x00000000
  R12 = 0x40442202 R13 = 0x003006c0 R14 = 0x004946a8 R15 = 0x00000000
  R16 = 0x00000000 R17 = 0x00000000 R18 = 0x00000000 R19 = 0x00000000
  R20 = 0x00000000 R21 = 0x00000000 R22 = 0x00000000 R23 = 0x00000000
  R24 = 0x00336560 R25 = 0x00000000 R26 = 0x002a41b0 R27 = 0x002784e8
  R28 = 0xcccccccd R29 = 0x002bdd8c R30 = 0x00494a10 R31 = 0x00000010
  CR  = 0x40442808
  CTR = 0x00254fe8
  XER = 0x20000000
  LR  = 0x0025d270
  DAR = 0xa5a5a5a9
  executing thread ID = 0x0b010010, name =
Stack Trace:
  IP: 0x0025d240, LR: 0x0025d270
--^ 0x00254f64--^ 0x00135648--^ 0x00135854--^ 0x002662f0--^ 0x0025de68
--^ 0x40842402--^ 0x00254f84--^ 0x00254ec4--^ 0x00135648--^ 0x00135854
--^ 0x001358ac--^ 0x001321f4--^ 0x0012aa4c--^ 0x0010b08c--^ 0x001039e8
--^ 0x000f502c--^ 0x000fd6cc--^ 0x0001360c--^ 0x0025f628--^ 0x00262f64
--^ 0x0025dd08

ADDR2LINE

line to Analyzed  0x00254f84 0x00254ec4 0x00135648 0x00135854 0x001358ac
ADDR: 0x00254f84 -> bsp_reset
/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/lib/libbsp/powerpc/motorola_powerpc/../../../../../../../../rtems/c/src/lib/libbsp/powerpc/motorola_powerpc/../../../../../../bsps/powerpc/motorola_powerpc/start/bspreset.c:16
ADDR: 0x00254ec4 -> bsp_fatal_extension
/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/lib/libbsp/powerpc/motorola_powerpc/../../../../../../../../rtems/c/src/lib/libbsp/powerpc/motorola_powerpc/../../../../../../bsps/shared/start/bspfatal-default.c:99
ADDR: 0x00135648 -> _User_extensions_Iterate
/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/cpukit/../../../../../rtems/c/src/../../cpukit/score/src/userextiterate.c:172
ADDR: 0x00135854 -> _Terminate
/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/cpukit/../../../../../rtems/c/src/../../cpukit/score/src/interr.c:38
ADDR: 0x001358ac -> _Workspace_Handler_initialization
/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/cpukit/../../../../../rtems/c/src/../../cpukit/score/src/wkspace.c:54
line to Analyzed  0x001321f4 0x0012aa4c 0x0010b08c 0x001039e8 0x000f502c
ADDR: 0x001321f4 -> _Thread_Dispatch_enable
/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/cpukit/../../../../../rtems/c/src/../../cpukit/score/src/threaddispatch.c:364
ADDR: 0x0012aa4c -> _POSIX_Get_error
/gem_base/targetOS/RTEMS/rtems/kernel/rtems/cpukit/include/rtems/posix/posixapi.h:70
ADDR: 0x0010b08c -> epicsEventWait
/builddir/build/BUILD/epics-base-git-13.a417dec/modules/libcom/src/O.RTEMS-mvme2700/../osi/os/posix/osdEvent.c:105
ADDR: 0x001039e8 -> epicsEventMustWait
/builddir/build/BUILD/epics-base-git-13.a417dec/modules/libcom/src/O.RTEMS-mvme2700/../osi/epicsEvent.cpp:127
ADDR: 0x000f502c -> errlogFlush
/builddir/build/BUILD/epics-base-git-13.a417dec/modules/libcom/src/O.RTEMS-mvme2700/../error/errlog.c:551
line to Analyzed  0x000fd6cc 0x0001360c 0x0025f628 0x00262f64 0x0025dd08
ADDR: 0x000fd6cc -> cantProceed
/builddir/build/BUILD/epics-base-git-13.a417dec/modules/libcom/src/O.RTEMS-mvme2700/../misc/cantProceed.c:69
ADDR: 0x0001360c -> pmacMbxReceiptISR
/home/framos/work/pmaclib.git/pmaclibApp/src/O.RTEMS-mvme2700/../drvPmacVme.c:130
ADDR: 0x0025f628 -> universeVMEISR
/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/lib/libbsp/powerpc/motorola_powerpc/../../../../../../../../rtems/c/src/lib/libbsp/powerpc/motorola_powerpc/../../../../../../bsps/powerpc/shared/vme/vmeUniverse.c:1978
ADDR: 0x00262f64 -> bsp_interrupt_handler_dispatch
/gem_base/targetOS/RTEMS/rtems/kernel/rtems/bsps/include/bsp/irq-generic.h:274
ADDR: 0x0025dd08 -> wrap_change_msr_done_std
/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/lib/libbsp/powerpc/motorola_powerpc/../../../../../../../../rtems/c/src/lib/libbsp/powerpc/motorola_powerpc/../../../../../../bsps/powerpc/shared/exceptions/ppc_exc.S:129
line to Analyzed
line to Analyzed  0x00254f64 0x00135648 0x00135854 0x002662f0 0x0025de68
ADDR: 0x00254f64 -> rtems_exception_frame_print
/gem_base/targetOS/RTEMS/rtems/kernel/rtems/cpukit/include/rtems/fatal.h:60
ADDR: 0x00135648 -> _User_extensions_Iterate
/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/cpukit/../../../../../rtems/c/src/../../cpukit/score/src/userextiterate.c:172
ADDR: 0x00135854 -> _Terminate
/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/cpukit/../../../../../rtems/c/src/../../cpukit/score/src/interr.c:38
ADDR: 0x001358ac -> _Workspace_Handler_initialization
/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/cpukit/../../../../../rtems/c/src/../../cpukit/score/src/wkspace.c:54
line to Analyzed  0x001321f4 0x0012aa4c 0x0010b08c 0x001039e8 0x000f502c
ADDR: 0x001321f4 -> _Thread_Dispatch_enable
/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/cpukit/../../../../../rtems/c/src/../../cpukit/score/src/threaddispatch.c:364
ADDR: 0x0012aa4c -> _POSIX_Get_error
/gem_base/targetOS/RTEMS/rtems/kernel/rtems/cpukit/include/rtems/posix/posixapi.h:70
ADDR: 0x0010b08c -> epicsEventWait
/builddir/build/BUILD/epics-base-git-13.a417dec/modules/libcom/src/O.RTEMS-mvme2700/../osi/os/posix/osdEvent.c:105
ADDR: 0x001039e8 -> epicsEventMustWait
/builddir/build/BUILD/epics-base-git-13.a417dec/modules/libcom/src/O.RTEMS-mvme2700/../osi/epicsEvent.cpp:127
ADDR: 0x000f502c -> errlogFlush
/builddir/build/BUILD/epics-base-git-13.a417dec/modules/libcom/src/O.RTEMS-mvme2700/../error/errlog.c:551
line to Analyzed  0x000fd6cc 0x0001360c 0x0025f628 0x00262f64 0x0025dd08
ADDR: 0x000fd6cc -> cantProceed
/builddir/build/BUILD/epics-base-git-13.a417dec/modules/libcom/src/O.RTEMS-mvme2700/../misc/cantProceed.c:69
ADDR: 0x0001360c -> pmacMbxReceiptISR
/home/framos/work/pmaclib.git/pmaclibApp/src/O.RTEMS-mvme2700/../drvPmacVme.c:130
ADDR: 0x0025f628 -> universeVMEISR
/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/lib/libbsp/powerpc/motorola_powerpc/../../../../../../../../rtems/c/src/lib/libbsp/powerpc/motorola_powerpc/../../../../../../bsps/powerpc/shared/vme/vmeUniverse.c:1978
ADDR: 0x00262f64 -> bsp_interrupt_handler_dispatch
/gem_base/targetOS/RTEMS/rtems/kernel/rtems/bsps/include/bsp/irq-generic.h:274
ADDR: 0x0025dd08 -> wrap_change_msr_done_std
/gem_base/targetOS/RTEMS/rtems/kernel/mvme2307/powerpc-rtems5/c/mvme2307/lib/libbsp/powerpc/motorola_powerpc/../../../../../../../../rtems/c/src/lib/libbsp/powerpc/motorola_powerpc/../../../../../../bsps/powerpc/shared/exceptions/ppc_exc.S:129

Regards, 




Replies:
Re: fatal code: 30 (INTERNAL_ERROR_BAD_THREAD_DISPATCH_DISABLE_LEVEL) Heinz Junkes via Tech-talk
Re: fatal code: 30 (INTERNAL_ERROR_BAD_THREAD_DISPATCH_DISABLE_LEVEL) Heinz Junkes via Tech-talk

Navigate by Date:
Prev: European XFEL job ad: Control Software Engineer (f/m/d) Steffen Hauf via Tech-talk
Next: Re: fatal code: 30 (INTERNAL_ERROR_BAD_THREAD_DISPATCH_DISABLE_LEVEL) Heinz Junkes via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024 
Navigate by Thread:
Prev: European XFEL job ad: Control Software Engineer (f/m/d) Steffen Hauf via Tech-talk
Next: Re: fatal code: 30 (INTERNAL_ERROR_BAD_THREAD_DISPATCH_DISABLE_LEVEL) Heinz Junkes via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024 
ANJ, 14 Sep 2022 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·