Helpful ReplyHot![SOLVED]Bug in the USART Harmony/FreeRTOS module

Author
BillP
Super Member
  • Total Posts : 292
  • Reward points : 0
  • Joined: 2014/09/28 07:53:35
  • Location: CA
  • Status: offline
2019/02/26 09:04:20 (permalink)
0

[SOLVED]Bug in the USART Harmony/FreeRTOS module

Platform: Curiosity PIC32MZ EF (100 pin) with 2 Clik board slots
WiFi4 Clik Board
MPLABX IDE v4.20
Harmony 2.06
XC32 v2.10
FreeRTOS v10.0.1 (the default version selected in MHC)

The WiFi4 uses a USART (Tx/Rx) interface to the PIC32MZ host. MHC configures the USART and a RESET signal to control the WiFi4 board.  The USART driver uses an interrupt with byte-mode at a 115200 baud rate.

I used MHC to configure 2 COM ports with the USB CDC library -- COM1 is a console port to enter AT commands for the WiFi4 Clik board and COM2 is a log port to display logging messages created during run-time.

I developed a superloop app that talks to the WiFi4 board.  All of the interfaces and features work as expected.  The logic running under the superloop is a little complicated because it relies on multiple state machines and a lot of flags to implement a command/response element.  [This is a standard interface characteristic in many serial interfaces, such as Modbus.]  

An RTOS simplifies the implementation of the command/response element by including delays and waits for the back-and-forth communications, so I converted the superloop logic to use the FreeRTOS module included in the Harmony distro. It crashed.  After many hours of checking configurations, debugging (sometimes) showed the PC was in the port_asm.S file (a FreeRTOS source file). No documentation on what that file does but it appears to deal with some interrupt vector locations.  

I was able to isolate the "bug" to be in the Rx and Error callbacks defined in the USART driver (byte-mode).  If I commented out the callbacks, the code did not crash (but it did not execute properly).  These same callbacks worked correctly in the superloop version.

Conclusion:  there appears to be a bug in the USART Harmony/FreeRTOS module.
post edited by BillP - 2019/03/04 08:50:27
#1
Twi
New Member
  • Total Posts : 2
  • Reward points : 0
  • Joined: 2019/02/22 01:39:12
  • Location: 0
  • Status: offline
Re: Bug in the USART Harmony/FreeRTOS module(?) 2019/02/26 11:08:57 (permalink)
0
Hey,
Maybe not? What's in those callbacks ?
#2
Paul PortSol
Super Member
  • Total Posts : 402
  • Reward points : 0
  • Joined: 2015/07/03 11:52:03
  • Location: 0
  • Status: offline
Re: Bug in the USART Harmony/FreeRTOS module(?) 2019/02/27 07:13:34 (permalink)
0
Anything in the HarmonyV206 Errata or the latest Errata for your PIC32 chip?
#3
BillP
Super Member
  • Total Posts : 292
  • Reward points : 0
  • Joined: 2014/09/28 07:53:35
  • Location: CA
  • Status: offline
Re: Bug in the USART Harmony/FreeRTOS module(?) 2019/02/27 08:24:03 (permalink)
0
I cannot find anything in the Harmony release notes.  I really doubt this is a Harmony problem since everything works in the superloop version.  It must be FreeRTOS doing something that interferes with the Harmony framework.  If no one in this forum has seen something like this before, then I will contact FreeRTOS and I will post any results.
#4
BillP
Super Member
  • Total Posts : 292
  • Reward points : 0
  • Joined: 2014/09/28 07:53:35
  • Location: CA
  • Status: offline
Re: Bug in the USART Harmony/FreeRTOS module(?) 2019/02/27 08:30:45 (permalink)
0
Here is the code in the Event Handlers.  This is the same in both the superloop and FreeRTOS. The RX event handler reads one byte and stores it in a circular buffer and returns. The check for an "OK" response is also done here to avoid more complex logic elsewhere.
 
void RXEventHandler(const SYS_MODULE_INDEX index)
{
    int pbm1;
    // a byte has been received. Store it in the rbuf
    rbuf[putbyte] = DRV_USART_ReadByte(serialData.usartHandle);
    // check for an "OK" response
    if(putbyte == 0) pbm1 = SERIAL_CIRCBUFSIZE;
    else pbm1 = putbyte - 1;
    if(rbuf[putbyte] == 'K' && rbuf[pbm1] == 'O') isOK = true;
    // update the counter and account for wrap-around
    putbyte++;
    if(putbyte >= SERIAL_CIRCBUFSIZE) putbyte = 0;
}

void ErrorEventHandler(const SYS_MODULE_INDEX index)
{
    serialData.isError = true;
}

#5
Twi
New Member
  • Total Posts : 2
  • Reward points : 0
  • Joined: 2019/02/22 01:39:12
  • Location: 0
  • Status: offline
Re: Bug in the USART Harmony/FreeRTOS module(?) 2019/02/28 01:36:18 (permalink)
0
Please provide FreeRTOS forum post link.
#6
BillP
Super Member
  • Total Posts : 292
  • Reward points : 0
  • Joined: 2014/09/28 07:53:35
  • Location: CA
  • Status: offline
Re: Bug in the USART Harmony/FreeRTOS module(?) 2019/02/28 12:11:45 (permalink)
#7
friesen
Super Member
  • Total Posts : 2036
  • Reward points : 0
  • Joined: 2008/05/08 05:23:35
  • Location: Indiana, USA
  • Status: offline
Re: Bug in the USART Harmony/FreeRTOS module(?) 2019/02/28 16:52:12 (permalink)
0
I kind of doubt it is a FreeRTOS problem. Have you isolated at all where the crash happens?
 
FreeRTOS is only a RTOS, it doesn't do as much as one may think.  You do have to be sure that the ISR's are set up to be compatible with their way of doing things, but beyond that I haven't run into much, other than that bootloaders need to be set up on 8192 instead of 4096 page boundaries for some unknown reason.

Erik Friesen
#8
BillP
Super Member
  • Total Posts : 292
  • Reward points : 0
  • Joined: 2014/09/28 07:53:35
  • Location: CA
  • Status: offline
Re: Bug in the USART Harmony/FreeRTOS module(?) 2019/03/04 08:52:23 (permalink)
0
Found it!  It is a Harmony bug, not FreeRTOS.  Richard Barry (FreeRTOS) doubted it was a FreeRTOS problem because the interrupt processing in FreeRTOS is a simple wrapper around the Harmony-generated code.  I reviewed his suggestions and the FreeRTOS code and I agree with him, so then I had to look at the USART driver.  I was confused because the application code ran fine in the superloop version, but locked up (not crashed) in the FreeRTOS version.  

The problem occurs in the USART interrupt/byte-mode driver.  The offending code is in the file framework/driver/usart/src/dynamic/drv_uasrt_byte_model.c.  The program hangs whenever a read or write is executed because there is some undocumented (surprise!) feature that tries to lock a mutex before executing the read or write.  I removed those mutex calls and the application works correctly.  

I am not suggesting this is the "universal" fix for the Harmony driver because I have no idea what MCHP had in mind when they put this logic in the driver.  Hopefully, they will review this and fix it in some future Harmony release.  For now, I will have to use my hack version (see below).

void DRV_USART_WriteByte( const DRV_HANDLE handle, const uint8_t byte)
{
    DRV_USART_CLIENT_OBJ * client;
    DRV_USART_OBJ * hDriver;

    /* Validate the client handle */
    client = _DRV_USART_DriverHandleValidate(handle);

    if(client == NULL)
    {
        SYS_DEBUG(0, "Invalid Driver Handle");
        return;
    }

    hDriver = client->hDriver;
    // Send one byte
    PLIB_USART_TransmitterByteSend(hDriver->moduleId, byte);
    _DRV_USART_InterruptSourceEnable(hDriver->txInterruptSource);

/***** -- v2.06 original code *****
    /* This function needs to be thread safe *

    if(OSAL_MUTEX_Lock(&(hDriver->mutexDriverInstance), OSAL_WAIT_FOREVER) == OSAL_RESULT_TRUE)
    {
        /* Send one byte
        PLIB_USART_TransmitterByteSend(hDriver->moduleId, byte);
        _DRV_USART_InterruptSourceEnable(hDriver->txInterruptSource);
        OSAL_MUTEX_Unlock(&(hDriver->mutexDriverInstance));
    }
    else
    {
        SYS_DEBUG(0, "Hardware Instance Mutex Time out in DRV_USART_WriteByte() function");
    }
****/
}

uint8_t DRV_USART_ReadByte( const DRV_HANDLE handle )
{
    DRV_USART_CLIENT_OBJ * client;
    DRV_USART_OBJ * hDriver;
    uint8_t readValue;

    /* Validate the client handle */
    client = _DRV_USART_DriverHandleValidate(handle);

    if(client == NULL)
    {
        SYS_DEBUG(0, "Invalid Driver Handle");
        return 0;
    }

    hDriver = client->hDriver;
    return PLIB_USART_ReceiverByteReceive(hDriver->moduleId);

/***** -- v2.06 original code *****

    /* This function needs to be thread safe
    if(OSAL_MUTEX_Lock(&(hDriver->mutexDriverInstance), OSAL_WAIT_FOREVER) == OSAL_RESULT_TRUE)
    {
        /* Read one byte
        readValue = PLIB_USART_ReceiverByteReceive(hDriver->moduleId);
        OSAL_MUTEX_Unlock(&(hDriver->mutexDriverInstance));
    }
    else
    {
        SYS_DEBUG(0, "Hardware Instance Mutex Time out in DRV_USART_ReadByte() function");
        return 0;
    }
 return readValue;
***/
}

#9
friesen
Super Member
  • Total Posts : 2036
  • Reward points : 0
  • Joined: 2008/05/08 05:23:35
  • Location: Indiana, USA
  • Status: offline
Re: Bug in the USART Harmony/FreeRTOS module(?) 2019/03/04 09:48:16 (permalink)
0
You really should figure out what is holding the mutex, or you'll probably end up running into some subtle bug thats rather hard to track down.

Erik Friesen
#10
friesen
Super Member
  • Total Posts : 2036
  • Reward points : 0
  • Joined: 2008/05/08 05:23:35
  • Location: Indiana, USA
  • Status: offline
Re: Bug in the USART Harmony/FreeRTOS module(?) 2019/03/04 09:51:15 (permalink)
0
However, if they are blocking on PLIB_USART_ReceiverByteReceive then..  Uh..  Do the troublesome ticket thing or move on.
 

Erik Friesen
#11
BillP
Super Member
  • Total Posts : 292
  • Reward points : 0
  • Joined: 2014/09/28 07:53:35
  • Location: CA
  • Status: offline
Re: Bug in the USART Harmony/FreeRTOS module(?) 2019/03/04 09:54:32 (permalink)
0
I agree, but since I identified the bug, I think it is up to MCHP to fix it so other subtle bugs are not created.
#12
NKurzman
A Guy on the Net
  • Total Posts : 17162
  • Reward points : 0
  • Joined: 2008/01/16 19:33:48
  • Location: 0
  • Status: offline
Re: Bug in the USART Harmony/FreeRTOS module(?) 2019/03/04 10:17:20 (permalink) ☄ Helpfulby lamdaelectronics 2019/03/05 04:32:49
5 (3)
BillP
I agree, but since I identified the bug, I think it is up to MCHP to fix it so other subtle bugs are not created.

That would be interesting to see.
The Harmony Team historically has not been interested in bug fixes. They are now on an ARM adventure. So you should insure their code is not your malfunction. The problems are yours.
#13
Jump to:
© 2019 APG vNext Commercial Version 4.5