Helpful ReplyHot!Live Update questions

Author
bblessing
Super Member
  • Total Posts : 653
  • Reward points : 0
  • Joined: 2008/12/04 06:44:21
  • Location: Cincinnati, OH
  • Status: online
2018/04/09 13:35:48 (permalink)
5 (1)

Live Update questions

Background: MPLAB X IDE v4.15, XC32 v2.05, Harmony v2.05, PIC32MZ2048EFH100
 
I should point out that I've successfully gotten the liveupdate to work on my application, complete with custom conditions to force the bootloader, etc. I am using Harmony's stock driver, though.
  • Is it true that the stock Harmony driver does not allow you to update a BFM? In other words, it only allows programming of a PFM. I get the impression that Harmony's driver does not, but hand rolled drivers can and do: http://www.microchip.com/forums/m926675.aspx
  • If the answer to the previous question is yes, then are there any plans to change this? The reason I ask is that it seems like it might be worthwhile to program both the BFM and the PFM of the unused panel at the same time through loadables. This way you could update the configuration bits relatively easily with less fear that things might lose consistency. I truly doubt that I'll have to change the configuration bit settings that often, if at all, so perhaps this isn't of that much value. Worse, it seems like it makes bricking the device easier as if the bootloader gets crapped up how do you get back? Drastically changing the configuration bits implies some serious changes anyway (at least to me), so it's probably best to just update the board through a traditional programmer in that event.
The more that I think about it, Harmony's setup actually makes a lot of sense if it can't update the BFM. The user can always update the PFM through the application, but the bootloader will be there as a fall back and it's more difficult to brick. Outside of the configuration bits, this allows for quite a bit of versatility.
#1
bblessing
Super Member
  • Total Posts : 653
  • Reward points : 0
  • Joined: 2008/12/04 06:44:21
  • Location: Cincinnati, OH
  • Status: online
Re: Live Update questions 2018/04/09 14:26:56 (permalink)
0
A potential issue that I've run into is using the bootloader in USART mode and having the web server enabled. Since a file system is involved with that, bootloader.c runs into trouble in the two places where the following condition is used to conditionally include code: SYS_FS_MAX_FILES > 0. This code is probably for USB MSD bootloaders, but is included in my case because there is nothing other than that basic check.
#2
bblessing
Super Member
  • Total Posts : 653
  • Reward points : 0
  • Joined: 2008/12/04 06:44:21
  • Location: Cincinnati, OH
  • Status: online
Re: Live Update questions 2018/04/09 15:13:31 (permalink)
0
One final thing: has anyone noticed just how SLOW the Unified Bootloader Application 0.1.8 is? Maybe it's because I'm using the UART liveupdate, but it takes like 6-8 minutes.
#3
bblessing
Super Member
  • Total Posts : 653
  • Reward points : 0
  • Joined: 2008/12/04 06:44:21
  • Location: Cincinnati, OH
  • Status: online
Re: Live Update questions 2018/04/12 12:26:05 (permalink)
0
Actually it doesn't seem like Harmony will allow you to use USB with Live Update. Can anybody verify these issues?
 

post edited by bblessing - 2018/04/12 12:30:11
#4
bblessing
Super Member
  • Total Posts : 653
  • Reward points : 0
  • Joined: 2008/12/04 06:44:21
  • Location: Cincinnati, OH
  • Status: online
Re: Live Update questions 2018/04/12 14:06:56 (permalink)
0
I must be taking crazy pills, as I removed the following line from BSP and I was allowed to eventually select something other than the USART:
 
set USE_BOOTLOADER to y
#5
bblessing
Super Member
  • Total Posts : 653
  • Reward points : 0
  • Joined: 2008/12/04 06:44:21
  • Location: Cincinnati, OH
  • Status: online
Re: Live Update questions 2018/04/16 10:18:30 (permalink)
0
Based on the speed of the USART liveupdate bootloader (115200 is the max if using the unified bootloader application 0.1.8, hereby referred to as UBA) and the fact that I'd have to either create my own application or modify the UBA, I am moving on to the UDP liveupdate bootloader. I think this will be more flexible in the long run and is most certainly faster (25 seconds or so to program). However, it is not without some issues. I'm hoping that someone from Microchip can comment on these issues.
 
The first issue is that bootloader.c has the following conditional code: SYS_FS_MAX_FILES > 0. It's in two places and apparently is only applicable to using liveupdate with USB host application. Since I use the file system with my web server, this code is triggered even if I don't use USB. This does not seem right.
 
The second issue stems from the use of the UDP liveupdate application, not the bootloader, which has been rock solid this entire time. As it currently stands, when the liveupdate application has finished programming the non-used panel, DATASTREAM_Close is called. This makes sense, but why does it shut down the entire TCPIP stack?!?! Closing the socket should suffice because the application doesn't actually reset itself. Sure the user could register a function with BOOTLOADER_ProgramCompleteRegister and have that reset the application, but what if the user made a mistake and wants to program again? I feel like there could be a little more flexibility programmed into this bootloader
 
The final issue is with the UBA itself. This thing seems only to operate in one-shot mode, probably related to the aforementioned problem in the previous paragraph.
 
Finally, since I continue to post regarding this issue, I want to thank Microchip for providing the liveupdate bootloader switcher/application framework. This has provided us, at least at this point, with a nearly unbrickable solution. At the same time, we're going to kill off using the MPFS2 application. Why would we need this if the entire application (including the web app) can be programmed into the other panel without stalling the CPU? The best part is that it ensures the client side scripting stays consistent with the server/PIC side custom_http_app.
 
EDIT: Obviously, we'll use MPFS2 to generate the correct files, but we'll no longer need to update the webpages independently from the application.
post edited by bblessing - 2018/04/16 10:29:38
#6
bblessing
Super Member
  • Total Posts : 653
  • Reward points : 0
  • Joined: 2008/12/04 06:44:21
  • Location: Cincinnati, OH
  • Status: online
Re: Live Update questions 2018/04/19 14:53:28 (permalink)
0
I promise this will be the last post on this thread, but please indulge me for a bit. I'm seeing something strange that I can't quite account for. I have two sets of firmware that are identical except that one sets a register to 4 and the other sets it to 5. These registers are accessed via a serial port and this helps me to determine which firmware I'm dealing with at a glance, kind of how the LEDs operate for the demo applications. I also display PFSWAP and BFSWAP. Both the bootloader (switcher) and the application work off of UDP as per Harmony's stack. In the application code, a function is registered via BOOTLOADER_ProgramCompleteRegister. This function simply turns off the interrupts and resets with the following logic:
 

int APP_Bootloader_ForceResetAfterProgram(void)
{
    PLIB_INT_Disable(INT_ID_0);
    SYS_RESET_SoftwareReset();
}

 
Suppose that at some point in time, I am reading 5 and PFSWAP is 1 (BFSWAP reads 0 every time, which is what I would expect). This is because the active panel contains the firmware that sets a register to 5. If I program the device with the Unified Bootloader Application and the application setting a register to 4, the reset occurs but I'm still reading 5 and 1. If I cycle power, then I'll read 4 and 0. If I now program the device with the application that sets the register to 5, the software reset occurs and I'm reading 5 and 1 WITHOUT a power cycle. I can go back and forth ad nauseam with the same results. Why would I need a power cycle when programming occurs such that PFSWAP is 1 and then 0, but not the other way around? Any suggestions?
#7
Ken_Pergola
Super Member
  • Total Posts : 2097
  • Reward points : 0
  • Joined: 2003/11/07 12:48:48
  • Status: offline
Re: Live Update questions 2018/04/19 17:25:36 (permalink)
0
bblessing

 
int APP_Bootloader_ForceResetAfterProgram(void)
{
    PLIB_INT_Disable(INT_ID_0);
    SYS_RESET_SoftwareReset();
}
 

 
 



Hello Bblessing,
 
I may be way off the mark here (in understanding the issue) but perhaps you could quickly try this for your reset code to see if there is any positive effect:
 
(I'm wondering if the Harmony reset MCU implementation is not performing the 'Prevent any unwanted code execution until the MCU reset actually occurs' snippet? Wild guess.)
 

int APP_Bootloader_ForceResetAfterProgram(void)
{
    //PLIB_INT_Disable(INT_ID_0);
    //SYS_RESET_SoftwareReset();
    
    uint32_t dummyRead = 0;

    // Disable interrupts
    __builtin_disable_interrupts();

    // TODO: Suspend DMA here if we ever use DMA in the future

    // The system unlock sequence must be performed before the
    // 'SWRST' bit can be written.
    SYSKEY = 0x00000000;
    SYSKEY = 0xAA996655;
    SYSKEY = 0x556699AA;

    // Prepare for the software-reset of the microcontroller
    RSWRSTSET = 1;

    // Once the 'SWRST' bit is set, any read of the 'RSWRST' register
    // will trigger the reset.
    dummyRead = RSWRST;

    // Prevent any unwanted code execution until the MCU reset actually occurs
    while(1)
    {
        dummyRead++;
    }    
}

 
 
 
This is the reset code I was using for the Bootloader I worked on:
 

static void SoftwareReset(void)
{
    uint32_t dummyRead = 0;

    // Disable interrupts
    __builtin_disable_interrupts();

    // TODO: Suspend DMA here if we ever use DMA in the future

    // The system unlock sequence must be performed before the
    // 'SWRST' bit can be written.
    SYSKEY = 0x00000000;
    SYSKEY = 0xAA996655;
    SYSKEY = 0x556699AA;

    // Prepare for the software-reset of the microcontroller
    RSWRSTSET = 1;

    // Once the 'SWRST' bit is set, any read of the 'RSWRST' register
    // will trigger the reset.
    dummyRead = RSWRST;

    // Prevent any unwanted code execution until the MCU reset actually occurs
    while(1)
    {
        dummyRead++;
    }
}

 
 
#8
bblessing
Super Member
  • Total Posts : 653
  • Reward points : 0
  • Joined: 2008/12/04 06:44:21
  • Location: Cincinnati, OH
  • Status: online
Re: Live Update questions 2018/04/20 06:58:41 (permalink)
0
Ken,
 
    Thanks for the reply! I gave it a shot, but the problem remains. It's just very strange: if PFSWAP is 0 and I program the other panel through the UBA 0.1.8 application, the device resets and PFSWAP shows as 1. If I do this again, the reset occurs, but PFSWAP is still 1. HOWEVER, if I cycle power at any point, PFSWAP will reflect the value that I would expect.
 
    Looking at the effects of a reset in the reference manual (section 52, live update), I see this tidbit:
 
Devices Resets other than a Power-on Reset (POR), reset the SWAP (or PFSWAP) bit in the
NVMCON register and the entire contents of the NVMPWP and NVMBWP registers (see Note).
All other register content persists through a non-POR reset.

 
    This seems like a lead. What I'm ultimately trying to accomplish is that after the device is programmed it resets, thus allowing the user to work with the new firmware. The user may not always be able to physically cycle power, as using UDP will allow the user to update a device, if need be, from the comfort of their office, as opposed to a hot, dirty, and noisy heat treating facility.
 
    As it currently stands, this only works when PFSWAP is 0 going to 1, not 1 going to 0. I think the initial decision in the bootloader as to whether the panels should be swapped is where I should look.
 
#9
bblessing
Super Member
  • Total Posts : 653
  • Reward points : 0
  • Joined: 2008/12/04 06:44:21
  • Location: Cincinnati, OH
  • Status: online
Re: Live Update questions 2018/04/20 10:06:14 (permalink)
5 (1)
I think Microchip needs to take a look at this. Even though I've found a solution to my problem, I believe it to be a pretty crappy fix because it violates a comment in the liveupdate reference (regarding the PFSWAP bit):
 
 
It is recommended that this bit is only changed by the code executing from the BFM region.

 
My code to fix the issue is this:
 
int APP_Bootloader_ForceResetAfterProgram(void)
{
    PLIB_INT_Disable(INT_ID_0);
    PLIB_NVM_MemoryModifyInhibit(NVM_ID_0);
    PLIB_NVM_MemoryOperationSelect(NVM_ID_0,NO_OPERATION);
    PLIB_NVM_FlashSwapLockSelect(NVM_ID_0,NVM_FLASH_SWAP_UNLOCKED);
    if (NVMCONbits.PFSWAP)
    {
        PLIB_NVM_FlashWriteKeySequence(NVM_ID_0, NVM_PROGRAM_UNLOCK_KEY1);
        PLIB_NVM_FlashWriteKeySequence(NVM_ID_0, NVM_PROGRAM_UNLOCK_KEY2);
        PLIB_NVM_ProgramFlashBank1LowerRegion(NVM_ID_0);
    }
    SYS_RESET_SoftwareReset();
    while (1);
}

 
I can now use software resets and any time that I update the code, the reset occurs, and the panel swap occurs correctly. Still, I don't like the fact that I'm updating the PFSWAP bit from the PFM. I'm also trying to understand exactly WHY this works. There has to be something that is persisting from a software reset that screws with the panel swapping on startup in the bootloader.
 
The example applications that Microchip provides all operate under the assumption that to get to the most recently programmed panel, a power cycle must occur. Unfortunately, many of us need to be able to trigger software resets in order to see the updated firmware.
post edited by bblessing - 2018/04/20 10:07:17
#10
bblessing
Super Member
  • Total Posts : 653
  • Reward points : 0
  • Joined: 2008/12/04 06:44:21
  • Location: Cincinnati, OH
  • Status: online
Re: Live Update questions 2018/04/22 09:39:55 (permalink)
0
This doesn't make any sense at all. It was my understanding that PFSWAP DID NOT SURVIVE A RESET UNDER ANY CIRCUMSTANCES. That is, it would power up as 0. I may have to post the crazy pills gif again because it seems like it can power up as 1, or there is something I simply don't understand.
 
I ended up editing bootloader.c, since I didn't like the fix above and the ultimate decision as to what panel should be used belongs in the bootloader. Armed with the possibility that PFSWAP could actually be 1 by the time the bootloader enters EnterApplication, I changed the SwapFlashPanels and EnterApplication functions. Basically I use PLIB_NVM_ProgramFlashBank1LowerRegion, PLIB_NVM_ProgramFlashBank2LowerRegion, and PLIB_NVM_ProgramFlashBank2IsLowerRegion, in addition to which flash id is larger, to make the decision as to how to swap the panels. My code is below. This seems to work.
 
If PFSWAP can indeed be 1 after a software reset, could someone please reference section 52.17 of the flash memory section and explain to me what I'm missing? It really seems, after reading, that PFSWAP is ALWAYS reset after any reset.
 

#if(BOOTLOADER_LIVE_UPDATE_SWITCHER == 1)
/********************************************************************
* Function:     SwapFlashPanels()
*
* Precondition:
*
* Input:        Boolean indicating whether the switch should go to panel 2
*
* Output:
*
* Side Effects:
*
* Overview:     The function will swap the flash panels in a safe
 *              place.
*
*
* Note:         This is placed in the .cache_init section in order
 *              to have it in boot flash rather than program flash.
********************************************************************/
void __longcall__ __attribute__ ((section (".cache_init"))) SwapFlashPanels(bool panel2)
{
    PLIB_NVM_MemoryModifyInhibit(NVM_ID_0);
    PLIB_NVM_MemoryOperationSelect(NVM_ID_0,NO_OPERATION);
    PLIB_NVM_FlashSwapLockSelect(NVM_ID_0,NVM_FLASH_SWAP_UNLOCKED);
    PLIB_NVM_FlashWriteKeySequence(NVM_ID_0, NVM_PROGRAM_UNLOCK_KEY1);
    PLIB_NVM_FlashWriteKeySequence(NVM_ID_0, NVM_PROGRAM_UNLOCK_KEY2);
    if (panel2)
    {
        PLIB_NVM_ProgramFlashBank2LowerRegion(NVM_ID_0);
    }
    else
    {
        PLIB_NVM_ProgramFlashBank1LowerRegion(NVM_ID_0);
    }
}
#endif

#if(BOOTLOADER_LIVE_UPDATE_STATE_SAVE != 1)
/********************************************************************
* Function:     Enter_Application()
*
* Precondition:
*
* Input:        None.
*
* Output:
*
* Side Effects: No return from here.
*
* Overview:     The function will program the checksum for the respective
 *              flash panel and jumps to application programmed in it.
*
*
* Note:
********************************************************************/
static void Enter_Application(void)
{
    void (*fptr)(void);

    /* Set default to APP_RESET_ADDRESS */
    fptr = (void (*)(void))APP_RESET_ADDRESS;

#if(BOOTLOADER_LIVE_UPDATE_SWITCHER == 1)
    T_FLASH_ID *lower_flash_id = LOWER_FLASH_ID_READ;
    T_FLASH_ID *upper_flash_id = UPPER_FLASH_ID_READ;

    SYS_DEVCON_InstructionCacheFlush();

    /* Application has been programmed first time into flash panel 1 by the bootloader */
    if( *(uint32_t *)LOWER_FLASH_ID_READ == FLASH_ID_CHECKSUM_CLR &&
        *(uint32_t *)UPPER_FLASH_ID_READ == FLASH_ID_CHECKSUM_CLR)
    {
        /* Program Checksum and initial ID's for both panels*/
        T_FLASH_ID low_flash_id = { 0 };
        T_FLASH_ID up_flash_id = { 0 };
 
        low_flash_id.flash_id = 1;
        low_flash_id.checksum_start = FLASH_ID_CHECKSUM_START;
        low_flash_id.checksum_end = FLASH_ID_CHECKSUM_END;

        up_flash_id.flash_id = 0;
        up_flash_id.checksum_start = FLASH_ID_CHECKSUM_START;
        up_flash_id.checksum_end = FLASH_ID_CHECKSUM_END;
        
        APP_NVMQuadWordWrite((void *)LOWER_FLASH_ID_BASE_ADDRESS, (uint32_t *)&low_flash_id);
        APP_NVMQuadWordWrite((void *)UPPER_FLASH_ID_BASE_ADDRESS, (uint32_t *)&up_flash_id);
    }
    /* If both the panels have proper checksum*/
    else if((lower_flash_id->checksum_start == FLASH_ID_CHECKSUM_START) &&
            (lower_flash_id->checksum_end == FLASH_ID_CHECKSUM_END) &&
            (upper_flash_id->checksum_start == FLASH_ID_CHECKSUM_START) &&
            (upper_flash_id->checksum_end == FLASH_ID_CHECKSUM_END))
    {
        if (upper_flash_id->flash_id > lower_flash_id->flash_id)
            SwapFlashPanels(!PLIB_NVM_ProgramFlashBank2IsLowerRegion(NVM_ID_0));
    }
    /* Fallback Cases when either of Panels checksum is corrupted*/
    else if((upper_flash_id->checksum_start == FLASH_ID_CHECKSUM_START) &&
            (upper_flash_id->checksum_end == FLASH_ID_CHECKSUM_END))
    {
        SwapFlashPanels(true);
    }
    /* Fallback Cases when either of Panels checksum is corrupted*/
    else if((lower_flash_id->checksum_start == FLASH_ID_CHECKSUM_START) &&
            (lower_flash_id->checksum_end == FLASH_ID_CHECKSUM_END))
    {
        SwapFlashPanels(false);
    }
#endif // BOOTLOADER_LIVE_UPDATE_SWITCHER == 1
    /* Disable Global Interrupts and Jump to Application*/
    PLIB_INT_Disable(INT_ID_0);
    if (bootloaderData.StartAppFunc != NULL)
        bootloaderData.StartAppFunc();
    fptr();
}
#endif // BOOTLOADER_LIVE_UPDATE_STATE_SAVE != 1

#11
Ken_Pergola
Super Member
  • Total Posts : 2097
  • Reward points : 0
  • Joined: 2003/11/07 12:48:48
  • Status: offline
Re: Live Update questions 2018/04/22 13:58:58 (permalink) ☄ Helpfulby bblessing 2018/04/22 17:10:54
5 (1)
Hello Bblessing,
 
bblessing
...It was my understanding that PFSWAP DID NOT SURVIVE A RESET UNDER ANY CIRCUMSTANCES. That is, it would power up as 0...

 
For the PIC32MZ2048EFH100 part you are using Data Sheet DS60001320 revision D, page 100 indicates:
That the nybble that contains 'PWSWAP' and 'BFSWAP' will contain an unknown value on reset (all resets).
 
(My gut feeling is perhaps this could be a documentation mistake. I would expect that this would be the case only on a POR (Power-On Reset). I would expect the 'PWSWAP' and 'BFSWAP' values to persist anyreset except a POR.)
 
I think this is a case in which you might have to ignore the documentation (for now) and run some test cases on the actual silicon to see how it behaves.
 
IMPORTANT: If you run some tests, just make sure that you you verify the 'PWSWAP' after you write to it (after unlocking) to ensure it actually changed to what you expect it to be -- you don't want to get fooled in thinking the change occurred in the case if it is locked and is not changeable.
 

...If PFSWAP can indeed be 1 after a software reset, could someone please reference section 52.17 of the flash memory section and explain to me what I'm missing? It really seems, after reading, that PFSWAP is ALWAYS reset after any reset.



Regarding '52.17 EFFECTS OF VARIOUS RESETS' (Section 52. Flash Memory with Support for Live Update, DS60001193 revision B-page 32):
 
My interpretation of this section is this (which unfortunately conflicts with the Data Sheet):
1) 'PWSWAP' is '0' for a POR (Power-On Reset).
2) 'PWSWAP' persists its preset state on any reset except a POR.
 
My advice (that I personally follow) would be to learn towards the veracity of the Data Sheet over the Reference Manual information. This is not an excuse, however, both should be crystal clear and exactly match so there is no customer confusion.
 
Bblessing, I think it would not be a bad idea to open up a Microchip Support ticket so that this can be cleared up/rectified by the responsible parties. The documentation between the Data Sheet and the Reference Manual should not be ambiguous like it currently is.
 
Reset state aside, ultimately the Reference Manual hints at the Boot Flash Memory code having the onus to configure the 'PWSWAP' to the desired/intended state:  
 
Section 52. Flash Memory with Support for Live Update, DS60001193 revision B-page 19:
"It is recommended that this bit is only changed by the code executing from the BFM region."
 
 
Does your code executing in the active Boot Flash Memory have a algorithm/strategy to regress to the older Application firmware image if the new Application firmware image (the one you want to run) is not a valid image (i.e., corrupted)?
 
I have not implemented Live Update (only Dual Boot) but it seems the "buck stops" at the Boot Flash Memory code to validate the new (desired to run) Application firmware image to determine whether to configure 'PWSWAP' to run from the new Application firmware image or fall back and run the older Application firmware image if it is deemed that the new Application firmware image is not "runnable". It's the final chance to catch and prevent a mishap.
 
So in this sense the reset value of 'PWSWAP' is immaterial because Boot Flash Memory code will always explicitly configure it to a '0' or a '1' before jumping to the Application space.
 
I hope this post helps somewhat.
 
Best regards,
 
Ken
 
P.S. Again, I have not implemented a Live Update (only Dual Boot) project in practice so I may be off base here in my advice. These are just some ideas I wanted to run past you to try to help you out.
post edited by Ken_Pergola - 2018/04/22 15:03:01
#12
Ken_Pergola
Super Member
  • Total Posts : 2097
  • Reward points : 0
  • Joined: 2003/11/07 12:48:48
  • Status: offline
Re: Live Update questions 2018/04/22 14:59:05 (permalink)
0
Bblessing,
 
It is interesting to note that, unlike the PIC32MZ (EF) Family, the newer PIC32MZ Graphics (DA) Family indicates in its Data Sheet (DS60001361 revision F, page 112) that the 'PWSWAP' and 'BFSWAP' bit will be '0' on all resets (POR-inclusive).
 
Again, the reset value is immaterial if my previous post stands on solid ground.
 
It still would not be a bad idea to open up a Microchip Support ticket so that the PIC32MZ EF Family Data Sheet can be fixed by the responsible parties if it is indeed contains incorrect information in this regard.
 
Best regards,
 
Ken
 
#13
bblessing
Super Member
  • Total Posts : 653
  • Reward points : 0
  • Joined: 2008/12/04 06:44:21
  • Location: Cincinnati, OH
  • Status: online
Re: Live Update questions 2018/04/22 17:08:45 (permalink)
0
Ken,
 
    I can't thank you enough for your nuanced replies! I missed that regarding the datasheet, and I'll definitely open a ticket.
 
    You can review the code I'm working from any time, as it's Harmony 2.05's stock liveupdate driver using UDP. It never looks at the PFSWAP bit, but compares the high and low flash "ID": each panel has it's ID incremented by two after successful programming.
    As for my setup, I have a dispswitch that forces the bootloader from both the application and within the bootloader. This way if everything gets gummed up, the user can flip the switch and get back to the bootloader. If the application is hosed, a power cycle may be necessary, but it's still possible to hop back to the bootloader. This part has worked flawlessly and I've yet to brick my board.
    The problem that I'm seeing is the following: suppose I program the bootloader from my Real ICE. If I then program the device through the bootloader, panel 1 is programmed and the device is running out of panel 1. Program the device again, and the programming occurs in panel 2. Resetting the device gets the device to run out of panel 2. If I now I program the device a third time, the programming occurs in panel 1. However, if I reset the device, it will still run out of panel 2.
    I should note for the previous paragraph, all resets are software resets in the application code. IF I perform a power cycle after each programming step, the device boots up to the correct firmware and panel every time without fail, at least from my observation. There is just some difference between the software resets and actual power cycles that the bootloader, which, in Harmony, does perform the "switching" operation, doesn't seem to account for. It's very bizarre, as Microchip's code is simple and the logic is sound. I think I'll do a bit of a deeper dive tomorrow with a fresh mind.
    Again, I can't thank you enough for your help!
#14
Ken_Pergola
Super Member
  • Total Posts : 2097
  • Reward points : 0
  • Joined: 2003/11/07 12:48:48
  • Status: offline
Re: Live Update questions 2018/04/22 18:31:55 (permalink)
0
 
Hello Bblessing.
 
You are very welcome. It sure will be nice to know what the root cause is. It will be a thrill for you when you are able to overcome this issue! I'll try my best to look at the code a bit more in detail when I am able to see if any red flags pop up. My time is kind of fragmented right now.
 
bblessing
 ...As for my setup, I have a dispswitch that forces the bootloader from both the application and within the bootloader. This way if everything gets gummed up, the user can flip the switch and get back to the bootloader. If the application is hosed, a power cycle may be necessary, but it's still possible to hop back to the bootloader. This part has worked flawlessly and I've yet to brick my board.

 
Nice job! You have a nice recovery plan there.
 
bblessing
...However, if I reset the device, it will still run out of panel 2...
...IF I perform a power cycle after each programming step, the device boots up to the correct firmware and panel every time without fail, at least from my observation. There is just some difference between the software resets and actual power cycles that the bootloader, which, in Harmony, does perform the "switching" operation, doesn't seem to account for...

 
Beyond some coding oversight that has not been found yet:
 
I'm grasping for straws a bit here but I am wondering if this might be a Data- or Instruction-cache coherency issue that is manifesting itself as the behavior you are seeing. It's worth throwing the question out there to rule it out. I'm not sure what the easiest/quickest way would be to try an experiment to turn off the Data and Instruction caches to see if this has a positive effect or not. You would have to force all data/instruction accesses from uncached KSEG1 address regions. I'm not a MIPS expert (it's been a while and I'm working with an ARM core lately) so maybe someone in the Forum Community can help out. I'd have to research this myself a bit -- I believe this can be done with XC32 attributes on a function basis but maybe this could be done easier and globally in the linker script itself. Or perhaps in the C startup code? It might be that you simply just have to change the cache policy to 'uncached' (I have not done this in practice so I would have to research this). Again, this would be a temporary modification as an experiment (as a goal to see if this is a cache coherency issue or perhaps it would lead to the a different root cause).
 
Bblessing, keep digging in and keep us posted on your findings. You've done a great job thus far -- you are very close!
 
Best regards,
 
Ken
post edited by Ken_Pergola - 2018/04/22 18:40:01
#15
Ken_Pergola
Super Member
  • Total Posts : 2097
  • Reward points : 0
  • Joined: 2003/11/07 12:48:48
  • Status: offline
Re: Live Update questions 2018/04/22 21:14:56 (permalink) ☄ Helpfulby bblessing 2018/04/23 07:12:29
4.67 (3)
A Forum Member in another thread posted this link which I am re-posting here. 
 
Completely disable cache
http://microchipdeveloper.com/32bit:mz-cache-disable
 
This should be the easiest way to disable the Data and Instruction caches completely to see if you might have a cache coherency issue.
 
Please note I am not implying that this is the cause or the solution -- just a debugging experimental data point that may help to identify the actual root cause so that a proper fix can be applied.
 
#16
bblessing
Super Member
  • Total Posts : 653
  • Reward points : 0
  • Joined: 2008/12/04 06:44:21
  • Location: Cincinnati, OH
  • Status: online
Re: Live Update questions 2018/04/23 08:32:02 (permalink)
0
So this will probably add more questions than answers, but here goes. I have been absolutely able to prove that some times PFSWAP will start up as 1. Ken's point about the datasheet stating that the initial value for PFSWAP being undefined got me thinking about my approach. Couple that with the MZ DA chips guaranteeing that PFSWAP is 0 at startup, I figured it would be better to FORCE PFSWAP to 0 on startup.
 
Anyway, for proof, I put this code in APP_Initialize:
 

if (PLIB_NVM_ProgramFlashBank2IsLowerRegion(NVM_ID_0))
    while (1);

 
This code will execute in SYS_Initialize and will occur before any calls to Bootloader_Tasks, which contains a call to Enter_Application: the only function that calls SwapFlashPanels, and thus the only function that calls PLIB_NVM_ProgramFlashBank2LowerRegion.
 
Sure enough the program would lock up in the while loop. This raises the following question: if PFSWAP is 1 at startup, what does this do to the following section of code in Enter_Application:
 

T_FLASH_ID *lower_flash_id = LOWER_FLASH_ID_READ;
T_FLASH_ID *upper_flash_id = UPPER_FLASH_ID_READ;

 
The macros expand to this (LOWER_FLASH_ID_BASE_ADDRESS = 0x9D000000 in my case):
 

#define LOWER_FLASH_ID_READ   ((T_FLASH_ID *)KVA0_TO_KVA1(LOWER_FLASH_ID_BASE_ADDRESS))
#define UPPER_FLASH_ID_READ   ((T_FLASH_ID *)KVA0_TO_KVA1(UPPER_FLASH_ID_BASE_ADDRESS))

 
The decision to swap panels in Enter_Application is based on the following:
 

if(upper_flash_id->flash_id > lower_flash_id->flash_id)
{
    SwapFlashPanels(true);
}

 
Can we conclude that, given the state of PFSWAP, that lower_flash_id and upper_flash_id are reading from the same spot in both cases? I don't know that we can, but I'm also not an expert regarding the virtual to physical macros.
 
If this poses a problem then the fix is simple: swap the panels such that PFSWAP is 0, as it would be on a power cycle. The beauty of this solution is this: if it's not a problem then the code will never be executed, and if it is then we know that PFSWAP will be in the correct state before any flash panel manipulation. Thus Microchip's panel switching logic in Enter_Application, which works very well as long as a power cycle is used, has the correct logic and need not be modified.
 
Here is what I have started using without issues; my above attempt didn't work as well (as I found this morning) because I think having PFSWAP set on start up screws with the flash ID logic. I modified the SwapFlashPanels function (similar to a previous post), Enter_Application (to reflect that SwapFlashPanels now takes a boolean argument), and Bootloader_Initialize (now contains a simple if statement that swaps the panels if PFSWAP is 1).
 
Thanks again Ken!
 

#if(BOOTLOADER_LIVE_UPDATE_SWITCHER == 1)
/********************************************************************
* Function:     SwapFlashPanels()
*
* Precondition:
*
* Input:        Boolean indicating Program Flash Bank 2 is to be mapped
*               to the lower mapped region.
*
* Output:
*
* Side Effects:
*
* Overview:     The function will swap the flash panels in a safe
 *              place.
*
*
* Note:         This is placed in the .cache_init section in order
 *              to have it in boot flash rather than program flash.
********************************************************************/
void __longcall__ __attribute__ ((section (".cache_init"))) SwapFlashPanels(bool panel2)
{
    PLIB_NVM_MemoryModifyInhibit(NVM_ID_0);
    PLIB_NVM_FlashWriteKeySequence(NVM_ID_0, NVM_PROGRAM_UNLOCK_KEY1);
    PLIB_NVM_FlashWriteKeySequence(NVM_ID_0, NVM_PROGRAM_UNLOCK_KEY2);
    if (panel2)
    {
        PLIB_NVM_ProgramFlashBank2LowerRegion(NVM_ID_0);
    }
    else
    {
        PLIB_NVM_ProgramFlashBank1LowerRegion(NVM_ID_0);
    }
}
#endif

#if(BOOTLOADER_LIVE_UPDATE_STATE_SAVE != 1)
/********************************************************************
* Function:     Enter_Application()
*
* Precondition:
*
* Input:        None.
*
* Output:
*
* Side Effects: No return from here.
*
* Overview:     The function will program the checksum for the respective
 *              flash panel and jumps to application programmed in it.
*
*
* Note:
********************************************************************/
static void Enter_Application(void)
{
    void (*fptr)(void);

    /* Set default to APP_RESET_ADDRESS */
    fptr = (void (*)(void))APP_RESET_ADDRESS;

#if(BOOTLOADER_LIVE_UPDATE_SWITCHER == 1)
    T_FLASH_ID *lower_flash_id = LOWER_FLASH_ID_READ;
    T_FLASH_ID *upper_flash_id = UPPER_FLASH_ID_READ;

    SYS_DEVCON_InstructionCacheFlush();

    /* Application has been programmed first time into flash panel 1 by the bootloader */
    if( *(uint32_t *)LOWER_FLASH_ID_READ == FLASH_ID_CHECKSUM_CLR &&
        *(uint32_t *)UPPER_FLASH_ID_READ == FLASH_ID_CHECKSUM_CLR)
    {
        /* Program Checksum and initial ID's for both panels*/
        T_FLASH_ID low_flash_id = { 0 };
        T_FLASH_ID up_flash_id = { 0 };
 
        low_flash_id.flash_id = 1;
        low_flash_id.checksum_start = FLASH_ID_CHECKSUM_START;
        low_flash_id.checksum_end = FLASH_ID_CHECKSUM_END;

        up_flash_id.flash_id = 0;
        up_flash_id.checksum_start = FLASH_ID_CHECKSUM_START;
        up_flash_id.checksum_end = FLASH_ID_CHECKSUM_END;
        
        APP_NVMQuadWordWrite((void *)LOWER_FLASH_ID_BASE_ADDRESS, (uint32_t *)&low_flash_id);
        APP_NVMQuadWordWrite((void *)UPPER_FLASH_ID_BASE_ADDRESS, (uint32_t *)&up_flash_id);
    }
    /* If both the panels have proper checksum*/
    else if((lower_flash_id->checksum_start == FLASH_ID_CHECKSUM_START) &&
            (lower_flash_id->checksum_end == FLASH_ID_CHECKSUM_END) &&
            (upper_flash_id->checksum_start == FLASH_ID_CHECKSUM_START) &&
            (upper_flash_id->checksum_end == FLASH_ID_CHECKSUM_END))
    {
        if(upper_flash_id->flash_id > lower_flash_id->flash_id)
        {
            SwapFlashPanels(true);
        }
    }
    /* Fallback Cases when either of Panels checksum is corrupted*/
    else if((upper_flash_id->checksum_start == FLASH_ID_CHECKSUM_START) &&
            (upper_flash_id->checksum_end == FLASH_ID_CHECKSUM_END))
    {
        SwapFlashPanels(true);
    }
#endif // BOOTLOADER_LIVE_UPDATE_SWITCHER == 1
    /* Disable Global Interrupts and Jump to Application*/
    PLIB_INT_Disable(INT_ID_0);
    if (bootloaderData.StartAppFunc != NULL)
        bootloaderData.StartAppFunc();
    fptr();
}
#endif // BOOTLOADER_LIVE_UPDATE_STATE_SAVE != 1
 
/******************************************************************************
  Function:
    SYS_MODULE_OBJ Bootloader_Initialize(const SYS_MODULE_INDEX   moduleIndex,
                              const SYS_MODULE_INIT    * const moduleInit)

  Summary:
    Initializes primitive data structures for the general features
    of the primitive layer.

  Description:
    Initializes external and internal data structure for the general
    features of the primitive layer.

    This function must be called at system initialization.

  Remarks:
    None.
*/
void Bootloader_Initialize ( const BOOTLOADER_INIT *drvBootloaderInit )
{
    #if(BOOTLOADER_LIVE_UPDATE_SWITCHER == 1)
    if (PLIB_NVM_ProgramFlashBank2IsLowerRegion(NVM_ID_0))
    {
        SwapFlashPanels(false);
    }
    #endif
    /* Place the App state machine in it's initial state. */
    bootloaderData.currentState = BOOTLOADER_CHECK_FOR_TRIGGER;
    bootloaderData.cmdBufferLength = 0;
    bootloaderData.streamHandle = DRV_HANDLE_INVALID;
    bootloaderData.datastreamStatus = DRV_CLIENT_STATUS_ERROR;
    bootloaderData.usrBufferEventComplete = false;

    bootloaderData.data = &data_buff;
    bootloaderData.type = drvBootloaderInit->drvType;
#ifdef BOOTLOADER_LEGACY
    BootloaderTriggerCheck = drvBootloaderInit->drvTrigger;
#endif
    
    bootloaderData.FlashEraseFunc = (BOOTLOADER_CALLBACK)NULL;
    bootloaderData.StartAppFunc = (BOOTLOADER_CALLBACK)NULL;
    bootloaderData.BlankCheckFunc = (BOOTLOADER_CALLBACK)NULL;
    bootloaderData.ProgramCompleteFunc = (BOOTLOADER_CALLBACK)NULL;
    bootloaderData.ForceBootloadFunc = (BOOTLOADER_CALLBACK)NULL;
    bootloaderData.softReset = (SYS_RESET_ReasonGet() & RESET_REASON_SOFTWARE) == RESET_REASON_SOFTWARE;
#if(BOOTLOADER_LIVE_UPDATE_STATE_SAVE != 1)
    SYS_RESET_ReasonClear(RESET_REASON_SOFTWARE);
    /* Delay to allow the internal pullups to stabilize */
    _CP0_SET_COUNT(0);
    while (_CP0_GET_COUNT() < SYS_CLK_FREQ / 5000);
#endif
}

 
 
#17
bblessing
Super Member
  • Total Posts : 653
  • Reward points : 0
  • Joined: 2008/12/04 06:44:21
  • Location: Cincinnati, OH
  • Status: online
Re: Live Update questions 2018/04/23 09:04:20 (permalink)
0
Actually, now that I reflect on it, there is no need to edit bootloader.c at all: just make sure that program flash panel 1 is in the lower region by forcing it in the function registered with BOOTLOADER_ForceBootloadRegister. This way, the stack need not be modified.
 

int APP_Bootloader_ForceEvent(void)
{
    /* Ensure that PFSWAP is actually 0 on startup*/
    if (PLIB_NVM_ProgramFlashBank2IsLowerRegion(NVM_ID_0))
    {
        PLIB_NVM_MemoryModifyInhibit(NVM_ID_0);
        PLIB_NVM_FlashWriteKeySequence(NVM_ID_0, NVM_PROGRAM_UNLOCK_KEY1);
        PLIB_NVM_FlashWriteKeySequence(NVM_ID_0, NVM_PROGRAM_UNLOCK_KEY2);
        PLIB_NVM_ProgramFlashBank1LowerRegion(NVM_ID_0);
    }
    
    /* Check the switch press to trigger bootloader */
    if (BSP_SWITCH_STATE_PRESSED == BSP_SwitchStateGet(BTL_TRIGGER_SWITCH))
    {
        return (1);
    }

    /* Check the trigger memory location and return true/false. */
    if (*(uint32_t *)APP_RESET_ADDRESS == 0xFFFFFFFF)
    {
        return (1);
    }
    
    return (0);
}

post edited by bblessing - 2018/04/23 09:11:01
#18
Jump to:
© 2018 APG vNext Commercial Version 4.5