Hot!USB flash drive/ file system writing issues since v2.03b

Author
Digital_Angel
Starting Member
  • Total Posts : 37
  • Reward points : 0
  • Joined: 2014/08/27 06:56:45
  • Location: Northampton, UK
  • Status: offline
2017/07/18 04:01:15 (permalink)
0

USB flash drive/ file system writing issues since v2.03b

I have a project using a PIC32MX795F512L that buffers MP3 audio it receives from an encoder and then writes it to a USB flash drive using a FAT file system, it repeats this up to a fixed amount of data for proof of concept.
 
Running the code on a custom board was successful, with a valid .mp3 being created on the USB media, until I changed flash drives. The newer ones were the same capacity, and have a faster write speed, but for some reason the data transfer to the drives when I run the code is much slower, and chunks of audio are lost as a result of the buffer overflowing.
The original flash drive is always fine, but the newer ones are sometimes lossy throughout the fixed amount of data transfer, and sometimes they sort themselves out, and the later part of the data transfer becomes flawless (all drives are Kingston DataTravelers)
 
Since migrating the project to Harmony v2.03b, this undesirable behaviour is now also apparent in the flash drive that always used to be ok, and this is 100% of the time.
Can anyone advise on where I should start looking for a solution to this problem? It was one thing when it occurred on some flash drives and not others, but it was consistent, and now to suddenly have an issue with the previously functioning drive there must be something in the software that would account for the issue.
 
I believe that this project was initially based on one of the Harmony file system example projects.
 
Please excuse the vagueness in my description, my current project was started by another engineer and I have not needed to delve into this portion of his code yet, so I am unfamiliar with the exact details of how the data transfer is done.
#1

12 Replies Related Threads

    Tez
    Moderator
    • Total Posts : 462
    • Reward points : 0
    • Joined: 2006/10/04 11:09:05
    • Location: 0
    • Status: offline
    Re: USB flash drive/ file system writing issues since v2.03b 2017/07/19 20:11:05 (permalink)
    3 (1)
    USB Pen Drives when operating at Full Speed (you are using the PIC32MX device which has full speed USB) are do not demonstrate the best write throughput. If you are looking for write throughput, you should try using High-Speed USB (checkout the PIC32MZ). 
     
    >> The newer ones were the same capacity, and have a faster write speed, but for some reason, the data transfer to the drives when I run the code is much slower, and chunks of audio are lost as a result of the buffer overflow.
     
    How did you confirm that the USB  speed? Did you try it out with a PC USB Host? If so, then this is not an apples to apples comparison. 
     
    Which version of Harmony were you using before v2.03?
    #2
    Digital_Angel
    Starting Member
    • Total Posts : 37
    • Reward points : 0
    • Joined: 2014/08/27 06:56:45
    • Location: Northampton, UK
    • Status: offline
    Re: USB flash drive/ file system writing issues since v2.03b 2017/07/24 06:48:52 (permalink)
    0
    I did indeed confirm write speed using a PC application, although all it really did was confirm my suspicions as to how different the device speeds were, as one drive has been around for a few years, and the others were purchased only a couple of months ago. The application suggested that the newer ones were about 3 x the speed of the older one.
     
    The last version of Harmony that wrote the mp3 successfully to the USB flash drive was v2.02b, so I believe that a change between that and v2.03b is likely responsible for the failings I'm now seeing in the data transfer, as I am using the same hardware (I have tried the 2.02 code again just now on the same PCB to be sure that the mp3 audio was solid with that version and the hardware I am using).
    #3
    Tez
    Moderator
    • Total Posts : 462
    • Reward points : 0
    • Joined: 2006/10/04 11:09:05
    • Location: 0
    • Status: offline
    Re: USB flash drive/ file system writing issues since v2.03b 2017/07/25 23:05:15 (permalink)
    3 (1)
    Are the newer pen drives working with v2.03b? Do you have USB analyzer with you? 
    #4
    Digital_Angel
    Starting Member
    • Total Posts : 37
    • Reward points : 0
    • Joined: 2014/08/27 06:56:45
    • Location: Northampton, UK
    • Status: offline
    Re: USB flash drive/ file system writing issues since v2.03b 2017/07/26 04:21:05 (permalink)
    0
    They are not, none of the sticks work with v2.03b.
    I don't have a USB analyzer unfortunately.
    #5
    RISC
    Super Member
    • Total Posts : 4593
    • Reward points : 0
    • Status: offline
    Re: USB flash drive/ file system writing issues since v2.03b 2017/07/26 04:24:14 (permalink)
    3 (1)
    Hi,
    Digital_Angel
    They are not, none of the sticks work with v2.03b.

     
    None of the USB stickes work on your board with Harmony v2.03b but I tested 2 different ones on Microchip PIC32MZ EF starter kit with the Harmony demos and both worked out of the box (writing a small file only)
    Regards
     
    #6
    Digital_Angel
    Starting Member
    • Total Posts : 37
    • Reward points : 0
    • Joined: 2014/08/27 06:56:45
    • Location: Northampton, UK
    • Status: offline
    Re: USB flash drive/ file system writing issues since v2.03b 2017/07/26 04:30:31 (permalink)
    0
    To clarify, I can write files to the USB sticks that can be read back on a Windows PC fine, it's just that the audio data is missing chunks compared to when I've used Harmony v2.02b.
    #7
    Digital_Angel
    Starting Member
    • Total Posts : 37
    • Reward points : 0
    • Joined: 2014/08/27 06:56:45
    • Location: Northampton, UK
    • Status: offline
    Re: USB flash drive/ file system writing issues since v2.03b 2017/09/20 07:55:38 (permalink)
    0
    I was hoping that v2.04 might have fixed this for me, but it hasn't.
    Does anyone have any idea what changed between v2.02 and v2.03 on either the USB or SPI drivers that could have caused this. It's critical to my project development, and the couple of Microchip support tickets that I've filed on it seem to have been completely ignored! sad: sad
    #8
    RISC
    Super Member
    • Total Posts : 4593
    • Reward points : 0
    • Status: offline
    Re: USB flash drive/ file system writing issues since v2.03b 2017/09/20 12:24:54 (permalink)
    0
    Hi,
     
    Is there any reson why you must migrate from v2.02b to a newer version ?
     
    According to your description, the issue is not with USB or file system.
    You mentionned that you can read the files properly on you PC.
    So it seems that the issue is related to SPI operation / real-time streaming.
    You need to do a deeper analysis to figure out what fails (reading ? / SPI frames / streaming ?)
     
    Regards
    #9
    qhb
    Superb Member
    • Total Posts : 6255
    • Reward points : 0
    • Joined: 2016/06/05 14:55:32
    • Location: One step ahead...
    • Status: offline
    Re: USB flash drive/ file system writing issues since v2.03b 2017/09/20 17:37:36 (permalink)
    0
    Just one other thought.
    Have you checked if the number of files on the drive affects your speed?
    #10
    Digital_Angel
    Starting Member
    • Total Posts : 37
    • Reward points : 0
    • Joined: 2014/08/27 06:56:45
    • Location: Northampton, UK
    • Status: offline
    Re: USB flash drive/ file system writing issues since v2.03b 2017/09/25 06:47:15 (permalink)
    0
    RISC
    Is there any reson why you must migrate from v2.02b to a newer version ?

     
    Hi RISC, 
    I need graphics controller and touch screen drivers that weren't present until after v2.02b.
     
     
    RISC
    According to your description, the issue is not with USB or file system.
    You mentionned that you can read the files properly on you PC.
    So it seems that the issue is related to SPI operation / real-time streaming.
    You need to do a deeper analysis to figure out what fails (reading ? / SPI frames / streaming ?)

    I had automatically ruled out the SPI as being the problem, purely because the previously working USB flash drive had now started to behave in the same way that the other flash drives had been, but of course there could be a completely different cause.
    Whatever it was, it was definitely introduced between Harmony versions. I hadn't spotted anything that I thought could be responsible in the USB changes, but I don't think I thought to check the SPI, so will do that next.
     
    Thank you!
     
     




    #11
    Digital_Angel
    Starting Member
    • Total Posts : 37
    • Reward points : 0
    • Joined: 2014/08/27 06:56:45
    • Location: Northampton, UK
    • Status: offline
    Re: USB flash drive/ file system writing issues since v2.03b 2017/09/25 06:48:59 (permalink)
    3 (1)
    qhb
    Have you checked if the number of files on the drive affects your speed?



    In all cases during my development I have been testing with a freshly formatted drive, and writing only one single file to it (anything more is for further down the line!), so this isn't a factor in this case.
     
    Thanks for the thought! :)
    #12
    Digital_Angel
    Starting Member
    • Total Posts : 37
    • Reward points : 0
    • Joined: 2014/08/27 06:56:45
    • Location: Northampton, UK
    • Status: offline
    Re: USB flash drive/ file system writing issues since v2.03b 2017/11/14 07:57:55 (permalink)
    0
    I'm back! and with a much better idea of what's going on.
     
    I have traced the problem of my missing data chunks to two causes:
     
    1.) The system tasks are taking much longer to execute, resulting in a slow build-up of data waiting to be read from my source (audio codec) as the state machine I was using had a number of breaks to allow system services to be dealt with, and when the source buffer overflows, it dumps all the data and starts again. Having reduced the number of breaks (a couple were unnecessary) and brought down the time taken to read a block of data, I can now keep up with the rate at which the data is made available to me - hurrah!
     
    2.) ... and this is the 'however'. I am still missing data due to apparent slow code execution when using particular USB flash drives. I have traced this (with some wiggling of I/O pins, a scope, and a lot of gradual narrowing down) to the SYS_FS_FileWrite function, which varies wildly in how long it takes to execute the below portion of code.
    fileStatus = obj->mountPoint->fsFunctions->write(obj->nativeFSFileObj, buffer, nbyte, &nosOfDataWritten);

    Beyond this I'm struggling to follow the code sequence deeper, as my C skills are limited, so I'm not sure what exactly is causing the huge delay.
     
    Any advice? Anyone with similar experiences?  I thought it may be USB 2/3 related, but I have a 2 that works, and a 2 and a 3 that don't.
    #13
    Jump to:
    © 2017 APG vNext Commercial Version 4.5