• AVR Freaks

Filesystem, USB MSD, long blocking writes

Author
campbellCustom
Starting Member
  • Total Posts : 69
  • Reward points : 0
  • Joined: 2014/08/30 14:35:35
  • Location: 0
  • Status: online
2019/10/10 11:50:49 (permalink)
0

Filesystem, USB MSD, long blocking writes

Microchip Harmony Filesystem, USB MSD, long blocking writes
PIC32MZEF (252MHz)… Harmony 2.06 …. MPLAB X 5.xx … XC32 2.xx
 
This project is using NO RTOS for performance reasons. I do realize I’m struggling to marry synchronous behavior with an asynchronous system. This trouble is compounded as this is my first experience with using SYS_FS, USB, and FAT. (I do know harmony from the TCP/IP side of things)
 
I’m using a USB drive to store data. I expect max read/write size to be 8192 bytes. I’d like to see the samples written every 32ms. I have the hardware and the file system calls ‘working’ in the sense that data can go in and out of the drive. After finding some performance curiosities (a.k.a. problems) it appears the Harmony file system will complete operations by waiting on the hardware.
 
The read times seem generally ok, but blocking longer than I’d like.
 
The write times are blocking longer than the read times. I understand that write times are longer for flash. On average over a long period of time the data throughput is ok, however there are occasional extra-long writes.
Every one or two seconds a write operation will take ~120ms. This puts kinks in my data. My reading indicates the file system may be going over some boundary in the hardware.
 
As of now the plan is to buffer samples in an interrupt to smooth over this issue. But I’m not sure how best to proceed, and have a few questions and thoughts.
 
As flash drives grow, will they write slower? If so how much slower? My reading says yes, but I’m curious if anyone out there has timed these things, or has insight on how to calculate. The ~120ms blocks are happening on a 32gig drive.
 
Is there a good source for USB keys with long term support? Providing a default drive is possible in this case. This hopefully ensures my timing parameters won’t change. I do expect consumer grade things to go obsolete constantly.
 
Why does harmony file system wait on hardware? Is there no flag to break out of the state machine, and wait for a read/write completion? Is there a warning in the help I missed? After being told repeatedly to put in a lot of ‘break-to-wait’ states in my own code, this FS library seems more at home in an RTOS.
 
What kind of data through put can be expected when using an RTOS with SYS_FS and USB  MSD? I have no insight on this, other than feeling pushed to learn RTOS if I continue with Harmony. I expect the RTOS will add overhead to the average speed of getting data into the drive.
 
To buffer for storage, would you use a fixed size buffer? Or dynamically allocate? As of now I’m trying to malloc a FIFO buffer sample in a timer interrupt. I’m just not sure where to cap memory usage, and figure there will be some weird bugs that depend on usb key speeds.
 
Thanks to all for any insight on my rambling ignorance.
-IC
#1

3 Replies Related Threads

    jtemples
    عُضْوٌ جَدِيد
    • Total Posts : 11347
    • Reward points : 0
    • Joined: 2004/02/13 12:31:19
    • Location: Southern California
    • Status: offline
    Re: Filesystem, USB MSD, long blocking writes 2019/10/10 12:50:52 (permalink)
    5 (1)
    Can you take your samples in an interrupt handler to maintain real-time?
     
    You're never going to get guaranteed performance specs on a USB drive, so I can't see any reason to try to quantify it.  Your next batch of "identical" drives could have completely different performance.
     
    Maybe add a second processor dedicated to USB?
    #2
    ric
    Super Member
    • Total Posts : 23893
    • Reward points : 0
    • Joined: 2003/11/07 12:41:26
    • Location: Australia, Melbourne
    • Status: online
    Re: Filesystem, USB MSD, long blocking writes 2019/10/10 15:04:33 (permalink)
    0
    jtemples
    Can you take your samples in an interrupt handler to maintain real-time?

    I think that is exactly what the OP is talking about doing, but they are trying to get a feel for how large a FIFO buffer they are going to need to not overflow while the file system is blocking.
     

    I also post at: PicForum
    Links to useful PIC information: http://picforum.ric323.co...opic.php?f=59&t=15
    NEW USERS: Posting images, links and code - workaround for restrictions.
    To get a useful answer, always state which PIC you are using!
    #3
    campbellCustom
    Starting Member
    • Total Posts : 69
    • Reward points : 0
    • Joined: 2014/08/30 14:35:35
    • Location: 0
    • Status: online
    Re: Filesystem, USB MSD, long blocking writes 2019/10/15 09:57:27 (permalink)
    0
    I found some threads on this here:
    https://www.microchip.com/forums/m1042740.aspx
    https://www.microchip.com/forums/m885813.aspx
    For whatever reason these search ok on google but don't come up on the forum search.
     
    I am not totally sure, but it looks like Harmony 3 is using the same blocking library for the filesystem.
     
     
    #4
    Jump to:
    © 2019 APG vNext Commercial Version 4.5