• AVR Freaks

SD Card Logger - Writing Speed Demos

Author
dogbertius
Senior Member
  • Total Posts : 146
  • Reward points : 0
  • Joined: 2009/02/13 10:26:01
  • Location: 0
  • Status: offline
2010/09/23 10:19:17 (permalink)
0

SD Card Logger - Writing Speed Demos

Good day all,

I am working on SD card loggers that need to write chunks of data in sizes of roughly 3100 bytes to 3500 bytes. With this in mind, I am using the "USB Mass Storage - SD Card Logger" demo from the Microchip Application Library, and can read/write files in my application and in Windows/linux as a mass storage device, using block sizes of 4096 bytes. Write speeds in my application however, write at very low speeds (ie: roughly 10K-bytes/s at best), using simple loops to have FSFwrite write a large array of size 10*4096 bytes at a time (would this constitute a true multiblock write?)

I have found posts on these forums where other people have achieved throughput of 400K-bytes/s with the Microchip FAT library and/or the Elm-Chan FatFS library.

My main concern is just simple, fast, SD card logging in my application (the USB functionality can be removed, but I still need a FAT/FAT32 library so I can remove the card and read it on a PC later). One insightful post (<http://www.microchip.com/...mp;mpage=1#523755>) noted I should use:
  • DMA +
  • SPI 16-bit mode instead of SPI 8-bit mode
  • Multi-block writes instead of single byte writes using FSFwrite
Does anyone have any working demos for PIC32mx695/PIC32mx795 (ie: written in C32, uses the newer plib funcitons, such as SPIChnOpen instead of OPENSPI, etc) or any resources on getting high read/write speeds using a PIC32 and an SD card for fast data logging? With the number of people posting comparative results between file systems, I figured at least a few other people have managed SD + Fat/Fat32 + PIC32 for decent write speeds.

Thank you all in advance!

PS: I've tried the FatFS demo from Microchip <http://www.microchip.com/...p;appnote=en539236> and can detect the SD card, yet I can't mount the filesystem or write files to it, and had to make a lot of modifications for using SPI channel 2A instead of channel 1.

"Do, or do not! There is no try"
#1

2 Replies Related Threads

    dogbertius
    Senior Member
    • Total Posts : 146
    • Reward points : 0
    • Joined: 2009/02/13 10:26:01
    • Location: 0
    • Status: offline
    Re:SD Card Logger - Writing Speed Demos 2010/10/08 15:53:01 (permalink)
    0
    Any luck yet from anyone else?

    At this point, I'd be fine with FatFS or the Microchip Fat implementation, so long as I can get decent throughput when writing directly from my main program code on the PIC, to the micro SD card. Many people in other threads have noted write speeds of 400KB/s+, while I'm lucky to get 9kB/s.

    Thanks!

    "Do, or do not! There is no try"
    #2
    asmallri
    Super Member
    • Total Posts : 1864
    • Reward points : 0
    • Joined: 2004/05/26 09:00:05
    • Location: Perth, Australia
    • Status: offline
    Re:SD Card Logger - Writing Speed Demos 2010/10/08 18:42:37 (permalink)
    0
    You sent me a PM but the link to my earlier post went nowhere so I do not know what you were referring to however....

    There are a number consideration with respect to performance for your data logger
    1. FAT16 is faster than FAT32
    2. FAT hits a speed bump every time a cluster boundary is crossed as it must search the FAT for available clusters, update FAT tables and directory entries each time a new cluster is required. Larger cluster sizes will improve performance.
    3. Based purely on my testing of lots of cards for my own data logger developments, "high performance cards" do not deliver high performance in SPI mode. My testing indicates several of these cards perform slower than standard cards in SPI mode.
    4. When writing to the file system what actually happens is you write to the file buffer and when this buffer is full, it is flushed to the media and the file system refills this buffer. Basically adding a complete write/read memory access sequence. 
    5. ELM some time ago added multiple drive support. This support comes with a CPU cycle cost
    6. The PIC32 is optimized for DWORD (32 bit) operations. Therefore performing DWORD read and writes reduces the number of reads and writes required to the file buffer however (from memory and I may be wrong) the driver accesses the buffer with byte wide operations. 

    Here is a suggestion for how you can increase the performance for your specific project.
    1. With newly formatted media, create a file of the maximum size of your data file. This will create a file occupying contiguous sectors on the media.
    2. Open the file with the file system driver, extract the start sector information and close the file. You now have the start sector.
    3. Using this start sector information, record your data to the media keeping track of total number of bytes written.
    4. When finished logging the data, flush the current file buffer to the media.
    5. Open the file for writing. Modify the filesize field in the file control block and close the file.

    Regards, Andrew




    Regards, Andrew

    http://www.brushelectronics.com/index.php?page=software
    Home of Ethernet, SD Card, and Encrypted Serial and USB Bootloaders for PICs!!
    #3
    Jump to:
    © 2019 APG vNext Commercial Version 4.5