• AVR Freaks

PIC32 SD card MSD performance unleashed

Author
zaphod
Starting Member
  • Total Posts : 41
  • Reward points : 0
  • Joined: 2010/07/11 03:58:17
  • Location: 0
  • Status: offline
2012/01/13 09:05:37 (permalink)
0

PIC32 SD card MSD performance unleashed

Hey all, as reported in many other threads, I also faced a very low performance when running the MDD application library to realize a SD card MSD device with my PIC32. With a SanDisk class8 8GB card (formatted with FAT32 - standard cluster size) and the SPI clocked at 20 MHz, the throughput was approximately 200 kB/s resp. 400 kB/s (write - read). A bit of search made me realize that these rates are extremely low for such an architecture. After several hours of code modification, I successfully unleashed the potential of the PIC32. Now read speed is 690 kB/s (345% gain) and write speed 780 kB/s (195% gain) !!! These are the modifications I applied: - DMA block transfers (512 Byte) instead of CPU byte transfers - multi-block access (significantly boosts write performance !) - USB double-buffering (partially interleaving the transfers with wait state polling) I'm quite satisfied with the performance now, but there is still one further possibility to speed it up even more: - full interleaving of USB / SD card transfers (using a double buffering scheme) If anyone has done the same modifications, it would be nice to get to know about the performance you were able to attain. In my case all measurements were done by writing 100 MB test data with the tool h2testw. BTW: If any Microchip staff member responsible for the application library implementation reads this, you should probably consider the following advice: Get a basic coding guideline briefing from an experienced senior developer (if there is any)!!! Greets Matthias
post edited by zaphod - 2012/01/13 09:10:22
#1

18 Replies Related Threads

    bnell
    Starting Member
    • Total Posts : 74
    • Reward points : 0
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2012/01/13 10:02:39 (permalink)
    0
    Nice, thanks for sharing.

    Yikes, I'm getting ~150 kB/s with a 100MB test file.  I've also had a support ticket in for a month about an infinite loop in one of the SPI calls from the MDD.  Sigh.

    Zaphod, any interest in sharing the code?
    #2
    jonm
    Senior Member
    • Total Posts : 119
    • Reward points : 0
    • Joined: 2010/06/23 15:51:53
    • Location: 0
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2012/01/13 12:45:07 (permalink)
    0
    Hi Matthias,
    I am curious, did you hack in the FatFS library into MDD, or modify MDD itself?  The new MDD library now has Multiblock support, and there are other threads which discuss this (http://www.microchip.com/forums/fb.ashx?m=591044) but I am still running on a stitched-together version of FatFS and MDD, and am wary about relying solely on the MDD code.  This solution works, but for what ever reason I cannot get my SD reads down to 64 bytes without bottlenecking and crashing when transferring over USB. (http://www.microchip.com/forums/m605726.aspx)
    #3
    erupter
    Senior Member
    • Total Posts : 179
    • Reward points : 0
    • Joined: 2011/05/31 15:15:52
    • Location: Rome, Italy
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2012/01/15 12:50:11 (permalink)
    0
    Jonm: when using MSD, MDD is not used.
    MSD passes direct control of the memory device to the controlling operating system.

    zaphod:
    nice, but I already did that a couple years ago.
    I also implemented a complete A2DP stack on a pic16 but Microchip never wanted it so I kept it to myself.
    ...
    Now if you really did that, which is quite possible, why not share and just brag about it? It seems a bit childish in my opinion.
    #4
    fmilano
    New Member
    • Total Posts : 16
    • Reward points : 0
    • Joined: 2011/09/09 08:17:31
    • Location: 0
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2012/01/17 10:55:03 (permalink)
    0
    @zaphod: I'm having exactly the same performance issues that you report. After some changes in the MDD SD code (enabled the enhanced buffering of SPI when transferring the 512 bytes sector) I've a reading performance of 470 KB/s instead of 400 KB/s. I would really like to have a look to your modifications; I can facilitate you mine (it is very simple, and the gain is moderate, but it is something).
    The problem is that if I test my modified MDD SD code without transferring through USB, I get 1.7MB/s reading sector contents (I'm not counting here the control bits sent for commands, CRC, waiting, etc). In both cases I'm working with a SPI clock of 20MHz and with SINGLE BLOCK reads. 
    I think your performance gains are great, but don't you think that it would be reasonable to get at least 1.1MB/s when using the USB MSD on a FAT32 formatted SD? 
    Does anybody know where the bottleneck(s) could be located?

    #5
    zaphod
    Starting Member
    • Total Posts : 41
    • Reward points : 0
    • Joined: 2010/07/11 03:58:17
    • Location: 0
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2012/01/21 18:28:32 (permalink)
    0
    Sorry, somehow the mail notification failed and I didn't have an eye on the thread the last days. I'm open to provide my modifications, if anyone is interested (which is obvisiously the case). You will find the files in the attached archive. As I already mentioned there is still some space for further improvements. In my case the MSD device is just a "nice to have" feature in a large project, so I focus on other things. But if you are able to crank up the speed even more, please also share your modifications in this thread ! @ erupter Just to clarify my intention, I'm not up for a "who has the bigger balls competition" it was just about sharing information ! So why don't you just tell your speeds instead of bragging about off-topic stuff ?
    post edited by zaphod - 2012/01/22 16:44:50
    #6
    Stampede
    Super Member
    • Total Posts : 400
    • Reward points : 0
    • Joined: 2006/10/04 05:59:28
    • Location: Germany
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2012/01/22 06:29:59 (permalink)
    0
    Hi, I was trying to test your files, but some definitions are missing: e.g.: MDD_MAX_MULTI_SECTOR, SDSPI_TX_DMA_SFR_ADDR, .... I think its located in your FSIO_perf.h :) Cheers Stefan
    #7
    erupter
    Senior Member
    • Total Posts : 179
    • Reward points : 0
    • Joined: 2011/05/31 15:15:52
    • Location: Rome, Italy
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2012/01/22 09:31:27 (permalink)
    0
    Zhapod I can't share anything because I haven't accomplished anything in this context.
    I was just using a figure of speech.

    My speed (default MSD) is 200 KB/s.
    I'll test yours although it seems, from the post above mine, that there are some missing files.
    #8
    zaphod
    Starting Member
    • Total Posts : 41
    • Reward points : 0
    • Joined: 2010/07/11 03:58:17
    • Location: 0
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2012/01/22 17:00:51 (permalink)
    0
    Everything should be included now. Please note that you have to adjust the "HardwareProfile.h" to your hardware configuration. You will see that I also stripped down the necessary defines. I'm curious if somebody takes the action to further enhance the implementation and squeezes out more speed. Replacing the FSIO implementation with FatFS would also be very nice, but I'm out of time right now.
    #9
    Stampede
    Super Member
    • Total Posts : 400
    • Reward points : 0
    • Joined: 2006/10/04 05:59:28
    • Location: Germany
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2012/01/23 05:47:11 (permalink)
    0
    Hi, I added ur functions, changed the hardware config accordingly. I always get an compile error, in the EVENT_TRANSFER_TERMINATED part. I needed to comment
    //USBMSDOutHandle = USBRxOnePacket(MSD_DATA_OUT_EP, (BYTE*)&msd_cbw, MSD_OUT_EP_SIZE);
    to make the hole thinr run. However, I get only 200kB writing and 500kB reading performance with the processor running at 80MHz. Any hints what could be wrong ?
    #10
    erupter
    Senior Member
    • Total Posts : 179
    • Reward points : 0
    • Joined: 2011/05/31 15:15:52
    • Location: Rome, Italy
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2012/01/23 05:58:56 (permalink)
    0
    I still haven't got around to testing it, but there is something unclear anyway.

    In my app, using default MC libraries, I get MSD functionality without the FSIO library.
    I'm perfectly able to read and write on the SD.

    So what does it have to do with it?
    #11
    zaphod
    Starting Member
    • Total Posts : 41
    • Reward points : 0
    • Joined: 2010/07/11 03:58:17
    • Location: 0
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2012/01/23 10:13:28 (permalink)
    0
    Check your definitions, you should have something like this in your usb_config.h:
     #define MSD_IN_EP_SIZE              64  #define MSD_OUT_EP_SIZE             64 #define MSD_DATA_IN_EP              2 #define MSD_DATA_OUT_EP             2 
    Also make sure that the source files FSIO.c and SD-SPI.c are replaced with the modified -perf.c files in your build tree. Sry for the missing line feeds, got the feeling it is due to a language issue. This silly forum somehow deletes all my inserted feeds.
    post edited by zaphod - 2012/01/23 10:18:48
    #12
    s4cral
    New Member
    • Total Posts : 4
    • Reward points : 0
    • Joined: 2012/08/12 00:53:45
    • Location: 0
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2013/08/09 01:16:21 (permalink)
    0
    Hi  Zaphod
     
    I had a quick look at your performance mod for PIC32 MSD.  I was busy integrating it with a more recent microchip library and got to the point where I can't see anywhere in your code where you trigger the DMA events to read or write data to SD.  So I assume there may have been an interrupt routine missed out from the posted src?  This thread is looking a bit old so I'm hoping you are still about.
     
    Cheers
     
     
    post edited by s4cral - 2013/08/09 01:40:05
    #13
    s4cral
    New Member
    • Total Posts : 4
    • Reward points : 0
    • Joined: 2012/08/12 00:53:45
    • Location: 0
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2013/08/09 23:35:07 (permalink)
    0
    Here's a few more details on the problem
    in the file SD-SPI_perf.c the code stalls at the following line when reading data
     
    while ( !(rxDmaSfrSet->DCHxINT.base & 0x00000023) )
     
    and what i've read so far tells me this
     
    A cell transfer is initiated in one of two ways:
    • Software can initiate a transfer by setting the channel CFORCE bit (DCHxECON<7>)
    • Interrupt event occurs on the device that matches the CHSIRQ interrupt and SIRQEN = 1
    (DCHxECON<4>). The user can select any interrupt on the device to start a DMA transfer
     
    Since there is no interrupt routine or anything to force the DMA transfer the code is stuck in that loop.
     
    Cheers
     
     
     
     
     
    #14
    escgmbh
    New Member
    • Total Posts : 20
    • Reward points : 0
    • Joined: 2009/05/05 02:10:46
    • Location: 0
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2013/09/17 09:02:27 (permalink)
    0
    Hi, 
     
    I know, it's already an older thread, but I've an issue with the implementation: 
    It works all fine, except the read performance is very bad on my 4GB card. 
    On the logic analyzer, I see that the waiting time (until the card send 0xfe after the 0x51 command), is about 3ms. Afterwards ,the 512 bytes are read with very good performance (512 bytes in 270us), but in total, I only got 512bytes every 3.3ms, which is not enough. 
    Do I simply have a bad card (4GB Sandisk SDHC mobile ultra)?
    Or is there a possibiltity to increase the read block size? 
    (I assume not, as it's a sector read...)
     
    Another 2GB card works faster (about 1.4ms access time).
     
    Did you measure your cards? What are reasonable values?
     
    best regards,
    Matthias
     
    #15
    s4cral
    New Member
    • Total Posts : 4
    • Reward points : 0
    • Joined: 2012/08/12 00:53:45
    • Location: 0
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2013/09/20 10:26:52 (permalink)
    0
    Hi Matthias
     
    I think the only way to speed the whole thing up is re-write the underlying USB libs.   I've modded the libs to use DMA read and write
    for single sector transfers but the performance gains were minimal.  If you are doing a lot of bulk data transfers then sorting out
    multi-block transfers should speed things up somewhat.   I'm doing lots of small data reads less than a sector in size so I'm stuck
    with the underlying io bottleneck.  The fatFS implementation for PIC32  has nice DMA code.  i see around 20 k/s write and  40 k/s read.
    Good luck with it.
     
    Cheers
     
     
     
    #16
    leaf05
    New Member
    • Total Posts : 20
    • Reward points : 0
    • Joined: 2012/03/07 08:24:45
    • Location: 0
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2013/09/20 12:05:50 (permalink)
    0
    By any chance can you share the code so we can all use it?
    Thanks a lot!
    Luis
    #17
    s4cral
    New Member
    • Total Posts : 4
    • Reward points : 0
    • Joined: 2012/08/12 00:53:45
    • Location: 0
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2013/09/21 10:33:19 (permalink)
    0
    Hi Luis
    I've just be testing the available code on the forums to this point.
    I have seriously slow data rates for SD mounted under windows.
    Its a huge amount of work to sort out this USB lag problem and right
    now I dont have time to do it.
     
    Cheers
    #18
    escgmbh
    New Member
    • Total Posts : 20
    • Reward points : 0
    • Joined: 2009/05/05 02:10:46
    • Location: 0
    • Status: offline
    Re:PIC32 SD card MSD performance unleashed 2013/09/22 10:17:12 (permalink)
    0
    Hi,
    I used the code published in this thread, it works great! The only thing was a missing extern declaration, then the code worked perfectly.
    My problem was effectively a slow card; I bought a new one (16GB sandisk); the typical sector access time is 10x faster now (0.3ms instead of 3ms before)!
    That solves all my issues. There seem to be really big differences regarding access time.
    One more detail: I get now about 640kB/s write speed according windows, using FAT32 formatting with the code published in this thread - great and many thanks!!! 
    best regards,
    Matthias
     
    post edited by escgmbh - 2013/09/23 02:33:37
    #19
    Jump to:
    © 2019 APG vNext Commercial Version 4.5