Helpful ReplyHot!FatFS implementation for PIC32

Page: 12 > Showing page 1 of 2
Author
Aiden.Morrison
Super Member
  • Total Posts : 729
  • Reward points : 0
  • Joined: 2005/02/25 11:18:31
  • Location: Canada
  • Status: offline
2011/03/04 10:23:13 (permalink)
5 (2)

FatFS implementation for PIC32

Hi All,

I'm working on a project where I need to stream data to SD at a high rate, and found that most of the openly available implementations I could find out there were just not up to the job - for example the microchip MDD library had the following performance -

Generic 512 M - FAT32/4096
8:52:41 to >9:03:00 - test cancelled - less than 13 kB/sec


Micro SD 4GB SDHC
9:21:18 to 9:27:00  - 342 seconds - 23kB/sec
9:27:00 to 9:32:40  - 340 seconds - 24kB/sec
9:32:40 to 9:38:21  - 341 seconds - 24kB/sec


Lexar 2GB fat32 formatted to 4kb allocation units
10:03:21 to 10:03:57 - 36 seconds to copy 8MB - 227kB/sec
10:03:57 to 10:04:33 - 36 seconds to copy 8MB - 227kB/sec
10:04:33 to 10:05:10 - 37 seconds to copy 8MB - 221kB/sec


Highly dependent on card type, and downright slow on many types.  I heard good things about FatFS
http://elm-chan.org/fsw/ff/00index_e.html
but couldn't find anyone who had implemented it for PIC32 that was willing to post code (fernando says he has too much proprietary stuff bolted on to share it, and dogbertius seems to be non communicative).

In the current implementation the benchmarks are now as follows:
Generic 512 M - FAT32/4096
seconds to copy 8MB
2:43:18 to 2:43:59 = 41 seconds to copy 8MB = 199.8 kB/sec
2:43:59 to 2:44:38 = 39 seconds to copy 8MB = 210.0 kB/sec
2:44:38 to 2:45:18 = 40 seconds to copy 8MB = 204.8 kB/sec

Micro SD 4GB SDHC
seconds to copy 8MB
2:36:30 to 2:36:54 = 24 seconds to copy 8MB = 341.33 kB/sec
2:36:54 to 2:37:16 = 22 seconds to copy 8MB = 372.36 kB/sec
2:37:16 to 2:37:37 = 21 seconds to copy 8MB = 390.09 kB/sec


Lexar 2GB fat32 formatted to 4kb allocation units
20 seconds =409.6 kB/sec
20 seconds =409.6 kB/sec
20 seconds =409.6 kB/sec



Performance can be improved further as this is simply a minimalistic implementation derived from the basic FatFS code, and using the pic24 example posted on the FatFS website.  I believe dogbertius was working on a version that would use DMA to do the bulk writes.

I likely won't be updating this code since I now have to roll it into a project that I shouldn't share, but my next tasks include trying to optimize the write and erase block sizes for each disk inserted by reading the card specific data and adapting to it (this should increase write speeds), as well as by utilizing DMA and SPI interrupts to arbitrate data streaming - I don't need to interleave reads with writes so this will destroy the generality of the file system.  


Some notes are included on the board used and what needs to be done to duplicate this simple setup.
Code should be attached here PIC32FatFS.zip
post edited by Aiden.Morrison - 2011/03/04 10:29:32
#1
Davis
New Member
  • Total Posts : 18
  • Reward points : 0
  • Joined: 2011/01/24 18:28:40
  • Location: 0
  • Status: offline
Re:FatFS implementation for PIC32 2011/03/06 06:27:22 (permalink)
0
UP!wink
#2
Aiden.Morrison
Super Member
  • Total Posts : 729
  • Reward points : 0
  • Joined: 2005/02/25 11:18:31
  • Location: Canada
  • Status: offline
Re:FatFS implementation for PIC32 2011/03/06 14:08:19 (permalink)
0
Two more notes:

1)  I'm still learning to use the file access controls like CREATE_ALWAYS etc. - the main.c in the rar archive doesn't properly do what I intended (create a new log file at each startup) - the updated main.c is attached, so it actually creates a new file at each startup for a logging application.

2) I forgot to mention that I disabled the timeout timer aswell, so while I haven't seen a read or a write timeout yet, there's no protection against this in the current code - originally ChaN indicates that 2 timer counters should be used that are decremented at 1kHz in a timer interrupt, but I just left them.

Hope these haven't tripped anyone up.
--An updated main.c is that addresses no.1 is attached.
post edited by Aiden.Morrison - 2011/03/06 14:12:03
#3
Aiden.Morrison
Super Member
  • Total Posts : 729
  • Reward points : 0
  • Joined: 2005/02/25 11:18:31
  • Location: Canada
  • Status: offline
Re:FatFS implementation for PIC32 2011/03/24 14:44:24 (permalink)
5 (1)
Bump.

Just wanted to mention that in this project there are two things you can change to get higher speed:

1) look up the definition of the fast clock setting for SPI - it currently sets SPI2BRG=2;  this can be set to SPIBRG=0, to raise the SPI clock from 6.7 MHz to 20 MHz.

2) changing the optimization level from O0 to O1 provides another small bump.

With those two changes the lexar 2G card now works at around 730 kB/sec write.
#4
mhark
Starting Member
  • Total Posts : 47
  • Reward points : 0
  • Joined: 2008/12/09 09:34:26
  • Location: 0
  • Status: offline
Re:FatFS implementation for PIC32 2011/04/04 11:03:11 (permalink)
0
Is this different from

http://www.microchip.com/...4&appnote=en539236


-mhark
#5
Aiden.Morrison
Super Member
  • Total Posts : 729
  • Reward points : 0
  • Joined: 2005/02/25 11:18:31
  • Location: Canada
  • Status: offline
Re:FatFS implementation for PIC32 2011/04/04 11:36:54 (permalink)
0
Yes, insofar as I couldn't get the microchip project to even compile.

It's from 2009 so my guess is it was built with tools that are now out of date...


This one is built with a 2011 version of FAT-FS as well.
#6
erupter
Senior Member
  • Total Posts : 179
  • Reward points : 0
  • Joined: 2011/05/31 15:15:52
  • Location: Rome, Italy
  • Status: offline
Re:FatFS implementation for PIC32 2011/10/05 06:41:53 (permalink)
5 (1)
I just wanted to thank Aiden.Morrison for sharing his code.
I've just tested it with my Sandisk 4GB SD.

 Microchip : don't event think of it
FatFS std : 62s - 394 KB/s
FatFS BRG0: 35s - 686 KB/s
FatFS OP1 : 27s - 872 KB/s


Now onto integrating this with MSD and possibly FreeRTOS.
Any suggestions here?
#7
rileonar
New Member
  • Total Posts : 5
  • Reward points : 0
  • Joined: 2010/03/05 06:24:45
  • Location: 0
  • Status: offline
Re:FatFS implementation for PIC32 2011/11/22 15:59:20 (permalink)
5 (1)
  Thanks, Aiden.Morrison! I've ported your code to to PIC32MX795F512L.
This chip is used into chipKIT Max32 board (an Arduino clone).
These are the changes made to your code:
  • device changed to PIC32MX795F512L (used into chipKIT Max32)
  • Header files included via "Include Search Path" option in project
  • System config performance optimization
  • LEDs removed (I don't have them in my "shield")
  • FatFs version updated to R0.09 September 6, 2011 (multi-partition)
  • get_fattime routine moved to main
  • disk_timerproc call added (handled by ISR)
  • parametrization of SPI port number
  • core_timer added for rtc and benchmarking purposes

Here is some benchmarking on old 64 MB MMC and 1 GB Kingston SD:

Type FileSystem BRG Optimization Seconds     KB/s
SD     FAT16         2         0                 37,43     641,16
SD     FAT16         1         0                 32,2       745,32
SD     FAT16         1         1                 24,14     994,04
SD     FAT16         1         s                 24,06     997,55
SD     FAT16         1         3                 22,52     1065,91
SD     FAT16         0         0                 24,48     980,43
SD     FAT16         0         1                 17,53     1369,08 (unstable)
SD     FAT32         2         0                 61,83     388,14
SD     FAT32         1         0                 56,69     423,36
SD     FAT32         0         0                 49,07     489,06
SD     FAT32         0         1                 41,71     575,37

MMC FAT16         2         0                  51,06     470,01
MMC FAT16         1         0                  45,63     525,96
MMC FAT16         1         1                  36,90     650,41
MMC FAT16         1         s                  37,70     636,55
MMC FAT16         1         3                  35,85     669,47
MMC FAT16         0         0                     n/a         n/a

Spreadsheet with above data is included into the project.
Now I begin thinking about your DMA solution...
 
Thanks again,
Riccardo
post edited by rileonar - 2011/11/22 16:17:10
#8
rcdeang
Starting Member
  • Total Posts : 44
  • Reward points : 0
  • Joined: 2011/01/25 13:07:10
  • Status: offline
Re:FatFS implementation for PIC32 2012/03/21 13:30:17 (permalink)
0
Thanks guys for all your work and sharing your code with everyone. I'm still reading through and studying everything.

I have a couple questions about USB and FatFs. Right now I have code that writes to a USB drive using the Microchip MDD file system library. I also need to be able to read/write to an SD card and I saw the Microchip's library can only access one drive at a time, which is why I'm looking into FatFs.

Can FatFs read/write to a USB drive in addition to an SD card? If so, would it be best to use Microchip's library for USB and FatFs for SD or just use FatFs for both?
#9
dogbertius
Senior Member
  • Total Posts : 146
  • Reward points : 0
  • Joined: 2009/02/13 10:26:01
  • Location: 0
  • Status: offline
Re:FatFS implementation for PIC32 2012/04/12 17:04:40 (permalink)
5 (1)
  Sorry about that. I've tried to at least keep up-to-speed on private messages, but have been swamped with work this last year. I hardly get any time for hobby electronics. It's sad really :( Glad to see the board members are still contributing actively. Personally I think MC should be hiring some of the board members so they can have demo apps that are both working and current.

"Do, or do not! There is no try"
#10
Markus Gritsch
New Member
  • Total Posts : 20
  • Reward points : 0
  • Joined: 2010/03/09 08:10:05
  • Location: 0
  • Status: offline
Re:FatFS implementation for PIC32 2012/04/12 23:40:56 (permalink)
5 (1)
Here is a working example of combining FatFs with the MCP USB stack. http://code.google.com/p/...rmware/FatFs_Test_MGr/ Markus
#11
spencoid
New Member
  • Total Posts : 2
  • Reward points : 0
  • Joined: 2013/01/06 12:00:54
  • Location: 0
  • Status: offline
Re:FatFS implementation for PIC32 2013/01/06 12:05:57 (permalink)
1 (1)
does anyone have sample arduino code to show the use of FatFS. 
#12
Andy123
Super Member
  • Total Posts : 567
  • Reward points : 0
  • Joined: 2005/04/25 14:20:54
  • Status: offline
Re:FatFS implementation for PIC32 2013/04/07 07:26:59 (permalink) ☄ Helpfulby scottdmorgan 2017/02/14 21:44:43
0
Want to bring this old theread back:
First, I want to say "thank you" to all contributers!
 
I can now get writing speeds that reported in this thread.
I am using modified application post #8 on custom PIC32MX795F512H

But the only way I can get it to work is to load and run Microchip MDDFS-SD applicaiton demo first.
Once it finished, I can load NBIIFS project and it works onlt once
Reset or another load will fail, forcing me to reload MDDFS-SD
I did debugging and it apperast that it's failing in disk_initialize on this line:
if (send_cmd(CMD0, 0) == 1) {   /* Enter Idle state */

this line never returns 1
Again, if I run MDDFS-SD first, then it works.
 
no-name SD is connected to SPI2,
PIC32 @80MHz,
MPLAB-X XC32 optimization 0 or 1
SD-CS is on RG9 pin 8
This is the only change I made:
/* Port Controls  (Platform dependent) */ 
#define CS_SETOUT() TRISGbits.TRISG9 = 0
#define CS_LOW()  _LATG9 = 0 /* MMC CS = L */
#define CS_HIGH() _LATG9 = 1 /* MMC CS = H */
// Change the SPI port number as needed on the following 5 lines
#define SPIBRG  SPI2BRG
#define SPIBUF  SPI2BUF
#define SPISTATbits SPI2STATbits
#define SPI_CHANNEL SPI_CHANNEL2
#define SPICONbits SPI2CONbits
 
  
I tried multiple slow BRGs, slow PIC speeds, slow peripherals clock.
I know I am misisng something simple and stupid :)
Any ideas?
post edited by Andy123 - 2013/04/07 08:14:57
#13
spencoid
New Member
  • Total Posts : 2
  • Reward points : 0
  • Joined: 2013/01/06 12:00:54
  • Location: 0
  • Status: offline
Re:FatFS implementation for PIC32 2013/04/07 09:21:40 (permalink)
1 (2)
Get an Arduino DUE. FatFS works fine on it without all the crap of dealing with the Digilent Board. I can read and write the SD card faster than I thought possible, all sizes speeds and formats. A lot more support on this platform even though ARM is pretty new for the Arduino. 
#14
Andy123
Super Member
  • Total Posts : 567
  • Reward points : 0
  • Joined: 2005/04/25 14:20:54
  • Status: offline
Re:FatFS implementation for PIC32 2013/04/07 13:01:04 (permalink)
0
spencoid:
My question was not directed to you, it was to the people who contributed to this post.
First, I do have DUE but I would post my question in a different forum about it, if I need to.
Next, why you decided that I am using Digilent board?
While I do have Digilent MAX32 board, I am not using it for this device and as I stated above I am using differnet PIC32 on the custom board the I can't change at this point.
 
I hope you will adhere to the rules of this forum - it is dedicated to and supported by Microchip, and if you have nothing to add, do not post anything please.
#15
Aiden.Morrison
Super Member
  • Total Posts : 729
  • Reward points : 0
  • Joined: 2005/02/25 11:18:31
  • Location: Canada
  • Status: offline
Re:FatFS implementation for PIC32 2013/04/09 04:04:50 (permalink)
0
Hi Andy123,
 
I just happened to log into the forums to see what is going on and am amazed/glad to see the thread still going, but sorry to hear you are having trouble.  I will need to use the code again in a project soon but it has been approx. 18 months since I last touched it and details are a little fuzzy.
 
One note on the code snippet you posted, I am not sure how the _LATGx macro works:
 
#define CS_LOW()  _LATG9 = 0 /* MMC CS = L */
#define CS_HIGH() _LATG9 = 1 /* MMC CS = H */
If this is manipulating the LAT register directly (and if this is code I originally wrote) then I owe everyone an apology as this is a bad idea as it can cause a read--> modify --> write issue with portG and interrupts.
 
try changing it to
#define CS_LOW()  LATGCLEAR = 512 /* MMC CS = L */
#define CS_HIGH() LATGSET = 512 /* MMC CS = H */ as these are atomic operations and happen in a single cycle.
 
Also, I can't remember clearly if the send_cmd(CMD0,0) initialization command is something that needs to be done once, needs to be done once or repeated until the card reports ready - maybe try adding a 5 second delay after powering up the card to make sure it is ready when the microcontroller goes to talk to it.
 
Good luck and let me know if either of those two things help.
 
-Aiden
 
 
#16
Andy123
Super Member
  • Total Posts : 567
  • Reward points : 0
  • Joined: 2005/04/25 14:20:54
  • Status: offline
Re:FatFS implementation for PIC32 2013/04/09 04:47:43 (permalink)
5 (1)
Aiden,
 
I will try LATGSET / LATGCLEAR later today,
 
I did more digging and it looks like the select() does not set "Succses" because of wait_ready(0) timeout  in Timer2
At this point I am trying to find differennces between MDDFS_SD that works and code posted in this thread.
 
 
#17
Aiden.Morrison
Super Member
  • Total Posts : 729
  • Reward points : 0
  • Joined: 2005/02/25 11:18:31
  • Location: Canada
  • Status: offline
Re:FatFS implementation for PIC32 2013/04/09 04:51:59 (permalink)
0
Hi Andy,
 
Again my memory is hazy but I think my code as posted had the timeout delays disabled, so it's possible I simply never noticed this problem. 
 
-Aiden
#18
Aiden.Morrison
Super Member
  • Total Posts : 729
  • Reward points : 0
  • Joined: 2005/02/25 11:18:31
  • Location: Canada
  • Status: offline
Re:FatFS implementation for PIC32 2013/04/09 05:27:23 (permalink)
0

Ok Andy123, I have done some digging in the pic32 header files and found that the definition of LATG macro is indeed directly manipulating the LAT register as would reasonably be expected.
This is not Read-Modify-Write safe and should not be used in an environment where there are any interrupts touching port G.
This is my fault as it is like this in my original code, and might very well be responsible to the intermittent failures I and others have noticed when using the code!
This probably isn't the source of your current problems but do change the lat lines to latset and latclear to be atomic/interrupt safe.
#19
Andy123
Super Member
  • Total Posts : 567
  • Reward points : 0
  • Joined: 2005/04/25 14:20:54
  • Status: offline
Re:FatFS implementation for PIC32 2013/04/09 08:07:57 (permalink)
0
Aiden,
I thin I can see what happened:
I am using modified code from post #8 that uses CoreTimerHandler that calls
disk_timerproc()
and as a result Timer2 updates.


 
your code does not have it, effectively disabling Timouts.
I will try to disable Timer2 updates (as well as Timer1) to see if it changes anything
 
Also, I got new 8GB SanDisk card today that I want to try instead of no-brand SD I have.
 
#20
Page: 12 > Showing page 1 of 2
Jump to:
© 2019 APG vNext Commercial Version 4.5