• AVR Freaks

Max USB Bulk transfer speed.

Page: 123 > Showing page 1 of 3
Author
Kiwi Mike AZ
Super Member
  • Total Posts : 2056
  • Reward points : 0
  • Status: offline
2009/04/16 13:54:05 (permalink)
0

Max USB Bulk transfer speed.

I have written some code for the PIC32MX460F512L and am using LibUSB on the PC side.
 
Testing on it own dedicated USB2.0 controller.
 
Have been transfering 1024bytes/ms
Tested 1024byte and also 512byte transfers
(512 byte transfers adds one packet gap in the middle of ~45ns - so the PIC code responds quickly)
Was hoping to get close to 1MByte/sec transfer rate.
I am able to get ~500KBytes/second transfer reliably.
 
Using a Beagle USB12 analizer I discovered that every second frame is being missed.
 
I know the PIC32 is fast enough and sohould not be the issue because when sending 512byte transfers it gets two of them in to one frame.
at 1024bytes per frame it uses 16 packets, leaving 4 packets spare for retrys etc (Which it do occur every now and then)
 
The Enumeration is set for 1ms, but being bulk it fills any time slot avaliable.
 
PC Code loop is basically the following using LibUSB
 


for (;;)
{
 retVal = device.BulkRead (0x82, buffer, INBUFFSIZE, 10); 
 if (retVal > 0)
 {
  nBytes += retVal;
 }
 if (nBytes >= 1048575) break; // Break at 1MByte (1024 x 1024)=1048576
}

 
INBUFFSIZE = 1024
 
I believe the PC side is the issue as it initiates any transaction.
 
Q: Is there anyone out there that has found this issue before and managed to resolve it?
Q: Any ideas what else to look for?
 
Thanks,
 
Mike
 
 
#1

59 Replies Related Threads

    xiaofan
    Super Member
    • Total Posts : 6247
    • Reward points : 0
    • Joined: 2005/04/14 07:05:25
    • Location: Singapore
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/04/16 16:49:59 (permalink)
    0
    You may have to use the asynchronous function of libusb-win32 or libusb 1.0 under Linux.

    libusb-win32's asynchronous function is not well documented.
    http://libusb-win32.svn.sourceforge.net/viewvc/libusb-win32/trunk/libusb/src/usb.h?view=markup


    374
    375 int usb_isochronous_setup_async(usb_dev_handle *dev, void **context,
    376 unsigned char ep, int pktsize);
    377 int usb_bulk_setup_async(usb_dev_handle *dev, void **context,
    378 unsigned char ep);
    379 int usb_interrupt_setup_async(usb_dev_handle *dev, void **context,
    380 unsigned char ep);
    381
    382 int usb_submit_async(void *context, char *bytes, int size);
    383 int usb_reap_async(void *context, int timeout);
    384 int usb_reap_async_nocancel(void *context, int timeout);
    385 int usb_cancel_async(void *context);
    386 int usb_free_async(void **context);


      USB_Links and libusb
    #2
    xiaofan
    Super Member
    • Total Posts : 6247
    • Reward points : 0
    • Joined: 2005/04/14 07:05:25
    • Location: Singapore
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/04/16 16:53:13 (permalink)
    0
    Last time I wrote a simple example here for the bare-minimum isochronous example. It will be similar for bulk transfer.


    <code>
    #include <usb.h>

    #define VERSION "0.1.0"
    #define VENDOR_ID 0x04D8
    #define PRODUCT_ID 0x0080
    #define INTERFACE 0
    #define BUFFER_SIZE 65

    usb_dev_handle *find_pickit2_isoc();

    usb_dev_handle* setup_libusb_access() {
    usb_dev_handle *pickit2_isoc;

    usb_set_debug(255);
    usb_init();
    usb_find_busses();
    usb_find_devices();

    if(!(pickit2_isoc = find_pickit2_isoc())) {
    printf("Couldn't find the mouse, Exiting\n");
    return NULL;
    }

    if (usb_set_configuration(pickit2_isoc, 1) < 0) {
    printf("Could not set configuration 1 : %s\n");
    return NULL;
    }

    if (usb_claim_interface(pickit2_isoc, INTERFACE) < 0) {
    printf("Could not claim interface: %s\n");
    return NULL;
    }

    return pickit2_isoc;
    }

    usb_dev_handle *find_pickit2_isoc()
    {
    struct usb_bus *bus;
    struct usb_device *dev;

    for (bus = usb_busses; bus; bus = bus->next) {
    for (dev = bus->devices; dev; dev = dev->next) {
    if (dev->descriptor.idVendor == VENDOR_ID &&
    dev->descriptor.idProduct == PRODUCT_ID ) {
    usb_dev_handle *handle;
    printf("pickit2_isoc with Vendor Id: %x and Product Id: %x
    found.\n", VENDOR_ID, PRODUCT_ID);
    if (!(handle = usb_open(dev))) {
    printf("Could not open USB device\n");
    return NULL;
    }

    return handle;
    }

    }
    }

    return NULL;
    }

    void test_isochronous_async(usb_dev_handle *dev)
    {
    unsigned char buf0[640];
    unsigned char buf1[640];
    unsigned char buf2[640];

    int i;

    /* use three contexts for this example (more can be used) */
    void *context0 = NULL;
    void *context1 = NULL;
    void *context2 = NULL;

    /* write transfer (stream) */
    /*
    usb_isochronous_setup_async(dev, &context0, 0x01);
    usb_isochronous_setup_async(dev, &context1, 0x01);
    usb_isochronous_setup_async(dev, &context2, 0x01);

    usb_submit_async(context0, buf0, sizeof(buf0));
    usb_submit_async(context1, buf1, sizeof(buf1));
    usb_submit_async(context2, buf2, sizeof(buf2));

    for(i = 0; i < 10; i++)
    {
    usb_reap_async(context0, 5000);
    usb_submit_async(context0, buf0, sizeof(buf0));

    usb_reap_async(context1, 5000);
    usb_submit_async(context1, buf1, sizeof(buf1));

    usb_reap_async(context2, 5000);
    usb_submit_async(context2, buf2, sizeof(buf2));
    }

    usb_reap_async(context0, 5000);
    usb_reap_async(context1, 5000);
    usb_reap_async(context2, 5000);

    usb_free_async(&context0);
    usb_free_async(&context1);
    usb_free_async(&context2);

    */

    /* read transfer (stream) */
    usb_isochronous_setup_async(dev, &context0, 0x81,64);
    usb_isochronous_setup_async(dev, &context1, 0x81,64);
    usb_isochronous_setup_async(dev, &context2, 0x81,64);

    usb_submit_async(context0, buf0, sizeof(buf0));
    usb_submit_async(context1, buf1, sizeof(buf1));
    usb_submit_async(context2, buf2, sizeof(buf2));

    for(i = 0; i < 10; i++)
    {
    usb_reap_async(context0, 5000);
    usb_submit_async(context0, buf0, sizeof(buf0));

    usb_reap_async(context1, 5000);
    usb_submit_async(context1, buf1, sizeof(buf1));

    usb_reap_async(context2, 5000);
    usb_submit_async(context2, buf2, sizeof(buf2));
    }

    for(i = 0; i < 640; i++) {
    printf(" %02x, %02x, %02x ", buf0[i],buf1[i],buf2[i]);
    // printf(" %02x, %02x, %02x ", buf0[i]);
    }

    usb_reap_async(context0, 5000);
    usb_reap_async(context1, 5000);
    usb_reap_async(context2, 5000);
    usb_free_async(&context0);
    usb_free_async(&context1);
    usb_free_async(&context2);

    /* release interface */
    usb_set_altinterface(dev, 0);
    usb_release_interface(dev, 0);
    }


    int main(void)
    {
    usb_dev_handle *pickit2_isoc;
    if ((pickit2_isoc = setup_libusb_access()) == NULL) {
    exit(-1);
    }
    test_isochronous_async(pickit2_isoc);
    usb_close(pickit2_isoc);

    return 0;
    }



      USB_Links and libusb
    #3
    xiaofan
    Super Member
    • Total Posts : 6247
    • Reward points : 0
    • Joined: 2005/04/14 07:05:25
    • Location: Singapore
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/04/16 16:54:20 (permalink)
    0
    http://www.microchip.com/forums/tm.aspx?m=270049&mpage=2

    You can also use debugview to check how libusb-win32 is up to.

      USB_Links and libusb
    #4
    xiaofan
    Super Member
    • Total Posts : 6247
    • Reward points : 0
    • Joined: 2005/04/14 07:05:25
    • Location: Singapore
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/04/16 16:55:17 (permalink)

      USB_Links and libusb
    #5
    FunnyNYPD
    Super Member
    • Total Posts : 1553
    • Reward points : 0
    • Joined: 2007/09/28 20:06:59
    • Location: USA & Canada
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/04/16 18:11:53 (permalink)
    0
    So, even the PIC32 can only run about 512K bytes/s reliably, I would imaging the 18F2550/4550 will run at a much slower rate.
    #6
    chinzei
    Super Member
    • Total Posts : 2250
    • Reward points : 0
    • Joined: 2003/11/07 12:39:02
    • Location: Tokyo, Japan
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/04/16 18:37:05 (permalink)
    0
    Using a Beagle USB12 analizer I discovered that every second frame is being missed....

    INBUFFSIZE = 1024

    Apply greater transfer size (INBUFFSIZE), either for synchronous or for asynchronous call. For example, 1M Bytes transfer at single call.
    The max transfer size depends on the device driver. I don't know libusb allows 1M Bytes transfer or not. But the max should be greater than 1K bytes.

    When synchronous call is applied repeatedly, gap between the calls occurs on the bus. Apply asynchronous call, and issue multiple forward calls, as xiaofan showed.

    at 1024bytes per frame it uses 16 packets, leaving 4 packets spare for retrys etc (Which it do occur every now and then)

    Theoretical max is 19 full-size packets / frame.
    But I've not seen 19 yet, 18 is the max I've seen.

    Do you see retry on the bus?
    Isn't it NAK?
    Retry is different from NAK.
    NAK is usual, but retry is a symptom of trouble.

    The Enumeration is set for 1ms

    What do you mean?
    If you mean the bInterval field of the endpoint descriptor, it is ignored for full-speed bulk endpoints.

    Tsuneo
    post edited by chinzei - 2009/04/16 19:07:18
    #7
    Kiwi Mike AZ
    Super Member
    • Total Posts : 2056
    • Reward points : 0
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/04/16 20:34:08 (permalink)
    0
    My colleague working on the same project just did this and is now getting about 888KByte/sec.... but has other devices on the same USB Controller. Changing that may get more.
    Will try these later tonight or tomorrow myself.

    Thanks,

    Mike


    #8
    xiaofan
    Super Member
    • Total Posts : 6247
    • Reward points : 0
    • Joined: 2005/04/14 07:05:25
    • Location: Singapore
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/04/17 07:23:38 (permalink)
    0

    ORIGINAL: mimemike
    My colleague working on the same project just did this and is now getting about 888KByte/sec....


    You've a new record for Microchip MCUs (without SPP). grin

    For PIC18F2550/4550, it is less than 700KB.
    http://www.microchip.com/forums/tm.aspx?m=173421
    http://www.microchip.com/forums/tm.aspx?m=165751

    With SPP, it is faster.
    http://www.microchip.com/forums/tm.aspx?m=251449



      USB_Links and libusb
    #9
    FunnyNYPD
    Super Member
    • Total Posts : 1553
    • Reward points : 0
    • Joined: 2007/09/28 20:06:59
    • Location: USA & Canada
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/04/17 07:32:27 (permalink)
    0
    So, a 700K~800K bytes/second rate is achievable reliably. Higher speed is either not stable or need special treatment, like the SPP interface, etc.
    #10
    Kiwi Mike AZ
    Super Member
    • Total Posts : 2056
    • Reward points : 0
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/04/17 10:01:34 (permalink)
    0
    Just broke that record!!!
     
    Test 1: 1032.853K Bytes/sec
    Test 2: 1024.725K Bytes/sec
    Test 3: 1024.187K Bytes/sec
    Test 4: 1033.399K Bytes/sec
    Test 5: 1024.725K Bytes/sec
     
     
    I would not want to do more than 1024 bytes per ms (16 Packets) as this leaves room for error corrections.
    Also to get this performance it is critical to make sure it is the ONLY device connected to that USB Host controller.
     
    Mike
    #11
    jmag99
    Super Member
    • Total Posts : 486
    • Reward points : 0
    • Joined: 2007/09/21 08:04:33
    • Location: RI, United States
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/04/17 10:55:58 (permalink)
    0
    Hi Mike,

    Do you think it's possible to extract data at this speed when the PIC is a host?  I want to get data off a USB thumbdrive at these speeds.
    #12
    Kiwi Mike AZ
    Super Member
    • Total Posts : 2056
    • Reward points : 0
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/04/17 11:01:21 (permalink)
    0
    Dont know about Host side..... thats what I am working on over the weekend.
     
    With the overhead for FAT etc I dont think 1MB/s will be possible...500K/sec maybe
     
    Will try and make a test.
     
    Mike
    #13
    chinzei
    Super Member
    • Total Posts : 2250
    • Reward points : 0
    • Joined: 2003/11/07 12:39:02
    • Location: Tokyo, Japan
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/04/17 17:52:12 (permalink)
    0
    xiaofan:

    You've a new record for Microchip MCUs (without SPP).

    For PIC18F2550/4550, it is less than 700KB.

    As mimemike works on PIC32, the comparison is not fair.
    For PIC18F, the process speed of the core limits the transfer speed. If PIC32 would do no more than PIC18F, it's rather a surprise for me.

    Tsuneo
    #14
    chinzei
    Super Member
    • Total Posts : 2250
    • Reward points : 0
    • Joined: 2003/11/07 12:39:02
    • Location: Tokyo, Japan
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/04/17 17:57:47 (permalink)
    0
    mimemike:

    I would not want to do more than 1024 bytes per ms (16 Packets) as this leaves room for error corrections.

    When the bulk endpoint (EP) is assigned enough bus bandwidth, transfer speed is tuned by the device side.
    For IN (device -> host) transfer, the rate, in which the firmware feeds packets to the bulk EP, determines the transfer speed.
    For OUT (host -> device) transfer, the rate, in which the firmware reads out the EP, regulates the transfer speed.

    In either direction, the host does NAKing to match the transfer speed. NAKing doesn't impact host CPU usage, because it is done just by the host controller without CPU intervention.

    mimemike:

    Also to get this performance it is critical to make sure it is the ONLY device connected to that USB Host controller.

    The performance will be achieved under these conditions.

    a) Single device on the bus
    Connect the device to the PC USB port directly, as you did.

    b) Over single TT (Transaction Translator) hub, without any other Full-/Low-speed device
    You can use a hub between the device and the PC. Rather, it speeds up the bulk transfer of the device.
    Hi-speed devices on the same hub doesn't disturb the Full-/Low-speed device.

    c) Over multiple TT hub
    In this case, you can use other Full-/Low-speed devices on the hub. The hub assigns full Full-/Low-speed bandwidth to each device on its ports.

    Most of Hi-Speed hubs on the market (usually, called as USB2.0 hub) are single-TT.

    Multi-TT hubs
    Belkin F5U234v1
    http://catalog.belkin.com/IWCatProductPage.process?Product_Id=159070
    IOGEAR GUH274
    http://www.iogear.com/product/GUH274/

    Tsuneo
    post edited by chinzei - 2009/04/17 18:01:00
    #15
    chinzei
    Super Member
    • Total Posts : 2250
    • Reward points : 0
    • Joined: 2003/11/07 12:39:02
    • Location: Tokyo, Japan
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/04/17 18:56:36 (permalink)
    0
    jmag99:

    Do you think it's possible to extract data at this speed when the PIC is a host? I want to get data off a USB thumbdrive at these speeds.


    I didn't check it on PIC, but I believe PIC32 will achieve this speed, which other full-speed host devices do for MSC (Mass Storage Class).

    Read: ~800 KBytes/sec
    Write: 300 - 400 KBytes/sec

    The point to speed up MSC transfer is reading / writing in the FAT cluster size.
    Microchip's host MSC implementation reads/ writes in sector (block) size - 512 bytes, to apply the same File System library to PIC18F MSC device. When it is optimized to cluster access for the USB host implementation, the overhead is reduced significantly (in the trade off of RAM requirement Smile ).

    "The Default Cluster Size for the NTFS and FAT File Systems"
    http://support.microsoft.com/kb/314878/

    Tsuneo
    post edited by chinzei - 2009/04/17 19:20:25
    #16
    daveinca
    Junior Member
    • Total Posts : 104
    • Reward points : 0
    • Joined: 2009/09/11 19:36:52
    • Location: Roseville, CA
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/11/15 10:30:31 (permalink)
    0
    Mike (or Tsuneo),

    How do you achieve these kinds of speeds? I've looked through all of the examples provided by Microchip and I don't see any examples of how you feed more than 255 bytes (the size field in calls like putUSBUSART() or USBTransferOnePacket() is always a single byte) to the USB buffers. Do you have to circumvent the USB stack and put your data directly into the USB DMA buffers or something of that nature?

    Tsuneo mentioned that the device transfer speed is determined by how much bandwidth you provide the bulk endpoint. Based on reading through the USB descriptors, I don't see any setting related to bandwidth. He does go on to say that the rate is based on how many packets you feed the bulk EP. Again, it seems as though the limitation is in the stack rather than how fast you can shove data at it.

    Of course, I've been using CDC as my model and that may be part of my problem. Based on other posts by Mike, it looks like if I want to achieve rates of 5MB/sec I will need to go to a custom class of device. Is that any example of how to create a high speed device? The amount of out data (host to device) is minimal (one control packet every few minutes kind of thing). But, my device will need to send large amounts of in data (device to host) but I don't understand how to do that. For instance, the code looks pretty much like this to send data:

        if (mUSBUSARTIsTxTrfReady()) {
             putUSBUSART((char*)&data_packets[send_index], 255);
        }

    Am I missing something or do I just need to dig a lot deeper into the stack to understand how to send more than 255 bytes every millisecond?
    #17
    Kiwi Mike AZ
    Super Member
    • Total Posts : 2056
    • Reward points : 0
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/11/15 12:27:33 (permalink)
    0

    1. You are using the CDC which has limitations
        I used Custom Class with BULK transfer and the Raw USB library calls (not USBUSART etc)

    2. You keep monitoring if the buffer has been sent and feed it as soon as the buffer is ready, need a tight loop to do this.

    3. I used the PIC32 specific stack

    Remember USB only sends 64 bytes at a time so you need to keep feeding the buffer the instant it is empty to fill all available time slots.

    Also at the PC end you need to read as much in one go, e.g. 1,048,576 bytes at a time (1MByte = 1024 x 1024)
    This PC buffer size may be limited depending on the library you used, I think the max I managed was 1,000,000 bytes at a time using LibUSB.

    Mike


    #18
    daveinca
    Junior Member
    • Total Posts : 104
    • Reward points : 0
    • Joined: 2009/09/11 19:36:52
    • Location: Roseville, CA
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/11/15 16:21:00 (permalink)
    0
    Thanks Mike. Can you tell me where those raw USB library calls are located (which file)? For instance, I would guess that you would use a call like USBGenWrite() to send some data out the USB port. I can't seem to find which file contains that function definition [or code].

    I am hoping that I can use a demo project as a starting point but I'm not sure which one to use. If you could give me a pointer or two as to which example code is best suited for this purpose, it would be very helpful. If there isn't anything on the Microchip web site, do you know of any other sites that may have examples that would be helpful?

    And finally, since you are using a tight loop, I would guess that we'd want to use polling rather than interrupts (#define USB_POLLING) to keep the loop even tighter. Is that correct?

    Thanks for you help on this. I am using the PIC32 USB stack (v2.5b I believe) as well so everything I need should be there. Given that I will need to poll some sensors [eventually], I may not get the performance you did but if I can get 5Mb/sec I will be very happy.

    On the PC side, I've downloaded the LibUSB files (v0.1.12.2) and will take a look at those. I've been using python to do my PC applications but I may need to move to C++ or C# given that I have to interface with the LibUSB API.
    #19
    daveinca
    Junior Member
    • Total Posts : 104
    • Reward points : 0
    • Joined: 2009/09/11 19:36:52
    • Location: Roseville, CA
    • Status: offline
    RE: Max USB Bulk transfer speed. 2009/11/17 10:20:54 (permalink)
    0
    Based on looking through the examples that come with the 2.5b USB stack, it looks like the following example would be a good starting point:
    USB Device - LibUSB - Generic Driver Demo
    Does that sound about right?

    I see that "functions" like USBGenWrite() and USBGenRead() are defined in the file usb_function_generic.h. They appear to just be wrappers for the function USBTransferOnePacket() which appears to be the basic function for reading from, or writing to, the USB endpoint.

    So I guess I'll just with the generic example and work from there.
    #20
    Page: 123 > Showing page 1 of 3
    Jump to:
    © 2020 APG vNext Commercial Version 4.5