Host interface to USB UVC compliant cameras using a PIC32 processor

Page: 12 > Showing page 1 of 2
Author
spcmicro
Starting Member
  • Total Posts : 58
  • Reward points : 0
  • Joined: 2009/04/20 02:58:29
  • Location: 0
  • Status: offline
2010/08/11 11:15:47 (permalink)
0

Host interface to USB UVC compliant cameras using a PIC32 processor

I am currently working on an interface for a PIC32 to UVC, USB Video Class, compliant web cameras. I only have to achieve 5 frames per second and a resolution of 320 x 240 so I believe the PIC32 is up to the task. I have most of the interface coded and running on an Explorer 16 board with the exception of the starting of the isochronous stream of video data. I can successfully set up the camera and select the desired interface. I know which endpoint I want to use and have successfully performed the probe and commit sequence. I attached the test camera to a PC and using a USB protocol analyzer I determined that I need to issue an ISOCH_TRANSFER with transfer flags set to USBD_TRANSFER_DIRECTION_IN ored with USBD_START_ISO_TRANSFER_ASAP. It appears to me as if the USBHostIssueDeviceRequest function may not support the use of the URB structure used for the transfer request.

Has anyone started an isochronous stream from the host side using the MAL USB stack and if so, can you point me in the right direction? I am using the latest version of the stack, version 2.7, which has a date of April 2010 on it.

Thanks in advance...
#1

39 Replies Related Threads

    DarioG
    Scheisse Menschen
    • Total Posts : 52257
    • Reward points : 0
    • Joined: 2006/02/25 08:58:22
    • Location: hi there
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2010/08/12 04:10:02 (permalink)
    0
    I only remember that somebody did consider writing such a host code, but AFAIK it was not carried on or at least that thread updated.

    Nice one, hope you succeed btw !

    Dario Greggio
    --
    Wouldn't it be wonderful, if you lost one son for every promise you don't keep?
    #2
    mzoran
    Super Member
    • Total Posts : 683
    • Reward points : 0
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2010/08/12 19:34:29 (permalink)
    0
     Here is the old thread:
    [link=http://www.microchip.com/forums/tm.aspx?m=498507&high]http://www.microchip.com/...aspx?m=498507&high[/link]=
     
    I didn't cary through with it because I got distracted by some other things and I also got discouraged that someone said that the newer cameras may only work at hispeed which few general purpose microcontrollers support.   I didn't feel like going on a fishing trip to find cameras that would work.
     
    I think alot of people are interested to know if this can be done, but I also suspect you are in uncharted waters as far as microcontrollers are conserned.   You may be able to find some help by looking at freeware code such as Linux or a Linux application.
     
     
     
     
    #3
    DarioG
    Scheisse Menschen
    • Total Posts : 52257
    • Reward points : 0
    • Joined: 2006/02/25 08:58:22
    • Location: hi there
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2010/08/13 01:17:44 (permalink)
    0
    Yep it was you MZoran Smile
    thx

    Dario Greggio
    --
    Wouldn't it be wonderful, if you lost one son for every promise you don't keep?
    #4
    spcmicro
    Starting Member
    • Total Posts : 58
    • Reward points : 0
    • Joined: 2009/04/20 02:58:29
    • Location: 0
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2010/08/23 08:18:25 (permalink)
    +3 (2)
    Update:

    I have most of the USB host interface for the USB Video Class driver working.

    1. I can view all of the descriptors for the device configuration, available interfaces, endpoints, etc... The Microchip USB host stack loads and parses them. I added debug code in the usb_host.c file to display them on the serial port debug output. You can plug in different UVC camera and all of the descriptors are parsed and displayed.
    2. I can set the camera's attributes such as brightness, contrast, etc...
    3. I can perform a probe and commit to select the desired interface and endpoint for streaming the video.
    4. I can start an isochronous video stream, compressed or uncompressed, and I receive packets, via transfer events, up to the number of buffers that I have allocated.

    Here is where I ran into a problem with the library. The PIC32 has 32K of memory and this is not enough to buffer a single frame of video data. The buffers you pass to the USB host library are not re-used and there appears to be no mechanism to tell the host library code that a buffer has been emptied on the client side and can be re-used. I have 22 buffers allocated and when the are all full, the host code just stops firing transfer events. The code just ignores the incoming packets. I have a request in with the support group for quite a while now but they have not responded yet. I can, and will, modify the usb_host.c source code to re-use the buffers in a ring mode but I'd rather have Microchip fix it. Without the re-use of buffer space the support of streaming devices with large amounts of data will not be possible. Maybe there is already a mechanism to tell the host code that previously filled buffers are ready for more data, but I can't find it. As it was, I had to dig into and step through the USB stack source code to answer some of my questions but I eventually resolved all of the problems I was having. I learned a lot by stepping the code, but I'm on a tight schedule and it would have been nice to have some better documentation. There are a couple of application notes from Microchip on the host stack but they are the usual bare minimum required to call it a document. In my opinion, Microchip could do a much better job at documenting the more complicated applications of their hardware and libraries.

    Some random notes:

    1. The application I'm writing transfers the video data to another processor, via SPI, that stores the images on an SD card. I only have to empty the buffers fast enough to keep up with the incoming stream. I think this will work at slow frame rates with smaller image sizes. My minimum target is 5 FPS at 320 x 240.

    2. The indispensable UVC document I used was here:

    http://www.usb.org/developers/devclass_docs/USB_Video_Class_1_1.zip

    There are 11 documents in the zip file but the chapter that was the most useful was the USB_Video_Class_1.1.pdf document. The payload documents are helpful when you get to that point and the FAQ had some tidbits in it that I also found useful.

    3. I purchased Jan Axelson's USB Complete book and it was very helpful with respect to USB host basics. Many thanks to Jan...

    4. I used a USB protocol analyzer to sniff packets with cameras hooked up to a PC.

    5. A very handy tool for viewing UVC descriptors for cameras is UVCView.x86.exe. It is from Microsoft and is described at:

    http://msdn.microsoft.com/en-us/library/ff554257%28VS.85%29.aspx

    You plug a camera into the PC and run the application and it displays all of the descriptors for the device. I used this to double check the descriptor information that I was displaying in the debug code I added to the usb_host.c source.

    More to come...



    #5
    mzoran
    Super Member
    • Total Posts : 683
    • Reward points : 0
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2010/08/23 17:33:56 (permalink)
    0
    That is a huge amount of information to give out.  It will help me if/when I ever get to this myself.

    As for the Microchip stack(I hope nobody deletes this post), I've always thought of it more of a starting point for my hobby projects rather then an end to end solution.   I've generally have always needed to make heavy modifications myself.  I think that is the whole point of the USB stack being open source( as long as you use it on MC products).  You can make your own changes.

    Thanks again for all the wonderful information.
    #6
    mzoran
    Super Member
    • Total Posts : 683
    • Reward points : 0
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2010/08/23 17:36:52 (permalink)
    0
    BTW, could you share out some of the USB camera models that you have been using?
    #7
    chinzei
    Super Member
    • Total Posts : 2250
    • Reward points : 0
    • Joined: 2003/11/07 12:39:02
    • Location: Tokyo, Japan
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2010/08/23 23:46:00 (permalink)
    0
    Here is where I ran into a problem with the library. The PIC32 has 32K of memory and this is not enough to buffer a single frame of video data. The buffers you pass to the USB host library are not re-used and there appears to be no mechanism to tell the host library code that a buffer has been emptied on the client side and can be re-used.

    It is already implemented on the V2.7 stack.
    The stack raises bfDataLengthValid of ISOCHRONOUS_DATA_BUFFER structure for the target buffer, when a transaction completes. The stack expects that user application drops bfDataLengthValid, when it finishes the data process on the buffer. And then, the stack recycles the buffers.
    For polling configuration, your user application should drop bfDataLengthValid, while polling single transaction completion.

    But a minor fix is required on usb_host.c, when you apply USB_ENABLE_TRANSFER_EVENT, instead of polling.
    The stack queues just the data buffer (pBuffer) to the USB_EVENT_DATA structure, when the transaction completes. The stack should pass the pointer of ISOCHRONOUS_DATA_BUFFER structure, so that user app can access to bfDataLengthValid field.
    Here is the modified code.
     
    usb_host.c

    void _USB_FindNextToken( void )
    {
        ...
        ...
        case TSUBSTATE_ISOCHRONOUS_READ_COMPLETE:
            ...
            #if defined( USB_ENABLE_ISOC_TRANSFER_EVENT )
                if (StructQueueIsNotFull(&usbEventQueue, USB_EVENT_QUEUE_DEPTH))
                {
                    USB_EVENT_DATA *data;

                    data = StructQueueAdd(&usbEventQueue, USB_EVENT_QUEUE_DEPTH);
                    data->event = EVENT_TRANSFER;
                    data->TransferData.dataCount        = pCurrentEndpoint->dataCount;
                //                                                                  <--------- modify this line as follows
                //  data->TransferData.pUserData        = ((ISOCHRONOUS_DATA *)(pCurrentEndpoint->pUserData))->buffers[((ISOCHRONOUS_DATA *)(pCurrentEndpoint->pUserData))->currentBufferUSB].pBuffer;
                    data->TransferData.pUserData        = (BYTE *)&(((ISOCHRONOUS_DATA *)(pCurrentEndpoint->pUserData))->buffers[((ISOCHRONOUS_DATA *)(pCurrentEndpoint->pUserData))->currentBufferUSB]);


    On the user application side,
     

    CLIENT_DRIVER_TABLE usbClientDrvTable[] =
    {
        {
            USBHostUVCInitialize,
            USBHostUVCEventHandler,
            0
        }
    };

    BOOL USBHostUVCEventHandler( BYTE address, USB_EVENT event, void *data, DWORD size );
    {
        ISOCHRONOUS_DATA_BUFFER * isoc_buf;

        switch ( event ) {
            case EVENT_TRANSFER;
                isoc_buf = (ISOCHRONOUS_DATA_BUFFER *)(((HOST_TRANSFER_DATA *)data)->pUserData);
                                                    // process data on the buffer for single transaction
                // isoc_buf->pBuffer                //   data buffer pointer
                // isoc_buf->dataLength             //   bytes on the buffer

                isoc_buf->bfDataLengthValid = 0;    // drop the valid flag of this buffer
                break;
        }
    }


    Tsuneo
    post edited by chinzei - 2010/08/24 01:29:09
    #8
    chinzei
    Super Member
    • Total Posts : 2250
    • Reward points : 0
    • Joined: 2003/11/07 12:39:02
    • Location: Tokyo, Japan
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2010/08/24 00:25:43 (permalink)
    0

    BTW, could you share out some of the USB camera models that you have been using?

    Mine is Elecom UCAM-DLO130, which is equipped Alcor micro AU3820.
    Unfortunately, both of the webcam and the chip has been discontinued. It was about two years ago, when I played with UVC. In those days, I've tried a couple of webcams, but only this model put images on Full-speed, too.

    I don't know about recent webcams on the market.
    I also expect spcmicro could share his webcam models.

    Tsuneo
    #9
    spcmicro
    Starting Member
    • Total Posts : 58
    • Reward points : 0
    • Joined: 2009/04/20 02:58:29
    • Location: 0
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2010/08/24 04:53:37 (permalink)
    0
    I have been using the Eyeball 2.0 camera from Blue Microphone. It only draws 128 mA which is a bit over the 100mA max for the Explorer 16 board. I haven't had a any problems yet, though I intend on modifying my USB PICtail board to supply up to 500 mA. The other camera I have used is a tiny one from Gigaware. The Gigaware camera is identified as a USB 2.0 camera from Sonix Technology. I have also sniffed a Microsoft Lifecam 2000, a Logitech QuickCam, and an iHome MyLife camera. These 3 cameras require up to 500 mA current and so I have not hooked them up to my development system yet. Most new Webcams are UVC compliant so that they can be used in Windows XP and Windows 7 without requiring a special driver disk. Installation disks are still provided for some UVC compliant cameras to enable extensions by vendors to support special features of their camera models.

    Many thanks to chinzei for the code change suggestions. I had been in this code and saw the bDataLengthValid flag. I had also tried clearing it in the host code, just for throughput testing, and it the host code didn't re-cycle the buffers. I must have done something wrong so I am going to re-test the changes today. I like your modification to pass the structure rather than just the buffer. I will make the changes today and get back to everyone.

    This should be the last piece of the puzzle. When I get this working I will be benchmarking the throughput and will publish the results.


    #10
    spcmicro
    Starting Member
    • Total Posts : 58
    • Reward points : 0
    • Joined: 2009/04/20 02:58:29
    • Location: 0
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2010/08/24 05:19:25 (permalink)
    0
    I just made the modifications that chinzei suggested, using transfer events, and the host code is now re-cycling the buffers. Thanks again chinzei...

    More to come... .. .

    #11
    spcmicro
    Starting Member
    • Total Posts : 58
    • Reward points : 0
    • Joined: 2009/04/20 02:58:29
    • Location: 0
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2010/10/13 04:11:44 (permalink)
    0
    Update:

    I now have the camera running reliably at 5 frames per second at 320 x 240. The camera is sending data at 15 frames per second and I'm throwing extra frames away. The second processor in the design buffers and stores data on an SD card. I had to use a class 10 card and the FATfs open source library but now I can save compressed JPEG frames that are between 16 and 32 K bytes and keep up with the camera. I'm having a problem with the format of the JPEG frames from the camera, they have a bug in their code that gets the Huffman table marker lengths wrong, but we are getting another camera that we've been told does not exhibit the problem.  I designed the interface to work with any UVC compliant camera so that as long as a camera supports the image size we need at the lower 15 FPS frame rate, we can use it.

    I'm thinking of writing an application note on Isochronous transfers using the Microchip USB host stack. Maybe when the activity on this project settle down. It's a good Winter project...

    #12
    ChrisPIC
    New Member
    • Total Posts : 4
    • Reward points : 0
    • Joined: 2011/01/19 08:08:14
    • Location: 0
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2011/01/19 08:30:32 (permalink)
    0
    Thanks for great discusion.
     
    i am just getting started trying something similar.  i have PIC32 USB eval board logitech c210 webcam.
     
    i'm confused by subclass for UVC and how to configure TPL (Target Peripheral List) in host stack.  seems there are three subclass's (A.2 of "USB_Video_Class_1.1.pdf").  SC_VIDEOCONTROL 0x01, SC_VIDEOSTREAMING 0x02, and SC_VIDEO_INTERFACE_COLLECTION 0x03.
     
    do i make three entries in TPL table, or just one? if one, which one?
     
    also, i have been modifying host stack code manually (TPL table).  just realized configuration tool exists (USBConfig.exe).  Should i use config tool only, or should i expect to need to modify code manually?
     
    Thanks
    Chris
    #13
    chinzei
    Super Member
    • Total Posts : 2250
    • Reward points : 0
    • Joined: 2003/11/07 12:39:02
    • Location: Tokyo, Japan
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2011/01/19 21:49:24 (permalink)
    0

    do i make three entries in TPL table, or just one? if one, which one?

    Read out the descriptors from the target webcam on a PC.
    And then, you'll know the class triad on the webcam.
    OR, you may assign VID/PID of the webcam to the TPL, instead of the class triad.

    Should i use config tool only, or should i expect to need to modify code manually?

    Microchip doesn't provide any host class driver for UVC.
    The config tool does nothing for not-supported class, unless you apply a generic one.

    Sound like it's your first project of host stack.
    I recommend you to play with other host examples on the stack for a while, until you become familiar with host coding. UVC implementation is really tough. I believe spcmicro is an experienced one. It's the reason why he finished his project just with such a trivial advises.

    Tsuneo
    #14
    ChrisPIC
    New Member
    • Total Posts : 4
    • Reward points : 0
    • Joined: 2011/01/19 08:08:14
    • Location: 0
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2011/01/23 19:59:35 (permalink)
    0
    Tsuneo, Thanks
     
    i used SC_VIDEOCONTROL 0x01 only (one entry in usbTPL) and it works so far (i get lots of descriptors).
     
    Yes, i understand i need to add UVC stuff.  i was/am confused weather it is ok to modify usb_host code myself or i should only use USBConfig.exe.  Not sure if USBConfig.exe does lots of magic stuff that i will struggle if i try to work with out it.  but i think i will continue without using USBConfig.exe unless someone tells me that is a mistake.
     
    chris
     
    #15
    ChrisPIC
    New Member
    • Total Posts : 4
    • Reward points : 0
    • Joined: 2011/01/19 08:08:14
    • Location: 0
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2011/02/09 22:45:44 (permalink)
    0
    I'm trying to figure out how to get video stream data to start occuring???
     
    I used the UVCView.x86.exe program and logged descriptor information.  very helpful!! (log attached)
    I'm using PIC32 USB, and logitech webcam c210.
     
    i enabled transfer events
         #define USB_SUPPORT_HOST
         #define USB_SUPPORT_INTERRUPT_TRANSFERS
         #define USB_ENABLE_TRANSFER_EVENT
    i can get/set attribures like brightness.
    i can set/get probe, commit, i think.
     
    but i don't know what next steps should be???  i tried setting various alternate settings to various interface numbers, and then wait
    for data to be recieved in event handler.  but does not seem like any video data arrives (events always come with endpoint = 0, not 81 or 87 like i hope/expect).
     
    any advise would be greatly appriciated.
     
    Thanks
    Chris
     
     
    #16
    chinzei
    Super Member
    • Total Posts : 2250
    • Reward points : 0
    • Joined: 2003/11/07 12:39:02
    • Location: Tokyo, Japan
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2011/02/11 09:02:00 (permalink)
    0

    i tried setting various alternate settings to various interface numbers, and then wait

    1) Enable one of alternate interfaces for video streaming using Set_Interface request, depending on expected screen size (or format)
    2) Start isoc IN tranfer sequence.

    - Enable isoc on usb_config.h
    #define USB_SUPPORT_ISOCHRONOUS_TRANSFERS

    - For the first-time capture, reserve isoc buffers using USBHostIsochronousBuffersCreate()
    - For second and later run, clear isoc buffer using USBHostIsochronousBuffersReset()

    - Start isoc IN transfer, call USBHostReadIsochronous()
    Every transaction completes, USBHostUVCEventHandler() (implemented by your code. See above post  http://www.microchip.com/forums/fb.ashx?m=520716 ) is called with EVENT_TRANSFER.

    The fix on above post is applied to v2010-10-19 stack, too.

    Tsuneo
    #17
    ChrisPIC
    New Member
    • Total Posts : 4
    • Reward points : 0
    • Joined: 2011/01/19 08:08:14
    • Location: 0
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2011/02/11 22:37:46 (permalink)
    0
    Tsuneo,
     
    Thanks so much for your reply.  I left work early to go home and try it.
     
    Yes, your answer was correct.  i now recieve video data.  Now i can figure out how to parse data,.... :-)
     
    in addition to USB_SUPPORT_ISOCHRONOUS_TRANSFERS as you recommended i also added USB_ENABLE_ISOC_TRANSFER_EVENT to usb_config.h.
      #define USB_SUPPORT_ISOCHRONOUS_TRANSFERS
      #define USB_ENABLE_ISOC_TRANSFER_EVENT
     
    Chris 
    #18
    chinzei
    Super Member
    • Total Posts : 2250
    • Reward points : 0
    • Joined: 2003/11/07 12:39:02
    • Location: Tokyo, Japan
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2011/02/12 02:26:20 (permalink)
    0
    Now i can figure out how to parse data

    One of my friend has a Logitech (LogiCool) C200, and he kindly lets me examine it closely for a while.
    For C200 and Logitech's latest driver on Win7 (x86), the outline of the start sequence of a scan is as follows,
    (the trace is taken over a USB1.1 hub)
    - a series of Probe requests (GET_CUR, MIN, MAX, RES, and SET_CUR), and then Commit request
    - Set_Interface for alternate
    - Clear_Feature( ENDPOINT_HALT, 0x87 )    <-- ??? I don't understand why this request is here
    - SET_CUR for one of EXTENSION
    - Isoc IN transactions for video frames

    A couple of first video frames are blank - the transactions hold just 12 bytes header.
    Even for non-blank video frame, the first transaction of the frame is often blank (just a header).

    C210 is a newer one than C200.
    I'm not sure if this observation on C200 is immediately applied to C210, or not. But hope so.

    Tsuneo
    post edited by chinzei - 2011/02/12 02:31:21
    #19
    paporas
    New Member
    • Total Posts : 1
    • Reward points : 0
    • Joined: 2011/03/29 00:23:12
    • Location: 0
    • Status: offline
    Re:Host interface to USB UVC compliant cameras using a PIC32 processor 2011/03/29 00:41:17 (permalink)
    0
    Hi Tsuneo, spcmicro and All,

    This is a great thread and thank you for all the information. 
    I want to interface a USB webcam to a PIC32, acquire data at about 10fps (320x240) ideally, and apply a motion detection algorithm to the stream. Since you already have experience on the subject, is the PIC32 capable of the task? From what I understand, the PIC can handle the acquisition (spcmicro was successful in acquiring 5fps), but does it also have the resources to process the images as they come (motion detection in a specific ROI) ?
    Just to explain the application better, when motion is detected, 10s of data should be recorder (any type of memory, non-permanent). Those data should afterwards be analysed by the PIC32, but this is something that is done without having to keep up with the video stream, so I am sure it can be done easily.

    Thank you in advance
    #20
    Page: 12 > Showing page 1 of 2
    Jump to:
    © 2017 APG vNext Commercial Version 4.5