continuously stream data over UDP?

Author
Griffin8116
New Member
  • Total Posts : 5
  • Reward points : 0
  • Joined: 2011/02/27 20:44:52
  • Location: 0
  • Status: offline
2011/03/01 09:42:22 (permalink)
0

continuously stream data over UDP?

Hi,

I'm trying to stream data from an Olimex MICRO-WEB (PIC18F67J60) and the TCP/IP stack v5.00. In essence, I'm using the ADCs on the PIC to take some measurements, filling a buffer and sending that information out as a UDP packet. The nature of the data is such that I need to be sampling the ADCs continuously and firing that data out, with no dead time while the UDP buffer is being flushed. Thus, I have created two 'buffers' for the data and implemented them such that one is being filled as the other is being read out. The trick is, is that the time that it takes to flush the buffer is larger than the time required to fill it (and introducing a third buffer does not solve the problem). 

I want to be able to send UDP packets in the background, and reliably be able to do other things (ie, sample the ADCs) on the 0.1s timescale. Taking data for 100ms and having a deadtime (ie, readout time) of 100ms would be fine, too, but there doesn't seem to be enough RAM available to do so. 

The actual data rate that I need to stream is 75,000 bytes / s, excluding UDP overhead.

Has anyone else tried to implement something similar / has any suggestions?

Thanks in advance,

Sean.


[font="'times new roman'"]

#1

10 Replies Related Threads

    bkamen
    Super Member
    • Total Posts : 822
    • Reward points : 0
    • Joined: 2010/01/26 12:39:27
    • Location: Central Illinois, USA
    • Status: offline
    Re:continuously stream data over UDP? 2011/03/01 10:44:47 (permalink)
    0

    I would ponder resizing the TX buffer space inside the ethernet module. There's 8K to use and since you're mostly sending, you might be able to fill that buffer and then when full, the "flush" command essentially tells the module to transmit while you've moved onto the next module.

    You may have to depart a little from what MCHP libraries exist to fulfill your application needs.

     -Ben

    p.s. Keep in mind UDP is a "best effort" protocol. Delivery is not guaranteed.
    #2
    sborden
    Super Member
    • Total Posts : 1932
    • Reward points : 0
    • Joined: 2010/08/05 02:12:53
    • Location: 0
    • Status: offline
    Re:continuously stream data over UDP? 2011/03/01 13:07:52 (permalink)
    0
    I have helped someone do something similar (acquire continuously and stream at 1kHz) with that chip.  What is your acquisition rate?  The TCP stack is not really that good for this kind of thing.  You need to carefully design your main loop software to avoid over-calling the stack functions, as they have a lot of overhead.
    #3
    Griffin8116
    New Member
    • Total Posts : 5
    • Reward points : 0
    • Joined: 2011/02/27 20:44:52
    • Location: 0
    • Status: offline
    Re:continuously stream data over UDP? 2011/03/02 09:11:22 (permalink)
    0

    Thanks for the responses.

    The acquisition rate is 2 bytes @ 25kHz plus 10 bytes @ 2.5kHz, triggered by means of an interrupt.
    We've figured out how large each of our buffers needs to be in order to keep the stack from crashing (turns out to be ~60 bytes) because it's being asked to execute too often.
    #4
    Griffin8116
    New Member
    • Total Posts : 5
    • Reward points : 0
    • Joined: 2011/02/27 20:44:52
    • Location: 0
    • Status: offline
    Re:continuously stream data over UDP? 2011/03/04 07:40:29 (permalink)
    0
    Does anyone know if there are any alternatives to using the TCP/IP stack that have less overhead?
    #5
    rusttree
    Super Member
    • Total Posts : 906
    • Reward points : 0
    • Joined: 2010/06/14 14:10:14
    • Location: Phoenix, AZ
    • Status: offline
    Re:continuously stream data over UDP? 2011/03/04 10:43:55 (permalink)
    0
    If you're ambitious, you can just write your own mini-stack.  If the ONLY purpose of your application is to stream UDP packets, you can forgo the majority of protocols and error checking.

    I believe all you would need is:
    1. DHCP capability to request an IP address.  Conveniently, DHCP uses UDP.
    2. ARP to retrieve the gateway's MAC address.
    3. Code to stream UDP packets.

    And that's pretty much it.  Of course, you will have no TCP capability, you won't be able to ping your device, etc. 
    #6
    sborden
    Super Member
    • Total Posts : 1932
    • Reward points : 0
    • Joined: 2010/08/05 02:12:53
    • Location: 0
    • Status: offline
    Re:continuously stream data over UDP? 2011/03/05 12:25:06 (permalink)
    0
    UART/SPI to TCP hardware module Smile
    #7
    Griffin8116
    New Member
    • Total Posts : 5
    • Reward points : 0
    • Joined: 2011/02/27 20:44:52
    • Location: 0
    • Status: offline
    Re:continuously stream data over UDP? 2011/03/08 08:53:55 (permalink)
    0
    Hi, all,

    @rusttree:

    The only purpose is, indeed, to stream UDP packets. I'm not even certain we need to implement DHCP capability because the plan is to have a computer dedicated to listening to the incoming data (we're going to have a dozen or so of these little gadgets sitting on a dedicated network). Thus, the target's MAC could even be hard-coded into the firmware. 

    This is how it's been implemented right now, we're just using the UDPPut(), UDPFlush(), and the socket state machine. Because of the overhead therein (presumably) we're getting rate issues and too much dead time. 

    The lack of TCP capability is not the end of the world, we'd like to implement the ability to get a timestamp from an NTP server so that we could synchronize our data, but this is currently on the back burner until the rate issues have been solved.

    @sborden: This actually sounds interesting, might be worth a gander. 




    #8
    rusttree
    Super Member
    • Total Posts : 906
    • Reward points : 0
    • Joined: 2010/06/14 14:10:14
    • Location: Phoenix, AZ
    • Status: offline
    Re:continuously stream data over UDP? 2011/03/08 09:27:56 (permalink)
    0
    You may be able to simplify it even more.  What is the hardware on the receiving end of the UDP datastream?
    #9
    Griffin8116
    New Member
    • Total Posts : 5
    • Reward points : 0
    • Joined: 2011/02/27 20:44:52
    • Location: 0
    • Status: offline
    Re:continuously stream data over UDP? 2011/03/14 06:22:31 (permalink)
    0
    The receiving end of our setup is a regular PC running Linux.
    #10
    rusttree
    Super Member
    • Total Posts : 906
    • Reward points : 0
    • Joined: 2010/06/14 14:10:14
    • Location: Phoenix, AZ
    • Status: offline
    Re:continuously stream data over UDP? 2011/03/14 09:37:34 (permalink)
    0
    I'd recommend implementing DHCP capability just because it's so simple, it uses UDP anyway, and if you ever want to get through to an NTP server, you'll need to make nice with your network router.

    I recently went through the exercise of writing my own bare-minimum mini-stack.  Let me know if you need some pointers.
    #11
    Jump to:
    © 2017 APG vNext Commercial Version 4.5