• AVR Freaks

Hot!How much heap do I really need for the TCP/IP Stack?

Author
jdeguire
Super Member
  • Total Posts : 469
  • Reward points : 0
  • Joined: 2012/01/13 07:48:44
  • Location: United States
  • Status: offline
2015/08/06 11:34:54 (permalink)
0

How much heap do I really need for the TCP/IP Stack?

This is something I've been wondering about for a while now.  With the MPLAB Harmony Configurator, no matter how big I make my default TCP or UDP buffers and no matter how many sockets I allocate for each or Ethernet MAC descriptors I allocate, the "TCP/IP Stack Dynamic RAM Size" value always stays at 39250 bytes.  Is there a formula or simple rule of thumb for figuring out just how much heap space the TCP/IP Stack actually needs?  In my case, I probably need less than the default since I reduce the number of available sockets (though I do bump up the default TCP window sizes), but this would be especially useful for making sure we don't use too much before running the app.
 
It'd be totally awesome if the MHC could figure out dynamically how much heap I need for the TCP/IP Stack, Ethernet MAC descriptors, and other drivers and set up the heap allocated by the linker based on that.  This probably isn't a simple task, however.
 
Thanks!
#1

8 Replies Related Threads

    roundrocktom
    Super Member
    • Total Posts : 1023
    • Reward points : 0
    • Joined: 2012/09/07 08:19:09
    • Location: Texas
    • Status: offline
    Re: How much heap do I really need for the TCP/IP Stack? 2015/08/07 05:56:37 (permalink)
    5 (1)
    Harmony Help is your friend
    I typed in "tcp stack size" and got:
     
    MPLAB Harmony TCP/IP Stack Heap Configuration
     

    MPLAB Harmony TCP/IP Stack Heap Configuration

    The MPLAB Harmony TCP/IP Stack uses dynamic memory for both initialization and run time buffers. Therefore, there are two requirements for a project containing the stack to work properly.

    The first requirement is that the TCP/IP stack should be configured with enough heap space. The amount of heap used by the stack is specified by the TCPIP_STACK_DRAM_SIZE parameter in the tcpip_config.h file.

    The value of the required TCP/IP heap for a project is application dependent. Some of the most important factors that impact the heap size are:

        Number of TCP sockets
            By default, each TCP_socket_requires 512 bytes for a RX buffer and 512 bytes for a TX buffer. These parameters can be adjusted by modifying the values specified in tcp_config.h or dynamically through the TCPIP_TCP_OptionsSet function.
        Number of UDP sockets:
            For each UDP_socket_that needs to initiate a transmission, the IP_layer will have to allocate the required space (suggested by the functions, UDPv4IsTxPutReady or UDPv4IsTxPutReady
            Once the UDP buffering will be added, each_socket_will have its own RX and TX buffers. These parameters can be adjusted by modifying the values specified in the file, udp_config.h, or dynamically through the TCPIP_UDP_OptionsSet function
        The type of Ethernet MAC that is used:If SSL is enabled, it will require additional buffering for encryption/decryption at run-time. The SSL module buffering requirement is dependent on the RSA key length specified in ssl_config.h.
            The PIC32MX6XX/7XX devices with a built-in 100 Mbps Ethernet controller use system memory for buffering incoming RX traffic. For sustained 100 Mbps operation, adequate RX buffer space must be provided. This parameter can be adjusted by modifying the values specified in the file, network_config.h, using the array, TCPIP_HOSTS_CONFIGURATION.

    The second requirement is that the project that includes the MPLAB Harmony TCP/IP Stack must be built with sufficient heap size to accommodate the stack (at a minimum, the project needs the TCPIP_STACK_DRAM_SIZE bytes of heap). This parameter is adjusted on the Linker tab in the project properties.

    In general, for a TCP/IP project running on a PIC32MX device with an embedded Ethernet controller, at least 8 KB of heap space is needed by the stack. However, the following is implied:

        100 Mbps traffic is not sustained
        No SSL
        Relatively few sockets

    A typical value for comfortably handling 100 Mbps Ethernet traffic would be 40 KB of TCP/IP heap space.

    The amount of the required


    [SNIP]

     
    #2
    jdeguire
    Super Member
    • Total Posts : 469
    • Reward points : 0
    • Joined: 2012/01/13 07:48:44
    • Location: United States
    • Status: offline
    Re: How much heap do I really need for the TCP/IP Stack? 2015/08/07 14:15:10 (permalink)
    0
    I've seen that section in the PDF guide, but it does not completely answer my question.
     
    Yes, it's pretty obvious that Tx and Rx buffers take up space.  However, if you actually look in the stack code, you'll see that other internal structures are also allocated on the heap.  For example, every TCP socket has its own Transmission Control Block structure that is allocated on the heap when the stack is initialized, though this is not mentioned in the document.  Most other modules have similar structures.  Extra space is also required for the PIC32 MAC packet descriptors as well as space for TCP and UDP headers that are allocated when a packet is sent.
     
    I did not see anything in the document that mentions the space used up by these internal structures.  Of course I can go delving into the code to get my answer, but I was hoping I wouldn't have to do that.  I could also just leave it at 40KB like the document suggests, but it'd be nice to actually know what I need.  The "heapinfo" command the guide mentions may be useful, at least.
    #3
    NorthGuy
    Super Member
    • Total Posts : 5544
    • Reward points : 0
    • Joined: 2014/02/23 14:23:23
    • Location: Northern Canada
    • Status: offline
    Re: How much heap do I really need for the TCP/IP Stack? 2015/08/07 15:49:32 (permalink)
    3 (1)
    As a rule of thimb, the hip should be at least twice as big as your real demand for memory. Start small and increase the size of the heap until you stop geting allocation errors at your max loads. Then double this size. This is to account for fragmentation loss which may develop over time.
     
    #4
    roundrocktom
    Super Member
    • Total Posts : 1023
    • Reward points : 0
    • Joined: 2012/09/07 08:19:09
    • Location: Texas
    • Status: offline
    Re: How much heap do I really need for the TCP/IP Stack? 2015/08/09 05:47:29 (permalink)
    4 (1)
    Two numbers are given in the Stack Heap Configuration:
    8K mininum (100Mbps not sustained, no SSL, just a couple of sockets);
    40K being comfortable
     
    The more heap space you have, the better for performance.  If you have a ton of traffic coming in, but the processor can not get to the buffer,  packets are either rejected (incoming) or deferred to a later time (out going).  You will impace performance.  With wireshark you could document this, but if it impacts less than 0.5% of the time, and I can save money with less RAM it seems reasonable trade off.  
     
    Since the buffer size is tied to overall system loading, there isn't a fixed answer.  I noticed our SW guys use 20K to 25K as a compromise.
     
    The Harmony Documentation has some good tips:
    By enabling the parameter, TCPIP_STACK_DRAM_DEBUG_ENABLE, in the tcpip_config.h file, the stack will output debug messages when it runs out of memory. Then, using the heapinfo command at the TCP/IP command processor prompt will return a snapshot of the current TCP/IP heap status and can help in early detection of problems.

    Optionally, enabling the parameter, TCPIP_STACK_DRAM_TRACE_ENABLE in the tcpip_config.h file, will instruct the TCP/IP heap allocation module to store trace information that will be displayed with the heapinfo command.

     
     
     
    #5
    jdeguire
    Super Member
    • Total Posts : 469
    • Reward points : 0
    • Joined: 2012/01/13 07:48:44
    • Location: United States
    • Status: offline
    Re: How much heap do I really need for the TCP/IP Stack? 2015/08/10 07:45:14 (permalink)
    0
    Thanks for the answers, you two!  It sounds like that the TCPIP_STACK_DRAM_TRACE_ENABLE and the "heapinfo" command will be my best bet.  The default value of 40KB is working just fine, so this was mostly to satisfy my curiosity.  I'll update this thread in the future if I decide to pursue my question further.
    #6
    NKurzman
    A Guy on the Net
    • Total Posts : 17615
    • Reward points : 0
    • Joined: 2008/01/16 19:33:48
    • Location: 0
    • Status: offline
    Re: How much heap do I really need for the TCP/IP Stack? 2015/08/10 14:39:40 (permalink)
    0
    Is there a way to get the TCP/IP  heapinfo numbers?  I see Command_HeapInfo(), but it only prints.
    TCPIP_HEAP_MaxSize() and TCPIP_HEAP_FreeSize() do not work standalone.
     
     I would like to monitor in the Program.
    #7
    quijote
    Starting Member
    • Total Posts : 89
    • Reward points : 0
    • Joined: 2005/01/27 04:06:37
    • Location: Spain
    • Status: offline
    Re: How much heap do I really need for the TCP/IP Stack? 2019/02/04 06:55:23 (permalink)
    0
    Hello All,
     
    I am trying to get real-time heap information about the TCP/IP stack using the heapinfo command, but I do not see where to call the 'TCP/IP Command Processor' from.
     
    Since my PIC has AppIO enabled, I am wondering if I can simply type this into the AppIO window?
     
    The Harmony manual unfortunately only has one reference to the TCP/IP Command Processor on page 6760 but does not document how to access it.
     
    Any help much appreciated.!
     
    #8
    rainad
    Moderator
    • Total Posts : 1205
    • Reward points : 0
    • Joined: 2009/05/01 13:39:25
    • Location: 0
    • Status: offline
    Re: How much heap do I really need for the TCP/IP Stack? 2019/02/04 09:09:32 (permalink)
    0
    The file tcpip_heap.h exports the TCP/IP heap API which you could use at run time in your app, exactly like the tcpip_command.c "heapinfo" does - there is no need to call the command itself in your code, that's just an example of how to use it.
    Get the heap handle with TCPIP_STACK_HeapHandleGet() and then you can call the TCPIP_STACK_HEAP_Size, TCPIP_STACK_HEAP_FreeSize, TCPIP_STACK_HEAP_HighWatermark, etc.
    You can display the values or use them as you see fit.
     
    #9
    Jump to:
    © 2019 APG vNext Commercial Version 4.5