• AVR Freaks

Hot!Socket always gets marked as "wasReset"

Author
David Green
Junior Member
  • Total Posts : 33
  • Reward points : 0
  • Joined: 2017/04/10 07:57:13
  • Location: 0
  • Status: offline
2019/04/11 08:26:51 (permalink)
0

Socket always gets marked as "wasReset"

Hi there,

PIC32MZ2048EFM
Harmony v2.06

I had problems with the HTTPNet Server when I stressed it with many requests in a row.
I sent 1000 Requests in one minute. The payload size of the request to the PIC was 54 Bytes, the returned payload was 116 Bytes.

For some reason the client sometimes (at various moments within this minute) reported that the connection was unexpectedly closed by the server.
I found out that the NET_PRES_SocketWasReset() function, called in the http_net.c returned true sometimes for no reason.
Disabling this query was a bad idea, since then the socket would not be reset properly when the client has actually reset the connection.

So I dug into the TCP-Stack to see when the socket is reset. I turned out that it ALWAYS is marked for reset, even when it was closed properly AND when the socket was opened freshly. The function "_TcpSocketSetIdleState" is responsible for that...

So I changed that. Within "_TcpSocketSetIdleState" the bSocketReset State is changed to 0 and every time the socket is closed by a reset, the Flag is set. (See attached file)
First tests showed that this change made my problems fade away.

So the question is: can someone confirm that the changes are valid and save for use?
If so: did I get all places the flag must be set?

If the changes are valid, please update the http_net server for the next release ;-)
#1

7 Replies Related Threads

    rainad
    Moderator
    • Total Posts : 1157
    • Reward points : 0
    • Joined: 2009/05/01 13:39:25
    • Location: 0
    • Status: online
    Re: Socket always gets marked as "wasReset" 2019/04/16 11:15:27 (permalink)
    0
    Can you please send some web pages/scenario which trigger the behavior you mention?
    Once I can reproduce it and understand what's going on, I'll be able to decide if the fix really addresses the root cause.
     
    #2
    David Green
    Junior Member
    • Total Posts : 33
    • Reward points : 0
    • Joined: 2017/04/10 07:57:13
    • Location: 0
    • Status: offline
    Re: Socket always gets marked as "wasReset" 2019/04/22 23:53:59 (permalink)
    0
    I can only provide a modified python script.
    Basically, I have a url-resource on the device that uses the dynamic variables to return a json, based on the request parameters. That way, I can read some informations stored in the device.
    Due to some external contraints, I need to split a read of a 65535 Byte into chunks of about 100 Bytes. That is why I cannot provide a webpage.
    It should be the same behaviour when you create a empty "example.htm" with a dynamic variable and fill that with a counter. Then poll that page without any pause like shown in the script.
     

    import requests
     
    ip_of_device = "192.168.100.115"
    url_on_device = "example.htm"

    def test_continuous_read(**kwargs):
        result = 1
        number_of_tests = 1000
        for run_index in range(number_of_tests):
            try:
                answer = requests.post("http://{}/{}".format(ip_of_device, url_on_device))
                if answer.status_code != 200:
                    print("Wrong result!")
                    result = 0
                    break
            except Exception as ex:
                print("Exception in run {} of {}: {}".format(run_index, number_of_tests, ex))
                result = 0
                break
        return result

    #3
    rainad
    Moderator
    • Total Posts : 1157
    • Reward points : 0
    • Joined: 2009/05/01 13:39:25
    • Location: 0
    • Status: online
    Re: Socket always gets marked as "wasReset" 2019/04/24 11:33:20 (permalink)
    0
    OK, pretty straightforward, I can give it a try and see what I get.
    The HTTP_NET connections are persistent by default. So after n passes (default n = 4) the script will probably stop not being able to open another connection.
    What setup do you have that allows 1000 connections, or after how many posts do you usually see the issue?
     
     
    #4
    David Green
    Junior Member
    • Total Posts : 33
    • Reward points : 0
    • Joined: 2017/04/10 07:57:13
    • Location: 0
    • Status: offline
    Re: Socket always gets marked as "wasReset" 2019/04/25 00:53:33 (permalink)
    0
    I do not open 1000 connections at the same time. The requests are sent one after another.

    But you are right, the connection should be persistent. I think it actually is because with the MSL timeout, it would run out of sockets quickly. (I have increased the number of sockets for TCP and NetPres to 30).

    The error happens at various points. Sometimes after 120 requests, somtimes after 500.

    Btw: I just realized that I did not mention that I use FreeRTOS...
    #5
    rainad
    Moderator
    • Total Posts : 1157
    • Reward points : 0
    • Joined: 2009/05/01 13:39:25
    • Location: 0
    • Status: online
    Re: Socket always gets marked as "wasReset" 2019/04/25 08:32:36 (permalink)
    4 (1)
    The fact that this happens under FreeRTOS is an important one, it could be a different problem all together. I'll do my tests using a multi-threaded app.
     
    Regarding the test, even with 30 HTTP connections, since they are persistent, the first 30 will remain open, so the step #31 of the test will lock up because the HTTP server won't reply, no more listening sockets.
    There is the possibility to configure the server with non-persistent connections: TCPIP_HTTP_NET_MODULE_FLAG_NON_PERSISTENT or to specify a small timeout: connTimeout/TCPIP_HTTP_NET_CONNECTION_TIMEOUT (default 60s);
    I was wondering what values for these parameters you have in your tests so I can run with the same.
     
    #6
    David Green
    Junior Member
    • Total Posts : 33
    • Reward points : 0
    • Joined: 2017/04/10 07:57:13
    • Location: 0
    • Status: offline
    Re: Socket always gets marked as "wasReset" 2019/04/25 08:56:20 (permalink)
    0
    TCPIP_HTTP_NET_MODULE_FLAG_NON_PERSISTENT is not set.
    but:
    #define TCPIP_HTTP_NET_CONNECTION_TIMEOUT                  0


    But from the description I would assume that this means that the connection will stay open forever and is re-used.
    I have attached the system_config.h in case you want to see the other settings.
    #7
    rainad
    Moderator
    • Total Posts : 1157
    • Reward points : 0
    • Joined: 2009/05/01 13:39:25
    • Location: 0
    • Status: online
    Re: Socket always gets marked as "wasReset" 2019/04/25 15:23:38 (permalink)
    4 (1)
    I've run 10000 cycles and no problem shows up.
    I'm using a FreeRTOS application.
    I have 100 TCP sockets, 100 NET_PRES sockets and 40 HTTP connections.
    Using buffer sizes different than yours though but probably this shouldn't make any difference.
     
    The only difference is with the persistent connections. On my system, when I run the script, the "request" connections remain active so that after exhausting the 40 available HTTP connections everything halts.
    To avoid that, I have set the TCPIP_HTTP_NET_CONNECTION_TIMEOUT to 2 sec, so that the HTTP server kills the inactive connections.
    Probably this is the difference, because otherwise I don't see any issues, the 10000 tests ran to completion.
    Do you have a wireshark log for the tests run on your machine? Maybe I can understand where the differences are.
     
     
    #8
    Jump to:
    © 2019 APG vNext Commercial Version 4.5