• AVR Freaks

Hot!SMTPC Module and TLS not working?

Author
mpgreen914
New Member
  • Total Posts : 15
  • Reward points : 0
  • Joined: 2015/03/27 09:41:45
  • Location: 0
  • Status: offline
2017/03/20 13:14:28 (permalink)
0

SMTPC Module and TLS not working?

I am using Harmony 1.10 and trying to get the new SMTPC module working properly in my app.
I am able to use a comcast email account to send emails with no problem if I set the SKIP TLS flag. (TCPIP_SMTPC_MAIL_FLAGS = 0x0020)
However, if I try to use TLS the whole things freezes up after the message query reports  status starting TLS communication. (TCPIP_SMTPC_MESSAGE_STAT_TLS) and I mean freezes the whole app hard. Not timing out, or even returning back to my APP_Tasks(). 
 
Everything seems to work fine in the app with TLS off.
 
I also tried a GMAIL account with the TLS on and got the exact same result. I can't get gmail to work without TLS, though, but I don't know even GMAIL will work without it. I just keep getting a FROM_REJECT result.
 
Anyone able to use SMTPC successfully with TLS?
#1

19 Replies Related Threads

    rainad
    Moderator
    • Total Posts : 1190
    • Reward points : 0
    • Joined: 2009/05/01 13:39:25
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/20 14:02:31 (permalink)
    0
    Please make sure that the project you use SMTPC in has the wolfSSL properly enabled and configured.
    I suggest testing with the project: web_net_server_nvm_mpfs.
    You could test first if it's possible to access your board web pages using https. That would make it clear that wolfSSL is up and running as expected.
    Once you're past this step, you could enable the "mail" command from the tcpip commands - if you have a console attached to our board. It's fairly easy to send mail using this command line utility.
     
    Note: wolf SSL needs a decent heap - probably a good idea to increase the one in your project by 10KB.
    Another thing is that it needs to access a NTP server. Make sure that access is not blocked.
     
    #2
    mpgreen914
    New Member
    • Total Posts : 15
    • Reward points : 0
    • Joined: 2015/03/27 09:41:45
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/21 08:41:13 (permalink)
    0
    Thank you for the reply.
    I added the wolfSSL library (via MHC) and used it's default settings. Never used this before so not sure if I need to modify these settings.
    I have SNTP up and verified to be running. i.e. connecting and getting valid UTC stamps.
    Also increased the heap size ...nearly doubled (under MHC project configuration XC32 (Global Options) )
    So far no change in the behavior of my app. Still locks up when trying to do TLS.
    I enabled debug messages and it turns out the app is throwing the following exception right after starting TLS:
    "General Exception Instruction bus error (cause=6, addr=0)"
     
     
    post edited by mpgreen914 - 2017/03/21 10:04:50
    #3
    rainad
    Moderator
    • Total Posts : 1190
    • Reward points : 0
    • Joined: 2009/05/01 13:39:25
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/21 10:40:57 (permalink)
    0
    Can you please try the web_net_server_nvm_mpfs demo app?
    That one has the wolfSSL and TLS properly configured.
    Most likely is a configuration problem somewhere.
     
    #4
    mpgreen914
    New Member
    • Total Posts : 15
    • Reward points : 0
    • Joined: 2015/03/27 09:41:45
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/21 12:10:34 (permalink)
    0
    I think you are right but, that project doesn't easily run on my hardware so I've been trying to compare the settings in there to the settings in my project.
     
    Do I have to use the HTTP NET Server versus the just the HTTP Server to have TLS work with emails only?
     
    Switching to the NET Server means a major code port to rewrite the custom_http_net_app.c file.
     
    Thanks again.
    #5
    rainad
    Moderator
    • Total Posts : 1190
    • Reward points : 0
    • Joined: 2009/05/01 13:39:25
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/21 13:41:22 (permalink)
    0
    No, I didn't mean to switch your project to http_net.
    I wanted just to check that if the wolfSSL is configured properly you have TLS up and running.
     
    If you don't have a ESK or the web_net_http_server demo is difficult to port on your board, then another way is to compare the system_config.h or the .mhc files between the 2 projects and check if you miss any configuration parameters.
    Check for WOLFSSL and NET_PRES symbols, maybe you spot some inconsistencies.
     
    #6
    mpgreen914
    New Member
    • Total Posts : 15
    • Reward points : 0
    • Joined: 2015/03/27 09:41:45
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/23 08:54:09 (permalink)
    0
    No change yet
    Just spent quite a bit of time trying to match the settings on web_net_server_nvm_mpfs project with mine. They seem to be identical except:
    -the example project uses the SMTP module rather than the SMTPC module and 
    -it uses the HTTP Net Server
     
    Also, in the example project I could not find any WOLFSSL settings in the system_config.h and only 2 NET_PRES settings that I match.
     
    Interestingly, there is a source/header file included under the system config called net_pres_enc_glue (.c .h) that in the example has a bunch of functions that were not generated in my project - maybe because I'm not using the HTTP Net Server. Just in case this was the issue, I added all the code in these files into my project. Everything compiled OK but I still get the exception when I try to use TLS.
     
    At this point, I'm still wondering if the SMTPC (not the SMTP) module has been tested to work with TLS properly. Wondering if anyone has gotten this to work?
    post edited by mpgreen914 - 2017/03/23 08:57:53
    #7
    martinw7750
    Super Member
    • Total Posts : 295
    • Reward points : 0
    • Joined: 2013/05/28 01:31:19
    • Location: UK
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/23 09:38:00 (permalink)
    0
    you need to add the functions in net_pres_enc_glue.c - they are the interfacet between tpc/ip and wolf ssl
     
    I found that by comparing a working demo ssl project with mine after mhc generator had been run.  mhc doesn't add those funtions.
     
    there are also some tables that may need updating in system_init.c, they are the funtions that interface to net_pres.
     
    my change was to .pProvObject_sc
     
    static const NET_PRES_INST_DATA netPresCfgs[] =
    {
        {
            .pTransObject_ss = &netPresTransObject0SS,
            .pTransObject_sc = &netPresTransObject0SC,
            .pTransObject_ds = &netPresTransObject0DS,
            .pTransObject_dc = &netPresTransObject0DC,
            .pProvObject_ss = NULL,
            //.pProvObject_sc = NULL,
            .pProvObject_sc = &net_pres_EncProviderStreamClient0,            
            .pProvObject_ds = NULL,
            .pProvObject_dc = NULL,
        },
    };
     
    at least, that's what I had to do!
     
    #8
    Adrian B
    Super Member
    • Total Posts : 80
    • Reward points : 0
    • Joined: 2014/04/16 10:03:07
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/23 10:37:10 (permalink)
    3 (1)
    I am using smtpc.c and works fine with gmail on both ports 465 and 587. There are no example apps for smtpc but you can read the documentation, set the config params as needed, prepare the mail handler with all fields. Then call the send function and verify the result. The error codes will tell you what the problem is.
     
    Be sure the number of sockets declared in system_config.h is enough, you will need additional sockets for that.
    #9
    rainad
    Moderator
    • Total Posts : 1190
    • Reward points : 0
    • Joined: 2009/05/01 13:39:25
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/23 11:49:59 (permalink)
    0
    mpgreen914
     
    Interestingly, there is a source/header file included under the system config called net_pres_enc_glue (.c .h) that in the example has a bunch of functions that were not generated in my project - maybe because I'm not using the HTTP Net Server.
    ...
    At this point, I'm still wondering if the SMTPC (not the SMTP) module has been tested to work with TLS properly. Wondering if anyone has gotten this to work?





    The network presentation (NET_PRES) layer is needed and the glue code that you mention is added by MHC if you select it. As mentioned above, this is the layer that allows the TCP/IP stack to use the wolfSSL encryption.
    And you need to select wolfSSL as well - see the 3rd party configuration in the project.
    For all these settings I suggested that you give a try first to the standard web_net_server... demo app.
    On top of this, SMTPC needs to be selected for the project, by default just SMTP is enabled.
     Look in the .mhc file, there should be lots of symbols starting with CONFIG_WOLFSSL_ and NET_PRES_. You should have those in your project too.
    And yes, the SMTPC module has been tested.
     
     

    Attached Image(s)

    #10
    mpgreen914
    New Member
    • Total Posts : 15
    • Reward points : 0
    • Joined: 2015/03/27 09:41:45
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/24 09:01:57 (permalink)
    0
    Thank you for the replies.
     
    Since SMPTC is new, knowing that TLS has been successfully made to work properly with it is helpful.
     
    After restarting the project for the 4th time and, I guess, getting the configuration right finally, the appropriate code was linked in. No longer getting the exception!
     
    Of course, now TLS is failing with the result being reported as TCPIP_SMTPC_RES_TLS_FAILED  - "TLS Session negotiation failed". Same result for several different mail services using port 587. Port 465 just times out.
     
    Anyone care to help a poor hardware guy (me) and give me some hints about what might be going wrong here? I was hoping not to have to dig deep into how TLS functions to get this all working.
     
    Thanks again for the replies!
    #11
    Adrian B
    Super Member
    • Total Posts : 80
    • Reward points : 0
    • Joined: 2014/04/16 10:03:07
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/24 09:09:27 (permalink)
    4 (1)
    If you are on the gmail account, be sure that the account is configured to "Allow less secure apps", otherwise it works only with OAuth-2. This setting is in gmail account, in security settings. I don't know about Comcast account, it may have something similar where it accepts to connect only from trusted devices.
    #12
    mpgreen914
    New Member
    • Total Posts : 15
    • Reward points : 0
    • Joined: 2015/03/27 09:41:45
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/24 09:25:20 (permalink)
    0
    Thanks Adrian. I checked this setting and yes it is on (allows less secure apps).
     
    I'm wondering if there is something further that has to be done with keys/encryption or if once you link in the code it's all handled automatically.
    #13
    Adrian B
    Super Member
    • Total Posts : 80
    • Reward points : 0
    • Joined: 2014/04/16 10:03:07
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/24 09:37:57 (permalink)
    4 (1)
    Did you select a 2048 key? Gmail won't accept lesser values.
    Otherwise, while there are no examples using smtpc in the apps directory, there are some examples in the stack itself. Do a search for the sender function in framework directory and you should find it. There is some code there sending email with smtpc.
    #14
    mpgreen914
    New Member
    • Total Posts : 15
    • Reward points : 0
    • Joined: 2015/03/27 09:41:45
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/24 10:21:51 (permalink)
    0
    Sheesh, every time I try to implement a new software feature with harmony I'm reminded of two things:
     1) what a sw novice I am  ....and 2) how insufficient documentation is.
     
    I have looked back carefully over the SMTPC module documentation and can't find any reference to key size. Also can't seem to see any place in MHC where this is explicitly set. Where would I set this?
     
    I did set the option in Presentation Layer -SUPPORT CLIENT CERTIFICATE - Variable Name .... to client_cert_der_2048 but I assumed this is just for HTTP stuff.
     
    In general, it seems WolfSSL might need to be configured but I'm not sure where how to do that. I will go search for the examples you mention and see if that has any hints.
     
    (TGIF sad: sad)
     
    Thanks much.
     
     
    #15
    Adrian B
    Super Member
    • Total Posts : 80
    • Reward points : 0
    • Joined: 2014/04/16 10:03:07
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/24 12:15:52 (permalink)
    0
    You should have some file net_pres_cert_store.c which selects the certificates and keys. The certificates themselves are in a file certs_test.h (for example application). But it looks like you used the 2048 one.
     
    As about configuring the smtpc, in system_init.c you should declare a TCPIP_SMTPC_MODULE_CONFIG constant structure, initialized with proper values. Then you should add it to your TCPIP_STACK_MODULE_CONFIG table which is initialized by TCPIP_STACK_Init() function.
     
    When sending the e-mails set the flag TCPIP_SMTPC_MAIL_FLAG_FORCE_TLS for port 587 and TCPIP_SMTPC_MAIL_FLAG_CONNECT_TLS for port 465
    #16
    mpgreen914
    New Member
    • Total Posts : 15
    • Reward points : 0
    • Joined: 2015/03/27 09:41:45
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/28 09:16:18 (permalink)
    0
    Thanks again for the responses.
     
    I have tried the FORCE_TLS flag (and just about every other combination of flags and ports) but no change. I still get the TLS session negotiation failed result.
     
    There is a TCPIP_SMTPC_MODULE_CONFIG struct in the system_init.c and it is initialized with the #defines in the config header... presumably all generated by the MHC.
     
    I went back and found an old Ethernet Starter Kit and ran both the web_net_server_nvm_mpfs and the web_server_nvm_mpfs demos with Harmony 1.10 and I can't get either of these to send any email (even non-TLS)  but I remember getting them to work with previous versions of harmony. Don't want to waste too much time debugging with demo code and demo hardware since my hardware works and already does send mail with Non TLS connections.
     
    You mentioned making sure enough sockets are declared. I have TCPIP_TCP_MAX_SOCKETS defined as 10. Is this the value you are referring to and is this high enough?
     
    Thanks.
     
     
    #17
    Adrian B
    Super Member
    • Total Posts : 80
    • Reward points : 0
    • Joined: 2014/04/16 10:03:07
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/28 09:34:23 (permalink)
    3 (1)
    Yes, TCPIP_TCP_MAX_SOCKETS. 10 was not enough for me when running few servers on board. It failed to open more http server sockets. But, since it says it fails to negotiate TLS sessions, it seems to pass this step of opening a socket.
    Maybe you should post your net glue code here.
    #18
    mpgreen914
    New Member
    • Total Posts : 15
    • Reward points : 0
    • Joined: 2015/03/27 09:41:45
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2017/03/31 11:17:30 (permalink)
    0
     
    I'm assuming you mean the code generated in net_pres_enc_glue.h and .c? I compared this to the code in sample project web_net_server_nvm_mpfs. It looks like the same code is generated. Here's the header:
     
    extern NET_PRES_EncProviderObject net_pres_EncProviderStreamServer0;
    extern NET_PRES_EncProviderObject net_pres_EncProviderStreamClient0;
    bool NET_PRES_EncProviderStreamServerInit0(struct _NET_PRES_TransportObject * transObject);
    bool NET_PRES_EncProviderStreamServerDeinit0();
    bool NET_PRES_EncProviderStreamServerOpen0(uintptr_t transHandle, void * providerData);
    bool NET_PRES_EncProviderStreamServerIsInited0();
    bool NET_PRES_EncProviderStreamClientInit0(struct _NET_PRES_TransportObject * transObject);
    bool NET_PRES_EncProviderStreamClientDeinit0();
    bool NET_PRES_EncProviderStreamClientOpen0(uintptr_t transHandle, void * providerData);
    bool NET_PRES_EncProviderStreamClientIsInited0();
    NET_PRES_EncSessionStatus NET_PRES_EncProviderServerAccept0(void * providerData);
    NET_PRES_EncSessionStatus NET_PRES_EncProviderClientConnect0(void * providerData);
    NET_PRES_EncSessionStatus NET_PRES_EncProviderConnectionClose0(void * providerData);
    int32_t NET_PRES_EncProviderWrite0(void * providerData, const uint8_t * buffer, uint16_t size);
    uint16_t  NET_PRES_EncProviderWriteReady0(void * providerData, uint16_t reqSize, uint16_t minSize);
    int32_t NET_PRES_EncProviderRead0(void * providerData, uint8_t * buffer, uint16_t size);
    int32_t NET_PRES_EncProviderReadReady0(void * providerData);
    int32_t NET_PRES_EncProviderPeek0(void * providerData, uint8_t * buffer, uint16_t size);
     
    Frankly, I've hit a wall finding easy solutions. Next I'll have to start capturing packets with Wireshark to try to see where it's breaking down, but ugh, that's tedious.
     
    Thanks for the help. Much appreciated.
    #19
    arpatel
    Starting Member
    • Total Posts : 60
    • Reward points : 0
    • Joined: 2019/01/09 23:24:14
    • Location: 0
    • Status: offline
    Re: SMTPC Module and TLS not working? 2019/07/12 00:58:07 (permalink)
    0
    hey guys
    CAn you please tell me where to see for net_pres_enc_glue.h?
    I cant find in my project and its showing error for missing that one.
     
    #20
    Jump to:
    © 2019 APG vNext Commercial Version 4.5