• AVR Freaks

Hot!Best advice to keep memory usage low

Page: 123 > Showing page 1 of 3
Author
swmcl
Super Member
  • Total Posts : 254
  • Reward points : 0
  • Joined: 2014/05/10 13:54:42
  • Location: Queensland
  • Status: offline
2019/06/10 15:34:14 (permalink)
0

Best advice to keep memory usage low

Hello again,
I am using a PIC18F47K42 which has 8k of SRAM.  In the last couple of days I ran out of memory I believe after I converted a bunch of structs and union-structs to arrays so that I could process them more efficiently.
 
My question is: What should I do in a general sense to reduce memory usage when it comes to variables ?
 
I did have a good number of union structs like this one,
 

union Local_Analogue_Sources
{
    struct
    {
        bool VBatt_01:1; // this is the lowest bit - bit 0
        bool VBatt_02:1;
        bool ANA_01:1;
        bool ANA_02:1;
        bool ANA_03:1;
        bool Blank_01:1;
        bool Blank_02:1;
        bool Blank_03:1; // this is the highest bit - bit 7

    };
    uint8_t LASByte;
};

 
There are at least 3 options.  I can make an array, I can use the struct or I could have a bunch of individual variables.
As I understand it, the array will process faster than the struct but the array consumes about 20% more memory.
 
Some of my routines are about 6 screens of searching and sorting for the one input so what I was trying to do was make arrays which could then have a generic procedure and reduce my screen count dramatically.
 
I do not aim to be a software guru.  I want the code to be easily readable to the next person who does not need great skills but I am on the limit for this amount of memory.  No other 8-bit PIC with XLP and 5V has more memory so I am stuck I think. 
 
Before things blew up, I had a 87% usage on the RAM and 37% usage on the program memory.
 
Some recommendations for general practice would be helpful because Internet searches assume an environment that has virtually unlimited memory.
 
Thanks,
 
 
#1

48 Replies Related Threads

    ric
    Super Member
    • Total Posts : 22647
    • Reward points : 0
    • Joined: 2003/11/07 12:41:26
    • Location: Australia, Melbourne
    • Status: online
    Re: Best advice to keep memory usage low 2019/06/10 15:41:48 (permalink)
    +1 (1)
    I don't know if this will help, but do as the manual says, and use "unsigned" as the base type for the bitfield, not "bool".
     

    I also post at: PicForum
    Links to useful PIC information: http://picforum.ric323.co...opic.php?f=59&t=15
    NEW USERS: Posting images, links and code - workaround for restrictions.
    To get a useful answer, always state which PIC you are using!
    #2
    swmcl
    Super Member
    • Total Posts : 254
    • Reward points : 0
    • Joined: 2014/05/10 13:54:42
    • Location: Queensland
    • Status: offline
    Re: Best advice to keep memory usage low 2019/06/10 15:52:23 (permalink)
    0
    Thanks ric I'll give it a go.
    #3
    NorthGuy
    Super Member
    • Total Posts : 5479
    • Reward points : 0
    • Joined: 2014/02/23 14:23:23
    • Location: Northern Canada
    • Status: offline
    Re: Best advice to keep memory usage low 2019/06/10 16:03:33 (permalink)
    +1 (1)
    If you have an array of bool, each element will take a byte, which is 8 times more than a bit. You can use bit arrays, but they're not supported in C, so you'll need to do custom processing and it'll be much slower than regular arrays.
    #4
    swmcl
    Super Member
    • Total Posts : 254
    • Reward points : 0
    • Joined: 2014/05/10 13:54:42
    • Location: Queensland
    • Status: offline
    Re: Best advice to keep memory usage low 2019/06/10 17:27:51 (permalink)
    0
    Yes, I think I'm going to go back to at least the structs and union-structs.  It will take a while to undo my code though (several days).  There will probably be a better way than what I am doing which I am yet to find.
     
    I'm currently looking at http://www.xargs.com/pic/c18-sections.html which is my introduction to sections within memory.  I can't understand my .map file as yet but what it tells me is to be a bit more careful about initialization of variables.  Sometimes I have a function to initialize them and sometimes I initialize them at declaration.  I didn't appreciate that they get stored in different places because of that.  Now I need to find out whether I am filling these sections of memory too much also.
     
    I do so want to just write a bunch of code and not know so much about all these intricacies - like a computer programmer ...
     
    :-)
    #5
    ric
    Super Member
    • Total Posts : 22647
    • Reward points : 0
    • Joined: 2003/11/07 12:41:26
    • Location: Australia, Melbourne
    • Status: online
    Re: Best advice to keep memory usage low 2019/06/10 17:30:30 (permalink)
    +1 (1)
    Are you using C18, or XC8?
    You don't need any of that section stuff if you're using XC8.
     

    I also post at: PicForum
    Links to useful PIC information: http://picforum.ric323.co...opic.php?f=59&t=15
    NEW USERS: Posting images, links and code - workaround for restrictions.
    To get a useful answer, always state which PIC you are using!
    #6
    NorthGuy
    Super Member
    • Total Posts : 5479
    • Reward points : 0
    • Joined: 2014/02/23 14:23:23
    • Location: Northern Canada
    • Status: offline
    Re: Best advice to keep memory usage low 2019/06/10 20:28:43 (permalink)
    +1 (1)
    swmcl
    Yes, I think I'm going to go back to at least the structs and union-structs.  It will take a while to undo my code though (several days).  There will probably be a better way than what I am doing which I am yet to find.



    Install Git and use it to keep track of your code. Then it's very easy to roll-back anything.
    #7
    jtemples
    عُضْوٌ جَدِيد
    • Total Posts : 11208
    • Reward points : 0
    • Joined: 2004/02/13 12:31:19
    • Location: Southern California
    • Status: offline
    Re: Best advice to keep memory usage low 2019/06/10 21:13:41 (permalink)
    +1 (1)
    The union you've shown will take one byte, so unless you have thousands of those, it's not your problem.  I'm pretty sure the PIC18F47K42 is not supported by C18, so the article you came across isn't relevant.
     
    the array will process faster than the struct but the array consumes about 20% more memory

     
    Show an example that makes you believe a struct uses more memory.
     
    Sometimes I have a function to initialize them and sometimes I initialize them at declaration.  I didn't appreciate that they get stored in different places because of that.  Now I need to find out whether I am filling these sections of memory too much also.


    What "sections of memory" are you talking about?  With such a huge amount of RAM for an 8-bit PIC, it would be very unlikely one would need to worry about running out unless you've created some large arrays.
    #8
    swmcl
    Super Member
    • Total Posts : 254
    • Reward points : 0
    • Joined: 2014/05/10 13:54:42
    • Location: Queensland
    • Status: offline
    Re: Best advice to keep memory usage low 2019/06/11 01:21:47 (permalink)
    0
    Guys,
    I'm using XC8 with MPLAB X and the next-to-latest versions.
     
    jt, I believe the array uses the more memory and the struct is the slower of the two.
     
    I've wound back most of my changes now.  The SRAM is down to under 87% and the Program code is at 37% ish.
     
    Now I'll try ric's suggestion.
    #9
    swmcl
    Super Member
    • Total Posts : 254
    • Reward points : 0
    • Joined: 2014/05/10 13:54:42
    • Location: Queensland
    • Status: offline
    Re: Best advice to keep memory usage low 2019/06/11 01:31:38 (permalink)
    0
    Ric,
     
    I read this in my version of the C Compiler User Guide (DS50002737A):
     
    4.4.5
    Structures and Unions
    MPLAB XC8 C Compiler supports struct and union types. Structures and unions
    only differ in the memory offset applied to each member.
    These types will be at least 1 byte wide. The members of structures and unions cannot
    be objects of type __bit, but bit-fields and _Bool objects are fully supported.
    Structures and unions can be passed freely as function arguments and function return
    values. Pointers to structures and unions are fully supported.
     
    So I'm assuming the examples they give will not change much. 
    post edited by swmcl - 2019/06/11 01:34:04
    #10
    ric
    Super Member
    • Total Posts : 22647
    • Reward points : 0
    • Joined: 2003/11/07 12:41:26
    • Location: Australia, Melbourne
    • Status: online
    Re: Best advice to keep memory usage low 2019/06/11 02:00:42 (permalink)
    +1 (1)
    swmcl
    I read this in my version of the C Compiler User Guide (DS50002737A):
    ...
    So I'm assuming the examples they give will not change much.

    The whole point of my question was about your attempt to use "sections", which are irrelevant for XC8.
     
    swmcl
    jt, I believe the array uses the more memory and the struct is the slower of the two.

    Why?
    Guesswork, or actual tests?
    We'd need to see both versions of your code to comment if you're doing it the best way.
     

    I also post at: PicForum
    Links to useful PIC information: http://picforum.ric323.co...opic.php?f=59&t=15
    NEW USERS: Posting images, links and code - workaround for restrictions.
    To get a useful answer, always state which PIC you are using!
    #11
    PICcie
    New Member
    • Total Posts : 14
    • Reward points : 0
    • Joined: 2018/03/02 12:15:35
    • Location: 0
    • Status: offline
    Re: Best advice to keep memory usage low 2019/06/11 02:44:27 (permalink)
    -1 (1)
    I know that in asm, you can directly work on bit with BSF, BCF, BTFSC ....
    It take the same number of cycle than in 8bit (SETF, CPFSEQ ...)
     
    But I don't know how xc8 compile struct.
    If it use Bit oriented instructions, it will the same time.
    If it use logic code like that, I think it will be slower.
    uint8_t var;
    var = var | 0b00010000; //Set bit 4
    var = var & 0b11101111; //Clear bit 4
    if(var & 0b00010000)    //test bit 4

    #12
    ric
    Super Member
    • Total Posts : 22647
    • Reward points : 0
    • Joined: 2003/11/07 12:41:26
    • Location: Australia, Melbourne
    • Status: online
    Re: Best advice to keep memory usage low 2019/06/11 03:24:53 (permalink)
    +2 (2)
    XC8 will code all those expressions using bit instructions.
     

    I also post at: PicForum
    Links to useful PIC information: http://picforum.ric323.co...opic.php?f=59&t=15
    NEW USERS: Posting images, links and code - workaround for restrictions.
    To get a useful answer, always state which PIC you are using!
    #13
    NorthGuy
    Super Member
    • Total Posts : 5479
    • Reward points : 0
    • Joined: 2014/02/23 14:23:23
    • Location: Northern Canada
    • Status: offline
    Re: Best advice to keep memory usage low 2019/06/11 06:47:26 (permalink)
    +2 (2)
    PICcie
    If it use Bit oriented instructions, it will the same time.
    If it use logic code like that, I think it will be slower.
    uint8_t var;
    var = var | 0b00010000; //Set bit 4
    var = var & 0b11101111; //Clear bit 4
    if(var & 0b00010000)    //test bit 4




    The compiler should figure out that these are bit operations.
     
    Setting bits is very efficient when you know which bits to set at the compile time. If you don't (for example you want to access an array of bits using an index which is not known at compile time), it is impossible to use bit access instructions and then logic operations with masks is the only choice.
     
    For example, this code:
     
    void set_bit(uint8_t idx) {
      holder_byte |= 1 << idx;
    }

     
    cannot use bit-setting instructions.
     
    Thus, everything depends on what you're doing.
    #14
    crennolet
    Super Member
    • Total Posts : 128
    • Reward points : 0
    • Joined: 2012/03/15 09:51:58
    • Location: 0
    • Status: offline
    Re: Best advice to keep memory usage low 2019/06/11 14:19:29 (permalink)
    0
    I'm a bit perplexed by the question. If the issue is memory required for structures that are bits, then bitfields are the obvious solution, and  they provide the minimum-memory footprint, as 1 bit per 1 bit is about as low as you'll get. Unless, of course, you are using the bitfields to store data, in which case you could use some simple compression (a modified RLL method might be useful - I used a Huffman encoding at one time to get around a problem like this.) But of course that won't help the "efficiency" (by which I presume you mean "speed") of the code.
    Another thing you could do (which depends greatly on what you are doing, which we don't know) is use "automatic" variables instead of global or "static" variables. These automatic variables live on the stack, which means the storage they occupy can be reused for other variables at other times. Of course, that might require some extra work to initialize them when the stack frame is created.
     
     
    #15
    swmcl
    Super Member
    • Total Posts : 254
    • Reward points : 0
    • Joined: 2014/05/10 13:54:42
    • Location: Queensland
    • Status: offline
    Re: Best advice to keep memory usage low 2019/06/11 20:14:33 (permalink)
    0
    Thanks for the replies.
    It started out that I had structures and unions like I posted in the starting post.  Then when I was endlessly scrolling through my code I decided to try to implement arrays instead of the structures.  It made my code very much simpler to use arrays and I could make a generic function into which I would pass an input like say, Check_Inputs(uint8_t Input1);
    It turns out that arrays chew up much more memory than do structures and unions (which I didn't appreciate) and so my memory ran out. 
    So now I am back to the old code and I have saved about 20% of my memory.
     
    The code I have is using a good number of ring buffers which each have arrays in them.  Does anyone have experience in using ring buffers that use only structures ?  Is it worth converting (if the ring buffer would still work with a structure-only configuration) ?
     
    Here is an example of a ring buffer with an array of floats:
     

    typedef struct
    {
        float buf[NUM_ELEMENTS_IN_ADCC_SHORT_TERM_RING_BUFFER];
        uint8_t head;
        uint8_t count;
    }ANA_SHORT_TERM_CIRC_Q_t;

     
    Perhaps I could create something like:
     

    typedef struct
    {
        float element1;
        float element2;
        float element3;
        float element4;
        float element5;
        float element6;
        float element7;
        float element8;
        uint8_t head;
        uint8_t count;
    }ANA_SHORT_TERM_CIRC_Q_t;

     
    I'm using a ring buffer for each analogue input for the short-term sample, the medium-term average and the long-term average at this stage.  These ring buffers are aplenty so consume a lot of memory.
     
    Rgds,
    #16
    ric
    Super Member
    • Total Posts : 22647
    • Reward points : 0
    • Joined: 2003/11/07 12:41:26
    • Location: Australia, Melbourne
    • Status: online
    Re: Best advice to keep memory usage low 2019/06/11 20:32:34 (permalink)
    +1 (1)
    Arrays of what?
    At the start you were talking about bit fields, now you're talking about floats.
     

    I also post at: PicForum
    Links to useful PIC information: http://picforum.ric323.co...opic.php?f=59&t=15
    NEW USERS: Posting images, links and code - workaround for restrictions.
    To get a useful answer, always state which PIC you are using!
    #17
    LdB_ECM
    Senior Member
    • Total Posts : 106
    • Reward points : 0
    • Joined: 2019/04/16 22:01:25
    • Location: 0
    • Status: offline
    Re: Best advice to keep memory usage low 2019/06/11 21:44:39 (permalink)
    +1 (1)
    To do averaging on an ADC samples there is no requirement except to hold one float value and the number of samples it represents. You don't need to hold each sample at all and it is crazy to do so.
     
    int SampleCount = 0;  // You start the sample count as zero
    float SampleTotal = 0.0f; // You start the sample total as zero
     
    /* For each sample */
    SampleCount++;
    SampleTotal += ADC_Sample;
     
    /* At any point the average is simply */
    float Ave = SampleTotal/SampleCount;

     
    All you need to track is SampleCount if you want to do intermediate values and when you reach some designated maxium SampleCount just zero both values to start again.
     
    Usually you do this trick with long int or long long int on the raw ADC counts you don't even bother converting to floats until you have the average based solely on speed of adding operations.
    #18
    swmcl
    Super Member
    • Total Posts : 254
    • Reward points : 0
    • Joined: 2014/05/10 13:54:42
    • Location: Queensland
    • Status: offline
    Re: Best advice to keep memory usage low 2019/06/11 21:58:59 (permalink)
    0
    Thanks LdB.
     
    At the moment, I am calculating an average from the short-term (very regular) samples and then passing this to be the 'sample' for a medium-term array of values.  This medium term array is also averaged to get the 'sample' for the long-term array.
    I think I actually do in maths what you suggest but I do hold the values which I can see is silly.  It is a while since I revised the code and I think it started out as a rolling average where I would hold the last 7 samples.  In the rolling code I needed to add the new sample and delete the oldest sample before taking the average.
     
    I'm hearin' ya.  I'll go back to the drawing board on this area.  It will probably save me a fairly significant amount of memory.
     
    #19
    jtemples
    عُضْوٌ جَدِيد
    • Total Posts : 11208
    • Reward points : 0
    • Joined: 2004/02/13 12:31:19
    • Location: Southern California
    • Status: offline
    Re: Best advice to keep memory usage low 2019/06/11 22:06:49 (permalink)
    +1 (1)
    It turns out that arrays chew up much more memory than do structures

     
    You keep saying this, but you haven't provided an example that demonstrates what you're seeing.
     
    #20
    Page: 123 > Showing page 1 of 3
    Jump to:
    © 2019 APG vNext Commercial Version 4.5