• AVR Freaks

Filtering in real-time

Author
Marino
Starting Member
  • Total Posts : 74
  • Reward points : 0
  • Joined: 2006/06/09 16:57:37
  • Location: 0
  • Status: offline
2007/05/13 03:26:56 (permalink)
0

Filtering in real-time

Hi,
 
I would like to permanently filter new incoming values in real-time based-on the previous values. A new value comes every 1.38 millisecond and is something between 0x0000 and 0x7FFF. What is your advice? What is the best method?
 
Long ago, I purchased the Digital-Filter-Design-Lite software v1.0. The included manual from 2002 contains some design examples and that’s it. I would appreciate a document that contains some more. Do you have any document tips?
 
Thank You
#1

12 Replies Related Threads

    Olin Lathrop
    Super Member
    • Total Posts : 7463
    • Reward points : 0
    • Joined: 2004/02/26 17:59:01
    • Location: Littleton Massachusetts
    • Status: offline
    RE: Filtering in real-time 2007/05/13 06:36:08 (permalink)
    0
    I would like to permanently filter

    Huh?  As apposed to what?  Temporarily filtering them and then letting them revert back to their original values?  I don't see what information you are trying to convey with "premanently".
     
    new incoming values in real-time based-on the previous values. A new value comes every 1.38 millisecond and is something between 0x0000 and 0x7FFF.

    1.38mS is a long time in the scheme of things, and your range of values works nicely for a dsPIC.  However you didn't say how you want to filter the values or what you want the filter to accomplish.
     
    What is your advice?

    Figure out what you want the filter to do first.
     
    What is the best method?

    Best method for what?  You haven't provided any spec on what you want to accomplish.
    #2
    Marino
    Starting Member
    • Total Posts : 74
    • Reward points : 0
    • Joined: 2006/06/09 16:57:37
    • Location: 0
    • Status: offline
    RE: Filtering in real-time 2007/05/13 08:47:49 (permalink)
    0
    Thank you for the response.
     
    I didn’t want to bother the reader with too many details, but some details are necessary to understand my question:
     
    I’ve implemented an application on a dsPIC4012; this application calculates three different results every 1.38 millisecond. The problem remaining; the results have sometimes some annoyance due to environment impacts. Know I know how quick the results should change and would like to filter any other changes.
     
    For Example: One result of the three results shouldn’t change quickly and I made it lazy this way: The new incoming value is in W1 and will go to RAM 0x8DA after filtering:
     
                            mov     0x8DA, W0
                            cpslt     W0, W1                                  ;
                            bra       LazyDecC2                             ;
     
    LazyIncC2:                                                                  ;                      
                            add      #0xFF, W0                              ; max 0x3FF
                            cpsgt    W0, W1                                  ;
                            mov     W0, W1
                            mov     W1, 0x8DA
                            bra       LazyC2End
     
    LazyDecC2:                                                                 ;
                            sub       #0xFF, W0                              ; max 0x3FF
                            cpslt     W0, W1                                  ;
                            mov     W0, W1
                            mov     W1, 0x8DA
     
    LazyC2End:
     
    Anyhow, this is just an example how I’m trying to solve the problem. The solution must be filtering the new incoming values immediately based-on the previous values. And for one result I need a kind of band pass.

    #3
    Marino
    Starting Member
    • Total Posts : 74
    • Reward points : 0
    • Joined: 2006/06/09 16:57:37
    • Location: 0
    • Status: offline
    RE: Filtering in real-time 2007/05/13 09:35:12 (permalink)
    0
    After looking at my code (attempt) to solve the problem, I found a bug: W0 will overflow if the result is bigger than 0x7F00. Pleas, forget about my attempt to solve the problem. It’s just an example to illustrate, what I need.
     
    I just need the new results; I can save the previous results in order to know how to filter annoyance from the new results.
    post edited by Marino - 2007/05/13 09:45:47
    #4
    Olin Lathrop
    Super Member
    • Total Posts : 7463
    • Reward points : 0
    • Joined: 2004/02/26 17:59:01
    • Location: Littleton Massachusetts
    • Status: offline
    RE: Filtering in real-time 2007/05/13 15:03:18 (permalink)
    0
    The problem remaining; the results have sometimes some annoyance due to environment impacts. Know I know how quick the results should change and would like to filter any other changes.

    So you are looking for some sort of low pass filter?  What settling time to what error level do you need?  You apparently need the output every 1.38mS (725Hz rate), but can you sample the input more often?  What kind of hardware filtering do you have before the digitization?
    #5
    Marino
    Starting Member
    • Total Posts : 74
    • Reward points : 0
    • Joined: 2006/06/09 16:57:37
    • Location: 0
    • Status: offline
    RE: Filtering in real-time 2007/05/14 07:25:49 (permalink)
    0
    The analogical signals to the dsPIC are already filtered outside the dsPIC. The dsPIC samples the filtered analogical signals and calculate the results. I need the results at least every 2 millisecond and I need approximately 1800 cycle to calculate them (The code is already very effective). So, at the moment I work with 6 MHz (1.5 MIPS) and calculate 721.15 times per second (2080 cycle for every calculation) and still have approximately 200 cycle to filter the results.
     
    By considering the results signals; I’m looking for some sort of low pass filter and for some sort of band pass filter; to filter  frequencies in Hz range (0-10Hz) by a sample rate of 500-1000 Hz (721 Hz at the moment).
     
    I have no experience with digital filters and don’t even know if digital filter is the appropriate tool for this task and need an advice to start working in the right direction.
    #6
    Olin Lathrop
    Super Member
    • Total Posts : 7463
    • Reward points : 0
    • Joined: 2004/02/26 17:59:01
    • Location: Littleton Massachusetts
    • Status: offline
    RE: Filtering in real-time 2007/05/14 15:16:37 (permalink)
    0
    I’m looking for some sort of low pass filter and for some sort of band pass filter; to filter  frequencies in Hz range (0-10Hz) by a sample rate of 500-1000 Hz (721 Hz at the moment).

    Finally, at least a partial real spec.  But once again, what settling time can you tolerate?  I'm not going to ask a third time.
     
    I don't feel like playing twenty questions.  I ask questions for a reason and I expect answers if you want my help, whether you think you understand the reason for the questions or not.
    #7
    Marino
    Starting Member
    • Total Posts : 74
    • Reward points : 0
    • Joined: 2006/06/09 16:57:37
    • Location: 0
    • Status: offline
    RE: Filtering in real-time 2007/05/14 16:27:21 (permalink)
    0
    Sorry, maybe I wasn’t sure about the meaning of settling time. I think; settling time is the time delay between the input and output response. I can tolerate a big time delay of 10 milliseconds, if this time delay is always the same (linear). I also can tolerate a big error level; the application is working perfect, if there are no environment impacts. I just want to make it robust when the environment impacts are present and want to try "soft-filtering" first. I also can raise the MIPS and calculate more than 721 times or get more than 200 cycles to filter, but the application is battery-powered.
     
    Thank You
    #8
    Marino
    Starting Member
    • Total Posts : 74
    • Reward points : 0
    • Joined: 2006/06/09 16:57:37
    • Location: 0
    • Status: offline
    RE: Filtering in real-time 2007/05/14 18:20:56 (permalink)
    0
    I wanted to answer very quickly and it’s the wrong answer. Pleas don’t hit me again.
     
    Settling time is the time necessary to converge to the final value of a step input, isn’t it?
     
    I want to remove the step inputs, and I can tolerate a big settling time, the annoyance is short step inputs (5 to 10 calculations / samples)
    #9
    Oznog
    Super Member
    • Total Posts : 1011
    • Reward points : 0
    • Joined: 2004/10/14 21:54:10
    • Status: offline
    RE: Filtering in real-time 2007/05/15 01:13:27 (permalink)
    0
    ORIGINAL: Marino
     
    I have no experience with digital filters and don’t even know if digital filter is the appropriate tool for this task and need an advice to start working in the right direction.


    I'm going on the interpretation that by "permanently" you mean "continuously".

    The keywords you need to research are "Finite Impulse Response Filter" (FIR filter) and "Infinite Impulse Response Filter" (IIR filter).  Wikipedia and Google are a good start.  Not knowing anything about what you need, the FIR is generally a somewhat more likely general-purpose choice.  Well, it's simpler in that it's always stable.  Start there.

    The FIR or IIR can be run on a large buffer of samples or an entire audio file, or it can be run on each sample as it comes in- which sounds like what you need.  For example, an FIR with 64 taps needs to remember the last 63 ADC readings in a RAM buffer, calculate with them along with the current reading and produce a single filter output sample to provide to a DAC.  As such there is only a single sampling cycle of "latency".  However, settling time of the filter is different than latency.

    You will need to find a tool to calculate the FIR coefficients or learn the methods by which filter coefficients are created.

    You may also want to pay attention to the imperfections of digital filters' response.  Whereas an analog lowpass will dropoff and the response will always diminish with increasing freq, the digital filter has minor lobes in its response that pop up outside the official cutoff freq.
    post edited by Oznog - 2007/05/15 01:30:47
    #10
    Olin Lathrop
    Super Member
    • Total Posts : 7463
    • Reward points : 0
    • Joined: 2004/02/26 17:59:01
    • Location: Littleton Massachusetts
    • Status: offline
    RE: Filtering in real-time 2007/05/15 07:17:52 (permalink)
    5 (1)
    Settling time is the time necessary to converge to the final value of a step input, isn’t it?

    Yes.  For example you might say you need 90% settling in 100mS.

    I want to remove the step inputs, and I can tolerate a big settling time

    Words like "big" are useless in specs.  How am I supposed to know what you consider "big"?

    Anyway, let's take a stab at your filter.  As Oznog mentioned, there are two broad classes of filters, FIR (finite impulse response) and IIR (infinite inpulse response).  From a practical application point of view, especially for beginners, think of FIR as table-driven filters (the fancy mathematical word is convolution) and IIR as equation-implemented filters.  I won't go into the pros and cons of each type here and just say that I disagree with Oznog and in your case a simple IIR (equation type) filter will be fine for your purposes, easier to implement, and I think easier for you to understand.

    A simple single pole low pass filter can be realized by:

    FILT <-- FILT + FF(NEW - FILT)

    Where NEW is the new input sample each iteration, FILT is the running filltered value, and FF is the "filter fraction".  FF = 0 is a infinitely "heavy" filter in that the output never changes.  FF = 1 is no filter at all since the input is just copied to the output.  Useful values of FF are obviously somewhere in between.

    Look at this equation a bit and try to understand it.  Each iteration NEW - FILT is the amount to add to FILT to bring it all the way to the input value.  If you added this whole amount to FILT for each new value of NEW, then FILT would just follow NEW.  This is why FF = 1 causes just a pass thru filter.

    Now think what happens when FF = 1/2 as a example.  Each time the output is brought only half way towards the new input.  If everything starts out at 0 and then the input suddenly changed to 1, the first time FILT would be 1/2 (half way from its old value of 0 to the NEW value of 1).  If NEW then stayed at 1 for subsequent iterations, the next time FILT would be 3/4, then 7/8, then 15/16, etc.  If you plotted these values as a function of input sample iterations, you would see the output has a exponential decay approaching the input value.  In this case the 50% settling time is one iteration and random input noise is attenuated by 2.  Think of everything being 0 and then a single blip comes in with the value 1, then back to 0.  This single input has a amplitude of 1, but the filter output will only go to 1/2, then decay back down to 0.

    Now go back and look at the equation again.  Note how it only requires a subtract, a multiply, and a add.  The dsPIC can perform each of these in a single cycle.  On other processors without such a nice multiply capability a common trick is to chose a value of FF that is 1/2**N, like 1/4, 1/8, etc.  The multiply by FF can then be accomplished by a right shift of N bits.  However on a dsPIC you don't need this trick.

    Back to your problem.  It looks like you are really just interested in reducing random noise, and are willing to put up with some finite settling time in return, although your value of 10mS makes no sense when you already said you don't care about frequencies above 10Hz, so I'll just ignore that.

    To pick something as a example, let's see what happens when FF = 1/16.  That gives you a 50% settling time in 11 iterations and a 95% settling time of about 47 iterations, which is 65mS with your 721Hz sample rate.  FF = 1/8 would yield 50% settling time in 5 iterations (6.9mS) and 95% settling time in 23 iterations (32mS), but at only half the random noise attenuation of the FF = 1/16 case.

    There is a fixed tradeoff between settling time and FF value for the filter shown above.  However you can get faster settling at the same random noise attenuation (or more noise attenuation at the same settling time) by using more computation.  This is done by concatenating multiple of the filters shown above in series.  In other words, the output of the first filter becomes the input of the next.  In fancy filter lingo, each individual filter according to the equation is called a "pole", and stringing multiple of them together is a multi-pole filter.  The random noise attenuation of the whole multi-pole filter is all the individual FF values multiplied together.  Due to the fact that the filters settle slowly, the settling times of multiple poles in series are overlapped somewhat.  This allows for a faster overall settling time for the same combined overall random noise attenuation.

    For example, using a two pole filter with each FF = 1/8 yields a total random noise attenuation of 64x, a 50% settling time of 12 iterations (17mS), and a 95% settling time of 35 iterations (49mS).

    I use these types of simple IIR low pass filters routinely on periodic A/D readings.  I often set up a periodic interrupt to read the A/D faster than the value is needed, then apply some low pass filtering to it.  The final low pass filtered value is left around in a global variable that the foreground code can read whenever it wants the current input value.  To make it easier to see the tradeoffs between the number of poles, FF values, and step response time, I have created the FILTBITS program and the PLOTFILT wrapper for it.  FILTBITS calculates the unit step response given the FF value (actually expressed as the number of bits to shift, in other words N instead of FF in the equation FF = 1 / 2**N), and PLOTFILT shows a plot of the data.  Both these programs are part of the PIC development tools release available at http://www.embedinc.com/pic/dload.htm.  See their documentation files for details.  For example, here is the PLOTFILT output for the 2 pole filter with each FF = 1/8 as described above:



     
    Edit: Removed thumbnail and substituted real plot image once it was uploaded to the forum server.
    post edited by Olin Lathrop - 2007/05/15 07:24:46

    Attached Image(s)

    #11
    Marino
    Starting Member
    • Total Posts : 74
    • Reward points : 0
    • Joined: 2006/06/09 16:57:37
    • Location: 0
    • Status: offline
    RE: Filtering in real-time 2007/05/15 16:25:44 (permalink)
    0
    You guys have made my day! Thank You!
     
    Without your help I would probably try to cut the noise with some adventurous-code for many days and nights. Just take a look at the lazy-code above and you can imagine in what direction I was trying.
     
    I still have some questions, but first I will learn the digital filter basics and try to find the answers and if I still have any questions, I will ask with an academic language.
     
    Thank You!
    #12
    JimDrew
    Super Member
    • Total Posts : 342
    • Reward points : 0
    • Joined: 2003/11/07 12:37:26
    • Status: offline
    Re: RE: Filtering in real-time 2017/03/17 12:37:39 (permalink)
    0
    Thanks for the info, Olin.  Took some tinkering, but I got a nice IIR filter that replaced the FIR I was using.
     
    post edited by JimDrew - 2017/03/18 18:28:00
    #13
    Jump to:
    © 2019 APG vNext Commercial Version 4.5