• AVR Freaks

Some experience how to handle fixed point <libq.h>

Author
Nikolay_Po
Super Member
  • Total Posts : 1941
  • Reward points : 0
  • Joined: 2012/04/01 13:49:27
  • Location: Russia, Novorossiysk
  • Status: offline
2014/04/05 05:37:27 (permalink)
5 (1)

Some experience how to handle fixed point <libq.h>

Hello peoples!
I'm a beginner in PIC24 and C both. Have spent some time figuring out how handle the <libq.h> trigonometry functions and how fast they are on non-ds PIC.
Here is some code and test results of Q15sinPI() function in XC16. Hope this saves the time for somebody and helps to implement a quite fast calculations:
  • For full sine wave generation the Q15sinPI() function is needed.
    Libq library have Q15sin() function which is appropriate only for the tasks where the argument (the angle in radians) is limited by -1..1. For full four quadrant operation Q15sinPI() function should be used.
    Argument range of Q15sinPI() is -PI..+PI which is mapped from the range of Q0_15 fixed point number:
    -32768 is -Pi, 32767 is +0.999969482421875*Pi.
  • _Q0_15 fixed point type is different from recently introduced XC16 1.20 fixed point type _Fract. Actually _Q0_15 is defined as signed short integer in libq.h: "typedef short _Q0_15;". There is no cast needed for handling _Q0_15 type as signed short integer.
  • <libq.h> library header should be included.
  • -lq parameter should be added in "Additional options" field of xc16-ld part of XC16 in MPLABX project parameters. As I've understood this is a linker option for libq library support.
  • Small test loop of argument incrementing and _Q15sinePI() computing takes 69 instruction cycles  for every time except 13 cycles for -PI and 0 argument values.
  • The sine is symmetric both in time and in amplitude.
  • The sine is almost perfect. Artifacts on the spectrum are below -97dB form full sine power. See attachments.
  • PIC24HJ_Start.zip in attachments is packaged MPLABX XC16 _Q15sinePI() example project for PIC24HJ64GP202 chip. The code starts the transmission of _Q15sinePI() output through an UART in forever loop: 115200-odd-2, little endian.
  • PIC_Output.zip is twice (7Zip and ZIP) packed archive of 109 sine cycles (above 2^22 samples) recorded from hardware UART output.
  • The spectrum comparison on PNG-image attached. The white is _Q15sinePI() output spectrum. The green is a spectrum of "ideal" 16bit per sample wave generated by Audacity 2.0.5 without a dithering. The frequency of sine on diagram is 192000/65536=2.9296875Hz because of 65536 samples period.
post edited by Nikolay_Po - 2014/04/05 05:38:32

Attached Image(s)

#1

17 Replies Related Threads

    DavidBLit
    Super Member
    • Total Posts : 1585
    • Reward points : 0
    • Joined: 2012/02/18 13:08:48
    • Location: The Land of Confusion
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2014/04/05 05:51:39 (permalink)
    +1 (1)
    Excellent work, as always!  cool: [8)]
    #2
    viki2000
    Super Member
    • Total Posts : 344
    • Reward points : 0
    • Joined: 2013/05/08 16:54:07
    • Location: Germany
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2017/05/02 06:12:16 (permalink)
    0
    I bought few PIC24HJ64GP202 and decided to replicate your proposed test.
    I setup the MPLABX with your project files and “libq.h” using XC 1.26. as we discussed here:
    http://www.microchip.com/forums/m955282.aspx#982363
    The PIC is programmed with PickKit 3 and seems that works.
    I use a USB to RS232 adapter and I can see the data on PC using different serial monitor/logger software.
    My question to you is: what kind of software did you use to capture the data and have it in that .raw format?
     
    I use for example RealTerm to see and capture or RS232 Data Logger to capture and I have tested several other programs for serial port. My problem is that I cannot get the data captured in the right format.
    If I take your .raw zipped data and I opened it in Audacity, then I am able to see the generated sine.
    If I understand right, the data speed and format is 115200 baud rate, 8 bit, odd, 2 bit of stop.
    My problem is how do you capture the data in the proper format to be able to import it in Audacity or why not Excel?
    #3
    Nikolay_Po
    Super Member
    • Total Posts : 1941
    • Reward points : 0
    • Joined: 2012/04/01 13:49:27
    • Location: Russia, Novorossiysk
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2017/05/02 08:42:16 (permalink)
    +1 (1)
    I don't remember. It was under Windows. May be I've imported raw data into CRAN R software as raw bytes. I had no experience with sprint() that days. Currently I'd use sprintf() with small one sample string buffer and streaming the data as integers with \r\n for new line.
    #4
    Nikolay_Po
    Super Member
    • Total Posts : 1941
    • Reward points : 0
    • Joined: 2012/04/01 13:49:27
    • Location: Russia, Novorossiysk
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2017/05/02 08:44:29 (permalink)
    +1 (1)
    What if you try to import the stream as raw mono 16bit audio right into Audacity? Should work!
    #5
    viki2000
    Super Member
    • Total Posts : 344
    • Reward points : 0
    • Joined: 2013/05/08 16:54:07
    • Location: Germany
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2017/05/02 11:31:10 (permalink)
    0
    Yes, I alreday did that your raw data in zip+7zip file. I went to Audacity, Import, Raw Data and was fine.
    My problem is from UART of PIC to RS232 TTL to USB PC to a serial port.
    Now, how do you capture that serial port raw data in PC?
    You cannot connect Audacity to import directly from serial port.
    post edited by viki2000 - 2017/05/02 11:32:20
    #6
    viki2000
    Super Member
    • Total Posts : 344
    • Reward points : 0
    • Joined: 2013/05/08 16:54:07
    • Location: Germany
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2017/05/02 11:37:54 (permalink)
    0
    Just to make it clear for myself:
    -  UART settings, so the same settings for serial port in PC is: 115200 baud rate, 8bit of data, odd, 2bit stop, sending the low byte first and the high byte afterwards.
    -  The data x rolls over as follows: -32768 is -Pi, 32767 is +0.999969482421875*Pi. Then y=sin(x) and that y is sent to UART.
    Is that right?
     
    What resolution and what frequency has the generated sine wave?
    I see mentioned “109 sine cycles (above 2^22 samples)” and then “The frequency of sine on diagram is 192000/65536=2.9296875Hz because of 65536 samples period.”
     
    If I calculate 2^22=4194304 and then 4194304/109=38479.85, so we can say 38480 samples for 1 cycle. Then how do we match that we “65536 samples period”?
    post edited by viki2000 - 2017/05/02 11:42:59
    #7
    CinziaG
    die fucking humans
    • Total Posts : 3145
    • Reward points : 0
    • Joined: 2016/12/07 14:20:36
    • Location: Wien
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2017/05/02 11:40:42 (permalink)
    0
    x should range from -3.14 to +3.14, so I suppose you mean
    y= ( x*3.141592653589 (by memory :D ) / 32768 )

    in 2018 you signed for your annihilation. in 2019 it will come ;) I promise
    my most wonderful creations here
    https://www.youtube.com/c...dPFRvtwsbSTXp6Sk6azGOQ
    #8
    viki2000
    Super Member
    • Total Posts : 344
    • Reward points : 0
    • Joined: 2013/05/08 16:54:07
    • Location: Germany
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2017/05/02 11:46:34 (permalink)
    0
    I quoted the explanations from main.c of the project:
    // Transmitting the sine function results over UART

    #include <xc.h> //The base for compiler
    #include "custom_header.h" //Custom project-specific functions and definitions
    #include <libq.h> //Fixed point math

    int main(void) {
        _Q0_15 X,Y; //Argument and Result
        unsigned short int Z; //Transmitted symbol counter
        
        Initialization(); //Setitng up default pins states and configuring the clock
        UART1_initialization(); //Configuring and enabling UART1

        X=0; //X rolls over. -32768 is -Pi, 32767 is +0.999969482421875*Pi
        Z=0; //Symbol counter for TX LED indication

        do {
            Y=_Q15sinPI(X++); //Computing sine value then incrementing X
            UART1_wait_and_indicate(&Z); //Waiting for TX and flashing by LED
            U1TXREG = Y && 0xFF; //Transmitting low byte
            UART1_wait_and_indicate(&Z); //Waiting for TX and flashing by LED
            U1TXREG = Y >> 8; //Transmitting high byte
        }while(1);
    }

    #9
    CinziaG
    die fucking humans
    • Total Posts : 3145
    • Reward points : 0
    • Joined: 2016/12/07 14:20:36
    • Location: Wien
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2017/05/02 11:49:15 (permalink)
    0
    Oh sorry, it's fixed point, I missed that Smile
    Then I suppose you're right.

    in 2018 you signed for your annihilation. in 2019 it will come ;) I promise
    my most wonderful creations here
    https://www.youtube.com/c...dPFRvtwsbSTXp6Sk6azGOQ
    #10
    Nikolay_Po
    Super Member
    • Total Posts : 1941
    • Reward points : 0
    • Joined: 2012/04/01 13:49:27
    • Location: Russia, Novorossiysk
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2017/05/02 12:23:54 (permalink)
    0
    I've set 192000 samples per second rate for an "audio" file which is one of standard rate for audio processing software. This is the reason of virtual 2.93Hz frequency you may observe on that old picture. Just to analyze the purity of generated wave by native audio software way.
    #11
    viki2000
    Super Member
    • Total Posts : 344
    • Reward points : 0
    • Joined: 2013/05/08 16:54:07
    • Location: Germany
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2017/05/02 12:26:07 (permalink)
    0
    What about the other questions that I have? Any chance to remember details?
    #12
    Nikolay_Po
    Super Member
    • Total Posts : 1941
    • Reward points : 0
    • Joined: 2012/04/01 13:49:27
    • Location: Russia, Novorossiysk
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2017/05/02 13:38:35 (permalink)
    +1 (1)
    I've used Realterm under the Windows Vista. Currently I'm using Ubuntu 16.04LTS and the capturing is as simple as one string:
    cat /dev/ttyUSB0 >DataStream.csv

    I'm using sprintf() for generic cases and my own simple "C" code to transmit hexadeximals if I need a tens of ksps of integers data stream rate. Even lower end PIC24s are capable to transmit at 2Mbaud out of spec (should not receive that fast).
    #13
    viki2000
    Super Member
    • Total Posts : 344
    • Reward points : 0
    • Joined: 2013/05/08 16:54:07
    • Location: Germany
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2017/05/03 00:53:08 (permalink)
    +1 (1)
    I used too RealTerm to visualize and capture data:
    https://sourceforge.net/projects/realterm/
    Then also RS232 Data Logger from Eltima, the freeware version:
    https://sourceforge.net/projects/rs232portlogger/
     
    Here are the screenshots video recordings and the captured data:
    https://goo.gl/YIIjQD
     
    Then I have tried some trial version of Device Monitoring Studio and Advanced Serial Data Logger:
    https://www.hhdsoftware.com/device-monitoring-studio
    https://www.aggsoft.com/asdl-rs232-to-excel.htm
    Or simple Tera Term:
    https://ttssh2.osdn.jp/index.html.en
    The data is saved in a .txt or .log file or any other specific file, attached you may see the .txt captured files.
    The PIC sends continuous in a loop.
     
    As you can see above, I have no problem to capture the data, even though I do not know if I capture all or enough packages.
    My problem is to see/get/convert the data correctly, because the data comes in packages of 8bit and 1 data sample for sine wave is 16bit. That means I have to remake 1 sample by putting together, by concatenate 2 byte. That 1 sample is 1 point of the sine wave.
    These data loggers/captures will store in a file either continuously data without any delimitation between packages as I see in HEX file from RealTerm or 1 byte as ASCII string.
    I do not know how I should convert that in a usable number, 2 byte together, to get the points of the sine. It is just a conversion problem or perhaps setting of the capture serial software.
    I would like to have in captured file the same as I see on the RealTerm screen terminal when I set in16 and 2 byte for one frame, then I think I see the correct usable numbers. Then as second part, would be even more interesting to know how you arrived to that .raw file which I am able to see in Audacity.
    So, how do I convert what I capture at serial port in a usable format?
    #14
    Nikolay_Po
    Super Member
    • Total Posts : 1941
    • Reward points : 0
    • Joined: 2012/04/01 13:49:27
    • Location: Russia, Novorossiysk
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2017/05/03 03:56:05 (permalink)
    0
    Just import the data stream as raw 16 bit audio. If you can't guess endianess in Audacity then try to cut single byte from the beginning of the file. This will shift the bytes inside a 16-bit words and should correct your import. After you got correct waveform in Audacity you can export the file even in text format.
    By the way, your problems have nothing common with the topic. It will be better you file new topic about import/export the data. Why you stick with my project? Don't you see correct numbers in debugger or in terminal? If you need to import the data stream correctly by your setup then write your own transmit function. It is easier than make LibQ to work.
    #15
    viki2000
    Super Member
    • Total Posts : 344
    • Reward points : 0
    • Joined: 2013/05/08 16:54:07
    • Location: Germany
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2017/05/03 07:18:12 (permalink)
    0
    Actually you gave me a good idea and eventually I have sorted out.
    To answer your question: I stick with your project because you draw my attention, I consider it a nice start point to develop later more complex code or perhaps simpler.
    You are right; it has to do with import/export data, not with the PIC and code, but is related with the project, to understand it and see where people may stumble if someone wants to replicate your tests.
    I see the correct number in terminal, I guess. I see numbers but no idea if they are correct, because they come and go fast :) , anyway here is what I have done:
    - Used RealTerm, set up the serial port as 115200 baud rate, 8bit of data, odd, 2bit stop, then on Display int16, then went to Capture, set to capture in capture.txt for 60s, but not as HEX.
    - Then in Audacity I just went to Import Raw Data, Signed 16bit PCM, Little-endian, 1 Channel Mono, Sample Rate 44100 Hz, Start offset 0 bytes, Amount to Import 100%.
    - And I got the same wave as in your .raw file above, the same frequency, of course less cycles and not sync with yours, which is logical.
    - The same I could do it with Tera Term.
    It did not cross to my mind to try to import as Raw Data the .txt file; I thought it must be always .raw file.
    Everything is fine now, looks good. The exporting to .raw or text seems that involves saving first as uncompressed WAV file: https://www.hellomico.com/getting-started/convert-audio-to-raw/ . I will investigate that later.
    Next I went to Audacity, Analyze, Sample Data Export, I have tried 100K samples with Data (CSV) and also Time Indexed (txt) and I have imported that resulted file in Excel with Data, From Text and after I arranged the samples I got a nice sine wave.
     I am not hijacking your post; I just wanted to understand it in detail, with everything that is related with it, meaning replicating in detail and see the generated data, if it is genuine sine.
    I have tried also Adobe Audition, where we can see and analyze even nicer the waveform. I measured the frequency by “hand” placing the cursors and I got 0.342s, around 2.92Hz as you mentioned above.
    Do you mind if I come later with a different code, maybe simpler or better written, which does the same as yours or if I analyze the practical aspects/implications of the existing code?
     
    Here is the result:
    https://goo.gl/oUdAhP
    #16
    Nikolay_Po
    Super Member
    • Total Posts : 1941
    • Reward points : 0
    • Joined: 2012/04/01 13:49:27
    • Location: Russia, Novorossiysk
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2017/05/03 07:39:28 (permalink)
    +1 (1)
    It can't be simpler than a simple call of libq function. I had none intention to discuss data export operations. I wanted to prove the libq works and works good. That is all.
    #17
    viki2000
    Super Member
    • Total Posts : 344
    • Reward points : 0
    • Joined: 2013/05/08 16:54:07
    • Location: Germany
    • Status: offline
    Re: Some experience how to handle fixed point <libq.h> 2017/07/13 03:10:42 (permalink)
    0
    Actually it can be simpler than a simple call of libq function.
    Your example was an inspiration and a challenge for me.
    I will continue with explanations here: http://www.microchip.com/forums/m990068.aspx
    #18
    Jump to:
    © 2021 APG vNext Commercial Version 4.5