• AVR Freaks

Hot!How to display Chinese characters

Page: 123 > Showing page 1 of 3
Author
Kaisertoni
New Member
  • Total Posts : 23
  • Reward points : 0
  • Joined: 2019/05/23 23:30:00
  • Location: 0
  • Status: offline
2019/06/26 07:28:02 (permalink)
0

How to display Chinese characters

Hi, I'm developing a program on a display with multilanguage support, and I don't know how to handle non-ASCII characters! For example Chinese characters..
 
With GRC (Graphics Resource Converter) I generated the 3 files (.c,.h,.S) that allow the management of the characters I need and then I imported them into my program. How can I now display one of these characters on the display? For example this 更?
 
If I define a string like 
char Controls[] = "I am trying to print a Chinese character: 更";

On the screen I see "I am trying to print a Chinese character: ?", this because I probably can't declare the Chinese symbol that way inside the string. 
 
So how can I do it? Thank you..
 
P.S. I'm using the dsPIC33EP512MU810 microcontroller, and the compiler is XC16 compiler v1.23
#1

50 Replies Related Threads

    du00000001
    Just Some Member
    • Total Posts : 2906
    • Reward points : 0
    • Joined: 2016/05/03 13:52:42
    • Location: Germany
    • Status: offline
    Re: How to display Chinese characters 2019/06/26 07:42:18 (permalink)
    4 (1)
    Just some hint:
    "char" equals to bytes while the chinese character does require either unicode or some UTF format.
    ("Unfortunately", these days the editors have less issues with the required adaptations.)
     
    If you do not want to resort to single-character outputs, you will either have to adapt the output to e.g. UTF-8, UTF-16 or other. Quite some work to do...
     
    P.S.: This whole issue has a bit to do with XC16 (which might or might not accept UTF-8 coded source files) - more with the way your input is converted to binary.

    PEBKAC / EBKAC / POBCAK / PICNIC (eventually see en.wikipedia.org)
    #2
    Kaisertoni
    New Member
    • Total Posts : 23
    • Reward points : 0
    • Joined: 2019/05/23 23:30:00
    • Location: 0
    • Status: offline
    Re: How to display Chinese characters 2019/06/26 07:48:47 (permalink)
    0
    Unfortunately I have to add the Chinese language at all costs, so even if the work is long I will have to do it anyway sad: sad. How can I adapt the output to e.g. UTF-8, UTF-16 or other? Thanks..
    #3
    du00000001
    Just Some Member
    • Total Posts : 2906
    • Reward points : 0
    • Joined: 2016/05/03 13:52:42
    • Location: Germany
    • Status: offline
    Re: How to display Chinese characters 2019/06/26 08:11:35 (permalink)
    4 (1)
    Are you using standard functions like printf() etc.?
    Or just putting your text to some fancy display ?
     
    Any conversion might require some external tool, converting your strings to Unicode/UTF-16 integer coding.
    (Start reading e.g. at  https://en.wikipedia.org/wiki/Unicode )
    The resulting "strings" will no longer be readable (put the input string into the comments to retain a minimum of readability) - you'll get some form of integer initialization sequence similar to e.g.
    //                                    'E'     'r'     'r'     'o'     'r'
    unsigned short error_string_en[] = { 0x0045, 0x0072, 0x0072, 0x006F, 0x0072 };

    The advantage of Unicode/UTF-16 is that the first byte indicates the character table to use (256 entries each), the second byte is the character within the selected character table. The disadvantage is that it might be somewhat less space-efficient than UTF-8. Depending on your overall architecture, you may or may not be able to omit the "string-closing" \0, which would have to be coded as 0x0000 (a word, not a byte!).
     
    Good luck with that - this is really some work to do!

    PEBKAC / EBKAC / POBCAK / PICNIC (eventually see en.wikipedia.org)
    #4
    NKurzman
    A Guy on the Net
    • Total Posts : 17625
    • Reward points : 0
    • Joined: 2008/01/16 19:33:48
    • Location: 0
    • Status: offline
    Re: How to display Chinese characters 2019/06/26 09:58:09 (permalink)
    4 (1)
    First, How Many Chinese Chars?  I assume you found a Font.
    "I am trying to print a Chinese character: 更" That would most likely be 2 fonts.
    You need to select the Chinese Font you are using. I assume it ill be set in MHC.
    If you are using < 256 Chars you maybe able to collect the chars in to a compressed Font. ( Hard to use, but space efficient)  If not you will need to store the Full Font Set.  With the Full Font you just print.
     
    du00000001  I think the GCC Compiler has Unicode/UTF-16 support. But am not 100% sure.  
    #5
    LdB_ECM
    Senior Member
    • Total Posts : 125
    • Reward points : 0
    • Joined: 2019/04/16 22:01:25
    • Location: 0
    • Status: offline
    Re: How to display Chinese characters 2019/06/26 10:04:47 (permalink)
    4 (1)
    I have done this on a couple of commercial developments so I will tell you what I know.
     
    It isn't to bad if you only have to display text, if you have to allow typing is a nightmare. Allowing for selection in millions of times easier than actually allowing typing.
     
    For display you need to know if you can get away with the 60-120 common glyphs. If you need like uneducated level you will have around 1000 glyphs, for newspaper level reading 2000-3000 glyphs and for education around 8000. So you need to know the target market and next you are going to get a shock how big the font files are. 
     
    So now you have your font files you simply need to map them which comes down to how many glyphs you are up against. Usually the safest way is follow windows and call a glyph TCHAR and define it as uint16_t. Whatever you do I would stay the hell away from utf8 for chinese the encoding will be variable 2-4 bytes per character that means to even do a string length of how many characters are in a string you have to decode the UTF8. Try to work out how much you need to allocate or have buffer space for a string you need to decode the UTF8 ... I think you get the picture.
    #6
    Kaisertoni
    New Member
    • Total Posts : 23
    • Reward points : 0
    • Joined: 2019/05/23 23:30:00
    • Location: 0
    • Status: offline
    Re: How to display Chinese characters 2019/06/27 00:29:01 (permalink)
    0
    du00000001
    Are you using standard functions like printf() etc.?
    Or just putting your text to some fancy display ?
     
    Any conversion might require some external tool, converting your strings to Unicode/UTF-16 integer coding.
    (Start reading e.g. at  https://en.wikipedia.org/wiki/Unicode )
    The resulting "strings" will no longer be readable (put the input string into the comments to retain a minimum of readability) - you'll get some form of integer initialization sequence similar to e.g.
    //                                    'E'     'r'     'r'     'o'     'r'
     
    unsigned short error_string_en[] = { 0x0045, 0x0072, 0x0072, 0x006F, 0x0072 };
     

    The advantage of Unicode/UTF-16 is that the first byte indicates the character table to use (256 entries each), the second byte is the character within the selected character table. The disadvantage is that it might be somewhat less space-efficient than UTF-8. Depending on your overall architecture, you may or may not be able to omit the "string-closing" \0, which would have to be coded as 0x0000 (a word, not a byte!).
     
    Good luck with that - this is really some work to do!




    Yes I'm using standard functions like printf(), etc. 
    As a start I tried to do something like this:
    unsigned short error_string_en[] = { 0x0045, 0x0072, 0x0072, 0x006F, 0x0072, 0x0000 }; //Error
    printf("%s",error_string_en);

    but only the first letter is shown on the display, what am I doing wrong?
    #7
    Kaisertoni
    New Member
    • Total Posts : 23
    • Reward points : 0
    • Joined: 2019/05/23 23:30:00
    • Location: 0
    • Status: offline
    Re: How to display Chinese characters 2019/06/27 00:38:40 (permalink)
    0
    NKurzman
    First, How Many Chinese Chars?  I assume you found a Font.
    "I am trying to print a Chinese character: 更" That would most likely be 2 fonts.You need to select the Chinese Font you are using. I assume it ill be set in MHC.If you are using < 256 Chars you maybe able to collect the chars in to a compressed Font. ( Hard to use, but space efficient)  If not you will need to store the Full Font Set.  With the Full Font you just print. du00000001  I think the GCC Compiler has Unicode/UTF-16 support. But am not 100% sure.  



    I have attached an image in which there are some Chinese characters I must use. As you can see I think I need to use more than 256 characters, also because in addition to the Chinese ones I have to use Latin characters for Italian and English. I don't want to memorize all the Chinese characters but only those that I need, so I avoid filling the memory too much. I am aware that with GRC if I do not use all the characters I have to manage their mapping. For example if I use the characters a, b, c these correspond to 0x0061, 0x0062 and 0x0063, but if I excluded the b then a would correspond to 0x0061 and c to 0x0062. I already have a function in the program that handles these cases, but as I wrote in the previous post, if I try to print the chinese character, I get a question mark.

    Attached Image(s)

    #8
    Kaisertoni
    New Member
    • Total Posts : 23
    • Reward points : 0
    • Joined: 2019/05/23 23:30:00
    • Location: 0
    • Status: offline
    Re: How to display Chinese characters 2019/06/27 00:41:00 (permalink)
    0
    LdB_ECM
    I have done this on a couple of commercial developments so I will tell you what I know.
     
    It isn't to bad if you only have to display text, if you have to allow typing is a nightmare. Allowing for selection in millions of times easier than actually allowing typing.
     
    For display you need to know if you can get away with the 60-120 common glyphs. If you need like uneducated level you will have around 1000 glyphs, for newspaper level reading 2000-3000 glyphs and for education around 8000. So you need to know the target market and next you are going to get a shock how big the font files are. 
     
    So now you have your font files you simply need to map them which comes down to how many glyphs you are up against. Usually the safest way is follow windows and call a glyph TCHAR and define it as uint16_t. Whatever you do I would stay the hell away from utf8 for chinese the encoding will be variable 2-4 bytes per character that means to even do a string length of how many characters are in a string you have to decode the UTF8. Try to work out how much you need to allocate or have buffer space for a string you need to decode the UTF8 ... I think you get the picture.




    Hi, I only have to display text! I have attached an image in which there are some Chinese characters that I have to use, for me the total number of characters does not exceed 200-300. Do you have an example of how I can apply what you wrote to me? thanks a lot

    Attached Image(s)

    #9
    LdB_ECM
    Senior Member
    • Total Posts : 125
    • Reward points : 0
    • Joined: 2019/04/16 22:01:25
    • Location: 0
    • Status: offline
    Re: How to display Chinese characters 2019/06/27 00:53:04 (permalink)
    4 (1)
    printf has no idea what a string of unsigned short is, it only knows how to print characters and the moment it sees a zero it will terminate. So yes it will print one character and terminate.
     
    Some implementations of printf such as on linux or windows  understand UTF8 and you can try something like
    printf("%s\n", "\xE2\x98\xA0");
    If you get a strange looking character then it knows UTF8
     
    I don't know any embedded micro controllers that will understand UTF8 however :-)
     
    RE above: If you can stay below 256 characters you can simply encode the chinese into the characters above 128. Basically create a basic C bitmap font  0..127 English 128-255 Chinese.
    I assume you have seen C code bitmap file like these
    https://github.com/idispa...ob/master/font-16x32.c
    post edited by LdB_ECM - 2019/06/27 00:59:20
    #10
    du00000001
    Just Some Member
    • Total Posts : 2906
    • Reward points : 0
    • Joined: 2016/05/03 13:52:42
    • Location: Germany
    • Status: offline
    Re: How to display Chinese characters 2019/06/27 00:54:40 (permalink)
    4 (1)
    Re using printf():
    XC16's printf() is certainly not up to printing Unicode. Thus you would have to adapt all standard functions used to processing Unicode. Thus your "result" doesn't come unexpected.
     
    Provided Chinese is the only (or one of a very small number of) language(s) with non-latin symbols and a quite limited symbol set, an exception for chinese text output (then using direct graphical output) might be an alternative.

    PEBKAC / EBKAC / POBCAK / PICNIC (eventually see en.wikipedia.org)
    #11
    LdB_ECM
    Senior Member
    • Total Posts : 125
    • Reward points : 0
    • Joined: 2019/04/16 22:01:25
    • Location: 0
    • Status: offline
    Re: How to display Chinese characters 2019/06/27 01:04:48 (permalink)
    0
    du00000001 you seem to know the XC16 compiler do you know if it supports semihosting?
     
    That is can he just provide a new version of putc to override the standard.
    #12
    du00000001
    Just Some Member
    • Total Posts : 2906
    • Reward points : 0
    • Joined: 2016/05/03 13:52:42
    • Location: Germany
    • Status: offline
    Re: How to display Chinese characters 2019/06/27 01:13:51 (permalink)
    0
    Semihosting? Not that I knew. (Although I do not know each and everything. And semihosting is mainly a debugger topic - little to do with the compiler.)

    PEBKAC / EBKAC / POBCAK / PICNIC (eventually see en.wikipedia.org)
    #13
    LdB_ECM
    Senior Member
    • Total Posts : 125
    • Reward points : 0
    • Joined: 2019/04/16 22:01:25
    • Location: 0
    • Status: offline
    Re: How to display Chinese characters 2019/06/27 01:27:25 (permalink)
    0
    Semihosting is a lot more than debugging you can redirect the console output permanently we do it a lot.
     
    So if you take FreeRTOS for example it has no console so I semihost it on baremetal on the PI
    https://github.com/LdB-ECM/Raspberry-Pi/tree/master/FreeRTOSv10.1.1
     
    This is all done by senihosting the printf and giving it somewhere to go

    #14
    Kaisertoni
    New Member
    • Total Posts : 23
    • Reward points : 0
    • Joined: 2019/05/23 23:30:00
    • Location: 0
    • Status: offline
    Re: How to display Chinese characters 2019/06/27 01:52:18 (permalink)
    0
     
    LdB_ECM
    printf has no idea what a string of unsigned short is, it only knows how to print characters and the moment it sees a zero it will terminate. So yes it will print one character and terminate.

     
    Right, I hadn't thought of it..
     
    LdB_ECM
    Some implementations of printf such as on linux or windows  understand UTF8 and you can try something like
    printf("%s\n", "\xE2\x98\xA0");
    If you get a strange looking character then it knows UTF8
     
    I don't know any embedded micro controllers that will understand UTF8 however :-)

     
    Yes, I can't use this instruction printf("%s\n", "\xE2\x98\xA0") because the program crashes
     
    LdB_ECM
    RE above: If you can stay below 256 characters you can simply encode the chinese into the characters above 128. Basically create a basic C bitmap font  0..127 English 128-255 Chinese.
    I assume you have seen C code bitmap file like these
    https://github.com/idispa...ob/master/font-16x32.c



    is what I'm trying to do, from 0 to 131 I have the characters for Italian and English, and from 132 to 134 I have three Chinese symbols that I'm trying to print (夹具 重) without success. I don't know how to do it, because if I do printf ("夹具 重") the program returns me ???
     
    #15
    Kaisertoni
    New Member
    • Total Posts : 23
    • Reward points : 0
    • Joined: 2019/05/23 23:30:00
    • Location: 0
    • Status: offline
    Re: How to display Chinese characters 2019/06/27 01:57:38 (permalink)
    0
    du00000001
    Re using printf():
    XC16's printf() is certainly not up to printing Unicode. Thus you would have to adapt all standard functions used to processing Unicode. Thus your "result" doesn't come unexpected.

     
    How can i reuse printf? For example, I can create a function x_printf() that is able to print unicode characters, but I have no idea how to start doing it. I have just entered the world of work and I am still fairly inexperienced...
     
    du00000001
    Provided Chinese is the only (or one of a very small number of) language(s) with non-latin symbols and a quite limited symbol set, an exception for chinese text output (then using direct graphical output) might be an alternative.



    I still ask the same question, how?
     
     
     
    #16
    du00000001
    Just Some Member
    • Total Posts : 2906
    • Reward points : 0
    • Joined: 2016/05/03 13:52:42
    • Location: Germany
    • Status: offline
    Re: How to display Chinese characters 2019/06/27 02:10:34 (permalink)
    4 (1)
    It's not about "reusing" - it was "Re: 'using ...'"
    Dig up some source code for printf() (e.g. starting here:  https://stackoverflow.com/questions/4867229/code-for-printf-function-in-c ) and modify it according to your needs.
     
    @LdB_ECM
    The term Semihosting is usually applied for printing via the debugger interface of the Arms.
    In the wider sense it would be available on the dsPICs: there's "somewhere" a short doc what to do to redirect printf() - - - IIRC, it's hardly more than implementing some putc() function.

    PEBKAC / EBKAC / POBCAK / PICNIC (eventually see en.wikipedia.org)
    #17
    ric
    Super Member
    • Total Posts : 23269
    • Reward points : 0
    • Joined: 2003/11/07 12:41:26
    • Location: Australia, Melbourne
    • Status: offline
    Re: How to display Chinese characters 2019/06/27 02:22:53 (permalink)
    4 (1)
    Do you actually need formatted printing, or is this just to output fixed strings?
    i.e. would an equivalent to puts() do the job rather than printf() ?
    puts() is a hell of a lot easier to implement yourself than printf() is!

    I also post at: PicForum
    Links to useful PIC information: http://picforum.ric323.co...opic.php?f=59&t=15
    NEW USERS: Posting images, links and code - workaround for restrictions.
    To get a useful answer, always state which PIC you are using!
    #18
    LdB_ECM
    Senior Member
    • Total Posts : 125
    • Reward points : 0
    • Joined: 2019/04/16 22:01:25
    • Location: 0
    • Status: offline
    Re: How to display Chinese characters 2019/06/27 02:26:20 (permalink)
    0
    @du00000001
    Yeah I never get to fussed what other people call things, it is what we call it because it is historic to our programming history. Lots of new I just got my degree people like to redefine the past but we just show them the middle finger because we have been doing it before they were born.
     
    @Kaisertoni
    As luck would have it I released some of this to public
    If you look at this file
    https://github.com/LdB-ECM/Raspberry-Pi/blob/master/Arm32_64_USB/emb-stdio.c
    goto downto line 693 and there you shall find my version of printf because it does not exist on a generic arm compiler.
     
    It all rolls around this function _doprnt
    int _doprnt(const char *fmt, va_list ap, int (*putc_func) (int, void*), void* putc_arg)

    and specifically you provide a function you see it there .. putc_func
     
    So I am going to show you how to  attach a new putc function which will will understand unicode and I will see if I can get XC16 to semihost.
     
    I won't get a chance to look at it until saturday however I have work stuff tomorrow. 
    #19
    Kaisertoni
    New Member
    • Total Posts : 23
    • Reward points : 0
    • Joined: 2019/05/23 23:30:00
    • Location: 0
    • Status: offline
    Re: How to display Chinese characters 2019/07/01 23:54:57 (permalink)
    0
    ric
    Do you actually need formatted printing, or is this just to output fixed strings?
    i.e. would an equivalent to puts() do the job rather than printf() ?
    puts() is a hell of a lot easier to implement yourself than printf() is!


     
    Hi I have to produce fixed strings! I checked the number of Chinese characters and exceeded 255 so I have to use 2 bytes for each character. After storing the characters in the micro flash, I have to declare the strings (as suggested by @ du00000001) in the following way:
    unsigned short ChineseStr[] = { 0x00A6, 0x015B, 0x016C, 0x0000};

    I have not yet understood how to print all the characters of the vector. As already mentioned with the printf () function I only display the first character on the display.
     
    Can you explain to me what changes I need to make to be able to solve the problem? Thanks
    #20
    Page: 123 > Showing page 1 of 3
    Jump to:
    © 2019 APG vNext Commercial Version 4.5