• AVR Freaks

Hot!XC8 compile using all cores?

Author
Xavier_
New Member
  • Total Posts : 1
  • Reward points : 0
  • Joined: 2019/08/28 22:33:58
  • Location: 0
  • Status: offline
2019/08/29 22:48:23 (permalink)
0

XC8 compile using all cores?

I've been doing small codes for years, when compiling takes less than a minute. But lately I have gotten into bigger projects where I have codes of more than 10,000 lines. And with these the compilation becomes eternal, more than 4 minutes.
Observing the use of the processor I see that when it compiles it only uses a single core.
You can see the attached image in which my quadcore is almost all the time at 25% during compilation.
 
Microchip should improve the XC8 compiler to take advantage of all processor cores as CPUs with 8 to 16 cores are becoming fashionable.
 
Curiously, the XC32 does use all the cores.
 
By the way, the "-j8" only works for the tasks that MPLAB X does, not about the XC8 compilation.

Attached Image(s)

#1

9 Replies Related Threads

    du00000001
    Just Some Member
    • Total Posts : 3059
    • Reward points : 0
    • Joined: 2016/05/03 13:52:42
    • Location: Germany
    • Status: offline
    Re: XC8 compile using all cores? 2019/08/30 12:37:30 (permalink)
    +1 (1)
    Honestly, 4 minutes is hardly enough time to get another coffee.
    And usually you would "make", not "build all" (unless your programming paradigm is "try and error" - repeatedly modifying some central header file. As compiling is mostly a serial job, multi threading will only apply for projects with many files - where file-I/O might soon become the bottleneck.
     
    While it might be nice to learn that XC8 supports multi-threading, until now I do not really have a need for this.

    PEBKAC / EBKAC / POBCAK / PICNIC (eventually see en.wikipedia.org)
    #2
    Gort2015
    Klaatu Barada Nikto
    • Total Posts : 3289
    • Reward points : 0
    • Joined: 2015/04/30 10:49:57
    • Location: 0
    • Status: offline
    Re: XC8 compile using all cores? 2019/08/30 14:09:58 (permalink)
    0
    3000 lines of C code + 3000 lines of asm code compile take 2.5 seconds from clean.
    With VLC and webpages open.
    1TB SSD, i7-920 (3.6GHz) and 32GB DDR3, Win7 64bit (Of course)
     
    Checked in Task Manager, all 8 cores busy at max.
    Seems to start at around 25%, shoots up to 100% and then back to around 25%.
     
    Right click the running process -> Set priority to normal
    Right click -> Infinity...  tick all cores
     
    This can also be set to auto.
     
    Make sure all cores are enabled in the MB BIOS.

    MPLab X playing up, bug in your code? Nevermind, Star Trek:Discovery will be with us soon.
    https://www.youtube.com/watch?v=Iu1qa8N2ID0
    + ST:Continues, "What Ships are Made for", Q's back.
    #3
    Gort2015
    Klaatu Barada Nikto
    • Total Posts : 3289
    • Reward points : 0
    • Joined: 2015/04/30 10:49:57
    • Location: 0
    • Status: offline
    Re: XC8 compile using all cores? 2019/08/30 14:14:43 (permalink)
    0
    "Microchip should improve the XC8 compiler to take advantage of all processor cores as CPUs with 8 to 16 cores are becoming fashionable."
    Depends really on how the operating system distributes tasks to cores but what good would a 16 core cpu do to improve on your compile speed?
     
    Try setting up a ram-disk.
     
    post edited by Gort2015 - 2019/08/30 14:25:24

    MPLab X playing up, bug in your code? Nevermind, Star Trek:Discovery will be with us soon.
    https://www.youtube.com/watch?v=Iu1qa8N2ID0
    + ST:Continues, "What Ships are Made for", Q's back.
    #4
    Gort2015
    Klaatu Barada Nikto
    • Total Posts : 3289
    • Reward points : 0
    • Joined: 2015/04/30 10:49:57
    • Location: 0
    • Status: offline
    Re: XC8 compile using all cores? 2019/08/30 14:27:06 (permalink)
    0
    4 Minutes, maybe in 1990.

    MPLab X playing up, bug in your code? Nevermind, Star Trek:Discovery will be with us soon.
    https://www.youtube.com/watch?v=Iu1qa8N2ID0
    + ST:Continues, "What Ships are Made for", Q's back.
    #5
    Antipodean
    Super Member
    • Total Posts : 1752
    • Reward points : 0
    • Joined: 2008/12/09 10:19:08
    • Location: Didcot, United Kingdom
    • Status: offline
    Re: XC8 compile using all cores? 2019/08/30 14:43:13 (permalink)
    +1 (1)
    Xavier_
    I've been doing small codes for years, when compiling takes less than a minute. But lately I have gotten into bigger projects where I have codes of more than 10,000 lines. And with these the compilation becomes eternal, more than 4 minutes.
    Observing the use of the processor I see that when it compiles it only uses a single core.
    You can see the attached image in which my quadcore is almost all the time at 25% during compilation.



    This sounds like you have one big file, so the compiler has no advantage in using multiple cores. I suspect hat if your project was broken up into module files then each module will compile in parallel thereby using multiple cores.
     
     

    Do not use my alias in your message body when replying, your message will disappear ...

    Alan
    #6
    Gort2015
    Klaatu Barada Nikto
    • Total Posts : 3289
    • Reward points : 0
    • Joined: 2015/04/30 10:49:57
    • Location: 0
    • Status: offline
    Re: XC8 compile using all cores? 2019/08/31 11:02:32 (permalink)
    0
    10,000 lines of well structured top-down code ?
     
    Depending on how you write your code and any dependencies effects the speed of the compiler and linker.
     
    One little change can cause the compiler to compile lots of code but one small change in the right place does not cause any compile at all in your project and the linker will take care of that.
     
    i.e.
    My LCD library only gets compiled if I make a change and manual compile it and that takes a long time (nearly five seconds)
    The main code is only 50 lines of C  (1 < sec)
    post edited by Gort2015 - 2019/08/31 11:04:52

    MPLab X playing up, bug in your code? Nevermind, Star Trek:Discovery will be with us soon.
    https://www.youtube.com/watch?v=Iu1qa8N2ID0
    + ST:Continues, "What Ships are Made for", Q's back.
    #7
    NKurzman
    A Guy on the Net
    • Total Posts : 17839
    • Reward points : 0
    • Joined: 2008/01/16 19:33:48
    • Location: 0
    • Status: offline
    Re: XC8 compile using all cores? 2019/08/31 13:03:48 (permalink)
    0
    Is this even possible with XC8? With XC32 modules are compiled independently. So parallel compilation is not an issue.
    XC8 is OCG. So effectively or code modules are compiled at once. The use of multiple cores with the need to be built in to the compiler
    #8
    jtemples
    عُضْوٌ جَدِيد
    • Total Posts : 11338
    • Reward points : 0
    • Joined: 2004/02/13 12:31:19
    • Location: Southern California
    • Status: offline
    Re: XC8 compile using all cores? 2019/08/31 16:11:59 (permalink)
    +1 (1)
    Curiously, the XC32 does use all the cores.

     
    I'm pretty sure XC32, which is based on gcc, knows nothing about CPU cores.
     
    But the way that gcc-based compilers generate code lends itself to having multiple compiles running in parallel on different cores.  This is what the -j flag in "make" does.

    XC8, on the other hand, generates all the code for the executable in a single process.  You can pass -j8 to make when using XC8, and it will work just fine, but it will have little effect because the task you're spreading across multiple cores is just the code parsing, which is quite fast.
     
    And if your build is taking 4 minutes, I suspect you have a very slow PC.  I don't have a project quite that large, but an 8000 line project built with  -j1 --opt=all takes about 8 seconds,  with -j8 about 5.
    post edited by jtemples - 2019/08/31 18:02:43
    #9
    Gort2015
    Klaatu Barada Nikto
    • Total Posts : 3289
    • Reward points : 0
    • Joined: 2015/04/30 10:49:57
    • Location: 0
    • Status: offline
    Re: XC8 compile using all cores? 2019/09/01 04:17:46 (permalink)
    0
    You need a bigger Engine.
     

    MPLab X playing up, bug in your code? Nevermind, Star Trek:Discovery will be with us soon.
    https://www.youtube.com/watch?v=Iu1qa8N2ID0
    + ST:Continues, "What Ships are Made for", Q's back.
    #10
    Jump to:
    © 2019 APG vNext Commercial Version 4.5