• AVR Freaks

Hot!MPLabX v5.40: Disk Usage Analysis

Author
GlennP
Super Member
  • Total Posts : 749
  • Reward points : 0
  • Joined: 2009/03/29 15:04:55
  • Location: El Paso County, CO, USA
  • Status: offline
2020/05/20 08:22:48 (permalink)
5 (2)

MPLabX v5.40: Disk Usage Analysis

In the process of installing a partial version of 5.40 (8- and 16- bit MCUs only), I noted a significant increase in disk space usage compared to v5.35.  This is to be expected as time marches on (although v5.40 does not include MPASMX and v5.35 does).
 
What I found interesting is the number of identical files has increased significantly.  Here is a brief summary of v5.35's and v5.40's installed files (both partial installs: 8- and 16- bit MCUs only):
 
v5.35 (w/ MPASMX):
Files: Total =  35,227, Duplicates =   9,195 (~26%).
Bytes: Total = 5.735GB, Duplicates = 0.654GB (~11%).
 
v5.40 (w/o MPASMX):
Files: Total =  40,228, Duplicates =  12,760 (~32%).
Bytes: Total = 6.415GB, Duplicates = 1.095GB (~17%).
 
What I plan to do is to use Windoz's Hardlinks to decrease the installed size.  But Microchip could decrease the size and download time of the installers, decrease the installed size, and decrease the install time by detecting this before packaging.
 
Here are two v5.40 examples where files have identical contents:
:: Possible Duplicates with FileSize = 196,462:
:: Actual Duplicates:
:: Vol Cpy Lnk Cod     Size Name
:: --- --- --- --- -------- ----
::   1   1   1   F  196,462 "D:\Program Files (x86)\Microchip\MPLABX\v5.40\packs\Microchip\PIC18Fxxxx_DFP\1.1.19\edc\PIC18LF4525.PIC"
::   1   2   1   S  196,462 "D:\Program Files (x86)\Microchip\MPLABX\v5.40\packs\Microchip\PIC18Fxxxx_DFP\1.2.26\edc\PIC18LF4525.PIC"

 
:: Possible Duplicates with FileSize = 97,513:
:: Actual Duplicates:
:: Vol Cpy Lnk Cod     Size Name
:: --- --- --- --- -------- ----
::   1   1   1   F   97,513 "D:\Program Files (x86)\Microchip\MPLABX\v5.40\mplab_platform\mplab_ide\modules\docs\Readme for MPLAB PM3.htm"
::   1   2   1   S   97,513 "D:\Program Files (x86)\Microchip\MPLABX\v5.40\docs\Readme for MPLAB PM3.htm"

 
This is not totally unusual, I have found similar levels of identical files in other packages.  But I found the increase from v5.35 to v5.40 a bit of a surprise.
 
GlennP
post edited by GlennP - 2020/05/20 09:48:42
#1

6 Replies Related Threads

    g1l1t1
    Starting Member
    • Total Posts : 13
    • Reward points : 0
    • Joined: 2020/02/27 06:19:02
    • Location: 0
    • Status: offline
    Re: MPLabX v5.40: Disk Usage Analysis 2020/05/20 11:29:08 (permalink)
    +1 (1)
    I too think it would be very helpful to minimize duplication of files.
     
    Upon looking at the errata libraries vs. the ordinary non-errata ones, I see that some of the libraries with the same names are identical between the two versions, based on the .a file's hash values.
     
    I also suspect, but haven't actually checked carefully, that libraries with the same name, but different hash values, may still share many identical object files. But in such cases it would be harder to reduce the amount of duplication without a lot of work on other parts of xc16 and MPLAB X.
    #2
    GlennP
    Super Member
    • Total Posts : 749
    • Reward points : 0
    • Joined: 2009/03/29 15:04:55
    • Location: El Paso County, CO, USA
    • Status: offline
    Re: MPLabX v5.40: Disk Usage Analysis 2020/05/20 11:49:22 (permalink)
    +1 (1)
    gtackett...
    I also suspect, but haven't actually checked carefully, that libraries with the same name, but different hash values, may still share many identical object files. But in such cases it would be harder to reduce the amount of duplication without a lot of work on other parts of xc16 and MPLAB X.


    That's an excellent observation that I had not even considered.  [I don't use libraries as I'm an Assembler Troglodyte.]  I was looking from 50,000' (and using a tool I've used for years).

    But, as you say, it would be more work to "fix" in the library case.  It might even fail the "Net Gain" test as shown by one of my favorite xkcd comics:

    https://imgs.xkcd.com/com..._it_worth_the_time.png

    GP
    #3
    du00000001
    Just Some Member
    • Total Posts : 3675
    • Reward points : 0
    • Joined: 2016/05/03 13:52:42
    • Location: Germany
    • Status: offline
    Re: MPLabX v5.40: Disk Usage Analysis 2020/05/23 03:53:45 (permalink)
    0
    Prediction:
    The number of duplicates in the *_DFP area will keep increasing:
    • DFPs are distributed as a complete package - containing existing derivatives as well as new ones.
    • Some customers have to "freeze" some DFP revision, so automatic removal isn't an option.
    • WRT DF0s; you may decide to remove outdated ones manually (better check for differences - could be some updated file has been "de-proved".

    PEBKAC / EBKAC / POBCAK / PICNIC (eventually see en.wikipedia.org)
    #4
    NorthGuy
    Super Member
    • Total Posts : 6086
    • Reward points : 0
    • Joined: 2014/02/23 14:23:23
    • Location: Northern Canada
    • Status: online
    Re: MPLabX v5.40: Disk Usage Analysis 2020/05/23 07:28:06 (permalink)
    0 (2)
    The duplicates are bad because they increase the size of the download. This may be dealt with in the installer - you only need one copy of the source file to spread it over multiple locations.
     
    I have lots of disk space on all the PCs, so I don't worry much about this. I have many versions of MPLAB X installed as I never uninstall them. The MPLAB X installer is relatively fast compared to other modern software.
    #5
    du00000001
    Just Some Member
    • Total Posts : 3675
    • Reward points : 0
    • Joined: 2016/05/03 13:52:42
    • Location: Germany
    • Status: offline
    Re: MPLabX v5.40: Disk Usage Analysis 2020/05/23 07:38:56 (permalink)
    0
    NorthGuy
    The duplicates are bad because they increase the size of the download. This may be dealt with in the installer - you only need one copy of the source file to spread it over multiple locations.
     <...snipped...>



    One might consider reducing the download size. But in the times of live-view of HD videos . . .
    (I still recall the times when a 40 MB printer driver download crashed a number of times prior completion of an intact download. Can you imagine a printer driver as small as 40 MB these days? I still have a copy of the download manager up and running, but rarely use it these days.)

     

    PEBKAC / EBKAC / POBCAK / PICNIC (eventually see en.wikipedia.org)
    #6
    judd
    Starting Member
    • Total Posts : 43
    • Reward points : 0
    • Joined: 2019/04/15 22:30:22
    • Location: 0
    • Status: offline
    Re: MPLabX v5.40: Disk Usage Analysis 2020/05/23 08:48:40 (permalink)
    0 (2)
    du00000001
    Prediction:
    The number of duplicates in the *_DFP area will keep increasing:
    • DFPs are distributed as a complete package - containing existing derivatives as well as new ones.
    • Some customers have to "freeze" some DFP revision, so automatic removal isn't an option.
    • WRT DF0s; you may decide to remove outdated ones manually (better check for differences - could be some updated file has been "de-proved".

    Sure, more content will move into DFPs (and TPs), but they don't all have to be distributed with MPLAB installer.
    To "freeze" your DFP, just download it from https://packs.download.microchip.com and install it/archive it.
    (There is not much there right now, but http://packs.download.atmel.com contains a decent history of where this system came from...)
    #7
    Jump to:
    © 2020 APG vNext Commercial Version 4.5