I'm currently trying to reduce the bundle size of an npm-package I have created. And I managed to get to an unpacked size of about 210kb.
https://www.npmjs.com/package/#agile-ts/core/v/0.0.12 <- 210kb
https://www.npmjs.com/package/#agile-ts/core/v/0.0.11 <- 304kb (with comments)
One change I made was to remove all the comments with help of the 'tsconfig' file, which reduced my unpacked size about 100kb BUT the minified size stayed the same (57kb)?
https://bundlephobia.com/result?p=#agile-ts/core#0.0.12 <- 57kB
https://bundlephobia.com/result?p=#agile-ts/core#0.0.11 <- 57kB (with comments)
So I was wondering how the unpacked size does affect the minified size. Are in the minified size comments already removed? I found no answer to this question on the internet.
Another package I found has an unpacked size of about 325kb
https://www.npmjs.com/package/#hookstate/core <- 325kb
but a minified size of 16.7kB.
https://bundlephobia.com/result?p=#hookstate/core#3.0.6 <- 16.7kb
-> It is about 30% larger in the unpacked size but 70% smaller in the minified size?
The only difference I found, is that the package I just mention consists out of 10 files and my package consists out of 66 files.
So it's smaller than my package.. but then it should be smaller in the unpacked size too.
In case you have any idea how to reduce the package size.. feel free to contribute ^^ or to give me some advice
https://github.com/agile-ts/agile/issues/106
Thank you ;D
What should matter is NOT how much the package contains on disk, but how much it takes space in a final application after all the bundling and minification is applied. This process includes variable names renaming, removal of comments, tree shaking and removal of unused/unreferenced code. There are tools which allow to inspect final application size and size of dependencies. They are different depending on what bundler you use.
It is very bad idea to remove comments from source code for the purpose of minifying your package download size. As well as to remove other development supportive files like Typescript definitions, etc.
Related
As a developer working with several languages, I notice that in most modern languages, dependencies metadata files can change a lot.
For instance, in NodeJS (which in my opinion is the worst when it comes to package management), a change of dependencies or in the version of NPM (respectively yarn) version can lead to huge changes in package-lock.json (respectively yarn.lock), sometimes with tens of thousands of modified lines.
In Golang for instance, this would be go.sum which can have important changes (in smaller magnitude when compared to Node of course) when modifying dependencies or running go mod tidy at times.
Would it be more efficient to track these dependencies files with git-lfs? Is there a reason not to do it?
Even if they are text files, I know that it is advised to push SVG files with git-lfs, because they are mostly generated files and their diff has no reason to be small when regenerating them after a change.
Are there studies about what language and what size/age of a project that makes git-lfs become profitable?
Would it be more efficient to track these dependencies files with git-lfs?
Git does a pretty good job at compressing text files, so initially you probably wouldn't see much gains. If the file gets heavily modified often, then over time the total cloneable repo size would increase by less if you use Git LFS, but it may not be worth it compared to the total repo size increases, which could make the size savings negligible as a percentage. The primary use case for LFS is largish binary files that change often.
Is there a reason not to do it?
If you aren't already using Git LFS, I wouldn't recommend starting for this reason. Also, AFAIK there isn't native built in support for diffing versions of files stored in LFS, though workarounds exist. If you often find yourself diffing the files you are considering moving into LFS, the nominal storage size gain may not be worth it.
I made a custom extension task in VS Code. And, I'm facing a problem while uploading the extension package to Marketplace. It's showing an error: "The extension package size '32440237 bytes' exceeds the maximum package size '26214400 bytes" as my extension size is ~32MB.
When I looked deep, then I got like node_modules folder (where all packages are present) size is increasing if I install some external packages like:
azure-pipelines-task-lib
azure-pipelines-tasks-azure-arm-rest-v2
#pulumi/azure
I also tried the solution which was given in this. But, no luck.
I'm worried, is there any way to decrease or compress the size of node_modules or extension-package.
or,
How the size is increasing for the node_modules folder?
if anyone has knowledge on this, please let me know.
Thanks in Advance!!
Well, I got the answer to this.
First thing, in VS Marketplace, the default size of uploading the extension package is 25MB.
So, if your extension package exceeds the max size limit, then don't worry, just do one thing:
Mail your issue to this mail id: vsmarketplace#microsoft.com. Then, one of from support team will contact you within 24hrs-48hrs.
Lastly, they will ask you the reason for extending the size limit, then you have to give the proper reason. Then, bingo!!
Your issue will resolve within 2-3 days max.
I have a project packaged using electron-builder with NSIS target, producing as artifacts a 40 MB .exe file and a .exe.blockmap file (which is gzipped JSON, I know). The problem is that even if something as simple as version number changes, blockmaps start differing vastly (for example only 1756 chunks of 2032 matched) and each update ends up downloading multiple megabytes.
I understand it may not be easy to make a detailed file-by-file map of an NSIS .exe containing app-64.7z containing app.asar finally containing files, but does electron-builder even try? Can it be overridden to use some binary-diff as basis for the block splitting, to ensure minimum differences between consecutive versions? I can't seem to find any documentation on app-builder.exe's routines for blockmap creation.
The largest part of the solution was introducing shortVersion and shortVersionWindows configuration strings in electron-updater v22.3.1 (20 Jan 2020), letting the 50 MB application not be re-stamped with full versioning at every single version update.
Also, space-padding all -bundle.js files (if present) helps to keep any app.asar file length fields to stay the same, again limiting the scope of changes in the final file.
When doing a fresh install of Haskell Stack through the install script from here:
wget -qO- https://get.haskellstack.org/ | sh
Followed by:
stack setup
you will end up with a $HOME/.stack/ directory of 1.5 GB size (from just a 120+ MB download). Further if you run:
stack update
the size increases to 2.5 GB.
I am used to Java which is usually considered large/big (covers pretty much everything and has deprecated alternatives for backwards compatibility), but as a comparison: an IDE including a JDK, a stand alone JDK, and the JDK source is probably around 1.5 GB in size.
On the other hand, that Haskell which is a "small beautiful" language (from what I have heard and read, this is probably referring mostly to the syntax and semantics, but still), is that large/big, seems strange to me.
Why is it so big (is it related to this question?)?
Is this size normal or have I installed something extra?
If there are several (4?, 5?) flavors of everything, then can I remove all but one?
Are some of the data cache/temporary that can be removed?
The largest directories are: .stack/programs/x86_64-linux/ghc-tinfo6-nopie-8.2.2/lib/ghc-8.2.2 (1.3 GB) and .stack/indices/Hackage (980 MB). I assume the first one are installed packages (and related to stack setup) and the latter is some index over the Hackage package archive (and related to stack update)? Can these be reduced (as above in 3 or grabbing needed Hackage information online)?
As you can probably see by inspection, it is a combination of:
three flavors (static, dynamic, and profiled) of the GHC runtime (about 400 megs total) and the core GHC libraries (another 700 megs total) plus 100 megs of interface files and another 200 megs of documentation and 120 megs of compressed source (1.5 gigs total, all under programs/x86_64-linux/ghc-8.2.2* or similar)
two identical copies of the uncompressed Hackage index 00-index.tar and 01-index.tar, each containing the .cabal file for every version of every package ever published in the Hackage database, each about 457 megs, plus a few other files to bring the total up to 1.0 gigs
The first of these is installed when you run stack setup; the second when you run stack update.
To answer your questions:
It's so big because clearly no one has made any effort to make it smaller, as evidenced by the whole 00-index.tar, 00-index.tar.gz, and 01-index.tar situation.
That's a normal size for a minimum install.
You can remove the profile versions (the *_p.a files) if you never want to compile a program with profiling. I haven't tested this extensively, but it seems to work. I guess this'll save you around 800 megs. You can also remove the static versions (all *.a files) if you only want to dynamically link programs (i.e., using ghc -dynamic). Again, I haven't tested this extensively, but it seems to work. Removing the dynamic versions would be very difficult -- you'd have to find a way to remove only those *.so files that GHC itself doesn't need, and anything you did remove would no longer be loadable in the interpreter.
Several things are cached and you can remove them. For example, you can remove 00-index.tar and 00-index.tar.gz (saving about half a gigabyte), and Stack seems to run fine. It'll recreate them the next time you run stack update, though. I don't think this is documented anywhere, so it'll be a lot of trial and error determining what can be safely removed.
I think this question has already been covered above.
A propos of nothing, the other day, I saw a good deal on some 3-terabyte drives, and in my excitement I ordered two before realizing I didn't really have anything to put on them. It kind of puts a few gigabytes in perspective, doesn't it?
I guess I wouldn't expend a lot of effort trying to trim down your .stack directory, at least on a beefy desktop machine. If you're working on a laptop with a relatively small SSD, think about maybe putting your .stack directory on a filesystem that supports transparent compression (e.g., Btrfs), if you think it's likely to get out of hand.
Inside a particular quarantine is all of the stuff one needs to run an application (bin, share, lib, etc.). Ideally, the quarantine has no leaks, which means it's not relying on any code outside of itself on the system. A quarantine can be defined as a set of executables (and some environment settings needed to make them run).
I think it will be beneficial to separate the built packages enough such that upgrading to a newer version of the quarantine won't require rebuilding the whole thing. I'll be able to update just a few packages, and then the new quarantine can use some of old parts and some of the new parts.
One issue I'm wondering about is the environment variables I'll be setting up to use a particular quarantines.
Is there a hard limit on how big PATH can be? (either in number of characters, or in the number of directories it contains) Does path length affect performance?
There's a hard limit. It's something like 32MB.
Yes, you can get it long enough to affect performance easily. Number of entries is the primary limiting factor, followed by number of / characters (this should not show itself unless the path depth exceeds some outrageous number like 30).