I'm building a Cordova app where tight control over the build environment is part of the spec/business requirements.
I'd like to (a) figure out the version and binary file and url for EVERY dependency, and (b) make sure they NEVER update automatically.
I'd like to use this to set up a Docker image for a build, and/or a local mirror of all dependencies. If the Docker image can be regenerated from the mirror that is best; if the Docker image works fine without a network connection (including cordova platform rm/add...) that is a close second.
Unfortunately, the number of dependencies (including dependencies of dependencies etc) is massive, and worse, there are many completely different kinds of packages with different package management systems.
Some of the things I know about:
npm packages -> list in package.json, installed in node_modules/, downloaded from npmjs.com
cordova platforms -> list in config.xml, installed in platforms/ (from npm?)
cordova plugins -> list in config.xml, installed in plugins/ (from npm?)
gradle? from https://services.gradle.org/distributions/
bintray packages?
maven packages?
for Android - platform-tools, build-tools etc, downloaded using tools/bin/sdkmanager from ?
for iOS - Xcode and a bunch of other things?
How can I get a handle on all of these? Is this an already solved problem (and how)?
It seems like this would be something that everyone who cares about versioning but wants to use Javascript/Node-type tools would need to solve; however there seems to be an almost total lack of documentation on it.
Related
Coworker and I are trying to figure out why compilation times are different. We have the same exact Dell Laptop 7030 model, same SSD, same hard drive, same memory, specs. Our task manager process look similar.
Corporate orders the identical model computers.
We are downloading from Angular Git Repository, with config and package json, with same node size memory.
Question is, Initial build between our laptops is 2 min compared to 8 min.
When we edit a single word in same file, his only takes 5 seconds to recompile, my takes 20 seconds.
Only programs running on the identical computer is Angular command ng serve.
Does anyone have ideas to resolve this issue, and slow compile time?
Is there anything I can change in my work station to the compile speed similar?
Attemped the solutions for all coworkers, still slow
We have same Node.js versions
updated from Angular 8 to 10 in Company Project Git Repo
tried npm cache clear
attempted Uninstalling and Reinstalling Angular and Nodejs
ScanDisk from Windows does not show errors on SSD drive
compared package-lock.json with coworker, they are the exact same, compared in source control diff
turned on Windows Defender Exclusion on the Angular Git Folder
Resources:
Angular compilation slow
How to speed up the Angular build process
Update:
Just noticed my laptop really stalls on styles.scss file at 48%
As you described enough, All the things are the same, whether node or angular versions, Hardware models, software and configurations. You also ensured that build pipelines are the same.
The only thing which comes to mind is a few difference in dependencies, yes the version mismatch inside node_modules packages. Considering that when you dont specify exact versions like ^x.x.x (aka semantic versioning) includes everything greater than the particular version in the same major range for dependency to be installed by npm or yarn. There's a tool named npm semver to show this. In this way, You have to check the installed packages versions too, by opening each one and looking inside package.json file. Specially for those which you think affect the performance like sass loader. Performance for each release may be different.
If you are running exactly the same codebase.
I would with a high degree of confidence say that it is
node.js or different npm versions that is the problem
try running
node --version
npm --version
yarn --version
to see which versions of them you have installed.
for angular 9 I would suggest running node 12, which is the current LTS (Long Time Support) version.
until node 14 which is also out now, but it's LTS release is not until october 2020.
I find yarn to be magnitudes faster than npm, so if you want a speed boost try running that if you haven't tried it.
Also Try clearing the npm cache, as mentioned per comments.
npm cache clear
I have a VS 2015 solution which requires I have the newest version of nodejs installed externally, and added to the External Web Tools path. (web project using a packages.json and gulp file to retrieve css/js modules and compile ssas at build time)
I also have older solutions that uses older versions of modules, and worked with the default versions of npm(1.4.9) and node(0.10.31). Once I installed newer nodejs externally and added to External Web Tools path, npm now throws various errors. As I work through errors I tend to to end up at github issues where someone had opened an issue and the responses were "you're using too old a version of node/npm".
In an ideal world we'd work through the old projects to get them on newer versions but this has a larger impact than is feasible.
Is there a way to specify that some projects use the default node/npm while other projects use the newer? I.e. a way to make the External Web Tools paths a proj/sln level setting instead of being a global Visual Studio setting.
You can use something like nvm.
In your project root directory, you would setup a .nvmrc and you would just set the version number in there. For example my file would have v10.9.0, and then I would just go into the directory and from the command line type the following:
nvm use
You can also have nvm set your npm version as well. See this answer
You can use Node Version Manager(NVM) to solve your problem
NVM allow to run multiple node version in single machine, You can switch on any node verion using NVM
for more information you can refer
https://medium.com/appseed-io/how-to-run-multiple-versions-of-node-js-with-nvm-for-windows-ffbe5c7a2b47
We are behind a firewall.
We have a self-hosted central npm repository, that doesn't have all the packages (most, but not all).
It seems the grunt build process (to compile from source) relies on phantomjs to be built via npm. I have phantomjs 1.9.2 built from source and available, but the process specifies that it needs to be installed via npm to work.
My goal is to build so I can use the IDE in my environment. There is a .app, and a .msi available for Wndows and Mac, but for Linux I have to compile from source (we have a .deb, but I'm on CentOS).
Any help is appreciated.
It looks like you don't actually need PhantomJS for building Brackets -- I see it's a dependency in the package.json, but I don't actually see a mention in the Brackets source code where it may be called.
So could you just try building Brackets without having PhantomJS installed and see if it works?
I have a project that runs on Debian and uses many packages provided from the Debian repositories.
Because of demand, I've looked into porting the project to CentOS, but found that many of the packages I require are completely missing - at least 10 dependencies would have to be compiled manually at install time on the users machine.
My question is, what is the best way to create an installer for the user's machine? Should I use automake tools (with the standard ./configure, make, make install), to compile the required libraries, or is this a non-standard approach. Note that my app doesn't actually need to be compiled since it is written in Python, so is it weird to do a "make", when you're not compiling your own app?
Should the configure script just warn the user that package X is missing, and let them handle the rest?
Should I roll my own dependency checker by runng pkg-config manually a few times for each library required, and exit if something is missing?
I'm quite new to this, so any tips to get me moving in the right direction are appreciated.
Edit: I am familiar with RPM and yum for red hat base distros, but CentOS is missing many multimedia packages that I require. An example of one of my package dependencies is "liquidsoap" which is a programmable audio engine: http://savonet.sourceforge.net/
This is available on Debian, but not Redhat/Centos
See this link on CentOS package management.
http://wiki.centos.org/PackageManagement/Yum
CentOS is redhat based and does not use .deb packages by default. However apt package management has been ported to tons of platforms, you may be able to use a port for centOS
If you use YUM whatever packages you need will be there for your application as redhat distros need all the same things that any other distro does.
EDIT: To get the details out of comments
Packages not available on the target platform either have to be built (possibly as a port) on the target platform and then shipped in the ported package (in this case YUM), or code needs to be modified and forked to use packages which already are available on the target platform. The choice depends on which is worse, or which is even possible given your constraints.
I have installed Node.js on an Ubuntu 64bit server using a standard apt-get and would like to profile scripts through the "--prof" flag.
Web searching shows there should be a tool to process the v8.log output located in "deps/v8/tools/linux-tick-processor" but I don't seem to have any of those directories. Do they come with the native install? should they be installed separately? if so how?
Thank you
You need to download the source package with sudo apt-get source nodejs.The path you mentioned is in there.
You'll need to scons prof=on d8 in deps/v8 to build the debugger first, which might have some trouble on a 64-bit machine (v8 is 32-bit only), see here for more info.
Here's how I did it for Node.js 0.10.25 and 0.10.26:
I downloaded the source for Node.js that corresponds to the binaries I'm using. (I'm on Debian testing, which is a bit behind the releases from the Node.js web site.)
I checked the version of v8 bundled in the node sources. (Look at deps/v8/ChangeLog. It was 3.14.5 for Node.js 0.10.25 and 0.10.26.)
I downloaded this exact version of v8 from the v8 site.
Why? I tried running make native in Node.js deps/v8 directory but the Makefile was complaining about a missing test directory. From this we can infer that the Node developers are not including the entire v8 distribution. Once upon a time, with an earlier version of Node (0.8.something) I did build v8 from what was available in deps/v8 but this time I decided to use a different approach.
As explained in v8's build/README.txt, in the top level of the source tree for v8, I did:
$ svn co http://gyp.googlecode.com/svn/trunk build/gyp
(Linking my installed gyp to build/gyp as suggested in OrangeDog's answer did not work. That's why I did the above.)
I ran:
$ CXX=g++-4.7 make native
Why the CXX setting? I ran into a compilation problem right away when I tried with the default gcc. I checked the version. It was 4.8 and I remembered a story on Slashdot about how 4.8 was giving people trouble. So I installed 4.7. Worked fine.
I linked out/native/d8 to a location which is in my PATH. This is because the linux-tick-processor script does a poor job at finding d8. The simplest solution was to make it available in my PATH. Your mileage may vary.
After all this, linux-tick-processor can be used with the v8.log files that Node produces.
Either install the source package - sudo apt-get source nodejs, or switch to the official source as the ubuntu packages are very out of date.
To build d8, go to the deps/v8 directory.
Create a symlink at build/gyp to the directory where gyp can be found (e.g. /usr/bin).
Run make native.
Copy/symlink out/native/d8 to somewhere on your PATH.