I use electron-builder as part of my build pipeline to build nodegit as a native dependency. It seems that every time I execute yarn install this dependency is compiled from scratch which takes way too long. Is there any way to use a cache for this?
$ electron-builder install-app-deps
• electron-builder version=22.9.1
• loaded configuration file=package.json ("build" field)
• rebuilding native dependencies dependencies=nodegit#0.27.0 platform=win32 arch=x64
• rebuilding native dependency name=nodegit version=0.27.0
Done in 150.42s.
I tried prebuild and compiled nodegit. I got out a nodegit-v0.27.0-node-v83-win32-x64.tar.gz but I don't really know where I can hook this in so this is used instead of compiling it over and over.
Afaik, I can bring S3 somehow into the game where I can host this file, but this is unnecessary overkill for my use case. I would be fine if I can "offer" this file somehow through my own in-house FTP or gitlab instance.
I'm trying to deploy Haskell application to Heroku using the following buildpack:
https://github.com/mfine/heroku-buildpack-stack
The build is going well until it hits the build time limit with the following error:
-----> Timed out running buildpack Haskell
! Push failed
Is there any way to increase this limit or to speed up the build?
Just split dependencies in half and make a push with only part of the dependencies enabled. Then make a push with all of them enabled. The latter build will use cached dependencies from the former build. If the project is super heavy on dependencies it would probably require more than two steps though.
I faced a similar issue with my angular project in a VPS with 1GB ram. I solved it by building the project in my local environment first and then uploading build files to the server.
I am interested in understanding how to install node-sqlite3, but provide by own precompiled package of sqlite3: I just want to install the Node client and skip the build phase entirely during install.
How can I do this?
(Reasoning: I am going to test the module in multiple environments and have already read countless posts of people having build issues in various environments, so I'd rather manually compile myself.)
It turns out that I was looking for a package like this one:
dblite on npm, GitHub
It happens occasionally that the development version of a module works in my development workspace and passes on Travis-CI but after publishing to npm it turns-out the end-user package is broken.
For example if you use a sub module that should be in dependencies but had it in devDependencies then CI will pass (but there are plenty other possible breakages).
How do you automate testing this? Do you use external rigging? Is there a secret module? Do you have a user acceptance test suite?
I use Github with Travis-CI but the standard setup uses the development install.
Once upon a time I discovered that npm would let me publish packages that are uninstallable. So I've added a target to my Gruntfile that does this:
Issue npm pack to create a package from my source.
Into a directory created (automatically by my Gruntfile) just for testing install the new package using npm install <path to the package created in the previous step>.
I have a target for publishing a new version that will publish only if the steps above are successful.
The steps above would not catch the dependency problem you mentioned in the question but they could easily be extended to catch it. To do this, I'd add one or more tests that cause the package installed in step 2 above to call require with all that it depends on.
I would suggest to set up your own CI server that does essentially one thing, npm install package ; cd node_modules/package ; npm test. This would ensure that your package is installable at least on your server.
I heard that Jenkins is good for this (at least, that's what node.js core team seems to be using), but don't have any first hand experience yet. We're just planning to set in up in a couple of weeks.
Also, having some external module that depends on you and testing it helps a bit. :)
I am new to Node.js programming and I have recently created a sample working web application using (express, backbone & other complimentary view technologies, with mongoDB). Now i am at a point where I want to deploy the same on a staging environment and I am not sure how to package this application and distribute the same. [I can take care of mongoDb and setting it up seperately]
I am from Java world and in there we create jars for reusable libs and war/ear packages for web applications which is deployed in a servlet container. Now in this case since node.js itself acts as a web container as well, how do i package my webapp?
Is there any standard format/guidelines of packaging node webapps built using express? (Is there a similar jar/war packaging systems for node apps?)
How do I deploy it once packaged? Would it become an exe, since it is also its own container?
PS: As of now I am thinking of just manually copying all the required source files into the staging environment and run npm commands to download all dependencies on that machine and then use 'forever' or some other mechanism to run my server.js. (Also, add some sort of monitoring, just in case app crashes and forever fails) I am not sure if that is the right way? I am sure there must be some standardized way of addressing this problem.
Deploying Node.js applications is very easy stuff. In maven, there is pom.xml. Related concept in Node.js is package.json. You can state your dependencies on package.json. You can also do environmental setup on package.json. For example, in dev environment you can say that
I want to run unit tests.
but in production;
I want to skip unit tests.
You have local repositories for maven under .m2 folder. In Node.js, there is node_modules folder under your Node.js project. You can see module folders with its name.
Let's come to the grunt part of this answer. Grunt is a task manager for your frontend assets, html, javascript, css. For example, before deployment you can minify html, css, javascript even images. You can also put grunt task run functions in package.json.
If you want to look at a sample application, you can find an example blog application here. Check folder structure and package.json for reference.
For deployment, I suggest you heroku deployment for startup applciations. You can find howto here. This is simple git based deployment.
On project running part, simply set your environment NODE_ENV=development and node app.js. Here app.js is in your project.
Here is relative concept for java and nodejs;
maven clean install => npm install
.m2 folder => node_modules(Under project folder)
mvn test => npm test(test section on package.json)
junit, powermock, ... => mocha, node-unit, ...
Spring MVC => Express.JS
pom.xml => package.json
import package => require('module_name')
There is no standardized way, but you're on the right track. If your package.json is up to date and well kept, you can just copy/zip/clone your app directory to the production system, excluding the node_modules.
On your production system, run
npm install to install your dependencies, npm test if you have tests and finally NODE_ENV=production node server.js
Some recent slides I considered to be quite helpful that also include the topic of wrappers like forever, can be found here.
Hope this might be helpful for somebody looking for the solution,Packaging of Node js apps can be done using "npm pack" command.It creates a zip file of your application which can be run in production/staging environment.
Is there any standard format/guidelines of packaging node webapps
built using express? (Is there a similar jar/war packaging systems for
node apps?)
Yes, the CommonJS Packages specification:
This specification describes the CommonJS package format for
distributing CommonJS programs and libraries. A CommonJS package is a
cohesive wrapping of a collection of modules, code and other assets
into a single form. It provides the basis for convenient delivery,
installation and management of CommonJS components.
For your next question:
2. How do I deploy it once packaged? Would it become an exe, since it is also its own container?
I second Hüseyin's suggestion to deploy on Heroku for production. For development and staging I use Node-Appliance with VirtualBox and Amazon EC2, respectively:
This program takes a Debian machine built by build-debian-cloud or
Debian-VirtualBox-Appliance and turns it into a Node.js "appliance",
capable of running a Node application deployed via git.
Your webapp will not become an exe.
few ways to approach this:
Push your code into Git repository, excluding everything that isn't your code (node_modules/**), then pull it in your staging environment, run npm install to restore all dependencies
create an NPM package out of it , install it via npm in your staging environment (this should also take care of all of the dependencies)
manual copy/ssh files to your staging environment (this can be automated with Grunt), than restore your dependencies via npm
I used zeit's pkg module. It can create cross platform deliverables for linux/win/macos. Actually used it in production and works fine without any issues.
It takes in all the js scripts and packages it into a single file.
The reason I used it is because it helps in securing your source code. That way in production at customers environment they will have access to application but not the source code.
Also one of the advantages is that at production environment, you do not actually need to have the customer install node.js as the node binaries also get packaged inside the build.
https://www.npmjs.com/package/pkg