I have a Node.js Express app that I'm hosting on Nodejitsu. I'm already using env switches in Express for dev and prod, but obviously I'm going to need to push to different dev and prod servers.
For Nodejitsu, pushing to these servers requires a simple jitsu deploy, which looks to my package.json for app and domain/subdomain information.
I'm trying to think of the best way to handle the various pushes to dev and prod (and/r perhaps testing, staging, etc. in the future) given the package.json restriction. Clearly I would like to avoid maintaining several git repos, each with its own slightly different package.json.
What kind of workflow would be ideal for handling this?
We are't using Jitsu, but what we do is keep a couple of different version of the confs around in a confs folder. When we go to deploy our scripts copy the appropriate conf/package.json file in to the root dir.
You may be interested in this: https://github.com/nodejitsu/jitsu/issues/134
As Małecki points out in that issue, this can be done with npm scripts. You can have a few script setup in the package.json file(s) that do the appropriate pre-deploy work.
Related
Two questions, all related to bundling. First server, then client.
Typically, I wrote my Express server code, using an app.js file to setup my express server, setup logging, routes, etc. I also ran tests against the functions associated with the routes.
When deploying my server code to AWS, I would push the code and a git hook would build the project by doing an npm install, run my task runner to bundle client files and to copy files to their dist directories.
For my client side code, I used to use browserfy, then moved to webpack to bundle all my client code to a single bundle or main.js.
I'm looking at using rollup now and appreciate its' ability for tree shaking.
As I'm now setting up a new project, I have a few questions/concerns:
Should I also bundle on the node/server-side? Is it more efficient for the server to run against an optimized, tree shaken, server bundle? Or once node interprets my unbundled app.js, it's pulled into memory along with all the require files and would run similar to the same bundled/tree shaken version.
I currently deploy code to an aws server via a git push. That push triggers a full build server side, moves client files to dist directories and starts the nginx and node servers. Should I instead run the build process locally and only push dist files to the server? Thinking about a CI/CD process, I'm thinking no... but it could happen... But is it worth it? Depends a lot on answer 1, and maybe some hybrid approach.
Maybe someone could outline their production deploy steps at a high level? I can also talk more on mine if interested.
I'm using a node package to run a web server (among other benefits) for my project. The catch is, my project is only loaded on the server if it's within a directory of the node package. In other words, my directory structure looks like this:
<npm_pkg>/
<npm_pkg_src>/
clients/
<my_project_name>/
<my_project_src>
I would like to be able to use standard deployment processes for my project (e.g. gcloud app deploy, Travis continuous deployment, etc.), but I need to run my project from within a subdirectory of the larger package. Is there an easy way to force a git clone <pkg> during a build step and deploy my project in the target subdirectory?
I'm pretty new to CI/CD, but I tried to search around for similar examples and couldn't find any. Note: the parent project is not owned by me and thus I can't just use submodules without forking it (and I have no intention to alter it in any way). I also strictly just want to be able to trigger deploys based on my actual project's repository, if possible, whereas submodules would involve maintaining two and committing features twice (from what I understand).
Any help is much appreciated.
Edit: I forgot to mention that as part of this configuration I also need to run my server script from the root of the parent package. IOW, my package.json's start script will look like "start": "cd ../.. && npm start". Just in case it's relevant.
This might be what you’re looking for: CI/CD with App Engine
Clone from the repo and deploy from the subdirectory it is located, and Cloud Source Repositories can automate the whole process for you
I would also suggest you keep services separate, this will make things clearer for you and others that will/might be working on the project with you
I have developed a node.js application and it is working fine locally.
Now, after that I deployed the application in Heroku I am getting some HTTP 404 errors in the browser console.
My understanding is that the reason for this issue is caused by the node package.json modules. In fact, the outer package.json is installed successfully but the node modules mentioned in the client folder is not installed.
Q: Any idea about what is necessary to modify (or to do) to make the client node modules as installed?
The image below depicts my folder structure.
Each time you push your code to Heroku, Heroku will look for the package.json file in the ROOT of your project. Heroku will then install those dependencies.
In your case, because you have multiple package.json files, Heroku isn't seeing the ones nested in sub-folders of your project.
The best option you have is to list all dependencies in your top-level package.json file. You could also create your own Heroku Buildpack to customize the Heroku deployment procedure, but I strongly recommend AGAINST doing this (it will be a nightmare to maintain over time).
Another option in your case (which might be good depending on how large your team is) would be to move your client-side code into it's own Git project, and have that deployed separately to a static hosting provider like Amazon S3, or something similar. This would then let you deploy your backend project directly to Heroku, while not having to worry about any front-end logic at all.
I have a repo which consists of several "micro-services" which I upload to AWS's Lambda. In addition I have a few shared libraries that I'd like to package up when sending to AWS.
Therefore my directory structure looks like:
/micro-service-1
/dist
package.json
index.js
/micro-service-2
/dist
package.json
index.js
/shared-component-1
/dist
package.json
component-name-1.js
/shared-component-2
/dist
package.json
component-name-2.js
The basic deployment leverages the handy node-lambda npm module but when I reference a local shared component with a statement like:
var sharedService = require('../../shared-component-1/dist/index');
This works just fine with the node-lambda run command but node-lambda deploy drops this local dependency. Probably makes sense because I'm going below the "root" directory in my dependency so I thought maybe I'd leverage gulp to make this work but I'm pretty darn new to it so I may be doing something dumb. My strategy was to:
Have gulp deploy depend on a local-deps task
the local-deps task would:
npm build --production to a directory
then pipe this directory over to the micro-service under the /local directory
clean up the install in the shared
I would then refer to all shared components like so:
var sharedService = require('local/component-name-1');
Hopefully this makes what I'm trying to achieve. Does this strategy make sense? Is there a simpler way I should be considering? Does anyone have any examples of anything like this in "gulp speak"?
I have an answer to this! :D
TL;DR - Use npm link to link create a symbolic link between your common component and the dependent component.
So, I have a a project with only two modules:
- main-module
- referenced-module
Each of these is a node module. If I cd into referenced-module and run npm link, then cd into main-module and npm link referenced-module, npm will 'install' my referenced-module into my main-module and store it in my node_modules folder. NOTE: When running the second npm link, the name of the project is the one you find in your package.json, not the name of the directory (see npm link documentation, previously linked).
Now, in my main-module all I need to do is var test = require('referenced-module') and I can use that to my hearts content. Be sure to module.exports your code from your referenced-module!
Now, when you zip up main-module to deploy it to AWS Lambda, the links are resolved and the real modules are put in their place! I've tested this and it works, though not with node-lambda yet, though I don't see why this should be a problem (unless it does something different with the package restores).
What's nice about this approach as well is that any changes I make to my referenced-module are automatically picked up by my main-module during development, so I don't have to run any gulp tasks or anything to sync them.
I find this is quite a nice, clean solution and I was able to get it working within a few minutes. If anything I've described above doesn't make any sense (as I've only just discovered this solution myself!), please leave a comment and I'll try and clarify for you.
UPDATE FEB 2016
Depending on your requirements and how large your application is, there may be an interesting alternative that solves this problem even more elegantly than using symlinking. Take a look at Serverless. It's quite a neat way of structuring serverless applications and includes useful features like being able to assign API Gateway endpoints that trigger the Lambda function you are writing. It even allows you to script CloudFormation configurations, so if you have other resources to deploy then you could do so here. Need a 'beta' or 'prod' stage? This can do it for you too. I've been using it for just over a week and while there is a bit of setup to do and things aren't always as clear as you'd like, it is quite flexible and the support community is good!
While using serverless we faced a similar issue, when having the need to share code between AWS Lambdas. Initially we used to duplication the code, across each microservice, but later as always it became difficult to manage.
Since the development done in Windows Environment, using symbolic links was not an option for us.
Then we came up with a solution to use a shared folder to keep the local dependencies and use a custom written gulp task to copy these dependencies across each of the microservice endpoints so that the dependency can be required similar to npm package.
One of the decisions we made is not to keep two places to define the dependencies for microservices, so we used the same package.json to define the local shared dependencies, where gulp task passes this file and copy the shared dependencies accordingly also installing the npm dependencies with a single command.
Later we made the code open source as npm modules serverless-dependency-install and gulp-dependency-install.
I have a project that consists of several nod.js backend applications. The apps are using the same modules (which a placed outside of each ap folder in shared location). The aps they are to be deployed on differnt environments (servers), some code is for test, some for debug as usual.
If I choosed a platform (for example PaaS nodejitsu) for one of my apps, how I'm supposed to send there only production code for one of my apps? I deployed on nodejitsu and it just sends the app folder and uses package.json to configure the app. But there are a bunch of code that is not need (tests) for example and some code is external. And what If I want to obstruct server code too? How this issues are supposed to be soleved?
For front-end applications has a tons of methods to be build for production. I understand that the requirements are different, but didn't find any infromation on best practices fo how correctly to prepare node.js back end application for deploy.
Read section "Keeping files out of your package" in the NPM Developer page. It states following
Use a .npmignorefile to keep stuff out of your package. If there's no .npmignore file, but there is a .gitignore file, then npm will ignore the stuff matched by the .gitignore file. If you want to include something that is excluded by your .gitignore file, you can create an empty .npmignore file to override it.
Add those test files in .gitignore
or make another branch for production in git and push the production branch.