How do I update a live Node.js Heroku App remotely? - node.js

I have a Node.js app that is live on Heroku
The Node.js folders/files that I uploaded on Heroku also reside on my computer
Whenever I update my Node.js folders/files on my computer, I want these updates to also be applied to the folders/files that are live on Heroku.
I want to be able to do that without having to stop, update and restart my Heroku app every time.
What I'm describing is basically a setup equivalent to that of the standard ftp connection that we all use whenever we make a local to remote update of static files of some standard website.
The git support that apparently Heroku offers doesn't do that. It requires for the app to be stopped (by running the appropriate commands on the terminal), then I need to make a git push (using the terminal) that updates all of the files (which takes forever) and not just the ones that need to be updated, and then the app needs to be restarted (again using the terminal). This is extremely frustrating for an app that is still in development, requires constant updates and cannot be tested locally (for a number of reasons).
Whenever a Node.js app is tested locally, the app can be started by calling supervisor app.js instead of node app.js.
What this does is it allows for the app to be updated and as soon as that happens (i.e. as soon as I hit "save") supervisor automatically restarts the app locally.
I'm looking for something similar to the above, i.e. linking my local app folder to my remote app folder and starting my remote app (on Heroku) using some supervisor mode so that as soon as my local folder is updated, my remote folder is also changed and the app automatically restarted.
It's extremely frustrating trying to test a Heroku app (that obviously needs constant updates) currently.
Testing it locally and then publishing it on Heroku (for good) will not do because some apps simply cannot be tested on localhost.
Any help would be much appreciated!

The git support that apparently Heroku offers doesn't do that. It
requires for the app to be stopped (by running the appropriate
commands on the terminal), then I need to make a git push (using the
terminal) that updates all of the files (which takes forever) and not
just the ones that need to be updated, and then the app needs to be
restarted (again using the terminal). This is extremely frustrating
for an app that is still in development, requires constant updates and
cannot be tested locally (for a number of reasons).
First, you don't need to stop your app before running git push heroku master. Just push, and the platform will build, and then restart your app with the new code, automatically. Second, git uses a diffing algorithm, so you aren't pushing all of the files - you're in fact just pushing the differences (assuming you're using git correctly). Third, you don't need to do that final, manual restart - the platform has already done this for you on push. Finally, I would advise that if your app is impossible to test locally, you might want to reconsider the architecture of that app. It sounds very un-portable. Perhaps refer to 12factor.net for a good architecture checklist.
Testing it locally and then publishing it on Heroku (for good) will not do because some apps simply cannot be tested on localhost.
What type of app are you building that would be impossible to test outside of a production environment?
In any case, the closest thing I'm aware of to what you're looking for is Dropbox Sync:
https://devcenter.heroku.com/articles/dropbox-sync

Related

How does Galaxy Meteor hosting for windows work?

I have a node.js application I have adopted from a more senior developer. I want to deploy it, and I know it will work because he already deployed it several times. I am reading these instructions:
https://galaxy-guide.meteor.com/deploy-quickstart.html
I use windows, as did he.
How does deployment work?
Take these instructions:
Windows If you are using Windows, the commands to deploy are slightly
different. You need to set the environment variable first, then run
the deployment command second (the syntax is the same as everything
you’d put for meteor deploy).
In the case of US East, the commands would be:
$ SET DEPLOY_HOSTNAME=galaxy.meteor.com
$ meteor deploy [hostname]
--settings path-to-settings.json
Am I just supposed to go to the source directory on my laptop and run these commands? What then happens? Is the source uploaded to their server from my laptop and then their magic takes care of the rest?
What about when I want to make a change to the code? Do I just do the same thing, poiting to an existing container and, again, they do the magic?
Am I just supposed to go to the source directory on my laptop and run these commands? What then happens? Is the source uploaded to their server from my laptop and then their magic takes care of the rest?
It is not magic. You basically go to your dev root and enter these commands. Under the hood it builds your app for production (including minification and prod flags for optimization) and once complete opens a connection to the aws infrastructure and pushes the build bundle.
See: https://github.com/meteor/meteor/blob/devel/tools/meteor-services/deploy.js
On the server there will be some install and post install scripts that set up all the environment for you and, if there are no errrors in the process, start your app.
These scripts have if course some automation, depending on your account settings and the commands you have entered.
What about when I want to make a change to the code? Do I just do the same thing, poiting to an existing container and, again, they do the magic?
You will have to rebuild (using the given deploy command) again but Galaxy will take care of the rest.

ReactJS: How to deploy on local server

I have a web app developed with a NodeJS + Express + GraphQL + MongoDB back-end and a ReactJS + Apollo front-end. I would like to deploy this application locally. Is that even possible?
I have come across dozens of "how to deploy to Heroku," "how to deploy to Digital Ocean", "how to deploy to Github", etc. But none that explains how to deploy locally.
Right now, I run: nodemon server for the back-end, and npm start for the front-end. I see the application running on http://localhost:3000/ (I use cors to connect the front end with the server running on port 3001).
I would like to just go to http://localhost:3000/ and see the app without having to execute the commands npm start and nodemon server. Is this possible? If so, how do I do that?
To my knowledge, our local server is not a WAMP server (our OS is Windows though). The IT department told me that it is a
[...] plain, regular old server. The address is localhost running on
port 3000. You can open up another port on 3001 if you need it. Just
drop your stuff on the C: drive and you should be good to go. I've
never heard of Node or React so I can't help if you have questions.
Any ideas? Many thanks in advance for your help!
UPDATE
There seems to be a bit of confusion surrounding what I am looking for. I am trying to deploy this locally.
Let's say, on your local computer (your laptop at home) you go to localhost:3000 on your favorite browser. Unless you are serving something to localhost in that moment nothing is going to show up, it will say "refused to connect" or something. What I want is to be able to open any machine on the network whenever I go to localhost:3000 and my react site appears and functions...does that make more sense?
I don't want this is development mode. I want a build of this project on localhost...I'm starting to think this isn't possible.
As i understood, you want to deploy it on a local server, not locally on your developing device.
I thought about doing that...but I'm not so sure IT will be okay with it always running... :(
How can you use a server if its not running? Just like WAMP (which runs apache), or whatever you got rolling there, it must be running. So, just make it a background process like slawomir suggested.
PS I dont think you understand node server properly though.
Read this to understand why node server needs reloading. After that you need to understand that no hot reload tool is perfect, and you gonna need to restart your server from time to time.
PPS I dont know what this means
[...] plain, regular old server. The address is localhost running on port 3000.
if there is a server running on 3000, youll need to change port for your server to smth else (most common is 9000)
To solve the problem you can create a startup script, which executes npm start and nodemon server. Then make sure to keep it hidden, so that your server will be always running. Keep in mind though, that any errors thrown will stop your server and unless you configure it, the server won't reload by itself.
I would try following:
build your app with the production environment variables set
get all files from dist folder and deploy them in your server
now access your app using localhost/
Maybe what you are looking for is something like ngrok which creates a socks tunnel to your localhost, effectivelly deploying from localhost, as I understand it, allowing you to access your localhost through a url like ldiuhv093.ngrok.io, or even a custom subdomain if you pay for a subscription fee.
If I have this wrong, someone please tell me!
To solve the problem first of need to create a batch file with .bat or .cmd extension and under that file add the following 2 command
nodemon server
npm start
Then follows the following steps to add it as a startup script for windows OS.
Create a shortcut to the batch file.
Once the shortcut has been created, right-click the file and select
Cut.
Press the Start button and type Run and press enter.
In the Run window, type shell:startup to open the Startup folder.
Once the Startup folder has been opened, click the Home tab at the
top of the folder and select Paste to paste the shortcut into the
folder.
Above steps are for example to create a batch file and add it as a startup script for Windows 8 and 10 users.
For better clarity or reference follows the following link.reference-link
There's no option to reload the server while keeping it running. You could, technically, have your 'main' file monitor another file for changes. This would be the file where you actually keep your sever program. Then, on changes, you discard your current logic and start executing that. That said, doing it that way would be very fragile and a very round-about way to do it. It also wouldn't fix your front-end for which you'd need a similar solution.
Instead, you could hook into your favorite editor's save event, and run those two console commands, so that every time you save, the server is automatically brought up. (Make sure to also clean up existing servers)
Run on Save for VSCode
save-commands for Atom
I know this post has been two years. But, I think the solution to your second desired outcome is to use concurrency. https://www.npmjs.com/package/concurrently.
This will allow you to do one NPM START to start two all three processes.
and to your first question, I think the solution is to add Electron to your app so you can package it to an executable application. When you start the app, your express server will start running in the background.
Most people probably don't understand why there is a need for this. Running on local server (computer) allows access to local file system and can even run SQL queries inside the proxy which would require IT involvement if hosted on outside server.
From what I have understand, that you want to deploy your app on local server that means you want to deploy it on the network that you are connected to.
Check ip from the command prompt
To deploy it locally,
Run: HOST=ip npm run start
It will get deploy on your local server. And everyone connected to the server can access the url
If this worked for you, kindly upvote
You need to do npm start There may be other ways of starting it but, all will result in the same. You can read this article on Freecodecamp on deploying on DigitalOcean. You can manipulate it to your localhost. Shouldn't be too different.FCC Tut on Deploying

Why won't Node `https.get` request work on `heroku local` but work when deployed?

I have a Node.js app that I wrote and successfully deployed to a Heroku app via Bash. From my computer running Ubuntu at home it runs fine locally too.
However, I have just cloned the repository to a Windows machine and while I've successfully managed to push updates to the Heroku remote, which work, whenever I run heroku local from Windows and try to access the local version (localhost:5000), that page serves an error and my command line returns TypeError: Request path contains unescaped characters and layer.js:95.5.
The app uses https.get and this seems to be the problem line, locally.
I'm reluctant to try adapting my code since it works fine on Heroku itself and works fine on my Ubuntu machine -- so can only assume that something needs configuring on my Windows machine.
Any idea what the problem might be?
tl;dr
Be careful which variable keys you choose for config/.env in Heroku. Windows has some reserved keys (like user and path) which will mean trying to set your own values against these keys will not work.
The detail
It turns out the issue is with my choice of variable names as used in my .env file.
When you have data that you don't want to commit to a repository (in my case, an authentication key and details for my account) you can add them to Heroku as "config vars" -- key=value pairs which Heroku keeps separate to your code so you can version and share your repository as needed and others can add their own details.
When running Heroku locally however using the heroku local command from the toolbelt, these variables need to come from somewhere else. Heroku's help recommends setting them up as key=value pairs in a file called .env (which you can then add to .gitignore to prevent accidental committing.
Unfortunately, in my case it was my choice of variable keys in .env that caused the problem. I had created dependencies on variables called user and path but these seem to be reserved on Windows and have a special purpose and therefore could not be overwritten by what was in my .env file. This is why what worked for me on my Ubuntu machine would not work directly on Windows.
You can see this in action by calling console.log(process.env.user, process.env.path) from Node on a Windows machine.
Now that I have changed the variable names to something non-reserved heroku local works fine. Calling the programme via node command will still not work as it is not set up to pick up the variables from .env as Heroku.
Hope this helps someone else.

Deployment race condition causing CDN to cache old or broken files

Our current deploy process goes something like this:
Use grunt to create production assets.
Create a datestamp and point files at our CDN (eg /scripts/20140324142354/app.min.js).
Sidenote: I've heard this process called "versioning" before but I'm not sure if it's the proper term.
Commit build to github.
Run git pull on the web servers to retrieve the new code from github.
This is a node.js site and we are using forever -w to watch for file changes and update the site accordingly.
We have a route setup in our app to serve the latest version of the app via /scripts/*/app.min.js.
The reason we version like this is because our CDN is set to cache JavaScript files indefinitely and this purposely creates a cache miss so that the code is updated on the CDN (and also in our users' browsers).
This works fine most of the time. But where it breaks down is if one of the servers lags a bit in checking out the new code.
Sometimes a client hits the page while a deploy is in progress and tries to retrieve the new JavaScript code from the CDN. The CDN tries to retrieve it but hits a server that isn't finished checking out the new code yet and caches an old or partially downloaded file causing all sorts of problems.
This problem is exacerbated by the fact that our CDN has many edge locations and so the problem isn't always immediately visible to us from our office. Some edge locations may have pulled down old/bad code while others may have pulled down new/good code.
Is there a better way to do these deployments that will avoid this issue?
As a general rule of thumb:
Don't do live upgrades. (unless the language supports it, but even then think twice)
Pulling code using git pull and then waiting for the app to notice changes to files sounds a lot like the 90's: uploading php files to an apache web server using ftp (or sftp if you are cool) and waiting for apache to notice that they were updated. It can't happen atomically, so of course there is a race condition. Some users WILL get a half built and broken site.
I recommend only upgrading your live and running application while no one is using it. Hopefully you have a pool of servers behind a load balancer of some sort, which will allow you to remove them one at a time and upgrade them.
This will mean that users will be able to use both the old and the new site at the same time depending on how and when they access it, but that is much better then not being able to access it at all.
Ideally you would be able to spin up copies of each of the web servers that you have running with the new version of the site. Check that the new version does work, and then atomically update the load balancer so that everyone gets bumped to the new site at the same time. And only once everything is verified to be working perfectly the old machines are shut down and decommissioned, or reused.
step 4 in your procedure should be:
git archive --remote $yourgithubrepo --prefix=$timestamp/ | tar -xf -
stop-server
ln -sf $timestamp current
start-server
your server would use the current directory (well, a symlink) at all times. no matter how long the deploy takes, your application is in a consistent state.
I'll go ahead and post our far-from-ideal monkey-patch that we're using right now.
We deploy once which may or may not go as planned, once we're sure the code is deployed on all the servers we do another build where the only thing that changes is the version number.
Then we deploy again server by server.
The race condition still exists but because the application code between the two versions is the same this masks the issue since no matter which server the CDN hits it gets the "latest" code.

How do I support multiple versions in Node.js

I have a web-site that needs to be up all the time. I also, of course, need to do new releases. Each page tends to be very long-lived, with lots of JavaScript doing AJAX calls to the server.
What I do is build a new WAR file and put it in Tomcat's webapps directory, which ends up looking like this:
20110701-7f077d 20110711-aa8db4 20110715-6f4a12
20110701-7f077d.war 20110711-aa8db4.war 20110715-6f4a12.war live
The war file is named after the date of its release and the first few characters of its GIT commit-id, just so I can keep track of everything. Tomcat automatically unpacks the war file into a directory of the same name. The live directly just contains a file giving the name of the "live" version.
This way, each user can continue using the version of the back-end that works with the version of the front-end that he has loaded into his browser. And obviously, version upgrade and reversion is painless.
Now, I'm switching to node.js and I want to do the same thing. I am reliably informed that node.js doesn't support independent applications in one instance. So, what to do?
The only thing I can thing of is to designate n slots (where n is some small number like 10 or 100), and each slot corresponds to a port (i.e., slot 1 is 8001 and so on), put Apache in front of several node.js instances, each representing a slot, and Apache would use mod_proxy or mod_redirect to proxy requests like '/slot01' to port 8081. "live" would point to the current slot.
This would be clumsy and error prone and require an otherwise useless Apache instance and most of all I cannot believe that node.js doesn't have a good solution to what seems like a near-universal problem.
You can use node-http-proxy and write some code to monitor your 'deployment directory' for new versions and when such versions are found you can start the corresponding script and proxy it under the directory name (to make myself clear if you find a new directory 'version-11-today' your parent node-http-proxy script could start the new script assigning it a port passed as a parameter and then proxy to the new app under the path '/version-11-today').
A similar solution could be done with nginx only in this case you could write a script to monitory the deployment directory and generate some new nginx configuration when new apps are found.
If you are afraid you might run out of ports I believe both node.js and nginx can run on and proxy unix sockets besides inet sockets.
An advantage of the above is that each app runs in its own process protecting the other apps from crashes and enabling individual app restarts.
A third solution if you are not afraid some bug will crash your app is to have a parent script that loads all the app versions in the same process and maps them under different paths depending on the directory they were found in. You can still restart your server without downtime such as in this example http://codegremlins.com/28/Graceful-restart-without-downtime

Resources