I just installed all necessary dependencies on a new AWS EC2 Linux 2 server and launch my long_running_script.py. This script essentially performs a few operations then sleeps for a few hours on a never ending loop.
When I launched the script initially, I saw the correct output and all was fine. I disconnected from the instance, and when I reconnected I expected to see the same output as before.
Instead I can't seem to see any script output or see it running after typing in the 'ps aux' command.
Did disconnecting from the instance somehow abort the script? If so, how can I make sure it stays running?
Appreciate your help.
Did disconnecting from the instance somehow abort the script? If so, how can I make sure it stays running?
Yes it did. There are many ways to solve this. You can launch it using tmux or screen. Lunching your program in these "shells" will keep it running after you log out.
There is also nohup and pm2 which could also be helpful.
Related
I don't have Sudo access, so currently i can't install 'Forever' https://www.npmjs.com/package/forever
Instead i am simply using 'Screen'.
I am running a node.js server, at a random point, the node server stops, and screen exits. I cannot seem to collect any error data on this. I seem to be completely unaware of why its happening and cannot think of a way to catch what is happening. It doesn't happen often (maybe 1 time per day). When i load putty back up and login to my Apache server through terminal, i type screen -x or screen -r and it tells me there are no screens attached. The node server process definitely stops because the app it runs stops working.
Obviously i can't post all the code here, there is tons of it. But everything appears to work wonderfully, except every now and then, something goes wrong and it closes the attached screen.
If there was a problem with the node server, i would expect a crash, and the attached screen would stay attached. There would be an error outputted to the terminal for me to see when i open it. But in this case, it totally closes the attached screen.
Does anybody know what kind of error can cause this?
On a side note, is there an alternative to 'Forever' that can be installed without Sudo access?
My node version wasn't correct which is why Forever wasn't installing. I didn't need SUDO after all. I am now using Forever and hopefully this will shed light on what is going on as i have a out.log file which should catch whatever the problem is. :-)
I am running some application in NodeJS that retrieves data from a database and serve at port 3000.
It is running fine most of the time but sometimes it just errors out (it could be too many connections or network issue or some sql injection etc.). This happens every 2-3 days randomly.
While I am figuring out root cause - what is the best way to have NodeJS always 'ON' - meaning if it stops running - some process kicks it and it starts again?
You can use forever as stated by the previous answer, but as I've worked with forever before, I'd suggest using the pm2 process manager. It has, on top of what you need, some more advanced functionalities and is well-documented.
Try this:
forever start -w <filename>
-w is for watch. it watches the file changes and restarts if any changes found.
Also, if the script is unexpectedly stopped, it will try to restart it.
forever list for details of the process.
you will see the PID, filename, log filename.
in the log file, you can see the log of your script.
Hope it helps. :)
A process has been started remotely through a SSH session. The output stream (text) is displayed OK thru SSH. I would like to display the results locally without interrupting the running process.
Is there a way to attach to a running process and 'piggyback' a stream?
A Linux-only solution is acceptable.
Thanks!
Use reptyr:
reptyr is a utility for taking an existing running program and
attaching it to a new terminal. Started a long-running process over
ssh, but have to leave and don't want to interrupt it? Just start a
screen, use reptyr to grab it, and then kill the ssh session and head
on home.
Or retty:
retty is a tiny tool that lets you attach processes running on other
terminals.
I am deploying my node.js program on a remote machine (Ubuntu 14.04) and the program terminates after ssh session even if I deploy it as background process.
node app.js &
I understand that using forever can solve this, which I have tried and it pretty much works. There is already a thread here that describes the good solutions to it, and there are many other threads all describes good tools and solutions for it.
But, I would like to understand why does the node.js process stops at
first place even if it runs as background process?
Since you are connecting through SSH all the processes belong to that session.
Unless specified by a command like
nohup
or no hang up for short, all the processes that belong to your session through SSH will die with the session.
It's like logging in with a user opening chrome and logging out. The chrome thread will be release once the owner was logged out.
I'm having a slightly weird, repeatable, but unexplainable problem with screen.
I'm using ansible/vagrant to build a consistent dev environment for my company, and as a slightly showy finishing touch it starts the dev server running in a screen session so the frontend devs don't need to bother logging in and manually starting the process, but backend devs can log in and take control.
However, one of the systems - despite being built from scratch - ends up with an immediately dead screen (it doesn't log anything to screenlog). Running the command manually works fine.
(the command being)
screen -L -d -m bash -c /home/vagrant/run_screen_server.sh
I've even gone to the point of nuking everything vagrant/virtualbox related on the system, making sure it's installing a clean, nightly box. Exactly the same source box works all the other machines.
Are there any other debugging steps I can be taking or is there something I'm missing?
I'm right now trying to do the same with my setup and hit the same problem.
Further testing has shown, that sleep 1 right after calling the screen helped. It seems the ssh script that ansible calls exits before the screen call is fully detached (or something else, that would explain that the sleep 1 helps)
I've also found Can't get Fabric's detached screen session example to work with the same suggestion.