I'm working on optimizing some of my workflow and was wondering if anyone has run into a similar issue as mine. I've been struggling to figure out how to start multiple "nohup" shell scripts at the same time. For example, I have several scripts that look like this:
start.sh
rm nohup.out
nohup python -u script.py args
I've tried running them with a script like this:
start_option_1.sh
process_directory_1/start.sh & process_directory_2/start.sh ... (3-5 more of these)
And like this:
start_option_2.sh
process_directory_1/start.sh && process_directory_2/start.sh ... (3-5 more of these)
but no dice... the scripts won't even start. Any ideas/help would be greatly appreciated!! Using python3.6 if that's important too (but seems like it's more of a nohup issue).
There is a big difference between using '&' and using '&&'. The first will run each of the scripts in the background. The second will executed them in sequence, as long as each scripts will return success ('exit 0', or equivalent).
From the context of 'start.sh', looks like you want the first option (start all scripts together). Each script is executing a python program 'script.py'. The post did not specify if there is one script at the initial working directory, or if there are multiple 'script.py', one in each folder. Probably the second option.
If that case, you want to launch you scripts from the process_directory_* folder. Consider making a change to:
( cd process_directory_1 && exec ./start.sh) &
( cd process_directory_2 && exec ./start.sh) &
...
Notes:
All scripts are launched at the same time.
Each script is executed in a different folder, to access the script.py in that folder.
Each job will leave log in the run folder 'nohup.log'.
Related
I have this crontab #reboot "/home/pi/Desktop/TV Scraper 2.0/run.sh" set up and for whatever reason it doesn't seem to run the bash file on reboot.
Typing "/home/pi/Desktop/TV Scraper 2.0/run.sh" on the terminal actually runs the script, so I know it's correct.
This is what's inside run.sh just in case:
#!/bin/bash
cd "/home/pi/Desktop/TV Scraper 2.0"
node ./app.js
I've also tried using #reboot root sh "/home/pi/Desktop/TV Scraper 2.0/run.sh" as well, but it doesn't work either.
How can I move forward with this? My knowledge of Linux is very limited. All I need is to have some Node and Python3 scripts run on every reboot. On Windows that's such an easy task: I've tried CRON, rc.local and autostart, nothing works.
My guess is that node is not available via cronjob, since its containing directory is not in your PATH environment variable. When you execute the script manually, it's probably available via PATH.
An easy fix for this is to use the full path, which you can get by executing which node. The result should be something like /usr/bin/node. Then you can use that, instead of just node.
For debugging purpose you can also redirect stdout and stderr to a file, so the last line in your script would look like this:
/usr/bin/node ./app.js &>/tmp/cron-debug.log
If that doesn't fix it, i would rename the directory "TV Scraper 2.0" and replace the whitespace characters with something like an underscore. Directory and file names are less likely to cause problems if you avoid whitespaces.
I have some fairly time consuming python scripts to run ~3 hours or so on my machine. I don't want to run them concurrently since it might crash my machine. Alone I have more than enough memory but running 5 or so might cause an issue. I am running them remotely so I ssh into my server and run them like this:
nohup python my_script.py > my_output.txt &
That way if my connection gets interrupted I can re-establish the connection and my result is right there. I want to run the same python script a couple times with different command line arguments sequentially so I can run everything I need without me needing to set up the next one every few hours. I could manually code all of the arguments into a python script and do it that way but it seems inelegant. I don't want to have to fiddle with my python script every time I do this. Is there some sort of listener I could use to trigger the next one when one of them finishes?
I'd suggest writing a bash script that runs the python jobs sequentially:
#!/bin/bash
python3 my_script1.py > my_output1.txt
python3 my_script2.py > my_output2.txt
Then nohup that:
nohup ./driver.sh &
You really want to read up on utilities like tmux or screen and just script the while thing.
When beginning to work, I have to run several commands:
source work/tools
cd work/tool
source tool
setup_tool
Off course, doing this a few times a day is really annonying, so I tried to make a bash script tool where I put these commands and put it in /user/bin to run it with command
tool
However, there is a problem. When i run the script and then try to work by typing some of the tool-based commands, it does not work.
I figured out, that it is fine, since if I make a script and then run it, the script seems to run in the same terminal window, but what it really does is, that it behaves as if it created a "hidden window" for its execution and after termination of the script, the "hidden window" terminates too. So I am asking - is there a way to automatize the source command?
I have tried using xterm -hold -e command, but it runs the programmed script in the new window. Obviously, I don't want that. How can I achieve running it in the current window?
Don't put files like that in /usr/bin. As a general rule you don't want to mess with the distribution owned locations like that. You can use /usr/local/bin if you need a system-wide location or you can create a directory in your home directory to hold things like this that are for your own usage (and add that to the $PATH).
What you've noticed is that when run as a script on its own (tool, /path/to/tool, etc.) that the script runs in its own shell session (nothing to do with terminal windows as-such) and you don't want that (as the changes the script makes don't persist to your current shell session).
What you want to do instead is "source"/run the script in your current session. Which you are already doing with that set of commands you listed (source work/tools is doing exactly that).
So instead of running tool or /path/to/tool instead use source /path/to/tool or . /path/to/tool.
As fedorqui correctly points out you don't even need a script for this anywhere as you can just make a shell function for this instead (in your normal shell startup files .bashrc, etc.) and then just run that function when you need to so that setup.
Be careful to use full paths for things when you do this though since you, presumably, want this to work no matter what directory you happen to be in when you run it.
It doesn't create a new hidden window, nor does it create a terminal. What happens is that if you're running a script, normally it runs on a new shell process. The script you're running is supposed to modify the shell environment, but if you're running the script in a new shell process, that shell process's environment is the one that gets modified, instead of your shell environment.
Scripts that needs to modify the current shell environments usually must be run with the source command. What you need to do is to run the script in the current shell. So you should do source /path/to/tool.
If you want to be able to source the script with just tool, put this in your alias file/shell startup (check your distro doc where the file is, but it's usually either .bash_aliases or .bashrc):
alias tool="source /path/to/tool"
I am trying to create an alias that will execute a script. When i cd into the directory where the script is located... lets say /usr/local/bin/startscript then the script runs as expected and starts the application i want it to.
SO. i went into my bashrc file and added an alias
alias startscript='/usr/local/bin/startscript'
The goal is to be able to run the script by simply typing "startscript" from any directory.
However, when i try to use the alias to run the script, it does not work properly as the application that should start, does not.
My script starts with
#!/bin/sh
and then goes from there
any ideas? Thanks
SCRIPT:
#!/bin/sh
#- Check for user 'user'
if [[ "`whoami`" != "user" ]]; then
echo "This script can only be executed by user 'user'."; exit
fi
. /usr/local/bin/etctrx/startscriptdirectory/startscriptsetup
#- Kill manager to avoid multiple processes
pkill -f 'JavaApp.jar'
#- Start
nohup java -classpath /usr/local/bin/etctrx/startscriptdirectory/RequiredJars/ojdbc5.jar:/usr/local/bin/etctrx/startscriptdirectory/RequiredJars/activation.jar:/usr/local/bin/etctrx/startscriptdirectory/RequiredJars/mail.jar -jar /usr/local/bin/etctrx/startscriptdirectory/JavaApp.jar > ${JAVAAPPLOGS}/startscript.log 2>&1 &
If the script runs as expected while in /usr/local/bin by simply typing startscript, but from another directory the script runs (does not return an error), but doesn't produce the desired results, then the issue is with how you reference the application from within the script.
As others have noted, you shouldn't need an alias for something in /usr/local/bin and if it runs from that directory, obviously your executable permissions are correct too. If the application you're trying to run is also in /usr/local/bin then your script probably assumes it's in the same directory, which wouldn't be the case elsewhere, so you would need to either ad a cd to /usr/local/bin within the script or specify the full application path.
I am able to call the script if i do this, but it still won't give me the
results I want,(application being started) like i do when I run the script from
the directory it lives in
It would appear that the "application" in question is in the same directory as the script, /usr/local/bin, which we have established is already on your PATH. For the script to run correctly but not the application means you might be calling the application wrong, for example
./application
This would fail unless you were calling from /usr/local/bin. Fix would be like this
application
I wrote an init script to execute last that will start some pythjon script. The Python script will just run and never terminate and this makes my little linux box (getty terminal on tty) to just outpout the script but never the login prompt. I made the mistake to not assign a fix ip so i basically had to start over again (re-download the initial build onto the flash). However now I'm wondering what different possibilities I have, is it enough to launch the script in my init script with a & at the end or do I need a nohup/ What's the best way to resolve this?
Thank you!
Ron
I got this resolved by directing the output of the script to /dev/null and adding an ampersand at the end kinda like
myscript.sh > /dev/null &
this will return control to the shell and keep executing the script in the back without reporting the results to stdout.
Yes, you need to make sure you init startup script returns. It's good to follow the LSB documentation and have a script that excepts standard actions like start, stop, status and make it return proper return codes.
Launching the python script in the background with an & at the end of the command should be sufficient. You may also want to take a look at the command start-stop-daemon to start your process.