I want to pass the result of a command as a value of an argument of the next command. I know there is xargs and pipe | but those don't really help.
I want to run the command tail -f --pid=$REMOTE_PID logs where REMOTE_PID is the PID of a program which is running on a remote server. It writes digits from 1 to 30 in a log file and sleep(1). So I want to display simultaneously the digits coming from the log file on the local machine. All this is done in a script, not manually !!
Here is what I have done so far but can't get the correct PID. In the first command, I put the & to release the shell, so that I can run the next command
ssh user#host 'nohup sh showdigits.sh' &
ssh user#host 'PID=`pgrep -f showdigits.sh` && tail --pid=$PID -f logs'
These commands work but I get several PIDs before gitting the right one:
tail: cannot open '8087' for reading: No such file or directory
tail: cannot open '8109' for reading: No such file or directory
==> logs <==
1
2
3
...
I tried another code :
ssh user#host 'nohup sh showdigits.sh' &
ssh user#host "ps -ef | awk '/[s]howdigits.sh/{print $2}' > pid && tail --pid=`cat pid` -f logs"
I get this error :
cat: pid: No such file or directory
tail: : invalid PID
I want to have the only one PID of the script showdigits.sh and pass it to tail. Maybe is there a simpler solution ?
Thank you
Your strategy is:
Start a process
Disconnect and forget about it
Reconnect and try to find it
Follow it
You can simplify it by dropping step 2 and 3 entirely:
ssh user#host '
nohup sh showdigits.sh &
tail --pid=$! -f logs
'
(NB: using sh to run a script is a worst practice)
your grep is matching more than 1 result. first match is assigned to --pid= and others are interpreted as file names.
you have to pass it through head (or tail) before processing (depending on which pid you want)
PID=$(pgrep -f showdigits.sh | head -n1)
Related
How can I wait for a program to complete and then start another program automatically in another console?
EDIT: The first program can be long running and the other program should start right after the completion of the first program.
tail -f --pid=xyzu /dev/null && second_program
for example, in one console (terminal) invoke cat. In another console, use pstree -p | grep cat to find its process id. Then, in that other console, type tail -f --pid=6169 /dev/null && ls (replace the number with the correct one and use the needed command instead of ls). Then, end cat with CTL-C.
EDIT: Another example is, you want to shutdown the computer automatically when a long running program has completed:
first, find the pid of the long running program:
pstree -p | grep long_running_program
For instance, you find the pid 3373. Now, use the command:
sudo tail -f --pid=3373 /dev/null && shutdown -h now
The following command runs well when i run it localy on the dsired machine:
app_name=*some_proccess_name*
pid=`pgrep $app_name | tail -n 1`
But when i run it in the following way, from a remote pc using ssh it dosn't work:
pid=$(ssh $USER_NAME#$HOST_NAME "echo `pgrep $app_name | tail -n 1`")
The value of pid afterwards is just blank. I am not sure what's wrong (just to clarify i have tried several processes names that are all running on the target pc- that's not the problem ).
P.S when i run the command without echo, like that, i just get stuck inside the remote pc and have to use exit in order to get unstuck and return to my local pc:
pid=$(ssh tester#mir1 "`pgrep indicator-apple | tail -n 1`")
Less is more
pid=$(ssh tester#mir1 pgrep indicator-apple \| tail -n 1)
am trying to execute the following command:
$ ssh root#10.10.10.50 "tail -F -n 1 $(ls -t /var/log/alert_ARCDB.log | head -n1 )"
ls: cannot access /var/log/alert_ARCDB.log: No such file or directory
tail: cannot follow `-' by name
notice the error returned, when i login to ssh separately and then execute
tail -F -n 1 $(ls -t /var/log/alert_ARCDB.log | head -n1 )"
see the below:
# ls -t /var/log/alert_ARCDB.log | head -n1
/var/log/alert_ARCDB.log
why is that happening and how to fix it. am trying to do this in one line as i don't want to create a script file.
Thanks a lot
Shell parameter expansion happens before command execution.
Here's a simple example. If I type...
ls "$HOME"
...the shell replaces $HOME with the path to my home directory first, then runs something like ls /home/larsks. The ls command has no idea that the command line originally had $HOME.
If we look at your command...
$ ssh root#10.10.10.50 "tail -F -n 1 $(ls -t /var/log/alert_ARCDB.log | head -n1 )"
...we see that you're in exactly the same situation. The $(ls -t ...) expression is expanded before ssh is executed. In other words, that command is running your local system.
You can inhibit the shell expansion on your local system by using single quotes. For example, running:
echo '$HOME'
Will produce:
$HOME
So you can run:
ssh root#10.10.10.50 'tail -F -n 1 $(ls -t /var/log/alert_ARCDB.log | head -n1 )'
But there's another problem here. If /var/log/alert_ARCDB.log is a file, your command makes no sense: calling ls -t on a single file gets you nothing.
If alert-ARCDB.log is a directory, you have a different problem. The result of ls /some/directory is a list of filenames without any directory prefix. If I run something like:
ls -t /tmp
I will get output like
file1
file2
If I do this:
tail $(ls -t /tmp | head -1)
I end up with a command that looks like:
tail file1
And that will fail, because there is no file1 in my current directory.
One approach would be to pipe the commands you want to perform to ssh. One simple way to achieve that is to first create a function that will echo the commands you want executed :
remote_commands()
{
echo 'cd /var/log/alert_ARCDB.log'
echo 'tail -F -n 1 "$(ls -t | head -n1 )"'
}
The cd will allow you to use the relative path listed by ls. The single quotes make sure that everything will be sent as-is to the remote shell, with no local expansion occurring.
Then you can do
ssh root#10.10.10.50 bash < <(remote_commands)
This assumes alert_ARCDB.log is a directory (or else I am not sure why you would want to add head -n1 after that).
I am opening a ssh tunnel with ssh -f -N -L ...
There can be multiple instances opened like this at once.
I would like to add information to the ssh command so I can find it with ps piped to grep.
Is there a way to add data to the command so that it shows up on `ps aux
Wouldn't it be sufficient to find it by its PID? (In bash, PID can be simply returned like this:)
ssh -f -N -L ... & echo $!
I'm writing a script to read from a input file, which contains ~1000 lines of host info. The script ssh to each host, cd to the remote hosts log directory and cat the latest daily log file. Then I redirect the cat log file locally to do some pattern matching and statistics.
The simplified structure of my program is a while loop looks like this:
while read host
do
ssh -n name#$host "cd TO LOG DIR AND cat THE LATEST LOGFILE" | matchPattern
done << EOA
$(awk -F, '{print &7}' $FILEIN)
EOA
where matchPattern is a function to match pattern and do statistics.
Right now I got 2 questions for this:
1) How to find the latest daily log file remotely? The latest log file name matches xxxx2012-05-02.log and is newest created, is it possible to do ls remotely and find the file matching the xxxx2012-05-02.log file name?(I can do this locally but get jammed when appending it to ssh command) Another way I could come up with is to do
cat 'ls -t | head -1' or
cat $(ls -t | head -1)
However if I append this to ssh, it will list my local newest created file name, can we set this to a remote variable so that cat will find the correct file?
2) As there are nearly 1000 hosts, I'm wondering can I do this in parallel (like to do 20 ssh at a time and do the next 20 after the first 20 finishes), appending & to each ssh seems not suffice to accomplish it.
Any ideas would be greatly appreciated!
Follow up:
Hi everyone, I finally find a crappy way do solve the first problem by doing this:
ssh -n name#$host "cd $logDir; cat *$logName" | matchPattern
Where $logName is "today's date.log"(2012-05-02.log). The problem is that I can only use local variables within the double quotes. Since my log file ends with 2012-05-02.log, and there is no other files ends with this suffix, I just do a blindly cat *2012-05-02.log on remote machine and it will cat the desired file for me.
For your first question,
ssh -n name#$host 'cat $(ls -t /path/to/log/dir/*.log | head -n 1)'
should work. Note single quotes around the remote command.
For your second question, wrap all the ssh | matchPattern | analyse stuff into its own function, then iterate over it by
outstanding=0
while read host
do
sshMatchPatternStuff &
outstanding=$((outstanding + 1))
if [ $outstanding -ge 20 ] ; then
wait
outstanding=$((outstanding - 1))
fi
done << EOA
$(awk -F, '{print &7}' $FILEIN)
EOA
while [ $outstanding -gt 0 ] ; do
wait
outstanding=$((outstanding - 1))
done
(I assume you're using bash.)
It may be better to separate the ssh | matchPattern | analyse stuff into its own script, and then use a parallel variant of xargs to call it.
for your second question, take a look at parallel distributed shell:
http://sourceforge.net/projects/pdsh/
If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you can do this:
parallel -j0 --nonall --slf <(awk -F, '{print $7}' servers.txt) 'cd logdir; cat `ls -t | head -1` | grep pattern'
This way you get the matching done on the remote server. If you prefer to transfer the full log file and do the matching locally, simply move the grep outside:
parallel -j0 --nonall --slf <(awk -F, '{print $7}' servers.txt) 'cd logdir; cat `ls -t | head -1`' | grep pattern
You can install GNU Parallel simply by:
wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
chmod 755 parallel
cp parallel sem
Watch the intro videos for GNU Parallel to learn more:
https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1