I am using a for loop to connect to a list of servers and perform some simple commands. If the server is not accessible then stderr is written to a file. I then grep that file for the server name. It seems relatively simple and for some reason it isn't working. For troubleshooting purposes I have narrowed my server list to two servers and only run simple commands.
for i in $(cat serverlist)
do
nexec -i $i hostname 2>>errorlog.txt
if grep -q $i errorlog.txt; then echo "error accessing" $i
else echo "was able to connect to" $i
fi
done
So in the serverlist I have defined two incorrect hosts for troubleshooting purposes. Nexec tries to connect to each and perform the hostname command. If it is unable to connect an error message is printed to errorlog.txt
e.g.,
nexec: Error accessing host test1
Since both servers are incorrectly specified I am not able to connect to either. Again for troubleshooting purposes.
When grep runs the first time against $i which is the first server in the list it doen't find any matches in error.txt. However, it should. If I cat the results instead of grepping it is there.
I am actually doing this in bladelogic so the rules are a bit different. It should still work.
while read -r i <&3; do
nexec -i "$i" hostname 2>>"errorlog.$i.txt" || {
echo "nexec for $i exited with status $?" >&2
continue
}
# check for case where it claimed success but actually failed
# if nexec is written correctly, you don't need any of this logic
# ...and can completely remove the rest of the loop.
if grep -q -e "$i" "errorlog.$i.txt"; then
echo "error accessing $i" >&2
else
echo "was able to connect to $i" >&2
fi
done 3<serverlist
# and combine all the individual logs into one file:
cat errorlog.*.txt >errorlog.txt && rm -f -- errorlog.*.txt
Not familiar with nexec, but I imagine something like this is what you are looking for
for i in $(cat serverlist)
do
if [ ! "$(nexec -i $i hostname)" ]
then echo "error accessing" $i
else echo "was able to connect to" $i
fi
done
Related
I read various answers on similar topic, but I still can't deal with my problem. Namely, on the remote computer I have a .bashrc file with a bunch of custom made functions. I would like to check if that function exists in that file. Just to add that the script constantly reports that there is a specified function on the remote computer even though it is not. This is what I have done so far:
echo "Enter IP addres of the remote PC [def host#XX.XX.XX.XX]"
read ip
ip=${ip:-host#XX.XX.XX.XX}
$(ssh $ip "[ '$(type -t $1)' = function ]")
if [ $? -eq 0 ]; then
echo "function exist"
else
echo 'function doesnt exist'
fi
$(...)is expanded localy inside " quotes. Reseach difference between single and double quotes.
the_function_you_want_to_check=something
ssh "$ip" '[ "$(type -t "'$the_function_you_want_to_check'")" = function ]'
Do not use $?. Just:
if ssh stuff...; then
echo yes
else
echo no
fi
Thank you for your prompt response. Please note that $1 is actually the first parameter of the bash functions that I run on my local computer. Now, the change you suggested reports that there is no function on the remote computer even though it exists. More complete function that I run on the local machine is:
appendFunction_to_remotePC(){
echo "Enter the IP addres of the PC [def host#XX.XX.XX.XX]"
read ip
ip=${ip:-host#XX.XX.XX.XX}
if ssh "$ip" '[ "$(type -t "'$1'")" = function ]'; then
echo yes
else
echo no
fi
}
I call the function on the local computer in the usual way:
$ appendFunction_to_remotePC "test"
Please explain to me why the very last echo statement is blank? I expect that XCODE is incremented in the while loop to a value of 1:
#!/bin/bash
OUTPUT="name1 ip ip status" # normally output of another command with multi line output
if [ -z "$OUTPUT" ]
then
echo "Status WARN: No messages from SMcli"
exit $STATE_WARNING
else
echo "$OUTPUT"|while read NAME IP1 IP2 STATUS
do
if [ "$STATUS" != "Optimal" ]
then
echo "CRIT: $NAME - $STATUS"
echo $((++XCODE))
else
echo "OK: $NAME - $STATUS"
fi
done
fi
echo $XCODE
I've tried using the following statement instead of the ++XCODE method
XCODE=`expr $XCODE + 1`
and it too won't print outside of the while statement. I think I'm missing something about variable scope here, but the ol' man page isn't showing it to me.
Because you're piping into the while loop, a sub-shell is created to run the while loop.
Now this child process has its own copy of the environment and can't pass any
variables back to its parent (as in any unix process).
Therefore you'll need to restructure so that you're not piping into the loop.
Alternatively you could run in a function, for example, and echo the value you
want returned from the sub-process.
http://tldp.org/LDP/abs/html/subshells.html#SUBSHELL
The problem is that processes put together with a pipe are executed in subshells (and therefore have their own environment). Whatever happens within the while does not affect anything outside of the pipe.
Your specific example can be solved by rewriting the pipe to
while ... do ... done <<< "$OUTPUT"
or perhaps
while ... do ... done < <(echo "$OUTPUT")
This should work as well (because echo and while are in same subshell):
#!/bin/bash
cat /tmp/randomFile | (while read line
do
LINE="$LINE $line"
done && echo $LINE )
One more option:
#!/bin/bash
cat /some/file | while read line
do
var="abc"
echo $var | xsel -i -p # redirect stdin to the X primary selection
done
var=$(xsel -o -p) # redirect back to stdout
echo $var
EDIT:
Here, xsel is a requirement (install it).
Alternatively, you can use xclip:
xclip -i -selection clipboard
instead of
xsel -i -p
I got around this when I was making my own little du:
ls -l | sed '/total/d ; s/ */\t/g' | cut -f 5 |
( SUM=0; while read SIZE; do SUM=$(($SUM+$SIZE)); done; echo "$(($SUM/1024/1024/1024))GB" )
The point is that I make a subshell with ( ) containing my SUM variable and the while, but I pipe into the whole ( ) instead of into the while itself, which avoids the gotcha.
#!/bin/bash
OUTPUT="name1 ip ip status"
+export XCODE=0;
if [ -z "$OUTPUT" ]
----
echo "CRIT: $NAME - $STATUS"
- echo $((++XCODE))
+ export XCODE=$(( $XCODE + 1 ))
else
echo $XCODE
see if those changes help
Another option is to output the results into a file from the subshell and then read it in the parent shell. something like
#!/bin/bash
EXPORTFILE=/tmp/exportfile${RANDOM}
cat /tmp/randomFile | while read line
do
LINE="$LINE $line"
echo $LINE > $EXPORTFILE
done
LINE=$(cat $EXPORTFILE)
I'm using scp command to copy file from one Linux host to another.
I run scp commend on host1 and copy file from host1 to host2. File is quite big and it takes for some time to copy it.
On host2 file appears immediately as soon as copying was started. I can do everything with this file even if copying is still in progress.
Is there any reliable way to find out if copying was finished or not on host2?
Off the top of my head, you could do something like:
touch tinyfile
scp bigfile tinyfile user#host:
Then when tinyfile appears you know that the transfer of bigfile is complete.
As pointed out in the comments, this assumes that scp will copy the files one by one, in the order specified. If you don't trust it, you could do them one by one explicitly:
scp bigfile user#host:
scp tinyfile user#host:
The disadvantage of this approach is that you would potentially have to authenticate twice. If this were an issue you could use something like ssh-agent.
On sending side (host1) use script like this:
#!/bin/bash
echo 'starting transfer'
scp FILE USER#DST_SERVER:DST_PATH
OUT=$?
if [ $OUT = 0 ]; then
echo 'transfer successful'
touch successful
scp successful USER#DST_SERVER:DST_PATH
else
echo 'transfer faild'
fi
On receiving side (host2) make script like this:
#!/bin/bash
SLEEP_TIME=30
MAX_CNT=10
CNT=0
while [[ ! -e successful && $CNT < $MAX_CNT ]]; do
((CNT++))
sleep($SLEEP_TIME);
done;
if [[ -e successful ]]; then
echo 'successful'
rm successful
# do somethning with FILE
fi
With CNT and MAX_CNT you disable endless loop (in case file successful isn't transferred).
Product MAX_CNT and SLEEP_TIME should be equal or greater expected transfer time. In my example expected transfer time is less than 300 seconds.
A checksum (md5sum, sha256sum ,sha512sum) of the local and remote files would tell you if they're identical.
For the situation where you don't have SSH access to the remote system - like an FTP server - you can download the file after it's uploaded and compare the checksums. I do this for files I send from production scripts at work. Below is a snippet from the script in which I do this.
MD5SRC=$(md5sum $LOCALFILE | cut -c 1-32)
MD5TESTFILE=$(mktemp -p /ramdisk)
curl \
-o $MD5TESTFILE \
-sS \
-u $FTPUSER:$FTPPASS \
ftp://$FTPHOST/$REMOTEFILE
MD5DST=$(md5sum $MD5TESTFILE | cut -c 1-32)
if [ "$MD5SRC" == "$MD5DST" ]
then
echo "+Local and Remote files match!"
else
echo "-Local and Remote files don't match"
fi
if you use inotify-tools,
then the solution will looks like this:
while ! inotifywait -e close $(dirname ${bigfile_fullname}) 2>/dev/null | \
grep -Eo "CLOSE $(basename ${bigfile_fullname})$">/dev/null
do true
done
echo "File ${bigfile_fullname} closed"
After some investigation, and discussion of the problem on other forums I have found one more solution. Maybe it can help somebody.
There is a command "lsof". It lists open files. During copying the file will be opened, so the command
lsof | grep filename
will return non empty result.
So you might want to make a while loop to wait until lsof returns nothing and proceed with your task.
Example:
# provide your file name here
f=<nameOfYourFile>
lsofresult=`lsof | grep $f | wc -l`
while [ $lsofresult != 0 ]; do
echo still copying file $f...
sleep 5
lsofresult=`lsof | grep $f | wc -l`
done; echo copying file $f is finished: `ls $f`
For the duplicate question, How to check if file has been scp 100% to the remote location , which was for an expect script, to know if a file is transferred completely, we can add expect 100% .. .. i.e something like this ...
expect -c "
set timeout 1
spawn scp user#$REMOTE_IP:/tmp/my.file user#$HOST_IP:/home/.
expect yes/no { send yes\r ; exp_continue }
expect password: { send $SCP_PASSWORD\r }
expect 100%
sleep 1
exit
"
if [ -f "/home/my.file" ]; then
echo "Success"
fi
If avoiding a second SSH handshake is important, you can use something like the following:
ssh host cat \> bigfile \&\& touch complete < bigfile
Then wait for the "complete" file to get created on the remote end.
I have a file named "host.txt" with listing IP addresses of two systems.
~] cat hosts.txt
10.1.1.10
10.1.1.20
Using below script I am trying to login to each system, check status of a service and print the output of each system. The script prompts to login, however does not continue to execute the /opt/agent.sh status command. Can someone please help fix this script?
#!/bin/bash
for HOST in `cat hosts.txt`
do
ssh root#$HOST
STATUS=`/opt/agent.sh status | awk 'NR==1{print $3 $4}'`
echo $STATUS
if [ $STATUS! == "isrunning" ]; then
echo "$host == FAIL"
else
echo "$host == PASS"
fi
You script does not continue until the ssh command completes, which does not happen the interactive shell on $HOST that you started with ssh exits. Instead, you want to execute a script on $HOST.
(Also, note the correct way to iterate over the contents of hosts.txt.)
#!/bin/bash
while read HOST; do
do
if ssh root#$HOST '
STATUS=`/opt/agent.sh status | awk 'NR==1{print $3 $4}'`
[ "$STATUS" = "isrunning" ]
'; then
echo "$HOST == FAIL"
else
echo "$HOST == PASS"
fi
done < hosts.txt
The remote script simply exits with the result of comparing $STATUS to "isrunning". An if statement on the local host outputs a string based on the that result (which is the result of the ssh command itself). This saves the trouble of having to pass the value of $HOST to the remote host, simplifying the quoting required for the remote script.
I want to create a bash file (.sh) which does the following:
I call the script like ./download.sh www.blabla.com/bla.jpg
the script has to echo then if the file has downloaded or not...
How can I do this? I know I can use errorlevel but I'm new to linux so...
Thanks in advance!
Typically applications in Linux will set the value of the environment variable $? on failure. You can examine this return code and see if it gets you any error for wget.
#!/bin/bash
wget $1 2>/dev/null
export RC=$?
if [ "$RC" = "0" ]; then
echo $1 OK
else
echo $1 FAILED
fi
You could name this script download.sh. Change the permissions to 755 with chmod 755. Call it with the name of the file you wish to download. ./download.sh www.google.com
You could try something like:
#!/bin/sh
[ -n $1 ] || {
echo "Usage: $0 [url to file to get]" >&2
exit 1
}
wget $1
[ $? ] && {
echo "Could not download $1" | mail -s "Uh Oh" you#yourdomain.com
echo "Aww snap ..." >&2
exit 1
}
# If we're here, it downloaded successfully, and will exit with a normal status
When making a script that will (likely) be called by other scripts, it is important to do the following:
Ensure argument sanity
Send e-mail, write to a log, or do something else so someone knows what went wrong
The >&2 simply redirects the output of error messages to stderror, which allows a calling script to do something like this:
foo-downloader >/dev/null 2>/some/log/file.txt
Since it is a short wrapper, no reason to forsake a bit of sanity :)
This also allows you to selectively direct the output of wget to /dev/null, you might actually want to see it when testing, especially if you get an e-mail saying it failed :)
wget executes in non-interactive way. This means that wget work in the background and you can't catch de return code with $?.
One solution it's to handle the "--server-response" property, searching http 200 status code
Example:
wget --server-response -q -o wgetOut http://www.someurl.com
sleep 5
_wgetHttpCode=`cat wgetOut | gawk '/HTTP/{ print $2 }'`
if [ "$_wgetHttpCode" != "200" ]; then
echo "[Error] `cat wgetOut`"
fi
Note: wget need some time to finish his work, for that reason I put "sleep 5". This is not the best way to do but worked ok for test the solution.