Shell script to compare remote directories - linux

I have a shell script that I am using to compare directory contents. The script has to ssh to different servers to get a directory listing. When I run the script below, I am getting the contents of the server that I am logged into's /tmp directory listing and not that of the servers I am trying to ssh to. Could you please tell me what I am doing wrong?
The config file used in the script is as follows (called config.txt):
server1,server2,/tmp
The script is as follows
#!/bin/sh
CONFIGFILE="config.txt"
IFS=","
while read a b c
do
SERVER1=$a
SERVER2=$b
COMPDIR=$c
`ssh user#$SERVER1 'ls -l $COMPDIR'`| sed -n '1!p' >> server1.txt
`ssh user#$SERVER2 'ls -l $COMPDIR'`| sed -n '1!p' >> server2.txt
done < $CONFIGFILE
When I look at the outputs of server1.txt and server2.txt, they are both exactly the same - having the contents of /tmp of the server the script is running on (not server1 or 2). Doing the ssh +dir listing on command line works just fine. I am also getting the error "Pseudo-terminal will not be allocated because stdin is not a terminal". Adding the -t -t to the ssh command isnt helping either
Thank you

I have the back ticks in order to execute the command.
Backticks are not needed to execute a command - they are used to expand the standard output of the command into the command line. Certainly you don't want the output of your ssh commands to be interpreted as commands. Thus, it should work fine without the backticks:
ssh user#$SERVER1 "ls -l $COMPDIR" | sed -n '1!p' >>server1.txt
ssh user#$SERVER2 "ls -l $COMPDIR" | sed -n '1!p' >>server2.txt
(provided that double quotes to allow expansion of $COMPDIR are used).

first you need to generate keys to login to remote without keys
ssh-keygen -t rsa
ssh-copy-id -i ~/.ssh/id_rsa.pub remote-host
then try to ssh without pass
ssh remote-host
then try to invoke in your script but first make sanity check
var1=$(ssh remote-host) die "Cannot connect to remote host" unless $var1;

Related

Get file count at remote location during FTP in shell script on linux server

Requirement : Need to get file count based on wildcard entry present on remote location(Linux server) and store it in variable for validation purpose
Tried the below code
export ExpectedFileCount=$(ftp -inv $FTPSERVER >> $FTPLOGFILE <<END_SCRIPT
user $FTP_USER $FTP_PASSWORD
passive
cd $PATH
ls -ltr ${WILDCARD}*xml| wc -l | sed 's/ *//g'
quit
END_SCRIPT)
But the code is storing the code snippet in the variable and and executing the commands every time I call the variable.
Please suggest the changes in the script to execute the script once and store the value in the variable
This seems to work (on Ubuntu, no promises about portability):
export ExpectedFileCount=`ftp -in $FTPSERVER << END_SCRIPT | tee -a $FTPLOGFILE | egrep -c '\.xml$'
user $FTP_USER $FTP_PASSWORD
passive
cd $REMOTE_PATH
ls -l
quit
END_SCRIPT`
Issues:
$REMOTE_PATH used in place of $PATH for remote directory (as $PATH has a special meaning)
only a simple ls -l is performed inside the ftp session, and the output parsed locally, as it does not support arbitrary shell commands
I can't see how to capture the output of a command with a heredoc using $(...), but it seems to work with backticks if the closing backtick is after the final delimiter

Bash for loops on a remote server

I am attempting to run multiple commands via a bash script on a remote server. specifically, the for loop to be run on the remote server is giving me issues. I suspect it is because I don't know how to properly escape characters or use $().
Below is the code.
ssh (user)#(server) <<EOF
sudo su - (username)
whoami
'for e in $(`ls -lrt /usr/jboss/jbosseap | awk '{print $9}' | grep multichannel`);
do
echo "$e";
done'
Removing user and server names for obvious reasons. Just concentrate on the for loop. when I run that for loop command line (without the $()) its works fine. Just not sure how to nest it in a remote call.
Thanks very much for any and all help!
If you've got a complex script that you're trying to run over ssh you're going to be better off putting that script in a file and piping that file into ssh like:
cat remote_script.sh | ssh user#host
or:
cat remote_script.sh | ssh user#host sudo -u username
And now you don't have to worry about N levels of escaping.
You can run it as below .
here file "list " includes your list of nodes and script should be present in all nodes
for i in $(cat list ) ;do ssh -o StrictHostKeyChecking=no $i "/path/your_script" ;done

Remote SSH commands not working in Linux

Regardless of why, I am trying to write a script that will let me send a command to various addresses. There is a shared key for the user, so there is no need for logging in. But this isn't working.
So, the following will not work...
#!/bin/bash
ip=$1
shift
args="'$#'"
cmd="ssh user#$ip -C $args"
output=$($cmd)
If I execute it with the following:
./myscript.sh 10.0.1.2 /bin/ls -l /var
I get the error of "ls -l /var: No such file or directory"
If I run that command (ssh user#10.0.1.2 -C '/bin/ls -l /var'), it works fine.
What am I doing wrong? These are the same installs of RHEL6.
Apparently, the quotes were confusing bash. The following works...
ip=$1
shift
$(ssh -o ConnectTimeout=1 User#$ip "$#")

SSH find and replace string in a list of filenames

We have a repository of about 3000 MP3 files. Many of these files have our old domain name within their name.
For example: somesong_oldDomainName.mp3
I need to SSH into the site, find all instances of those files and rename them with the new domain name.
Example: somesong_NEWDomainName.mp3
I know the basic SSH commands but not something advanced like this.
Pretty sure it'll be a combination of multiple commands.
Assuming you get an interactive shell when you ssh into your linux server, this might be a possible way:
ssh user#machine-name-or-ip
then you will get some sort of terminal like
user#machine-name:~$
where you enter the commands to execute on that remote machine.
As mentioned in the comments, the answer here might just fit very well:
Bash: Rename small part of multiple files in middle of name
user#machine-name:~$ for i in *.mp3; do mv "$i" "$(echo "$i" | sed 's/_oldDomainName/_NEWDomainName/g')"; done
This assumes, your current directory is the one with all the MP3 files in it.
If you dont want interactivly operate on your files, e.g. because they change very often and you want a script to perform this action, SSH can also execute a command and/or shell script remotely.
To pass the command directly with the SSH call:
SSH error when executing a remote command: "stdin: is not a tty"
To pipe a local shell script into the SSH connection: How to use SSH to run a shell script on a remote machine?
Run a remote shell script via SSH: how to run a script file remotely using ssh
Edit:
Assume you are connected via SSH to your remote machine and have somewhat similar versions of bash and sed, it should work like this:
$ ls
bar_chosefil.mp3 boo_chosefil.mp3 foo_chosefil.mp3
$ for i in *.mp3; do mv $i $(echo $i | sed 's/chosefil/tamasha/g'); done
$ ls
bar_tamasha.mp3 boo_tamasha.mp3 foo_tamasha.mp3
Versions involved:
bash: 4.2.25
sed: 4.2.1
mv: 8.13
Edit 2:
Updated the command to work with blanks in filenames
$ ls
asd chosefil.mp3 bar_chosefil.mp3 boo_chosefil.mp3 foo_chosefil.mp3
$ for i in *.mp3; do mv "$i" "$(echo "$i" | sed 's/chosefil/tamasha/g')"; done
$ ls
asd tamasha.mp3 bar_tamasha.mp3 boo_tamasha.mp3 foo_tamasha.mp3

find files on remote machine

Hi i am trying to execute the following command on a remote machine how do i do this
ssh login.com find /nfs/repo/ -name ".user_repo.log" | \
xargs cat | awk '{$NF=""; print $0}' | \
sed "1i Owner RepoName CreatedDate" | column -t
I get the following error message
cat: /nfs/repo/new1/info/.user_repo.log: No such file or directory
cat: /nfs/repo/new2/info/.user_repo.log: No such file or directory
cat command is trying to find file on the local system while these files are present on the remote machine.How do i handle this
If you do:
ssh host command1 | command2
Then the shell will break at the pipe, so you'll get "ssh host command1" run as one command (i.e. remotely), and then "command2" run as another command (i.e., locally.) You can force all the commands to run remotely by enclosing in quotes:
ssh host "command1 | command2"
Note, since you already have quotes in the command, you might have to get creative with escaping.
Or, you might put all those commands in a short shell script and then just run the script:
ssh host myscript.sh

Resources