how to sed replace whole line with the string as variable ?
#!/bin/bash
ssh $1 ssh-keyscan -t rsa $1 > /tmp/$1
RSA=$(cat /tmp/$1)
echo $RSA
sed -i 's:'^"$1".*':'"$RSA"':' /etc/ssh/ssh_known_hosts
cat /etc/ssh/ssh_known_hosts | grep $1
It is storing the variable in RSA but not replacing, not sure what's wrong with sed part.
You can use the following command.
#!/bin/bash
ssh $1 ssh-keyscan -t rsa $1 > /tmp/$1
RSA=$(cat /tmp/$1)
echo $RSA
sed -i -e "/$1/ d" -e "/^$1/ a $RSA" /etc/ssh/ssh_known_hosts
cat /etc/ssh/ssh_known_hosts | grep $1
I modified as per your requirement to add if not line exists and replace if it exists.
Related
Follow are known. Possible it helps:
Get the filename.extension incl. fullpath:
Script: /path1/path2/path3/path4/path5/bashfile.sh
#!/bin/bash
echo $0
read -r
Output:
/path1/path2/path3/path4/path5/bashfile.sh
Get filename.extension:
Script: /path/path/path/path/path/bashfile.sh
#!/bin/bash
echo ${0##*/}
read -r
Output:
bashfile.sh
Question:
How to get directory name and file name of a bash script by bash ?
Script: `/path1/path2/path3/path4/path5/bashfile.sh`
Wanted output:
/path5/bashfile.sh
Remark:
Perhaps its possible, if you look from right side, remove all left from "/*/"
Little bit shorter than the first fitting solution:
Script: /path1/path2/path3/path4/path5/bashfile.sh
#!/bin/bash
n=$(($(echo $0 | tr -dc "/" | wc -m)+1))
echo "/""$(echo "$0" | cut -d"/" -f$(($n-1)),$n)"
read -r
Output:
/path5/bashfile.sh
Perhaps they are a shorter solution.
readlink -f $0 |awk -F"/" '{print "/"$(NF-1)"/"$NF}'
# or
awk -F"/" '{print "/"$(NF-1)"/"$NF}' <(readlink -f $0)
# or
awk -F"/" '{print "/"$(NF-1)"/"$NF}' <<<$(readlink -f $0)
# or
sed -E 's/^(.*)(\/\w+\/\w+\.\w+$)/\2/g' <(readlink -f $0)
output
/path5/bashfile.sh
#!/bin/bash
echo "/$(basename "$(dirname "$0")")/$(basename "$0")"
echo
echo
read -r
Output:
/Dirname/Filname.Extension
I'm trying to write a script that builds a list of nodes then ssh into the first node of that list
and runs a checknodes.sh script which it's self is just a for i loop that calls checknode.sh
The first 2 lines seems to work ok, the list builds successfully, but then I get either get just the echo line of checknodes.sh to print out or an error saying cat: gpcnodes.txt: No such file or directory
MYSCRIPT.sh:
#gets the master node for the job
MASTERNODE=`qstat -t -u \* | grep $1 | awk '{print$8}' | cut -d'#' -f 2 | cut -d'.' -f 1 | sed -e 's/$/.com/' | head -n 1`
#builds list of nodes in job
ssh -qt $MASTERNODE "qstat -t -u \* | grep $1 | awk '{print$8}' | cut -d'#' -f 2 | cut -d'.' -f 1 | sed -e 's/$/.com/' > /users/issues/slow_job_starts/gpcnodes.txt"
ssh -qt $MASTERNODE cd /users/issues/slow_job_starts/
ssh -qt $MASTERNODE /users/issues/slow_job_starts/checknodes.sh
checknodes.sh
for i in `cat gpcnodes.txt `
do
echo "### $i ###"
ssh -qt $i /users/issues/slow_job_starts/checknode.sh
done
checknode.sh
str=`hostname`
cd /tmp
time perf record qhost >/dev/null 2>&1 | sed -e 's/^/${str}/'
perf report --pretty=raw | grep % | head -20 | grep -c kernel.kallsyms | sed -e "s/^/`hostname`:/"
When ssh -qt $MASTERNODE cd /users/issues/slow_job_starts/ is finished, the changed directory is lost.
With the backquotes replaced by $(..) (not an error here, but get used to it), the script would be something like
for i in $(cat /users/issues/slow_job_starts/gpcnodes.txt)
do
echo "### $i ###"
ssh -nqt $i /users/issues/slow_job_starts/checknode.sh
done
or better
while read -r i; do
echo "### $i ###"
ssh -nqt $i /users/issues/slow_job_starts/checknode.sh
done < /users/issues/slow_job_starts/gpcnodes.txt
Perhaps you would also like to change your last script (start with cd /users/issues/slow_job_starts)
You will find more problems, like sed -e 's/^/${str}/' (the ${str} inside single quotes won't be replaced by a host), but this should get you started.
EDIT:
I added option -n to the ssh call.
Redirects stdin from /dev/null (actually, prevents reading from stdin).
Without this option only one node is checked.
I declare three variables.
$1=`ssh <server_1> cat /etc/passswd|cut -f -d:|grep -e $IID -e $EID`
$2=`ssh <server_2> cat /etc/shadow|cut -f -d:|grep -e $IID -e $EID`
$3=`ssh <server_3> cat /etc/passwd}|cut -f -d:|grep -i $CID`
The above three variables are created by taking ssh to servers and checking the presence of the IDs which I give as input. If the ID doesn't exist already, the the variable is going to be null.
Now, how do I verify if all the three variables are null. I wanted to use the OR condition specified within an IF.
I tried,
if [ -s "$1" -o -s "$2" -o -s "$3"];then
echo -$1 $2 $3 "already exist(s)"
It didnt work. Please advise.
PS: I have just begun my career in Unix and correct me If am wrong anywhere.
Several points.
When you assign to a variable, don't use the dollar sign:
foo=xxx
Variables $1, $2 etc are already used for your command line arguments. Pick other names. But not $4please. :-)
When you specify a command for ssh, and it has arguments, it has to be quoted, because the command needs to be a single argument for ssh. In your case use double quotes, as you want variable expansion for $IID etc.
Most Unix utils are able to open input files themselves, so you don't need to start your pipeline with cat.
foo=`ssh <server_1> "cut -f -d: /etc/passwd | grep -e $IID -e $EID"`
Or something like that.
It was a typo in my question. I had actually declared it as,
1=`ssh <server_1> cat /etc/passswd|cut -f -d:|grep -e $IID -e $EID`
2=`ssh <server_2> cat /etc/shadow|cut -f -d:|grep -e $IID -e $EID` and so on.
And I tried it as ,
if [ -s "$1" -o -s "$2" -o -s "$3"];then
echo -e $1 $2 $3 "already exist(s)"
Since I had to Deliver my script today, I used the conventional method of,
ssh <server_1> "cat /etc/passswd|cut -f -d:|grep -e $IID -e $EID" > file1
ssh <server_2> "cat /etc/shadow|cut -f -d:|grep -e $IID -e $EID" > file2
ssh <server_3> "cat /etc/passwd|cut -f -d:|grep -ix $CID" > file3
if [ -s file1 -o -s file2 -o -s file3]; then
for i in `cat file1 file2 file3`
do
echo $i "already exists"
done
else
And I have now learnt from my first post, that -s to ensure that a file is not empty and -z is to ensure string is empty.
I am looking for a linux command that searches a string in a text file,
and highlights (colors) it on every occurence in the file, WITHOUT omitting text lines (like grep does).
I wrote this handy little script. It could probably be expanded to handle args better
#!/bin/bash
if [ "$1" == "" ]; then
echo "Usage: hl PATTERN [FILE]..."
elif [ "$2" == "" ]; then
grep -E --color "$1|$" /dev/stdin
else
grep -E --color "$1|$" $2
fi
it's useful for stuff like highlighting users running processes:
ps -ef | hl "alice|bob"
Try
tail -f yourfile.log | egrep --color 'DEBUG|'
where DEBUG is the text you want to highlight.
command | grep -iz -e "keyword1" -e "keyword2" (ignore -e switch if just searching for a single word, -i for ignore case, -z for treating as a single file)
Alternatively,while reading files
grep -iz -e "keyword1" -e "keyword2" 'filename'
OR
command | grep -A 99999 -B 99999 -i -e "keyword1" "keyword2" (ignore -e switch if just searching for a single word, -i for ignore case,-A and -B for no of lines before/after the keyword to be displayed)
Alternatively,while reading files
grep -A 99999 -B 99999 -i -e "keyword1" "keyword2" 'filename'
command ack with --passthru switch:
ack --passthru pattern path/to/file
I take it you meant "without omitting text lines" (instead of emitting)...
I know of no such command, but you can use a script such as this (this one is a simple solution that takes the filename (without spaces) as the first argument and the search string (also without spaces) as the second):
#!/usr/bin/env bash
ifs_store=$IFS;
IFS=$'\n';
for line in $(cat $1);
do if [ $(echo $line | grep -c $2) -eq 0 ]; then
echo $line;
else
echo $line | grep --color=always $2;
fi
done
IFS=$ifs_store
save as, for instance colorcat.sh, set permissions appropriately (to be able to execute it) and call it as
colorcat.sh filename searchstring
I had a requirement like this recently and hacked up a small program to do exactly this. Link
Usage: ./highlight test.txt '^foo' 'bar$'
Note that this is very rough, but could be made into a general tool with some polishing.
Using dwdiff, output differences with colors and line numbers.
echo "Hello world # $(date)" > file1.txt
echo "Hello world # $(date)" > file2.txt
dwdiff -c -C 0 -L file1.txt file2.txt
I have .csv file that contain 2 columns delimited with ,.
file.csv
word1,word2
word3,word4
word5,word6
.
.
.
.
word1000,1001
I want to create a new file from file.csv and insert sed command like this:
mynewfile
sed -e 's,word1,word2,gI' \
-e 's,word3,word4,gI' \
-e 's,word5,word6,gI' \
....
How can I make a script to add sed command?
You can use sed to process each line:
echo -n 'sed ' ; sed -e "s/^\(.*\)/-e 's,\1,gl'\ \\\/" file.csv
will produce as requested
sed -e 's,word1,word2,gl' \
-e 's,word3,word4,gl' \
-e 's,word5,word6,gl' \
Your goal seams to be performing custom replacements from a file. In this case, I would not generate a file containing a bash script to do the job, but I would generate a sed script to do the job:
sed -e 's/^/s,/' -e 's/$/,gI/' file.csv > sed_script
sed -f sed_script <<< "word1"
We can even avoid to use the sed_script file with bash's process substitution:
sed -f <(sed -e 's/^/s,/' -e 's/$/,gI/' file.csv) <<< "word1"
Update:
Simplifying the sed script generation, it becomes:
sed -e 's/.*/s,&,gI/' file.csv > sed_script
sed -f sed_script <<< "word1"
and
sed -f <(sed -e 's/.*/s,&,gI/' file.csv) <<< "word1"