I have been trying quite a few ways with no luck. I have a file named test.txt that has some lorem ipsum and the text [staging: production] I simply want to add a few lines that I have saved in a variable in before it.
If you could explain where I have gone wrong with any of the below it would be much appreciated!
#!/bin/bash
test="lala\
kjdsh"
sed '/^#$/{N; /[staging: production]/ i \
<Location /cgis> \
</Location>\
}' ./test.txt
sed -i -e 's/\[staging\: production\]/\$test/g' ./test.txt
#sed -i 's/Lorem/beautiful/g' test.txt
#awk -v data=$test '{A[NR]=$0}/\[staging\: production\]/{ print data }' test.txt > testfile.txt
#read -a text <<<$(cat test.txt)
#echo ${#text[#]}
#for i in ${text[#]};
#do
# echo -n $i;
# sleep .2;
#done
#ed -s test.txt <<< $'/\[staging\: production\]/s/lalalala/g\nw'
#awk -v data=$test '/\(/\[staging\: production\]\)/ { print data }' test.txt > testfile.txt
# && mv testfile.txt test.txt
#sed -i -e '/\(\[staging\: production\]\)/r/$test\1/g' test.txt
#sed "/\(\[staging\: production\]\)/s//$test\1/g" test.txt
sed -i -e 's/\[staging\: production\]/\$test/g' ./test.txt
won't work because inside singe quotes BASH will not expand \$test.
Therefore you don't need to escape the $.
If you want to substitute with the contents of the variable $test do:
sed -i -e 's/\[staging: production\]/'$test'/g' ./test.txt
You also do not need to escape :
To insert before your pattern works for me this way:
sed -i -e '/\[staging: production\]/ i '$test'' ./test.txt
However to preserve the linebreak inside the variable I needed to define:
test="lala\nkjdsh"
Please note the \n to encode the linebreak.
Try it in perl, it seems to work fine:
perl -pe '{$rep="what\nnow"; s/(\[foo foo2\])/$rep$1/}' file
This might work for you (GNU sed):
test="lala\\
kjdsh"
sed '/\[staging: production\]/i\'"$test" test.txt
N.B. \\ in the variable and the variable is surrouded by "'s in the sed command.
Related
I have two lists list1 and list2 with a filename on each line. I want a result with all filenames that are only in list2 and not in list1, regardless of specific file extensions (but not all). Using Linux bash, any commands that do not require any extra installations. In the example lists, I do know all file extensions that I wish to ignore. I made an attempt but it does not work at all, I don't know how to fix it. Apologies for my inexperience.
I wish to ignore the following extensions:
.x
.xy
.yx
.y
.jpg
list1.txt
text.x
example.xy
file.yx
data.y
edit
edit.jpg
list2.txt
text
rainbow.z
file
data.y
sunshine
edit.test.jpg
edit.random
result.txt
rainbow.z
sunshine
edit.test.jpg
edit.random
My try:
while read LINE
do
line2=$LINE
sed -i 's/\.x$//g' $LINE $line2
sed -i 's/\.xy$//g' $LINE $line2
sed -i 's/\.yx$//g' $LINE $line2
sed -i 's/\.y$//g' $LINE $line2
then sed -i -e '$line' result.txt;
fi
done < list2.txt
Edit: I forgot two requirements. The filenames can have . in them and not all filenames must have an extension. I know the extensions that must be ignored. I ammended the lists accordingly.
An awk solution might be more efficient for this task:
awk '
{ f=$0; sub(/\.(xy?|yx?|jpg)$/,"",f) }
NR==FNR { a[f]; next }
!(f in a)
' list1.txt list2.txt > result.txt
comm can do precisely this.
You can preprocess the input:
strip the suffices
sort (comm expects sorted input)
remove duplicates
ss()( sed 's/\.\(x\|xy\|yx\|y\|jpg\)$//' "$#" | sort -u )
comm -13 <(ss list1.txt) <(ss list2.txt) >result.txt
Your code was:
while read LINE
do
line2=$LINE
sed -i 's/\.x$//g' $LINE $line2
sed -i 's/\.xy$//g' $LINE $line2
sed -i 's/\.yx$//g' $LINE $line2
sed -i 's/\.y$//g' $LINE $line2
then sed -i -e '$line' result.txt;
fi
done < list2.txt
Some issues that immediately jump out:
syntax error - then/fi but no matching if
you never access list1
you don't quote variables when you use them, so whitespace and special characters will cause problems
while read ... sed ... sed ... sed ... is inefficient - multiple invocations of sed instead of just one, and a loop that sed would perform implicitly
sed expects file arguments not strings
sed -i will try to overwrite input file arguments
you use result.txt as both input and output to sed but never assign any contents to it
you try to use data ($line) as sed commands, instead of applying sed commands to that data
because you used single-quotes, sed -i -e '$line' will attempt to run a (non-existent) sed command line on the last line of input ($)
g option to s/// does nothing when search is anchored
I'd use join:
$ join -t. -j1 -v2 -o 2.1,2.2 <(sort list1.txt) <(sort list2.txt) | sed 's/\.$//'
rainbow.z
sunshine
(The bit of sed is needed to turn sunshine. into sunshine)
I've 95 files that looks like :
2019-10-29-18-00/dev/xx;512.00;0.4;/var/x/xx/xxx
2019-10-29-18-00/dev/xx;512.00;0.68;/xx
2019-10-29-18-00/dev/xx;512.00;1.84;/xx/xx/xx
2019-10-29-18-00/dev/xx;512.00;80.08;/opt/xx/x
2019-10-29-18-00/dev/xx;20480.00;83.44;/var/x/x
2019-10-29-18-00/dev/xx;3584.00;840.43;/var/xx/x
2019-10-30-00-00/dev/xx;2048.00;411.59;/
2019-10-30-00-00/dev/xx;7168.00;6168.09;/usr
2019-10-30-00-00/dev/xx;3072.00;1036.1;/var
2019-10-30-00-00/dev/xx;5120.00;348.72;/tmp
2019-10-30-00-00/dev/xx;20480.00;2033.19;/home
2019-10-30-12-00;/dev/xx;5120.00;348.72;/tmp
2019-10-30-12-00;/dev/hd1;20480.00;2037.62;/home
2019-10-30-12-00;/dev/xx;512.00;0.43;/xx
2019-10-30-12-00;/dev/xx;3584.00;794.39;/xx
2019-10-30-12-00;/dev/xx;512.00;0.4;/var/xx/xx/xx
2019-10-30-12-00;/dev/xx;512.00;0.68;/xx
2019-10-30-12-00;/dev/xx;512.00;1.84;/var/xx/xx
2019-10-30-12-00;/dev/xx;512.00;80.08;/opt/xx/x
2019-10-30-12-00;/dev/xx;20480.00;83.44;/var/xx/xx
2019-10-30-12-00;/dev/x;3584.00;840.43;/var/xx/xx
For some lines I've 2019-10-29-18-00/dev and for some other lines, I've 2019-10-30-12-00;/dev/
I want to add the ; before the /dev/ where it is missing, so for that I use this sed command :
sed 's/\/dev/\;\/dev/'
But How I can apply this command for each lines where the ; is missing ? I try this :
for i in $(cat /home/xxx/xxx/xxx/*.txt | grep -e "00/dev/")
do
sed 's/\/dev/\;\/dev/' $i > $i
done
But it doesn't work... Can you help me ?
Could you please try following with GNU awkif you are ok with it.
awk -i inplace '/00\/dev\//{gsub(/00\/dev\//,"/00;/dev/")} 1' *.txt
sed solution: Tested with GNU sed for few files and it worked fine.
sed -i.bak '/00\/dev/s/00\/dev/00\;\/dev/g' *.txt
This might work for you (GNU sed & parallel):
parallel -q sed -i 's#;*/dev#;/dev#' ::: *.txt
or if you prefer:
sed -i 's#;*/dev#;/dev#' *.txt
Ignore lines with ;/dev.
sed '/;\/dev/{p;d}; s^/dev^;/dev^'
The /;\/dev/ check if the line has ;/dev. If it has ;/dev do: p - print the current line and d - start from the beginning.
You can use any character with s command in sed. Also, there is no need in escaping \;, just ;.
How I can apply this command for each lines where the ; is missing ? I try this
Don't edit the same file redirecting to the same file $i > $i. Think about it. How can you re-write and read from the same file at the same time? You can't, the resulting file will be in most cases empty, as the > $i will "execute" first making the file empty, then sed $i will start running and it will read an empty file. Use a temporary file sed ... "$i" > temp.txt; mv temp.txt "$i" or use gnu extension -i sed option to edit in place.
What you want to do really is:
grep -l '00/dev/' /home/xxx/xxx/xxx/*.txt |
xargs -n1 sed -i '/;\/dev/{p;d}; s^/dev^;/dev^'
grep -l prints list of files that match the pattern, then xargs for each single one -n1 of the files executes sed which -i edits files in place.
grep for filtering can be eliminated in your case, we can accomplish the task with a single sed command:
for f in $(cat /home/xxx/xxx/xxx/*.txt)
do
[[ -f "$f" ]] && sed -Ei '/00\/dev/ s/([^;])(\/dev)/\1;\2/' "$f"
done
The easiest way would be to adjust your regex so that it's looking a bit wider than '/dev/', e.g.
sed -i -E 's|([0-9])/dev|\1;/dev|'
(note that I'm taking advantage of sed's flexible approach to delimiters on substitute. Also, -E changes the group syntax)
Alternatively, sed lets you filter which lines it handles:
sed -i '/[0-9]\/dev/ s/\/dev/;/dev/'
This uses the same substitution you already have but only applied on lines that match the filter regex
Hi i have a script which replaces the certain occurrences in the .sql files and after that writes it in the some new file.So,i m here unnecessarily creating extra files.Is there anyway by which i can write in the same file.
Below is the part of the script:
sed "s/v1/$value1/g" Save.sql >> CreateViewFinal1.sql
sed "s/v2/$value2/g" CreateViewFinal1.sql >> CreateViewFinal2.sql
sed "s/v3/$value3/g" CreateViewFinal2.sql >> CreateViewFinal3.sql
sed "s/v4/$value4/g" CreateViewFinal3.sql >> CreateViewFinal4.sql
sed "s/v5/$value5/g" CreateViewFinal4.sql >> CreateViewFinal5.sql
sed "s/v6/$value6/g" CreateViewFinal5.sql >> CreateViewFinal6.sql
sed "s/v7/$value7/g" CreateViewFinal6.sql >> CreateViewFinal7.sql
sed "s/v8/$value8/g" CreateViewFinal7.sql >> CreateViewFinal8.sql
sed "s/v9/$value9/g" CreateViewFinal8.sql >> CreateViewFinal9.sql
sed "s/a1/$value10/g" CreateViewFinal9.sql >> CreateViewFinal10.sql
sed "s/b1/$value11/g" CreateViewFinal10.sql >> CreateViewFinal11.sql
sed "s/c1/$value12/g" CreateViewFinal11.sql >> CreateViewFinal12.sql
sqlplus -S -L cimkroger/cimkroger#orcl #CreateViewFinal12.sql
Thanks in advance.
You can sed's inline editing and can avoid multiple sed commands with -e switch like this:
sed -i.bak -e "s/v1/$value1/g" -e "s/v2/$value2/g" -e "s/v3/$value3/g" Save.sql
Yes.
You can change a file directly with the -i option of sed.
Hence, sed -i .... file will replace something in the same file.
Moreover, instead of so many sed different lines, you can do multiple sed actions with the -e option. So instead of:
sed "s/v1/$value1/g" Save.sql >> CreateViewFinal1.sql
sed "s/v2/$value2/g" CreateViewFinal1.sql >> CreateViewFinal2.sql
sed "s/v3/$value3/g" CreateViewFinal2.sql >> CreateViewFinal3.sql
You can do
sed -i -e "s/v1/$value1/g" -e "s/v2/$value2/g" -e "s/v3/$value3/g" Save.sql
and so on.
Example
$ cat file
hello you
$ sed -i -e 's/hello/bye/g' -e 's/you/me/g' file
$ cat file
bye me
Here's what output looks like, basically:
? RESTRequestParamObj.cpp
? plugins/dupfields2/_DupFields.cpp
? plugins/dupfields2/_DupFields.h
I need to get the filenames from second column and pass them to rm. There's AWK script that goes like awk '{print $2}' but I was wondering if there's another solution.
If you have spaces between the ? and the filename then:
cut -c9-
If they're tabs then:
cut -f2
Placed your output in file
$> cat ./text
? RESTRequestParamObj.cpp
? plugins/dupfields2/_DupFields.cpp
? plugins/dupfields2/_DupFields.h
Edit it with sed
$> cat ./text | sed -r -e 's/(\?[\ \t]*)(.*)/\2/g'
RESTRequestParamObj.cpp
plugins/dupfields2/_DupFields.cpp
plugins/dupfields2/_DupFields.h
Sed in here is matching 2 parts of line -
? with tabs or spaces
Other characters until the end f the line
And then it changes whole line only with second part.
This might work for you:
echo "? RESTRequestParamObj.cpp" | sed -e 's/^\S\+/rm /' | sh
or using GNU sed
echo "? RESTRequestParamObj.cpp"| sed -r 's/^\S+/rm /e'
bash only solution, assuming your output comes from stdin:
while read line; do echo ${line##* }; done
use cut/perl instead
cut -f2 -t'\t'|xargs rm -rf
<your output>|perl -ne '#cols = split /\t/; print $cols[1]'|xargs rm -rf
how i can add Add text at the beginning of each line?
for example:- i have file contain:-
/var/lib/svn/repos/b1me/products/payone/generic/code/core
/var/lib/svn/repos/b1me/products/payone/generic/code/fees
/var/lib/svn/repos/b1me/products/payone/generic/code/2ds
i want it to become:-
svn+ssh://svn.xxx.com.jo/var/lib/svn/repos/b1me/products/payone/generic/code/core
svn+ssh://svn.xxx.com.jo/var/lib/svn/repos/b1me/products/payone/generic/code/fees
svn+ssh://svn.xxx.com.jo/var/lib/svn/repos/b1me/products/payone/generic/code/2ds
in other word i want to add "svn+ssh://svn.xxx.com.jo" at the beginning of each line of this file
One way to do this is to use awk.
awk '{ printf "svn+ssh://svn.xxx.com.jo"; print }' <filename>
If you want to modify the file in place, you can use sed with the -i switch.
sed -i -e 's_.*_svn+ssh://svn.xxx.com.jo&_' <filename>
Using sed:
printf "line1\nline2\n" | sed "s/^/new text /"
Using ex:
printf "line1\nline2\n" | ex -s +"%s/^/foo bar /e" +%p -cq! /dev/stdin
Using vim:
printf "line1\nline2\n" | vim - -es +"%s/^/foo bar /e" +%p -cq!
Using shell:
printf "line1\nline2\n" | while read line; do echo foo bar $line; done
ruby -pne 'sub(/^/,"svn+ssh://svn.xxx.com.jo")' file
Simple way:
sed -i 's_._svn+ssh://svn.xxx.com.jo_' <filename>
It can also be done with Perl:
perl -pe 's#^#svn+ssh://svn.xxx.com.jo#' input.file