I have one requirement.
I have one text file named as a.txt, which is having list of words -
GOOGLE
FACEBBOK
Now I have one another file named as b.txt , which is having content as
Company name is google.
Company name is facebook.
Like this n of lines are there with different different words.
Then I am writing script file -
FILENAME="a.txt"
SCHEMA=$(cat $FILENAME)
for L in $SCHEMA
do
echo "${L,,}"
sed -i -E "s/.+/\L&_/" b.txt
done
So after running script the output file of b.txt file I am expecting is
Company name is google_
Company name is facebook_
But the output after running that script I am getting is -
Company name is google.__
Company name is facebook.__
And this output will be saved in b.txt file as I mentioned in sed command
Note - In a.txt I am having the list of Words which I want to replace and in b.txt file I am having paragraphs of line in which I am having words like google. , facebook. and so on.
So that's why I am not able to give direct sed command for replacement.
I hope that you understand my requirement.
Thanks in advance!
You can use the following GNU sed solution:
FILENAME="a.txt"
while IFS= read -r L; do
sed -i "s/\($L\)\./\1_/gI" b.txt
done < $FILENAME
Or, the same without a loop as a single line (as used in anubhava's answer):
sed -i -f <(printf 's/\\(%s\\)\\./\\1_/gI\n' $(<"$FILENAME")) b.txt
With the script, you
while IFS= read -r L; do - read the file line by line, each line being assigned to L
sed -i "s/\($L\)\./\1_/gI" b.txt - replaces all occurrences of L (captured into Group 1 with the help of capturing \(...\) parentheses) followed with . (in a case insensitive way due to I flag) in b.txt with the same value as captured in Group 1 and _ appended to it.
-f allows passing a list of commands to sed
printf 's/\\(%s\\)\\./\\1_/gI\n' $(<"$FILENAME") creates a list of sed commands, in this case, it looks like
s/\(GOOGLE\)\./\1_/gI
s/\(FACEBOOK\)\./\1_/gI
Here is how you can do it in a single shell command without any loop using gnu-sed with printf in a process substitution:
sed -i -E -f <(printf 's/\\b(%s)\\./\\1_/I\n' $(<a.txt)) b.txt
cat b.txt
Company name is google_
Company name is facebook_
This would be far more efficient than running sed or awk in a loop esp if input files are big in size.
printf command is creating a sed command script that looks like this:
s/\b(GOOGLE)\./\1_/I
s/\b(FACEBOOK)\./\1_/I
sed -f runs that dynamically generated script
With a single awk reading 2 Input_files could you please try following.
awk '
FNR==NR{
a[tolower($0)]
next
}
($(NF-1) in a){
sub(/\.$/,"")
print $0"_"
}
' a.txt FS="[ .]" b.txt
Explanation: Adding detailed explanation for above solution.
awk ' ##Starting awk program from here.
FNR==NR{ ##Checking condition FNR==NR which will be TRUE when a.txt is being read.
a[tolower($0)] ##Creating array a with index of current line in lower case from a.txt here.
next ##next will skip all further statements from here.
}
($(NF-1) in a){ ##Checking condition if 2nd last field is present in array a then do following.
sub(/\.$/,"") ##Substituting last DOT with NULL here.
print $0"_" ##Printing current line with _ here.
}
' a.txt FS="[ .]" b.txt ##Mentioning a.txt and setting field separator as space and . for b.txt here.
2nd solution: Adding 1 more solution with awk here.
awk '
FNR==NR{
a[tolower($0)]
next
}
{
sub(/\.$/,"")
}
($NF in a){
print $0"_"
}
' a.txt b.txt
This might work for you (GNU sed):
sed 's#.*#s/(&)./\\1_/Ig#' a.txt | sed -i -Ef - b.txt
N.B. The match is case insensitive because of the I flag on the substitution command, however the replacement is from the original file i.e. if the original string is google the match is case insensitive to GOOGLE and replaced by google_.
I would like insert a file into another file using sed. Here is an example
I have a file input.txt which has the line:
#INCLUDE filename-to-import
Currently I am processing this with sed to create output.txt
sed -r -e "/^#INCLUDE (\w+)/r ${HOME}/inc/<filename>.sh" input.txt
How can I back reference the group captured in the regex in order to import the filename-to-import that is specified? I tried using \1 but that didn't work, I believe because it is outside of the regex scope. $1 didn't work either.
Is this possible with sed?
As said, I don't think this can be done with sed. Here is an awk one liner instead:
awk '$1 == "#INCLUDE" { while(getline line <$2){print line} }' input.txt
This might work for you (GNU sed):
sed -E '/^~INCLUDE (\w+)/{p;s##cat /inc/\1.sh#e}' file
Capture a back reference using the substitute command and in the RHS of it, evaluate a shell command, using the e flag.
I have a testfile similar to this one:
Bla: Blubb
Bla: blubb
Relevant stuff
So basically header information followed by a newline. I need everything after the newline as the number of header-lines varies.
Does anybody have an idea on how to achieve that?
Cheers and thank you
Fabian
With sed
sed -n '/^$/,$p' file
With awk (including the newline):
awk '/^$/{p=1};p' file
With awk (excluding the newline):
awk 'p;/^$/{p=1}' file
This might work for you (GNU sed):
sed '1,/^\s*$/d' file
I have a file: users.txt that looks like the below:
Name
Jack
Name
Jackson
Name
Peter
I need to remove EVERYTHING ABOVE the last match found Name so that the file looks like:
Name
Peter
I have tried searching the net, but only find awk command's that can achieve this.
I need it to be sed in order to use the -i option to save. I have come across where a file is sometimes empty when redirecting > standard output using awk.
Your time and assistance is greatly appreciated.
You can use tac twice with awk:
tac file | awk '{print} /Name/{exit}' | tac
Name
Peter
Through sed,
tac file | sed -n '0,/Name/p' | tac
Using sed without tac:
sed -n '/Name/h; //!H; ${g;p}' file
When Name is seen, replace the hold space with the current line
When Name is NOT seen, append the current line to the hold space
At the last line, replace the pattern space with the hold space and print.
Since we don't have sed in a pipeline, you can use -i to replace the file.
All in awk
awk 'x{x=x"\n"$0}/Name/{x=$0}END{print x}' file
This might work for you (GNU sed):
sed 'H;/Name/h;$!d;x' file
Hold matches in the hold space and at the end of file print the hold space with the last match.
I have lines in a file in linux, and i am trying print the line without the | and without some parameters
$cat file
2013-07-15,Provider 1.99,3|30000055|2347|0,12222,1,3,0,0,0,19,aaa,bbb
2013-07-15,Provider 1.99,3|30000055|2347|0,12222,44,12,0,0,0,33,aaa,bbb
and i need the output like:
2013-07-15,Provider,2347,12222,1,3,0,0,0,19,aaa,bbb
2013-07-15,Provider,2347,12222,44,12,0,0,0,33,aaa,bbb
and i am trying with awk, but i have some problems.
If your lines have similar pattern you would to retain then you can do:
awk 'BEGIN{FS=OFS=","}{$2="Provider";$3=2347}1' file
If you don't know what the patterns are then here is a more generic one:
awk 'BEGIN{FS=OFS=","}{split($2,a,/ /);split($3,b,/\|/);$2=a[1];$3=b[3]}1' file
If it doesn't solve your problem, I am pretty sure it would help you guide to get one.
Using sed:
sed 's/ [^|]*|[^|]*|\([^|]*\)|[^,]/,\1/' input
and some shorter version:
sed 's/ .*|\([^|]*\)|[^,]*/,\1/' input
and even shorter:
sed 's/ .*|\(.*\)|[^,]*/,\1/' input
Use awk, and let blank or comma or pipe be the field separators:
awk -F '[[:blank:],|]' -v OFS=, '{
print $1,$2,$6,$8,$9,$10,$11,$12,$13,$14,$15,$16
}' file
2013-07-15,Provider,2347,12222,1,3,0,0,0,19,aaa,bbb
2013-07-15,Provider,2347,12222,44,12,0,0,0,33,aaa,bbb