I have a file named abc.csv which contains these 6 lines:
xxx,one
yyy,two
zzz,all
aaa,one
bbb,two
ccc,all
Now whenever all comes in a line that line should be replaced by both one and two, that is:
xxx,one
yyy,two
zzz,one
zzz,two
aaa,one
bbb,two
ccc,one
ccc,two
Can someone help how to do this?
$ awk -F, -v OFS=, '/all/ { print $1, "one"; print $1, "two"; next }1' foo.input
xxx,one
yyy,two
zzz,one
zzz,two
aaa,one
bbb,two
ccc,one
ccc,two
If you want to stick to a shell-only solution:
while read line; do
if [[ "${line}" = *all* ]]; then
echo "${line%,*},one"
echo "${line%,*},two"
else
echo "${line}"
fi
done < foo.input
In sed:
sed '/,all$/{ s/,all$/,one/p; s/,one$/,two/; }'
When the line matches ,all at the end, first substitute all with one and print it; then substitute one with two and let the automatic print do the printing.
Related
My 1st file -
#cat 123
tom,123
jack,222
rock
google,908
mohan,323
ram,789
My 2nd file -
#cat www
vicky,tom,home
google,monk,uber
dhoom,monk,uber
ram,monk,uber
rock,monk,uber
jack,monk,uber
Desired output -
#cat match_output.txt
tom,123,vicky,tom,home
rock,rock,monk,uber
jack,222,jack,monk,uber
google,908,google,monk,uber
ram,789,ram,monk,uber
For now, I'm getting only this -
#cat match_output.txt
rock,monk,uber
My Script -
#!/bin/bash
# because File2 is bigger, it gets the main loop.
# read each line of File2
>/var/lib/jenkins/Jack/match_output.txt
while IFS=, read -r string; do
# read each line of File1.txt
while IFS=, read -r string2; do
# check match, and write if needed.
if [[ $string == *"$string2"* ]]; then
echo $string >> match_output.txt
echo "wrote "$string" to match_output.txt..."
fi
done < /var/lib/jenkins/Jack/123
done < /var/lib/jenkins/Jack/www
Not able to read 1st value of 1st file before the comma of each line and match with 2nd file line by line and print the output in a new file....
To get the first value before the comma, you can use the cut command.
With the following code
if [[ $string == *"$string2"* ]]; then
echo $string >> match_output.txt
echo "wrote "$string" to match_output.txt..."
fi
you compare the complete lines. If you want to compare $string with only the first value (before the comma) of $string2, you need to adjust this comparison.
string2FirstValue=`echo "$string2" |cut -d',' -f1`
if [[ $string == *"$string2FirstValue"* ]]; then
echo $string2,$string >> match_output.txt
fi
Safer and more efficient way is to use awk:
awk '
BEGIN {FS=OFS=","}
FNR == NR {
map[$1] = $0
next
}
{
for (i=1; i<=NF; ++i)
if ($i in map) {
print map[$i], $0
next
}
}' 123 www
tom,123,vicky,tom,home
google,908,google,monk,uber
ram,789,ram,monk,uber
rock,rock,monk,uber
jack,222,jack,monk,uber
I am trying to get the number of the line where the word "nel" comes as the variable "line" from the prueba.txt with help of the progtesis.awk command I am writing.
I am running this in the terminal:
awk -f progtesis.awk prueba.txt
And progtesis sees as follow:
line=$(awk -f '/nel/{print NR}' FILENAME}
echo "$line"
Any suggestions?
No need of an external awk script :
line=$(awk '/nel/ {print NR; exit}' "${filename}")
echo "${line}"
will display number of first line matching /nel/.
Otherwise if progtesis.awk contains
/nel/ {print NR; exit}
The bash commands can be
line=$(awk progtesis.awk "${filename}")
echo "${line}"
My input file has blocks like below. Please help me deleting the block and its contents using awk or sed
[abc]
para1=123
para2=456
para3=111
[pqr]
para1=333
para2=765
para3=1345
[xyz]
para1=888
para2=236
para3=964
now how do i delete a block and its parameters completely .Please help me achieve this with awk command.Thanks in advance
You can use RS for split blocks, (NOTE: NR>1 because awk generate a empty block in beginning)
awk -vRS='[' -v remove="pqr" '
NR>1 && $0 !~ "^"remove"]" {printf "%s", "["$0; }
' file
you get,
[abc]
para1=123
para2=456
para3=111
[xyz]
para1=888
para2=236
para3=964
Depends on how you want to filter. If you want to delete the block with the header '[pqr]'
awk '!/^\[pqr\]/' RS= ORS='\n\n' input
or
awk '$1 !~ "[pqr]"' RS= ORS='\n\n' input
If you want to omit the 2nd record (the same as above)
awk 'NR!=2' RS= ORS='\n\n' input
If you want to omit the record in which para2=765,
awk '$3 !~ "765"' RS= ORS='\n\n' input
Perl solution to remove block [abc]
perl -lne 'BEGIN{$/=""} print "$_\n" unless /^\[abc\]/' file
-n loop around every line of the input file, put the line in the $_ variable, do not automatically print the line
-l removes newlines before processing, and adds them back in afterwards
-e execute the perl code
$/ is the input record separator. Setting it to "" in a BEGIN{} block puts Perl into paragraph mode.
$_ is the current line.
/^/ is a regular expression which begins with the search term
output:
[pqr]
para1=333
para2=765
para3=1345
[xyz]
para1=888
para2=236
para3=964
This variation enables argument parsing with -s and passes [abc] to variable $b
perl -slne 'BEGIN{$/=""} print "$_\n" unless /^$b/' -- -b='\[abc\]'
I propose a slightly different solution using only shell.
#!/bin/sh
# specify the block to withhold
WITHHOLD=2
COUNT=1
INAWHITESP=0
while read i
do if [ -z "$i" -a "$INAWHITESP" -eq 0 ]
then COUNT=$(( COUNT + 1 ))
INAWHITESP=1
fi
if [ -n "$i" -a "$INAWHITESP" -eq 1 ]
then INAWHITESP=0
fi
if [ "$COUNT" -ne "$WITHHOLD" ]
then printf "%s\n" "$i"
fi
done < inputfile > outputfile
To remove block abc
awk 'BEGIN{RS=""} !/\[abc\]/'
How can I loop through a file, in bash, and echo the line and line number?
I have this, which has everything but the line number:
while read p;
do
echo "$p" "$LINE";
done < file.txt
Thanks for your help!
edit this will be run multi-thread using xargs, so i don't want to use a counter.
I would just use cat -n file
But if you really want to use a bash loop:
i=0
while read; do
printf '%d %s\n' $(( ++i )) "$REPLY"
done < file
Update: I now prefer nl to cat -n, as the former is standard. To get the same result as cat -n, use nl -b a "$file".
You can use awk:
awk '{print NR,$0}' file.txt
You can use awk:
awk '{print NR "\t" $0}' file.txt
Or you can keep a count of the line number in your loop:
count=0
while read line
do
((count+=1))
printf "%5d: %s\n" "$count" "$line"
done
Using printf allows you to format the number, so the lines in the file are all lined up.
Just for fun, bash v4
mapfile lines < file.txt
for idx in "${!lines[#]}"; do printf "%5d %s" $((idx+1)) "${lines[idx]}"; done
Don't do this.
Below is my comma separated input.txt file, i want to read the columns and write the lines in to the output.txt when any 1 column has a space.
Content of input.txt:
1,Hello,world
2,worl d,hell o
3,h e l l o, world
4,Hello_Hello,World#c#
5,Hello,W orld
Content of output.txt:
1,Hello,world
4,Hello_Hello,World#c#
is't possible to achieve using awk? Please help!
A simple way to filter out lines with spaces is using inverted matching with grep:
grep -v ' ' input.txt
If you must use awk:
awk '!/ /' input.txt
Or perl:
perl -ne '/ / || print' input.txt
Or pure bash:
while read line; do [[ $line == *' '* ]] || echo $line; done < input.txt
# or
while read line; do [[ $line =~ ' ' ]] || echo $line; done < input.txt
UPDATE
To check if let's say field 2 contains space, you could use awk like this:
awk -F, '$2 !~ / /' input.txt
To check if let's say field 2 OR field 3 contains space:
awk -F, '!($2 ~ / / || $3 ~ / /)' input.txt
For your follow-up question in comments
To do the same using sed, I only know these awkward solutions:
# remove lines if 2nd field contains space
sed -e '/^[^,]*,[^,]* /d' input.txt
# remove lines if 2nd or 3rd field contains space
sed -e '/^[^,]*,[^,]* /d' -e '/^[^,]*,[^,]*,[^,]* /d' input.txt
For your 2nd follow-up question in comments
To disregard leading spaces in the 2nd or 3rd fields:
awk -F', *' '!($2 ~ / / || $3 ~ / /)' input.txt
# or perhaps what you really want is this:
awk -F', *' -v OFS=, '!($2 ~ / / || $3 ~ / /) { print $1, $2, $3 }' input.txt
This can also be done easily with sed
sed '/ /d' input.txt
try this one-liner
awk 'NF==1' file
as #jwpat7 pointed out, it won't give correct output if the line has only leading space, then this line, with regex should do, but it has been already posted in janos's answer.
awk '!/ /' file
or
awk -F' *' 'NF==1'
Pure bash for the fun of it...
#!/bin/bash
while read line
do
if [[ ! $line =~ " " ]]
then
echo $line
fi
done < input.txt
columnWithSpace=2
ColumnBef=$(( ${columnWithSpace} - 1 ))
sed '/\([^,]*,\)\{${ColumnBef\}[^ ,]* [^,]*,/ d'
if you know the column directly (by example the 3):
sed '/\([^,]*,\)\{2}[^ ,]* [^,]*,/ d'
If you can trust the input to always have no more than three fields, simply finding a space somewhere after a comma is sufficient.
grep ',.* ' input.txt
If there can be (or usually are) more fields, you can pull that off with grep -E and a suitable ERE, but you are fast approaching the point at which the equivalent Awk solution will be more readable and maintainable.