I have two with AWK which works perfectly
myScript3.awk
#!/usr/bin/awk -f
BEGIN {
FS=">|</"
OFS=","
}
{
data[count++] = $2
ptint $2
}
END{
print data[2],data[6],data[3], FILENAME
}
The above script will scan the xml document and return the 2nd, 6th, 3rd element along with file name.
for filename in *.xml
do
awk -f myscript3.awk $filename >> out.txt
done
The above script will scan the entire folder and list down xml files and then execute myscript.
i have to merge these two scripts as one.
Thanks for your help
Note about calling conventions: if you're running the script as awk -f script you do not need the shabang (#!) line at the beginning. Alternatively you can run it with the shabang as ./script if script is executable.
Answer
awk has BEGINFILE and ENDFILE, replace BEGIN/END with them and give the xml files as an argument, the following should work:
Edit
As noted by Dennis in the comments below, there's no need for BEGINFILE. Also note that this requires a fairly recent version of GNU awk to work.
myScript3.awk
BEGIN {
FS=">|</"
OFS=","
}
{
data[count++] = $2
print $2
}
ENDFILE {
print data[2],data[6],data[3], FILENAME
}
Run it like this:
awk -f myscript.awk *.xml
#!/bin/bash
AWKPROG='BEGIN {FS=">|</" OFS=","}
{ data[count++] = $2; ptint $2 }
END {print data[2], data[6], data[3], FILENAME}'
for filename in *.xml;do awk $AWKPROG $filename; done >> out.txt
Warning: Untested.
Related
I wish print in the "END" of the second line(or also in the end the file) a specific word get from a variable.
it would be appreciated to have both methods.
I have a file my_file.txt with this content:
#Insert the names separeted by space
l.lovre p.jhonson p.backer t.thompson q.ward
I tried to coding right so:
#!/bin/bash
filename=/path/my_file.txt
read -p "Insert name: " name
#my attempt
awk -v myvar="$name" 'END{print myvar}' $filename
echo 'Name added'
But doesn't work
I would get this result:
my_file.txt:
#Insert the names separeted by space
l.lovre p.jhonson p.backer t.thompson q.ward **name**
Thanks in advance
sed is simpler:
sed -i "$ s/$/ $name/" my_file.txt
Edit: Make sure the last line in the file is not empty.
You could add it to the whole line at line 2 in the file with $0 = $0 FS myvar and keep your END block to print it after all the lines since you want that too. If you don't want it after all the lines, remove the END block.
awk -v myvar="$name" 'FNR==2 {$0 = $0 FS myvar}1; END{print myvar}' $filename
Output with name=Michele:
#Insert the names separeted by space
l.lovre p.jhonson p.backer t.thompson q.ward Michele
Michele
By default awk does not update the file. If you want to update the file you have a couple options:
######
# write to tmp file and then overwrite original file with tmp file
awk -v myvar="$name" 'END{print myvar}' "$filename" > tmpfile && mv tmpfile "$filename"
######
# if using 'GNU awk' you can use '-i inplace' to modify the file
awk -i inplace -v myvar="$name" 'END{print myvar}' "$filename"
As for the issue of appending the new name onto the end of the 2nd line, or creating a new line if there are no names in the current file: one idea:
######
# writing to tmp file
awk -v myvar="$name" 'FNR==2 { print $0,myvar; added=1; next} 1; END { if (added!=1) print myvar}' "$filename" > tmpfile
mv tmpfile "$filename"
######
# using 'GNU awk'
awk -i inplace -v myvar="$name" 'FNR==2 { print $0,myvar; added=1; next} 1; ENDFILE { if (added!=1) print myvar}' "$filename"
NOTES:
all input lines need to be echoed to stdout in order for them to 'remain' in the file, hence the standalone 1
END {} block processing takes place after a file has been processed; any output generated by the END {} block will go to stdout
if using GNU awk we have access to the ENDFILE {} block which is, in essence, the same as an END {} block except that processing in this case does apply to the file
Taking the GNU awk solution for a test drive:
$ filename=names.dat
$ cat "$filename"
# Insert the names separeted by space
$ name='b.bunny'
$ awk -i inplace -v myvar="$name" 'FNR==2 { print $0,myvar; added=1; next} 1; ENDFILE { if (added!=1) print myvar}' "$filename"
$ cat "$filename"
# Insert the names separeted by space
b.bunny
$ name='d.duck'
$ awk -i inplace -v myvar="$name" 'FNR==2 { print $0,myvar; added=1; next} 1; ENDFILE { if (added!=1) print myvar}' "$filename"
$ cat "$filename"
# Insert the names separeted by space
b.bunny d.duck
I'm having trouble with this exercise. So far I got this code:
#!/bin/bash
for file in *.txt
do
if grep -qv '[0-9]' $file; then
echo "$file"
fi
done
Unfortunately with no success. It lists all of my .txt files. As in the Title, I want to check the insides of all files in the directory, and echo the namefile if they contain no numbers.
The -v causes it to match any line that does not contain a digit. If a file contains even one such line, the file will match.
Instead of inverting the match with -v, use !:
for file in *.txt
do
if ! grep -q '[0-9]' "$file"; then
echo "$file"
fi
done
You could also do the whole thing with a single grep invocation and skip the loop. -L prints files that don't match the pattern.
grep -L '[0-9]' *.txt
With GNU awk for nextfile and ENDFILE:
awk '
/[0-9]/ { f=1; nextfile }
ENDFILE { if (!f) print FILENAME; f=0 }
' *.txt
With any awk:
awk '
/[0-9]/ { f[FILENAME] }
END { for (i=1; i<ARGC; i++) if ( !(ARGV[i] in f) ) print ARGV[i] }
' *.txt
The second script has to be written that way to correctly handle empty files.
i have a problem with this code.. i can't figure out what i have to write as condition to cut my file with awk.
i=0
while [ $i -lt 10 ]; #da 1 a 9, Ap1..Ap9
do
case $i in
1) RX="54:75:D0:3F:1E:F0";;
2) RX="54:75:D0:3F:4D:00";;
3) RX="54:75:D0:3F:51:50";;
4) RX="54:75:D0:3F:53:60";;
5) RX="54:75:D0:3F:56:10";;
6) RX="54:75:D0:3F:56:E0";;
7) RX="54:75:D0:3F:5A:B0";;
8) RX="54:75:D0:3F:5F:90";;
9) RX="D0:D0:FD:68:BC:70";;
*) echo "Numero invalido!";;
esac
echo "RX = $RX" #check
awk -F, '$2 =="$RX" { print $0 }' File1 > File2[$i] #this is the line!
i=$(( $i + 1 ))
done
the command echo prints correctly but when i use the same "$RX" as condition in AWK it doesn't work (it prints a blank page).
my File1 :
1417164082794,54:75:D0:3F:53:60,54:75:D0:3F:1E:F0,-75,2400,6
1417164082794,54:75:D0:3F:56:10,54:75:D0:3F:1E:F0,-93,2400,4
1417164082794,54:75:D0:3F:56:E0,54:75:D0:3F:1E:F0,-89,2400,4
1417164082794,54:75:D0:3F:5A:B0,54:75:D0:3F:1E:F0,-80,2400,4
1417164082794,54:75:D0:3F:53:60,54:75:D0:3F:1E:F0,-89,5000,2
could you tell me the right expression "awk -F ..."
thank you very much!
To pass variables from shell to awk use -v:
awk -F, -v R="$RX" '$2 ==R { print $0 }' File1 > File2[$i]
#Ricky - any time you write a loop in shell just to manipulate text you have the wrong approach. It's just not what the shell was created to do - it's what awk was created to do and the shell was created to invoke commands like awk.
Just use a single awk command and instead of reading File 10 times and switching on variables for every line of the file, just do it all once, something like this:
BEGIN {
split(file2s,f2s)
split("54:75:D0:3F:1E:F0\
54:75:D0:3F:4D:00\
54:75:D0:3F:51:50\
54:75:D0:3F:53:60\
54:75:D0:3F:56:10\
54:75:D0:3F:56:E0\
54:75:D0:3F:5A:B0\
54:75:D0:3F:5F:90\
D0:D0:FD:68:BC:70", rxs)
for (i in rxs) {
rx2file2s[rxs[i]] = f2s[i]
}
}
{
if ($2 in rx2file2s) {
print > rx2file2s[$2]
}
else {
print NR, $2, "Numero invalido!" | "cat>&2"
}
}
which you'd then invoke as awk -v file2s="${File2[#]}" -f script.awk File1
I say "something like" because you didn't provide any sample input (File1 contents) or expected output (File2* values and contents) so I couldn't test it but it will be very close to what you need if not exactly right.
I have code where awk is piped to a clearcase command where If else loop is not working.
code is below :
#!/bin/ksh
export dst_region=$1
cleartool lsview -l | gawk -F":" \ '{ if ($0 ~ /Global path:/) { if($dst_region == "ABC" || $dst_region -eq "ABC") { system("echo dest_region is ABC");}
else { system("echo dest_region is not ABC"); } }; }'
But when I execute the above script the I get incorrect output,
*$ ksh script.sh ABCD
dest_region is ABC
$ ksh script.sh ABC
dest_region is ABC*
Could anyone please help on this issue ?
It would be useful if you explained exactly what you are trying to do but your awk script can be cleaned up a lot:
gawk -F":" -vdst_region="$1" '/Global path:/ { if (dst_region == "ABC") print "dest_region is ABC"; else print "dest_region is not ABC" }'
General points:
I have used -v to create an awk variable from the value of $1, the first argument to the script. This means that you can use it a lot more easily in the script.
awk's structure is condition { action } so you're using if around the whole one-liner unnecessarily
$0 ~ /Global path:/ can be changed to simply /Global path:/
the two sides of the || looked like they were trying to both do the same thing, so I got rid of the one that doesn't work in awk. Strings are compared using ==.
system("echo ...") is completely unnecessary. Use awk's built in print
You could go one step further and remove the if-else entirely:
gawk -F":" -vdst_region="$1" '/Global path:/ { printf "dest region is%s ABC", (dst_region=="ABC"?"":" not") }'
I'm using the following shell script to find the contents of one file into another:
#!/bin/ksh
file="/home/nimish/contents.txt"
while read -r line; do
grep $line /home/nimish/another_file.csv
done < "$file"
I'm executing the script, but it is not displaying the contents from the CSV file. My contents.txt file contains number such as "08915673" or "123223" which are present in the CSV file as well. Is there anything wrong with what I do?
grep itself is able to do so. Simply use the flag -f:
grep -f <patterns> <file>
<patterns> is a file containing one pattern in each line; and <file> is the file in which you want to search things.
Note that, to force grep to consider each line a pattern, even if the contents of each line look like a regular expression, you should use the flag -F, --fixed-strings.
grep -F -f <patterns> <file>
If your file is a CSV, as you said, you may do:
grep -f <(tr ',' '\n' < data.csv) <file>
As an example, consider the file "a.txt", with the following lines:
alpha
0891234
beta
Now, the file "b.txt", with the lines:
Alpha
0808080
0891234
bEtA
The output of the following command is:
grep -f "a.txt" "b.txt"
0891234
You don't need at all to for-loop here; grep itself offers this feature.
Now using your file names:
#!/bin/bash
patterns="/home/nimish/contents.txt"
search="/home/nimish/another_file.csv"
grep -f <(tr ',' '\n' < "${patterns}") "${search}"
You may change ',' to the separator you have in your file.
Another solution:
use awk and create your own hash(e.g. ahash), all controlled by yourself.
replace $0 to $i and you can match any fields you want.
awk -F"," '
{
if (nowfile==""){ nowfile = FILENAME; }
if(FILENAME == nowfile)
{
hash[$0]=$0;
}
else
{
if($0 ~ hash[$0])
{
print $0
}
}
} ' xx yy
I don't think you really need a script to perform what you're trying to do.
One command is enough. In my case, I needed an identification number in column 11 in a CSV file (with ";" as separator):
grep -f <(awk -F";" '{print $11}' FILE_TO_EXTRACT_PATTERNS_FROM.csv) TARGET_FILE.csv