I am working on a shell script which contains following piece of code.
I don't understand these lines, mostly the cut command and export command. Can any one help me...
Also please point me to a better linux command reference.
Thanks in advance!
# determine sum of 60 records
awk '{
if (substr($0,12,2) == "60" || substr($0,12,2) == "78") \
print $0
}'< /tmp/checks$$.1 > /tmp/checks$$.2
rec_sum =`cut -c 151-160 /tmp/checks$$.2 | /u/fourgen/cashnet/bin/sumit`
export rec_sum
Inside my sumit script following is the code
awk '{ total += $1}
END {print total}' $1
Let me show my main script prep_chk
awk 'BEGIN{OFS=""} {if (substr($0,12,2) == "60" && substr($0,151,1) == "-") \
{ print substr($0,1,11), "78", substr($0,14) } \
else \
{ print $0 } \
}' > /tmp/checks$$.1
# determine count of non-header record
rec_cnt=`wc -l /tmp/checks$$.1`
rec_cnt=`expr "$rec_cnt - 1"`
export rec_cnt
# determine sum of 60 records
awk '{ if (substr($0,12,2) == "60" || substr($0,12,2) == "78") \
print $0 }'< /tmp/checks$$.1 > /tmp/checks$$.2
rec_sum=`cut -c 151-160 /tmp/checks$$.2 | /u/fourgen/cashnet/bin/sumit`
export rec_sum
# make a new header record and output it
head -1 /tmp/checks$$.1 | awk '{ printf("%s%011.11d%05.5d%s\n", \
substr($0,1,45), rec_sum, rec_cnt, substr($0,62)) }' \
rec_sum="$rec_sum" rec_cnt="$rec_cnt"
# output everything else sorted by tran code
grep -v "%%%%%%%%%%%" /tmp/checks$$.1 | cut -c 1-150 | sort -k 1.12,13
cut -c cuts text from a given position in a file, in this case characters 151 to 160 in the file /tmp/checks$$.2. This string is piped to some code called submit which produces some output.
That output is then assigned to the environment variable rec_sum. The export command makes this variable available to be used through the system, for example in another shell script.
Edit:
If that's all you have inside your submit script it simply adds on the string you pass it, which it seems must be a number, to some value total and prints the number it was passed. It seems like there must be some more code inside that script otherwise it would be a bit of an over complicated way to do it.
Related
First time attempting to tinker with AWK and use it to take input from a file like the following:
data.csv
James,Jones,30,Mr,Main St
Melissa,Greene,200,Mrs,Wall St
Robert,Krupp,410,Mr,Random St
and process it into a LaTeX Template
data.tex
\newcommand{\customertitle}{XYZ} %ggf. \
\newcommand{\customerName}{Max Sample} % Name \
\newcommand{\customerStreet}{Str} % Street \
\newcommand{\customerZIP}{12345} % ZIP
First I tried to replace the customer name this way
awk 'BEGIN{FS=","}{print $1 " " $2 }' data.csv | xargs -I{} sed "s/Max Sample/{}/" data.tex > names
which gave me a merged file.. and therefore I subsequently attempted to render the file as single .tex files by inserting a keyword "#TEST" at the end of the original file, so I could use it as a Record Separator to get me back to single files with the following command:
awk 'BEGIN {FS=RS="#TEST"} {i=1}{ while (i <= NF) {print $i >>"final"NR".tex"; i++}}' names
Even though that worked for this one field, for multiple fields it doesn't seem to be a proper solution as is though. (title, street, zip code)
That's why I'm now attempting to get it working with the gsub action in AWK.
Tried those different approaches. Based on what I could find regarding it thus far, that's what I came up with:
awk 'BEGIN {FS=","}NR==FNR{a[FNR]=$4;next}{gsub ("XYZ",a[FNR]);print}' data.csv data.tex
which replaces XYZ with nothing
awk 'BEGIN {FS=","}NR==FNR{a[FNR]=$4;next}RS="#TEST"{for (i in a) {gsub("XYZ",i);print}}' data.csv data.tex \
which counts four times to 7
Tried those also with the merged file, ie. the "names" output from the first command and didn't get it to work.
What am I missing? Can the gsub command not replace a string with an array? Is a loop required?
I'm stuck, hope someone can help out here
I hope it is your case.
Create a file csv_to_latex.awk and put this code:
BEGIN{
FS=","
while(getline < latex > 0) {
lax_array[$0]
}
}
{
name = $1" "$2
zip = $3
status = $4
street = $5
for (lax_key in lax_array)
{
if (lax_key ~ /XYZ/)
{
gsub("{XYZ}", "{"status"}", lax_key)
print lax_key
}
else if (lax_key ~ /Max Sample/)
{
gsub("{Max Sample}", "{"name"}", lax_key)
print lax_key
}
else if (lax_key ~ /Str/)
{
gsub("{Str}", "{"street"}", lax_key)
print lax_key
}
else if (lax_key ~ /12345/)
{
gsub("{12345}", "{"zip"}", lax_key)
print lax_key
}
}
}
To execute this code, use in terminal:
awk -v latex="data.tex" -f csv_to_latex.awk data.csv
The output:
\newcommand{\customertitle}{Mr} %ggf. \
\newcommand{\customerName}{James Jones} % Name \
\newcommand{\customerStreet}{Main St} % Street \
\newcommand{\customerZIP}{30} % ZIP
\newcommand{\customertitle}{Mrs} %ggf. \
\newcommand{\customerName}{Melissa Greene} % Name \
\newcommand{\customerStreet}{Wall St} % Street \
\newcommand{\customerZIP}{200} % ZIP
\newcommand{\customertitle}{Mr} %ggf. \
\newcommand{\customerName}{Robert Krupp} % Name \
\newcommand{\customerStreet}{Random St} % Street \
\newcommand{\customerZIP}{410} % ZIP
I feel like doing replaces is unnecessary and would probably approach it like this:
awk -F, -v po='\\newcommand{\\%s}{%s} %s \n' '{
printf po, "customertitle", $4, "%ggf. \\"
printf po, "customerName", $1" "$2, "% Name \\"
printf po, "customerStreet", $NF, "% Street \\"
printf po, "customerZip", $3, "% Zip"
}' data.csv
output:
\newcommand{\customertitle}{Mr} %ggf. \
\newcommand{\customerName}{James Jones} % Name \
\newcommand{\customerStreet}{Main St} % Street \
\newcommand{\customerZip}{30} % Zip
\newcommand{\customertitle}{Mrs} %ggf. \
\newcommand{\customerName}{Melissa Greene} % Name \
\newcommand{\customerStreet}{Wall St} % Street \
\newcommand{\customerZip}{200} % Zip
\newcommand{\customertitle}{Mr} %ggf. \
\newcommand{\customerName}{Robert Krupp} % Name \
\newcommand{\customerStreet}{Random St} % Street \
\newcommand{\customerZip}{410} % Zip
I'm building a script in bash for use on Linux, and I use the output of a executable to fill parameters:
version=$("${path_exec}" -version | awk '{if($1=="kernel" && $2=="release") print $3}')
patch=$("${path_exec}" -version | awk '{if($1=="patch" && $2=="number") print $3}')
This will run the executable defined in "path_exec" twice, which is time consuming. Is there a way to assign the version and path variable with a value using only one execution of "path_exec"?
An example of what I've tried to tackle this is shown below, but I don't think this will do what I want:
${path_hostexec} -version | awk '{if($1=="kernel" && $2=="release") {version_agent = $3;} else if($1=="patch" && $2=="number") {patch_agent = $3;}}'
Could you please try following. Since I didn't have output of path_exec command so couldn't test it.
myarr=($("${path_exec}" -version | awk '{if($1=="kernel" && $2=="release");val=$3} {if($1=="patch" && $2=="number") print val,$3}'))
#Now get every element in the array
for i in "${myarr[#]}"
do
echo $i
done
What I have done is:
Merged your both awk programs into one to make it run on a single time.
Now creating an array by output of awk command output(which should be val1 val2 as an example format)
Once an array created then we could get its all values by for loop or you could get its specific value by mentioning its index eg--> myarr[1] to print 2nd element.
Output both values on a single line, and let read separate the line into its two parts.
IFS=, read version patch < <($path_exec -version |
awk '/kernel release/ {v=$3}
/patch number/ {p=$3}
END {print v","p}
')
Thanks guys, I've managed to get it working thanks to your input:
version_patch_agent=$("${path_hostexec}" -version | awk '/kernel release/ {v=$3} /patch number/ {p=$3} END {print v" patch "p}')
This puts the version and patch number into a variable that I can just echo to get the info on the screen. Thanks again all!!
Edit: Long story short: I'm trying to count the amount of successful connections from a log file. If it reaches 8, I need it run some command, then start over and count again.
I try to loop through a file, and grep for a specific string, and count the lines of it with wc -l. If it reaches 8 I need it to run a command, then start again from that point.
Assuming I have a file like the example below:
I want to find "STRING" and if wc -l is equal to 8 I need to run a command and start over (append something and start from there) and count the other 8 that comes after them.
STRING ...................
STRING ...................
STRING ...................
STRING ...................
STRING ...................
STRING ...................
...
I wrote this:
while true
do
countz=$(cat file.txt | grep "STRING" | wc -l)
echo $countz;
if [ "$countz" -eq 8 ];
then #do_something
fi
done
Not very clear what you want to achieve but
seq 100 | xargs -n 1 echo STRING \
| awk 'BEGIN { c=0; } $0 ~ /^STRING / {c++; if (c==8) { print "found"; c=0; } }' \
| your_command
This will execute your_command for every 8-th line
Or maybe you can count the total number of occurences of STRING in a file and dived it by 8 and then run that number of commands ?
countz=(cat file.txt | grep "STRING" | wc -l)
total_runs=$[countz / 8]
for ((i=0;i<$total_runs;i++)); do your_command; done
UPDATE
Seems you want to edit log file.
Easiest will be to create a new file with "delimiters"
awk 'BEGIN { c=0; } {
print $0;
if ($0 ~ /^STRING /) {
c++;
if (c==8) { print "your delimiter"; c=0; }
}
}' logfile > processed_logfile
It's still not completely clear what you're trying to do so I'm guessing a bit but you can probably do something like this:
awk '(c += /STRING/) == 8 { system("some command here"); c = 0 }' file
This adds to the counter c whenever the pattern matches and then runs a command every time the counter reaches 8.
If instead of running some command, you want to print some additional output, the script can be modified slightly:
awk '1; (c += /STRING/) == 8 { print "8 matches since last time"; c = 0 }' file
The 1 is the shortest true condition to trigger a { print }, so every line is printed. After every 8 matches, the message is also printed.
I have the following file:
1,A
2,B
3,C
10000,D
20,E
4000,F
I want to select the lines having a count greater than 10 and less than 5000. the output should be E and F. In C++ or any other language is a piece of cake. I really wanted to know how can I do it with a linux command.
I tried the following command
awk -F ',' '{$1 >= 10 && $1 < 5000} { count++ } END { print $1,$2}' test.txt
But it is only givine 4000,F.
just do:
awk -F',' '$1 >= 10 && $1 < 5000' test.txt
you put boolean check in {....}, and don't use the result at all. it doesn't make any sense. You should do either {if(...) ...} or booleanExpression{do...}
useless count++
you have only print statement in END so only last line was printed out.
Your script does actually:
print the last line of the test.txt, no matter what it is.
I need to take a file and count the number of occurrences of $7 - I've done this with awk (because I need to run this through more awk)
What I want to do is combine this into one script - so far I have
#! /usr/bin/awk -f
# get the filename, count the number of occurs
# <no occurs> <filename>
{ print $7 | "grep /datasheets/ | sort | uniq -c"}
how do I grab that output and run it through more awk commands - in the same file
Eventually, I need to be able to run
./process.awk <filename>
so it can be a drop-in replacement for a previous setup which would take too much time/effor to to change -
if you want to forward the output of an awk script to another awk script, just pipe it to awk.
awk 'foobar...' file|awk 'new awkcmd'
and your current awk|grep|sort|uniq could be done with awk itself. save your 3 processes. you want to get the repeated counts, don't you?
awk '$7~=/datasheets/{a[$7]++;} END{for(x in a)print x": "a[x]' file
should work.
If you use Gawk, you could use the 2-way communications to push the data to the external command then read it back:
#!/usr/bin/gawk -f
BEGIN {
COMMAND = "sort | uniq -c"
SEEN = 0
PROCINFO[ COMMAND, "pty" ] = 1
}
/datasheets/ {
print $7 |& COMMAND
SEEN = 1
}
END {
# Don't read sort output if no input was provided
if ( SEEN == 1 ) {
# Tell sort no more input data is available
close( COMMAND, "to" )
# Read the sorted data
while( ( COMMAND |& getline SORTED ) > 0 ) {
# Do whatever you want on the sorted data
print SORTED
}
close( COMMAND, "from" )
}
}
See https://www.gnu.org/software/gawk/manual/gawk.html#Two_002dway-I_002fO