Integrating several shell scripts into one script - linux

Question

I would like to integrate a few short scripts into one script where I can update an argument for the input file from the command line. I am going through 22 files and counting lines where $5!="1".
Here is a sample head of the input file:
Currently, I have the following 3 short scripts:
CHROM POS N_ALLELES N_CHR {FREQ}
2 45895 2 162 0.993827 0.00617284
2 45953 2 162 0.993827 0.00617284
2 264985 2 162 1 0
2 272051 2 162 0.944444 0.0555556
1) count lines (saved as wcYRI.sh): $5!="1"{sum++}END{print sum}
2) apply linecount (saved as check-annos.sh): awk -f wcYRI.sh ~/folder$1/file$1
3) apply linecount for 22 files, sum the output:
for i in {1..22};
do sh check-annos.sh $i; done
| awk '{sum+=$1}END{print sum}'
Its relatively simple, but sometimes script 1 gets a little longer for data files that look like this:
Chr Start End Ref Alt Func.refGene Gene.refGene ExonicFunc.refGene AAChange.refGene LJB2_SIFT LJB2_PolyPhen2_HDIV LJB2_PP2_HDIV_Pred LJB2_PolyPhen2_HVAR LJB2_PolyPhen2_HVAR_Pred LJB2_LRT LJB2_LRT_Pred LJB2_MutationTaster LJB2_MutationTaster_Pred LJB_MutationAssessor LJB_MutationAssessor_Pred LJB2_FATHMM LJB2_GERP++ LJB2_PhyloP LJB2_SiPhy
16 101593 101593 C T exonic POLR3K nonsynonymous SNV POLR3K:NM_016310:exon2:c.G164A:p.G55E 0.000000 0.997 D 0.913 D 0.000000 D 0.999989 D 2.205 medium 0.99 5.3 2.477000 17.524
...and I am using an awk file like this (performing an array match) as input -f to script 2 above:
NR==FNR{
arr[$1$2];next
}
$1$2 in arr && $0~/exonic/&&/nonsynonymous SNV/{nonsyn++};
$1$2 in arr && $0~/exonic/&&/synonymous SNV/ && $0!~/nonsynonymous/{syn++}
END{
print nonsyn,"nonsyn YRI","\t",syn,"YRI syn"
}
My goal is to integrate this process a bit more so I don't need to go into script 2 and change the ~/folder$1/file$1 each time-- I'd like to be able to use ~/folder$1/file$1 as an input at the command line. However when I try to use something like this in a for-loop at the command line, it doesn't accept $1 the way it does when $1 is built into a separate script being called by the for-do-done loop (as in script 3 --i.e. script 3 will take script 2, but I can't just enter the contents of script 2 explicitly into the for-loop as an argument(s)).
I am actually not so concerned about having a separate AWK file to handle the line parsing, the main thing annoying me is that I am modifying script 2 for each folder/file set, and I would like to be able to do this from the command line so that the script knows when I tell it ~/folder$1/file$1, to cycle through numbers 1-22 and I so can save one universal script for this process, since I have many folder/file combinations to look at.
Any advice is appreciated for shortening the pipeline in general, but specifically the command line argument problem is bugging me a lot!

Answer 1

If I understand the problem correctly, I see two ways to handle it. If the path format is consistent (i.e. the number always occurs twice, in the same positions), you could make the script accept the parts of the path as two different parameters. The script would look like this:
#!/bin/bash
folderPrefix="$1"
filePrefix="$2"
for num in {1..22}; do
awk -f wcYRI.sh "$folderPrefix$num/$filePrefix$num"
done |
awk '{sum+=$1}END{print sum}'
... and then you'd run it with ./scriptname ~/folder file. Alternately, if you need to be able to define the folder/file path format more flexibly, you could do something like this:
#!/bin/bash
for num in {1..22}; do
eval "awk -f wcYRI.sh $1"
done |
awk '{sum+=$1}END{print sum}'
... and then run it with ./scriptname '~/folder$num/file$num'. Note that the single-quotes are needed here so that the $var references don't get expanded until eval forces them to be.
BTW, the file wcYRI.sh is an awk script, not a shell script, so I'd recommend changing its file extension to prevent confusion. Actually, the preferred way to do this (for both shell and awk scripts) is to add a shebang line as the first line in the script (see my examples above; for an awk script it would be #!/usr/bin/awk -f), then make the script executable, and then run it with just ./scriptname and let the shebang take care of specifying the interpreter (sh, bash, awk -f, whatever).