please explain the line below used in shell scripts,
awk -F\| -v src=$storekey 'src==$41' $SRC_Path >> $DST_Path
Thanks!
Ok first ${variable} is a shell variable, so those would be defined higher in your script i.e.
storekey = "1234" or something
you can try this on your shell (linux or command line terminal)
type:
$ storekey="foo"
$ echo $storekey
So most of your question is pertaining to the variables and the command line which confuses how they are used, if you replaced the variables on a command line to test, you could work test it out to find out what they are doing.
In essence Awk is a stream parsing tool, so if you had a file of say 10 columns with a known delimiter such as "," or "|" you could ask awk for a specific column to be printed or output. This is what is happening below, but it is being confused by the presence of custom shell variables.
then to break down the command line awk is parsing a "|" delimited input (-F\| ) defined by $storekey variable, taking the column where src== $41 (this has some reference to the data being input), from $SRC_PATH (a directory) to $DST_PATH (another directory or path).
If you could share more of the shell script I could provide a more in depth answer.
btw, you could also find out more information, using the commands
man awk
info awk
from your command line, however these are a bit arcane for those not so familiar with *nix variants.
Related
My python script can take a series of bitwise operators as one of its arguments. They all work fine except for "=<<" which is roll left, and "=>>" which is roll right. I run my script like ./script.py -b +4,-4,=>>10,=<<1, where anything after -b can be any combination of similar operations. As soon as the terminal sees "<<" though, it just drops the cursor to a new line after the command and asks for more input instead of running the script. When it sees ">>", my script doesn't process the arguments correctly. I know it's because bash uses these characters for a specific purpose, but I'd like to get around it while still using "=>>" and "=<<" in my arguments for my script. Is there any way to do it without enclosing the argument in quotation marks?
Thank you for your help.
You should enclose the parameters that contain special symbols into single quotation marks (here, echo represents your script):
> echo '+4,-4,=>>10,=<<1'
+4,-4,=>>10,=<<1
Alternatively, save the parameters to a file (say, params.txt) and read them from the file onto the command line using the backticks:
> echo `cat params.txt`
+4,-4,=>>10,=<<1
Lastly, you can escape some offending symbols:
> echo +4,-4,=\>\>10,=\<\<1
+4,-4,=>>10,=<<1
Ok, here I'm again, struggling with ssh. I'm trying to retrieve some data from remote log file based on tokens. I'm trying to pass multiple tokens in egrep command via ssh:
IFS=$'\n'
commentsArray=($(ssh $sourceUser#$sourceHost "$(egrep "$v" /$INSTALL_DIR/$PROP_BUNDLE.log)"))
echo ${commentsArray[0]}
echo ${commentsArray[1]}
commax=${#commentsArray[#]}
echo $commax
where $v is something like below but it's length is dynamic. Meaning it can have many file names seperated by pipe.
UserComments/propagateBundle-2013-10-22--07:05:37.jar|UserComments/propagateBundle-2013-10-22--07:03:57.jar
The output which I get is:
oracle#172.18.12.42's password:
bash: UserComments/propagateBundle-2013-10-22--07:03:57.jar/New: No such file or directory
bash: line 1: UserComments/propagateBundle-2013-10-22--07:05:37.jar/nouserinput: No such file or directory
0
Thing worth noting is that my log file data has spaces in it. So, in the code piece I've given, the actual comments which I want to extract start after the jar file name like : UserComments/propagateBundle-2013-10-22--07:03:57.jar/
The actual comments are 'New Life Starts here' but the logs show that we are actually getting it till 'New' and then it breaks at space. I tried giving IFS but of no use. Probably I need to give it on remote but I don't know how should I do that.
Any help?
Your command is trying to run the egrep "$v" /$INSTALL_DIR/$PROP_BUNDLE.log on the local machine, and pass the result of that as the command to run via SSH.
I suspect that you meant for that command to be run on the remote machine. Remove the inner $() to get that to happen (and fix the quoting):
commentsArray=($(ssh $sourceUser#$sourceHost "egrep '$v' '/$INSTALL_DIR/$PROP_BUNDLE.log'"))
You should use fgrep to avoid regex special interpretation from your input:
commentsArray=($(ssh $sourceUser#$sourceHost "$(fgrep "$v" /$INSTALL_DIR/$PROP_BUNDLE.log)"))
Here is an extract from a script showing the variables for the script
PathToPiconPNG="/var/OscamSrvidPicon/picon/19.2E/"
PathToOscamSrvid="/var/OscamSrvidPicon/picon/19.2E/oscam.srvid"
PathToPiconTPL="/var/OscamSrvidPicon/oscam_picons/"
PathToTmp="/tmp/"
I want to run this script numerous times replacing (for example) this line:
PathToPiconPNG="/var/OscamSrvidPicon/picon/19.2E/"
with this lines
PathToPiconPNG="/var/OscamSrvidPicon/picon/28.2E/"
I have tried using sed (I know this example is wrong but you might get what im trying to achieve)
sed 's/{PathToPiconPNG="/var/OscamSrvidPicon/picon/19.2E/"}/{PathToPiconPNG="/var/OscamSrvidPicon/picon/28.2E/"}/g' filename.txt > newfilenam.txt
If that is not possible, is there any way that I can set the variable externally from another script
sed -E 's/picon\/.+\//picon\/28.2E\//' filename.txt > newfilenam.txt
In linux, I am trying to select a variable from a specific column and row of CSV file and then use this variable as the end of a file location hierarchy. When I type the following into a bash terminal window, it seems to work by outputting the variable in correct row and column on screen.
awk -F "," 'FNR == 2 {print $8}' /sdata/images/projects/ASD_SSD/1/ruths_data/ruth/imaging\ study/imaging\ study\ working/delete2.csv
However, I am trying to go do the following substitution within a script, this fails to work...
r=2
c=8
s=awk -F "," 'FNR == $r {print $c}' /sdata/images/projects/ASD_SSD/1/ruths_data/ruth/imaging\ study/imaging\ study\ working/delete2.csv
I then try to use the s output as the end of a hierarchy file location. For example, /home/ork/js/s*
I keep getting the following error, so this looks like it's not creating the s variable and then not inserting it into the actual file location.
omitting directory `/home/ork/js/'
I have spent a few hours trying to figure out what is preventing this from working and am a new user (so I am sure it is something simple, sorry).
I hope I was clear enough, please let me know if this requires further clarification.
This is a common question here. The single quotes are protecting the variables from the shell, so they never get expanded. Also command substitution is needed when assigning to variable s. One way to do it would be:
s=$(awk -F, 'FNR==r{print c}' r="$r" c="$c" file)
I'm new to scripting and need a lot of help understanding how to load in a text file that will be passed in by a parameter. I'm afraid that you'll have to really dumb down your answers. Please explain like I am a 10 year old!
For example, how does one write a script that accepts two parameters, a file path and an integer.
Also how do you run the script? would you just call script.sh -f data.txt
Could you explain your requirement a bit in detail?
I assume you are looking to extract the Nth column from a given file. You don't need to parse command-line parameters within AWK, instead just pass them from sh(1) itself.
# foo.sh
awk "{ print $"$1" }" $2
Here I'm escaping out of AWK to get the first argument which is the position number.
If you find that too confusing you can manipulate ARGC, ARGV. Read awk(1).
Hope that helps.
You can do this by manipulating ARGC and ARGV to extract your integer from the argument list in the BEGIN block. You can put your script in a text file with the first line being #!/usr/bin/awk -f and then make the file executable with chmod a+x.
This page of the GNU AWK (gawk) manual describes how to process options as getopt would.
Include the function shown on that page in your script.