I have a wierd situation. I am trying to cut some info from a file and everything works fine when I run the command straight into the terminal, but as soon as I make it a variable in a script it returns a mixture of what it should cut and a list of the files in the current directory.
cat query.sql | cut -d':' -f3,4
works but...
QUERY_SQL="query.sql"
MYSQL_COMMAND=`cat $QUERY_SQL | cut -d':' -f3,4`
echo $MYSQL_COMMAND
returns the wierd output mentioned above.
What am I doing wrong?
EDIT:
The query file looks something like this...
email#somehwhere.com:3:SQL CODE
I suspect something in the contents of MYSQL_COMMAND is being interpreted as a filename glob pattern. Try changing
MYSQL_COMMAND=`cat $QUERY_SQL | cut -d':' -f3,4`
echo $MYSQL_COMMAND
to
MYSQL_COMMAND="$(cut -d: -f3,4 < "$QUERY_SQL")"
printf '%s\n' "$MYSQL_COMMAND"
Best defensive coding practice for shell is to put double quotes around every variable substitution, unless you know for a fact that you need word splitting and glob expansion to happen after a particular substitution. Changing echo to printf '%s\n' avoids a related set of problems. I can never remember whether you actually need double quotes around $(...) in a variable assignment, so I put them in just to be safe.
Related
I'm in the process of switching from zsh to bash, and I need to produce a bash script that can remove duplicate entries in $PATH without reordering the entries (thus no sort -d magic). zsh has some nice array handling shortcuts that made it easy to do this efficiently, but I'm not aware of such shortcuts in bash. I came across this answer which has gotten me 90% of the way there, but there is a small problem that I would like to understand better. It appears that when I run that awk command, the last record processed incorrectly matches the pattern.
$ awk 'BEGIN{RS=ORS=":"}!a[$0]++' <<<"aa:bb:cc:aa:bb:cc"
aa:bb:cc:cc
$ awk 'BEGIN{RS=ORS=":"}!a[$0]++' <<<"aa:bb:cc:aa:bb"
aa:bb:cc:bb
$ awk 'BEGIN{RS=ORS=":"}!a[$0]++' <<<"aa:bb:cc:aa:bb:cc:" # note trailing colon
aa:bb:cc:
I don't understand awk well enough to know why it behaves this way, but I have managed to work around the issue by using an intermediate array like so.
array=($(awk 'BEGIN{RS=":";ORS=" "}!a[$0]++' <<<"aa:bb:cc:aa:bb:cc:"))
# Use a subshell to avoid modifying $IFS in current context
echo $(export IFS=":"; echo "${array[*]}")
aa:bb:cc
This seems like a sub-optimal solution however, so my question is: did I do something wrong in the awk command that is causing false positive matches on the final record processed?
The last record in your original string is cc\n which is different from cc. When unsure what's happening in any program in any language, adding some print statements is step 1 to debugging/investigating:
$ awk 'BEGIN{RS=ORS=":"} {print "<"$0">"}' <<<"aa:bb:cc:aa:bb:cc"
<aa>:<bb>:<cc>:<aa>:<bb>:<cc
>:$
If you want the RS to be : or \n then just state that (with GNU awk at least):
$ awk 'BEGIN{RS="[:\n]"; ORS=":"} !a[$0]++' <<<"aa:bb:cc:aa:bb:cc"
aa:bb:cc:$
The $ in all of the above is my prompt.
Another possible workaround instead of your bash array solution
$ echo "aa:bb:cc:aa:bb:cc" | tr ':' '\n' | awk '!a[$0]++' | paste -sd:
aa:bb:cc
I'm trying to follow a guide found here, but I do not like what I see as a cop out. In the script they set PS1 to
PS1="<code> `cat /proc/loadavg | awk '{print $1}'` <more code>"
My problem with this is I would like to know if it is possible to write it with single quotes like:
PS1='<code> `cat /proc/loadavg | awk \'{print $1}\'` <more code>'
So it is evaluated every time I run the command, not just the once. It seems the presence of the single quotes in awk are forcing me to use double quotes. I would like to have this run after every prompt and I have another awk tidbit of code I would like to run here as well.
If this would be too cumbersome for bash to do, then I'm fine not having it, it's more for proof of concept anyways.
You can't put a single quote into single quotes, you have to end the single quotes, insert the quote, and start single quotes again:
PS1='$(code | awk '\''{print $1}'\'')'
# or
PS1='$(code | awk '"'"'{print $1}'"'"')'
This question already has answers here:
How do I split a string on a delimiter in Bash?
(37 answers)
Closed 4 years ago.
I am new to shell scripting, can you please help with below requirement, thanks.
$AU_NAME=AU_MSM3-3.7-00.01.02.03
#separate the string after last "-", with "." as delimiter
#that is, separate "00.01.02.03" and print/save as below.
major=00
minor=01
micro=02
build=03
First, note that you don't use $ when assigning to a parameter in the shell. Your first line should be just this:
AU_NAME=AU_MSM3-3.7-00.01.02.03
The $ is used to get the value of the parameter once assigned. And the bit after the $ can be an expression in curly braces with extra stuff besides just the name, allowing you to perform various operations on the value. For example, you can do something like this:
IFS=. read major minor micro build <<EOF
${AU_NAME##*-}
EOF
where the ##*- strips off everything from the beginning of the string through the last '-', leaving just "00.01.02.03", and the IFS (Internal Field Separator) parameter tells the shell where to break the string into fields.
In bash, zsh, and ksh93+, you can get that onto one line by shortening the here-document to a here-string:
IFS=. read major minor micro build <<<"${AU_NAME##*-}"
More generally, in those same shells, you can split into an arbitrarily-sized array instead of distinct variables:
IFS=. components=(${AU_NAME##*-})
(Though that syntax won't work in especially-ancient versions of ksh; in them you have to do this instead:
IFS=. set -A components ${AU_NAME##*-}
)
That gets you this equivalence (except in zsh, which by default numbers the elements 1-4 instead of 0-3):
major=${components[0]}
minor=${components[1]}
micro=${components[2]}
build=${components[3]}
In bash, you can do something like this:
version=$(echo $AU_NAME | grep -o '[^-]*$')
major=$(echo $version | cut -d. -f1)
minor=$(echo $version | cut -d. -f2)
micro=$(echo $version | cut -d. -f3)
build=$(echo $version | cut -d. -f4)
The grep call uses -o which outputs only the matching part of the line. The match itself is every non-hyphen character to the end of the line.
The cut command uses the delimeter . (-d.), and uses -f to select individual fields.
It's a little clunky. I'm sure there are probably better ways to achieve this, but you can do quite a lot with grep and cut alone so they're handy tools to have in your arsenal.
You can use parameter expansion and the special IFS variable.
#! /bin/bash
AU_NAME=AU_MSM3-3.7-00.01.02.03
IFS=. VER=(${AU_NAME##*-})
for i in {0..3} ; do
echo ${VER[i]}
done
major=${VER[0]}
minor=${VER[1]}
micro=${VER[2]}
build=${VER[3]}
BTW, in an assignment, do not start the variable on the left hand side with a dollar sign.
I am trying to trim this, which is stored in a variable called $line.
[2012-06-18 10:37:09,026 (there is a lot of text after this, i just cut it out)
I am new to shell scripting and this is the code that i have
errortime= $line | cut -c2-10;
it is giving me an error, what is the correct code to pull the date out of the variable $line.
did you want:
errortime=`echo $line | cut -c2-20`
instead?
Edit:
If you are using ksh, that line needs to look like this:
errortime=$(echo $line | cut -c2-20)
I am trying to assign output of a cut command to a variable, however I am running into a strange problem.
I am using tcsh shell.
$echo $0
tcsh
This is the command I am running:
$set a=`cut -f2 -d' ' test.txt`
Missing }. //This is the output I am getting
Now the file is real simple (well this is the not the file I was working on but I reduced the problem to this.)
Test.txt:
{ {corner
Thats it! This is the file. If I change the file to this:
{ {corner}
Statement works but "a" gets the following value:
$echo $a
corner //Please note its not {corner} but corner
Hence I think that shell is trying to execute {corner
as a command and since its missing the closing brace shell complains. Does anyone have any idea why its showing this behavior? My understanding is that it should just assign the output of cut to the variable but looks like its assigning it recursively!
Newbie
You have to wrap it around double quotes
set a="`cut -f2 -d' ' test.txt`"
Same applies to uses such as echo
echo "$a"
Output
{corner