There is such a script:
NAME = `echo "$QUERY_STRING" | sed -n 's/^.*post=\([^&]*\).*$/\1/p' | sed "s/%20/ /g"`
RES = `psql -U user -d db -t -c "SELECT tabl FROM tablica WHERE name = '$NAME'"`
echo $RES
Everything works fine (it means GET requests are fine). But the data from the database do not go.
The problem is that the value of the parameter in the query WHERE NAME is not being set, and I get a syntax error.
I have read many articles on the Internet, but found nothing about a variable inside backticks.
How can I fix this?
You're not allowed to put spaces around the equal sign in variable assignments. And you should generally not use backticks, but prefer the $() form, it is easier to deal with quoting with it.
NAME=$(echo "$QUERY_STRING" | sed -n 's/^.*post=\([^&]*\).*$/\1/p' | sed "s/%20/ /g")
RES=$(psql -U user -d db -t -c "SELECT tabl FROM tablica WHERE name = '$NAME'")
echo "$RES"
Note that what you're doing is pretty insecure, you need stronger validation for your inputs.
Related
I always write some magic numbers in my interactive shells and shell scripts.
For instance, If I want to list my users's names and shells, I'll write
cut --delimiter=: --fields=1,7 /etc/passwd
There exist two magic-numbers 1,7. And there are more and more magic-numbers in other circumstances.
Question
How to avoid magic-numbers in interactive shells and shell scripts?
Supplementary background
Our teacher told us using cut -d: -f1,7 /etc/passwd. But for new linux-users, they don't konw what's meaning of d,f,1,7.(not just for new linux-users，the whole system has so many configuration files that it is not easy for a person to remember every magic-numbers)
So, in interactive shells, we can use --delimiter, --fields,and the bash repl(or zsh,fish) has good tab completion to it.
How about the 1 and 7? In shell scripts, It's a good method to declare some const variables like LoginField=1 and ShellField=7 after reading the man 5 passwd. But when some one is writing in the interactive shells, it's not a good idea to open a new window and search the constants of LoginField=1,ShellField=7 and define it. how to using some thing like tab completion to simplify operations?
Use variables:
LoginField=1 ShellField=7
cut --delimiter=: --fields="$LoginField,$ShellField" /etc/passwd
Just like in other languages - by using variables. Example:
$ username_column=1
$ shell_column=7
$ cut --delimiter=: --fields="$username_column","$shell_column" /etc/passwd
The variables may be defined at the top of the script so that can be
easily modified or they can be set in an external config-like file
shared by multiple scripts.
The classic way to parse /etc/passwd is to read each column into an appropriately named variable:
while IFS=: read name passwd uid gid gecos home shell _; do
...
done < /etc/passwd
Use export:
export field_param="1,7"
(you can put it .bashrc file to have configured each time shell session is started). This export can be part of .sh script. It's a good practice to put them in the head/top of the file.
Then:
cut --delimiter=: --fields=$field_param /etc/passwd
This way you will need to edit the magic number in the only location.
Continuing from my comment, it's hard to tell exactly what you are asking. If you just want to give meaningful variable names, then do as shown in the other answers.
If however you want to be able to specify which fields are passed to cut from the command line, then you can use the positional parameters $1 and $2 to pass those values into your script.
You need to validate that two inputs are given and that both are integers. You can do that with a few simple tests, e.g.
#!/bin/bash
[ -n "$1" ] && [ -n "$2" ] || { ## validate 2 parameters given
printf "error: insufficient input\nusage: %s field1 field2\n" "${0##*/}"
exit 1
}
## validate both inputs are integer values
[ "$1" -eq "$1" >/dev/null 2>&1 ] || {
printf "error: field1 not integer value '%s'.\n" "$1"
exit 1
}
[ "$2" -eq "$2" >/dev/null 2>&1 ] || {
printf "error: field2 not integer value '%s'.\n" "$2"
exit 1
}
cut --delimiter=: --fields=$1,$2 /etc/passwd
Example Use/Output
$ bash fields.sh
error: insufficient input
usage: fields.sh field1 field2
$ bash fields.sh 1 d
error: field2 not integer value 'd'.
$ bash fields.sh 1 7
root:/bin/bash
bin:/usr/bin/nologin
daemon:/usr/bin/nologin
mail:/usr/bin/nologin
ftp:/usr/bin/nologin
http:/usr/bin/nologin
uuidd:/usr/bin/nologin
dbus:/usr/bin/nologin
nobody:/usr/bin/nologin
systemd-journal-gateway:/usr/bin/nologin
systemd-timesync:/usr/bin/nologin
systemd-network:/usr/bin/nologin
systemd-bus-proxy:/usr/bin/nologin
<snip>
Or if you choose to look at fields 1 and 3, then all you need do is pass those as the parameters, e.g.
$ bash fields.sh 1 3
root:0
bin:1
daemon:2
mail:8
ftp:14
http:33
uuidd:68
dbus:81
nobody:99
systemd-journal-gateway:191
systemd-timesync:192
systemd-network:193
systemd-bus-proxy:194
<snip>
Look things over and let me know if you have further questions.
Scraping the output of man 5 passwd for human-readable header names:
declare $(man 5 passwd |
sed -n '/^\s*·\s*/{s/^\s*·\s*//;y/ /_/;p}' |
sed -n 'p;=' | paste -d= - - )
See "how it works" below for what that does, then run:
cut --delimiter=: \
--fields=${login_name},${optional_user_command_interpreter} /etc/passwd
Which outputs the specified /etc/passwd fields.
How it works.
The man page describing /etc/passwd contains a bullet list of header names. Use GNU sed to find the bullets (·) and leading whitespace, then remove the bullets and whitespace, replace the remaining spaces with underlines; a 2nd instance of sed provides fresh line numbers, then paste the header names to the line numbers, with a = between:
man 5 passwd |
sed -n '/^\s*·\s*/{s/^\s*·\s*//;y/ /_/;p}' |
sed -n 'p;=' | paste -d= - -
Outputs:
login_name=1
optional_encrypted_password=2
numerical_user_ID=3
numerical_group_ID=4
user_name_or_comment_field=5
user_home_directory=6
optional_user_command_interpreter=7
And declare makes those active in the current shell.
I have this bash script:
databases=`mysql -h$DBHOST -u$DBUSER -p$DBPASSWORD -e "SHOW DATABASES;" | tr -d "| " | grep -v Database`
and the issue is when the password has all the characters possible. how can i escape the $DBPASSWORD in this case? If I have a password with '!' and given the fact that command is inside backticks. I have no experience in bash scripts but I've tried with "$DBPASSWORD" and with '$DBPASSWORD' and it doesn't work. Thank you
LATER EDIT: link to script here, line 170 -> https://github.com/Ardakilic/backmeup/blob/master/backmeup.sh
First: The answer from #bishop is spot on: Don't pass passwords on the command line.
Second: Use double quotes for all shell expansions. All of them. Always.
databases=$(mysql -h"$DBHOST" -u"$DBUSER" -p"$DBPASSWORD" -e "SHOW DATABASES;" | tr -d "| " | grep -v Database)
Don't pass the MySQL password on the command line. One, it can be tricky with passwords containing shell meta-characters (as you've discovered). Two, importantly, someone using ps can sniff the password.
Instead, either put the password into the system my.cnf, your user configuration file (eg .mylogin.cnf) or create an on-demand file to hold the password:
function mysql() {
local tmpfile=$(mktemp)
cat > "$tmpfile" <<EOCNF
[client]
password=$DBPASSWORD
EOCNF
mysql --defaults-extra-file="$tmpfile" -u"$DBUSER" -h"$DBHOST" "$#"
rm "$tmpfile"
}
Then you can run it as:
mysql -e "SHOW DATABASES" | tr -d "| " ....
mysql -e "SELECT * FROM table" | grep -v ...
See the MySQL docs on configuration files for further examples.
I sometimes have the same problem when automating activities:
I have a variable containing a string (usually a password) that is set in a config file or passed on the command-line, and that string includes the '!' character.
I need to pass that variable's value to another program, as a command-line argument.
If I pass the variable unquoted, or in double-quotes ("$password"), the shell tries to interpret the '!', which fails.
If I pass the variable in single quotes ('$password'), the variable isn't expanded.
One solution is to construct the full command in a variable and then use eval, for example:
#!/bin/bash
username=myuser
password='my_pass!'
cmd="/usr/bin/someprog -user '$username' -pass '$password'"
eval "$cmd"
Another solution is to write the command to a temporary file and then source the file:
#!/bin/bash
username=myuser
password='my_pass!'
cmd_tmp=$HOME/.tmp.$$
touch $cmd_tmp
chmod 600 $cmd_tmp
cat > $cmd_tmp <<END
/usr/bin/someprog -user '$username' -pass '$password'
END
source $cmd_tmp
rm -f $cmd_tmp
Using eval is simple, but writing a file allows for multiple complex commands.
P.S. Yes, I know that passing passwords on the command-line isn't secure - there is no need for more virtue-signalling comments on that topic.
I have a shell script of more than 1000 lines, i would like to check if all the commands used in the script are installed in my Linux operating system.
Is there any tool to get the list of Linux commands used in the shell script?
Or how can i write a small script which can do this for me?
The script runs successfully on the Ubuntu machine, it is invoked as a part of C++ application. we need to run the same on a device where a Linux with limited capability runs. I have identified manually, few commands which the script runs and not present on Device OS. before we try installing these commands i would like to check all other commands and install all at once.
Thanks in advance
I already tried this in the past and got to the conclusion that is very difficult to provide a solution which would work for all scripts. The reason is that each script with complex commands has a different approach in using the shells features.
In case of a simple linear script, it might be as easy as using debug mode.
For example: bash -x script.sh 2>&1 | grep ^+ | awk '{print $2}' | sort -u
In case the script has some decisions, then you might use the same approach an consider that for the "else" cases the commands would still be the same just with different arguments or would be something trivial (echo + exit).
In case of a complex script, I attempted to write a script that would just look for commands in the same place I would do it myself. The challenge is to create expressions that would help identify all used possibilities, I would say this is doable for about 80-90% of the script and the output should only be used as reference since it will contain invalid data (~20%).
Here is an example script that would parse itself using a very simple approach (separate commands on different lines, 1st word will be the command):
# 1. Eliminate all quoted text
# 2. Eliminate all comments
# 3. Replace all delimiters between commands with new lines ( ; | && || )
# 4. extract the command from 1st column and print it once
cat $0 \
| sed -e 's/\"/./g' -e "s/'[^']*'//g" -e 's/"[^"]*"//g' \
| sed -e "s/^[[:space:]]*#.*$//" -e "s/\([^\\]\)#[^\"']*$/\1/" \
| sed -e "s/&&/;/g" -e "s/||/;/g" | tr ";|" "\n\n" \
| awk '{print $1}' | sort -u
the output is:
.
/
/g.
awk
cat
sed
sort
tr
There are many more cases to consider (command substitutions, aliases etc.), 1, 2 and 3 are just beginning, but they would still cover 80% of most complex scripts.
The regular expressions used would need to be adjusted or extended to increase precision and special cases.
In conclusion if you really need something like this, then you can write a script as above, but don't trust the output until you verify it yourself.
Add export PATH='' to the second line of your script.
Execute your_script.sh 2>&1 > /dev/null | grep 'No such file or directory' | awk '{print $4;}' | grep -v '/' | sort | uniq | sed 's/.$//'.
If you have a fedora/redhat based system, bash has been patched with the --rpm-requires flag
--rpm-requires: Produce the list of files that are required for the shell script to run. This implies -n and is subject to the same limitations as compile time error checking checking; Command substitutions, Conditional expressions and eval builtin are not parsed so some dependencies may be missed.
So when you run the following:
$ bash --rpm-requires script.sh
executable(command1)
function(function1)
function(function2)
executable(command2)
function(function3)
There are some limitations here:
command and process substitutions and conditional expressions are not picked up. So the following are ignored:
$(command)
<(command)
>(command)
command1 && command2 || command3
commands as strings are not picked up. So the following line will be ignored
"/path/to/my/command"
commands that contain shell variables are not listed. This generally makes sense since
some might be the result of some script logic, but even the following is ignored
$HOME/bin/command
This point can however be bypassed by using envsubst and running it as
$ bash --rpm-requires <(<script envsubst)
However, if you use shellcheck, you most likely quoted this and it will still be ignored due to point 2
So if you want to use check if your scripts are all there, you can do something like:
while IFS='' read -r app; do
[ "${app%%(*}" == "executable" ] || continue
app="${app#*(}"; app="${app%)}";
if [ "$(type -t "${app}")" != "builtin" ] && \
! [ -x "$(command -v "${app}")" ]
then
echo "${app}: missing application"
fi
done < <(bash --rpm-requires <(<"$0" envsubst) )
If your script contains files that are sourced that might contain various functions and other important definitions, you might want to do something like
bash --rpm-requires <(cat source1 source2 ... <(<script.sh envsubst))
Based #czvtools’ answer, I added some extra checks to filter out bad values:
#!/usr/bin/fish
if test "$argv[1]" = ""
echo "Give path to command to be tested"
exit 1
end
set commands (cat $argv \
| sed -e 's/\"/./g' -e "s/'[^']*'//g" -e 's/"[^"]*"//g' \
| sed -e "s/^[[:space:]]*#.*\$//" -e "s/\([^\\]\)#[^\"']*\$/\1/" \
| sed -e "s/&&/;/g" -e "s/||/;/g" | tr ";|" "\n\n" \
| awk '{print $1}' | sort -u)
for command in $commands
if command -q -- $command
set -a resolved (realpath (which $command))
end
end
set resolved (string join0 $resolved | sort -z -u | string split0)
for command in $resolved
echo $command
end
I am trying to just echo a command within my bash script code.
OVERRUN_ERRORS="$ifconfig | egrep -i "RX errors" | awk '{print $7}'"
echo ${OVERRUN_ERRORS}
however it gives me an error and the $7 does not show up in the command. I have to store it in a variable, because I will process the output (OVERRUN_ERRORS) at a later point in time. What's the right syntax for doing this? Thanks.
On Bash Syntax
foo="bar | baz"
...is assigning the string "bar | baz" to the variable named foo; it doesn't run bar | baz as a pipeline. To do that, you want to use command substitution, in either its modern $() syntax or antiquated backtick-based form:
foo="$(bar | baz)"
On Storing Code For Later Execution
Since your intent isn't clear in the question --
The correct way to store code is with a function, whereas the correct way to store output is in a string:
# store code in a function; this also works with pipelines
get_rx_errors() { cat /sys/class/net/"$1"/statistics/rx_errors; }
# store result of calling that function in a string
eth0_errors="$(get_rx_errors eth0)"
sleep 1 # wait a second for demonstration purposes, then...
# compare: echoing the stored value, vs calculating a new value
echo "One second ago, the number of rx errors was ${eth0_errors}"
etho "Right now, it is $(get_rx_errors eth0)"
See BashFAQ #50 for an extended discussion of the pitfalls of storing code in a string, and alternatives to same. Also relevant is BashFAQ #48, which describes in detail the security risks associated with a eval, which is often suggested as a workaround.
On Collecting Interface Error Counts
Don't use ifconfig, or grep, or awk for this at all -- just ask your kernel for the number you want:
#!/bin/bash
for device in /sys/class/net/*; do
[[ -e $device/statistics/rx_errors ]] || continue
rx_errors=$(<"${device}/statistics/rx_errors")
echo "Number of rx_errors for ${device##*/} is $rx_errors"
done
Use $(...) to capture the output of a command, not double quotes.
overrun_errors=$(ifconfig | egrep -i "RX errors" | awk '{print $7}')
Your double quotes around RX errors are a problem. Try;
OVERRUN_ERRORS="$ifconfig | egrep -i 'RX errors' | awk '{print $7}'"
To see the commands as they are executing, you can use
set -v
or
set -x
For example;
set -x
OVERRUN_ERRORS="$ifconfig | egrep -i 'RX errors' | awk '{print $7}'"
set +x
The task requires that a bash script be written that will search the "who" command for a given user ID which will be provided via command line argument
This script will display whether or not this user ID is logged in
So far I know that to get the user ID, one can do:
who | cut -d' ' -f1 | grep "userIdToSearchFor"
This grep will display the user ID if it exists, or nothing if it doesn't, so it seems like a good method
I believe the $1 variable will hold the first command line argument
How can I implement this in a bash script file please?
EDIT:
Current working script looks like this
#!/bin/bash
userid=$(who | cut -d' ' -f1 | grep "$1")
if [ "$1" == "$userid" ]
then
echo "online"
else
echo "offline"
fi
This should work for you :
STRING=$(who | cut -d' ' -f1 | grep "$1")
if [ "$1" = "$STRING" ]
then
echo "online"
else
echo "offline"
fi
Some comments and suggestions :
No spaces on both sides of the = when you assign variables (that's where your error message come from).
To assign commands result to a variable, you must use the $( ) syntax. See command substitution for more.
Quote you vars in your test to prevent word splitting.
You should loop on the test, there could be multiple identical usernames.
Avoid caps in you variable names not to confuse with environment variables which are capitalized by convention.
Avoid to use the type of the var for its name, in your case username would be a better choice.
You're doing it the hard way.
$ cat user.sh
#!/bin/bash
# user.sh username - shows whether username is logged on or not
if who | grep --silent "^$1 " ; then
echo online
else
echo offline
fi
$ ./user.sh msw
online