This question already has answers here:
Difference between sh and Bash
(11 answers)
Closed 1 year ago.
I'm still figuring out some niches with bash scripting, I get the following error in my bash script:
updated_names=$(comm -23 <(echo ${current_run_names} | tr " " "\n" | sort) <(echo ${started_run_names} | tr " " "\n" | sort) )
updated_names =$(echo ${updated_names} | cut -d' ' -f1)
Not sure what I've been doing wrong, have deleted various tokens but still continue to get the same error.
How I run it
sh conversion.sh -r directory_location
Supposing that the first of these lines ...
updated_run_names=$(comm -23 <(echo ${current_run_names} | tr " " "\n" | sort) <(echo ${started_run_names} | tr " " "\n" | sort) )
update_run_names=$(echo ${new_run_names} | cut -d' ' -f1)
... is the line 72 referenced in the error message, the issue is most likely with your process substitutions. These are the two fragments of the form <( command ).
Process substitution is a Bash extension to the POSIX shell language. It is not recognized by most other shells, and it is not recognized by Bash itself when it runs in POSIX mode. There is more than one way to get Bash running in POSIX mode, but one of them is to invoke it via the name sh, which is what you are doing.
Note well that the shebang line at the top of your script has a functional role only when you launch the script directly, by making it executable and launching it by name:
./conversion.sh -r directory_location
When you instead launch it as shown in the question, by naming it as an argument to sh, the shebang line is just a comment, and Bash runs (as sh) in POSIX mode.
Note also that although it is conventional for some other language interpreters (Python, especially), it is not conventional to use env in shell script shebang lines.
Related
This question already has answers here:
Conditional step in a pipeline
(2 answers)
Can I make a shell function in as a pipeline conditionally "disappear", without using cat?
(1 answer)
Closed 4 months ago.
EMPTY_VAR=''
MMDDYYYY='6.18.1997'
PIPE_VAR=' | xargs echo "1+" | bc'
echo "$MMDDYYYY" | cut -d "." -f 2${EMPTY_VAR}
>> 18
Command above would give me correct output, which is 18, but if I try to use PIPE_VAR instead it would give me bunch of errors:
echo "$MMDDYYYY" | cut -d "." -f 2${PIPE_VAR}
cut: '|': No such file or directory
cut: xargs: No such file or directory
cut: echo: No such file or directory
cut: '"1+"': No such file or directory
cut: '|': No such file or directory
cut: bc: No such file or directory
OR:
echo "$MMDDYYYY" | cut -d "." -f 2"$PIPE_VAR"
cut: invalid field value ‘| xargs echo "1+" | bc’
Try 'cut --help' for more information.
What I'm really trying to find out is that even possible to combine commands like this?
You can't put control operators like | in a variable, at least not without resorting to something like eval. Syntax parsing comes before parameter expansion when evaluating the command line, so Bash is only ever going to see that | as a literal character and not pipeline syntax. See BashParsing for more details.
Conditionally adding a pipeline is hard to do well, but having a part of the pipeline conditionally execute one command or another is more straightforward. It might look something like this:
#!/bin/bash
MMDDYYYY='6.18.1997'
echo "$MMDDYYYY" | cut -d "." -f 2 |
if some_conditional_command ; then
xargs echo "1+" | bc
else
cat
fi
It looks like you're trying to calculate the next day. That's hard to do with plain arithmetic, particularly with month/year ends.
Let date do the work. This is GNU date. It can't parse 6.18.1997 but it can parse 6/18/1997
for MMDDYYYY in '2.28.1996' '2.28.1997'; do
date_with_slashes=${MMDDYYYY//./\/}
next_day=$(date -d "$date_with_slashes + 1 day" '+%-m.%-d.%Y')
echo "$next_day"
done
2.29.1996
3.1.1997
This question already has answers here:
Syntax error in shell script with process substitution
(4 answers)
syntax error near unexpected token `<'
(2 answers)
Closed 4 months ago.
See solution at end of post.
I'm running into a frustrating problem. Thanks for any help you can give. What I'm trying to do is get all occurrences of certain values in an error file. I'm using process substitution to parse error output files. I created a test script with the following code and it works perfectly.
#--------------------
# Get all values of affiliate
echo "Building array of affiliates with errors"
while read -r line; do
AffiliateName_array+=($line)
i=$(( ${#AffiliateName_array[#]} - 1 ))
echo " ScriptName_array | index --> $i | $line"
done < <(grep Running ${LogDir}/temp_python_errors.txt | cut -d' ' -f5 | tr '[:lower:]' '[:upper:]')
The echo output looks like this:
Building array of affiliates with errors
index --> 0 | affiliate1
index --> 1 | affiliate2
Then I cut and pasted that exact code into the final script and it throws the error:
path/AR_exit_process.sh: line 166: syntax error near unexpected token `<'
path/Affiliate_Remits/AR_exit_process.sh: line 166: ` done < <(grep Running ${LogDir}/temp_python_errors.txt | cut -d' ' -f5 | tr '[:lower:]' '[:upper:]')'
Here's the code from my final script (exactly the same as the test script since it was cut and pasted):
#--------------------
# Get all values of affiliate
echo "Building array of affiliates with errors"
while read -r line; do
AffiliateName_array+=($line)
i=$(( ${#AffiliateName_array[#]} - 1 ))
echo " ScriptName_array | index --> $i | $line"
done < <(grep Running ${LogDir}/temp_python_errors.txt | cut -d' ' -f5 | tr '[:lower:]' '[:upper:]')
NOTES:
All scripts have the #!/bin/bash bang line
The scripts run on the same server
The test script is called directly from the command line but
the final script is 2 layers deep. All calling scripts use #!/bin/bash
My main reference:
https://wiki.bash-hackers.org/syntax/expansion/proc_subst
SOLUTION: As Charles suggested I added a line to both the test and final scripts with the command: set -o I found that when running the test script, I got the value posix off. However, when I ran the final script with set -o I got posix on. What this means is that the test script was running under bash while the final script was running under sh. The solution is to add the line set +o posix to turn posix off before executing the code. What I'm still not clear about is how the posix setting is different on the same server. More research ...
i'm using some bash command to extract the current teamviewer id.
Therefore i use this:
#!/bin/bash
OUTPUT="$(teamviewer --info | grep "TeamViewer ID:" | tr -s " " | cut -d ":" -f$
TEAMVIEWERID="${OUTPUT}"
echo $TEAMVIEWERID
mysql --host=xxx --user=xxx --password=xxx xxx$
update table SET teamviewerID="$TEAMVIEWERID" WHERE client="$1";
EOF
echo "DONE"
if i run it:
pi#xxx:~/Documents/xxx/tv $ sudo ./tv.sh client_xxx
4975XXXXX
DONE
pi#xxx:~/Documents/xxx/tv $
ok everything seems to be fine BUT in mysql i receive the following thing:
[0m 4975XXXXX
I'm confused what is happening here...
thx for helping
The characters at the beginning ([0m) are a so called escape sequence. This specific one is used to clear all terminal formattings.
You can easily strip it by using sed.
Just replace your TEAMVIEWERID= line with the following:
TEAMVIEWERID=$(echo "$OUTPUT" | sed 's/\[0m\s//g')
Edit: If the TeamViewer ID always consists of numbers only, we can strip the unknown character by only allowing numbers:
TEAMVIEWERID=$(echo "$id" | sed 's/\[0m\s//g' | sed -re 's/[^0-9]+//g')
This will only allow numbers.
This question already has answers here:
Why does process substitution not work in a shell script?
(1 answer)
Bash script process substitution Syntax error: "(" unexpected
(3 answers)
Closed 5 years ago.
i running a file containning such a command
comm -3 <(cut -f 1 -d' ' <./atoz |sort) <(cut -f 1 -d' '
It succeeds when it was run outside of the file.
But i get error:"file1: 1: file1:Syntax error: "(" unexpected ", when i type
sh file1 file2 and try to run the command from the file1.
The file have no any #!/bin/bash or .sh suffix since i dont have the background to solve this kind of problem..
Does somebody know how to solve it? Thanks a lot
Process substitution (<()) is a bash feature. It is not supported with /bin/sh, which guarantees only features present in the POSIX sh specification (on platforms conformant with 1992-era or newer POSIX specifications; on old ones, it could be 1970s-era Bourne).
Use bash yourscript, or a #!/bin/bash shebang, to run this file.
I have a shell script of more than 1000 lines, i would like to check if all the commands used in the script are installed in my Linux operating system.
Is there any tool to get the list of Linux commands used in the shell script?
Or how can i write a small script which can do this for me?
The script runs successfully on the Ubuntu machine, it is invoked as a part of C++ application. we need to run the same on a device where a Linux with limited capability runs. I have identified manually, few commands which the script runs and not present on Device OS. before we try installing these commands i would like to check all other commands and install all at once.
Thanks in advance
I already tried this in the past and got to the conclusion that is very difficult to provide a solution which would work for all scripts. The reason is that each script with complex commands has a different approach in using the shells features.
In case of a simple linear script, it might be as easy as using debug mode.
For example: bash -x script.sh 2>&1 | grep ^+ | awk '{print $2}' | sort -u
In case the script has some decisions, then you might use the same approach an consider that for the "else" cases the commands would still be the same just with different arguments or would be something trivial (echo + exit).
In case of a complex script, I attempted to write a script that would just look for commands in the same place I would do it myself. The challenge is to create expressions that would help identify all used possibilities, I would say this is doable for about 80-90% of the script and the output should only be used as reference since it will contain invalid data (~20%).
Here is an example script that would parse itself using a very simple approach (separate commands on different lines, 1st word will be the command):
# 1. Eliminate all quoted text
# 2. Eliminate all comments
# 3. Replace all delimiters between commands with new lines ( ; | && || )
# 4. extract the command from 1st column and print it once
cat $0 \
| sed -e 's/\"/./g' -e "s/'[^']*'//g" -e 's/"[^"]*"//g' \
| sed -e "s/^[[:space:]]*#.*$//" -e "s/\([^\\]\)#[^\"']*$/\1/" \
| sed -e "s/&&/;/g" -e "s/||/;/g" | tr ";|" "\n\n" \
| awk '{print $1}' | sort -u
the output is:
.
/
/g.
awk
cat
sed
sort
tr
There are many more cases to consider (command substitutions, aliases etc.), 1, 2 and 3 are just beginning, but they would still cover 80% of most complex scripts.
The regular expressions used would need to be adjusted or extended to increase precision and special cases.
In conclusion if you really need something like this, then you can write a script as above, but don't trust the output until you verify it yourself.
Add export PATH='' to the second line of your script.
Execute your_script.sh 2>&1 > /dev/null | grep 'No such file or directory' | awk '{print $4;}' | grep -v '/' | sort | uniq | sed 's/.$//'.
If you have a fedora/redhat based system, bash has been patched with the --rpm-requires flag
--rpm-requires: Produce the list of files that are required for the shell script to run. This implies -n and is subject to the same limitations as compile time error checking checking; Command substitutions, Conditional expressions and eval builtin are not parsed so some dependencies may be missed.
So when you run the following:
$ bash --rpm-requires script.sh
executable(command1)
function(function1)
function(function2)
executable(command2)
function(function3)
There are some limitations here:
command and process substitutions and conditional expressions are not picked up. So the following are ignored:
$(command)
<(command)
>(command)
command1 && command2 || command3
commands as strings are not picked up. So the following line will be ignored
"/path/to/my/command"
commands that contain shell variables are not listed. This generally makes sense since
some might be the result of some script logic, but even the following is ignored
$HOME/bin/command
This point can however be bypassed by using envsubst and running it as
$ bash --rpm-requires <(<script envsubst)
However, if you use shellcheck, you most likely quoted this and it will still be ignored due to point 2
So if you want to use check if your scripts are all there, you can do something like:
while IFS='' read -r app; do
[ "${app%%(*}" == "executable" ] || continue
app="${app#*(}"; app="${app%)}";
if [ "$(type -t "${app}")" != "builtin" ] && \
! [ -x "$(command -v "${app}")" ]
then
echo "${app}: missing application"
fi
done < <(bash --rpm-requires <(<"$0" envsubst) )
If your script contains files that are sourced that might contain various functions and other important definitions, you might want to do something like
bash --rpm-requires <(cat source1 source2 ... <(<script.sh envsubst))
Based #czvtools’ answer, I added some extra checks to filter out bad values:
#!/usr/bin/fish
if test "$argv[1]" = ""
echo "Give path to command to be tested"
exit 1
end
set commands (cat $argv \
| sed -e 's/\"/./g' -e "s/'[^']*'//g" -e 's/"[^"]*"//g' \
| sed -e "s/^[[:space:]]*#.*\$//" -e "s/\([^\\]\)#[^\"']*\$/\1/" \
| sed -e "s/&&/;/g" -e "s/||/;/g" | tr ";|" "\n\n" \
| awk '{print $1}' | sort -u)
for command in $commands
if command -q -- $command
set -a resolved (realpath (which $command))
end
end
set resolved (string join0 $resolved | sort -z -u | string split0)
for command in $resolved
echo $command
end