I'm running a bash script on OSX Yosemite and Mavericks.
When I run the script the last line seems to be getting cut off part way. The last command loads a daemon - this is how it appears in the script:
sudo -u <USERNAME> launchctl load -w -F /Library/LaunchAgents/ExecuteScriptProcess.plist
It seems that once out of 10 or so times that I run this script the last command will not work. This is the output I'm receiving in the terminal:
: No such file or directoryocess.plist
Note how part of the filename ocess.plist was essentially concatenated to the end of the output and the name of the command does not appear before the colon.
This behaviour seems to show up randomly. Has anyone come across this before?
This is due to carriage returns in your script or data. The bash tag wiki has more information:
Check whether your script or data has DOS style end-of-line characters
Use cat -v yourfile or echo "$yourvariable" | cat -v .
DOS carriage returns will show up as ^M after each line.
If you find them, delete them using dos2unix (a.k.a. fromdos) or tr -d '\r'
Related
I have a niche requirement to run commands stored in another config file within a wrapper script for another utility. My wrapper script (below) works for every command in that config file that does not use "runuser" and also include arguments. If a command uses runuser and my "-c" command includes arguments, the script fails.
The wrapper script
#!/bin/bash
nagios_cmd=$(grep $1 /etc/nagios/nrpe.cfg | awk -F "=" {'print $2'})
exec=$($nagios_cmd)
if [ $? -eq 0 ]
then
#exitok
echo $exec
exit 0
else
#exitcritical
echo $exec
exit 1001
fi
The config file
command[check_crsdb_state]=sudo /usr/lib64/nagios/plugins/check_crsdb_state
command[check_crsasm_state]=sudo /usr/lib64/nagios/plugins/check_crsasm_state
command[check_ora1_tablespace_apex]=sudo /usr/sbin/runuser -l oracle -c '/check_oracle_tablespace APEX 32000'
command[check_ora1_tablespace_lob1]=sudo /usr/sbin/runuser -l oracle -c '/check_oracle_tablespace LOB1 32000'
Successful Script Run
[root#quo-mai-ora1 /]# ./rmmwrapper.sh check_crsasm_state
OK - All nodes report 'Started,STABLE'
[root#quo-mai-ora1 /]#
Failure Script Run
[root#quo-mai-ora1 /]# ./rmmwrapper.sh check_ora1_tablespace_apex
APEX: -c: line 0: unexpected EOF while looking for matching `''
APEX: -c: line 1: syntax error: unexpected end of file
[root#quo-mai-ora1 /]#
Failure Script Run (with bash -x)
[root#quo-mai-ora1 /]# bash -x ./rmmwrapper.sh check_ora1_tablespace_apex
++ grep check_ora1_tablespace_apex /etc/nagios/nrpe.cfg
++ awk -F = '{print $2}'
+ nagios_cmd='sudo /usr/sbin/runuser -l oracle -c '\''/check_oracle_tablespace APEX 32000'\'''
++ sudo /usr/sbin/runuser -l oracle -c ''\''/check_oracle_tablespace' APEX '32000'\'''
APEX: -c: line 0: unexpected EOF while looking for matching `''
APEX: -c: line 1: syntax error: unexpected end of file
+ exec=
+ '[' 1 -eq 0 ']'
+ echo
+ exit 1001
[root#quo-mai-ora1 /]#
The Problem
You can see in the bash -x output, that for some reason when the $nagios_cmd gets executed, it places single quotes before the spaces separate multiple args that are supplied to that resulting script (/check_oracle_tablespace). I've tried different ways of executing $nagios_cmd (using backticks instead etc. I've tried escaping the space characters by modifying the config file to look like this:
command[check_ora1_tablespace_apex]=sudo /usr/sbin/runuser -l oracle -c '/check_oracle_tablespace\ APEX\ 32000'
I've also tried encapsulating the command after -c on runuser in double quotes instead of single, or just no quotes at all.
I'm clearly missing something fundamentally wrong with bash. How can I get the script to just execute the contents of $nagios_cmd as it appears in plain text?
This looks like one of those rare cases where eval is actually the right answer. Try this:
exec=$(eval "$nagios_cmd")
Explanation: bash doesn't expand variables until fairly late in the process of parsing commands, so the string in the variable isn't parsed like it would be if it were actually part of the command. In this case, the problem is that it's expanded after quotes and escapes have been parsed, so it's too late for the single-quotes around the multi-word command to have their intended effect. See BashFAQ #50: "I'm trying to put a command in a variable, but the complex cases always fail!"
What eval does is essentially re-run the entire command parsing process from the beginning. So the variable gets expanded out to the command you want to run (including quotes, etc), and that gets parsed like it would be normally.
Note that I did put double-quotes around the variable; that's so it doesn't go through the partial-parsing process that is done to unquoted variable references, and then through the full parsing process. This one-and-a-half-times parsing process can have rare but really weird effects, so it's best avoided.
Also: eval has a well-deserved bad reputation (I've used the phrase "massive bug magnet" to describe it). This is because what it fundamentally does is treat data (e.g. the contents of variables) as executable code, so it's easy to find that you're doing things like executing parts of your filenames as commands. But in this case, your data is supposed to be a command, and is (hopefully) trusted not to contain malicious, invalid, etc content. In this case, you want the data to be treated as executable code.
I am curling commands for a bash script to run an installation on my linux system and then piping it to the bash console.
curl -G website/bashscript.sh | sudo bash
However the commands I receive from curl has an added
echo "Rebooting"
reboot
at the end.
This curl and sudo command is within my own bash script I am writing for automating an installation process.
How do I remove the echo and reboot command from this text and pipe it to bash? I would rather not save it to a file, edit it, then pass it to bash since I really don't want to remove the text file afterwards.
Thanks!
Remove the last line, if it is the word reboot:
sed '${/^reboot$/d}'
The sed syntax: $ selects the last line. { starts a block. /^reboot$/ selects lines, which contain only the word reboot (^ matches the beginning and $ matches the end of the line). d deletes the line in case of a match. } closes the block.
Remove the last line, if it is the phrase echo "Rebooting":
sed '${/^echo "Rebooting"$/d}'
All together:
curl -G website/bashscript.sh |
sed '${/^reboot$/d}' |
sed '${/^echo "Rebooting"$/d}' |
sudo bash
Another variant with grep, which removes the unwanted commands in any line:
curl … | grep -vx 'echo "Rebooting"
reboot' | …
I need to highlight certain keywords like "fail, failed, error, fatal, missing" over my terminal.
I need this with the output of ALL the commands, not any specific command. I assume I need to tweak my bashrc file for this.
To color I can use:
<input coming to terminal>|grep -P --color=auto 'fail|failed|error|fatal|missing|$'
I tried the following command but not helped:
tail -f $(tty) |grep -P --color=auto 'fail|failed|error|fatal|missing|$' &
[1]+ Stopped(SIGTTIN) tail -f $(tty) | grep -P --color=auto 'fail|failed|error|fatal|missing|$'
Searched SO for answers but could not find any question which provides desired an answer.
I don't think there's really an elegant way to do this using the shell. Ideally, you'd get a terminal emulator with this kind a keyword highlighting built in. You can get some of the way by piping the output of bash through a filter that adds ANSI colour escapes. Here is a sed script, that replaces "fail" with (red)fail(normal):
s/fail/\x1B[31m&\x1B[0m/
t done
:done
Run bash with its output piped through sed like this:
$bash | sed -f color.sed
This mechanism is not without problems, but it works in some cases. Usually it's better just to collect up the output you want, and then pipe it through sed, rather than working directly with the bash output.
I have made an install script which basically inserts some RewriteRule right after RewriteEngine On line by using sed inside a shell script
#!/bin/bash
REWRITE_RULE="RewriteRule \^catalog\/category\/view\/id\/([0-9]+)$ http:\/\/localhost\/app\/#?c=$1 [NE,L,R=301]"
FILE_PATH=".htaccess"
sed -i "s/RewriteEngine on/RewriteEngine on\n\n$REWRITE_RULE/g" $FILE_PATH
it does nothing but gives some error like
: No such file or directory
I tried same commands in shell and it worked without any issues and updated my .htaccess file
I am new to shell scripting so don't know the difference of using same command through shell and through script.. please guide
The script itself is fine. Are you sure that the lines posted above are exactly as you took them from your script?
The error message indicates that you provided an empty name as the filename, thus I suggest you put a
echo "FILE_PATH: $FILE_PATH"
directly before your sed command in order to check whether the variable is set correctly.
You'll find that your script contains carriage returns due to being saved with DOS end-of-line characters. Use tr -d '\r' <brokenscript >fixedscript to fix your script.
Here's a copy of a session with the problem reproduced and fixed:
$ cat file
var=foo
$ cat myscript
sed -i s/foo/bar/ file
$ bash myscript
: No such file or directory
$ shellcheck myscript
In myscript line 1:
sed -i s/foo/bar/ file
^-- SC1017: Literal carriage return. Run script through tr -d '\r' .
$ tr -d '\r' < myscript > fixedscript
$ bash fixedscript
$ cat file
var=bar
$
I am a windows user having basic idea about LINUX and i encountered this command:
cat countryInfo.txt | grep -v "^#" >countryInfo-n.txt
After some research i found that cat is for concatenation and grep is for regular exp search (don't know if i am right) but what will the above command result in (since both are combined together) ?
Thanks in Advance.
EDIT: I am asking this as i dont have linux installed. Else, i could test it.
Short answer: it removes all lines starting with a # and stores the result in countryInfo-n.txt.
Long explanation:
cat countryInfo.txt reads the file countryInfo.txt and streams its content to standard output.
| connects the output of the left command with the input of the right command (so the right command can read what the left command prints).
grep -v "^#" returns all lines that do not (-v) match the regex ^# (which means: line starts with #).
Finally, >countryInfo-n.txt stores the output of grep into the specified file.
It will remove all lines starting with # and put the output in countryInfo-n.txt
This command would result in removing lines starting with # from the file countryInfo.txt and place the output in the file countryInfo-n.txt.
This command could also have been written as
grep -v "^#" countryInfo.txt > countryInfo-n.txt
See Useless Use of Cat.