In order to identify if my installation has errors that I should notice, I am using grep command on the file and write the file using tee because I need to elevate permissions.
sudo grep -inw ${LOGFOLDER}/$1.log -e "failed" | sudo tee -a ${LOGFOLDER}/$1.errors.log
sudo grep -inw ${LOGFOLDER}/$1.log -e "error" | sudo tee -a ${LOGFOLDER}/$1.errors.log
The thing is that the file is created even if the grep didn't find anything.
Is there any way I can create the file only if the grep found a match ?
Thanks
You may replace tee with awk, it won't create file if there is nothing to write to it:
... | sudo awk "{print; print \$0 >> \"errors.log\";}"
But such feature of awk is rarely used. I'd rather remove empty error file if nothing is found:
test -s error.log || rm -f error.log
And, by the way, you may grep for multiple words simultaneously:
grep -E 'failed|error' ...
Related
I have below script which will append the file awslogs.conf on search of folder names under directory but if I re-execute this script, this appends again. How can I avoid doing this?
for NAME in $(ls -1p /home/ec2-user/src/ |grep -v "^_" | grep -v "/$" |cut -d. -f1);
do
sudo tee -a /etc/awslogs/awslogs.conf << END
[$NAME]
datetime_format = %b %d %H:%M:%S
file = /var/log/airflow/$NAME/*/*/*.log
buffer_duration = 5000
log_stream_name = $NAME
initial_position = start_of_file
log_group_name = CLOUDWATCH_LOGS
END
done
sudo service awslogsd start
sudo systemctl enable awslogsd
The tee command by default replaces the content of the file. -a overrides that to append instead. If you want to replace the content simply remove -a from the command.
If you want to append the content only if the file does not already contain it, that's an entirely different proposition. You could for example pull out the contents as a multi-line string and check for the entire thing using GNU grep:
contents="[$NAME]
datetime_format …"
if ! grep --fixed-strings --null --quiet "$contents" /etc/awslogs/awslogs.conf
then
tee -a …
I have a script which run this command successfully. I am using this command in another script which gives me error on this line (.md5: Permission denied).
I am running the previous script with sudo.
for i in ${NAME}*
do
sudo md5sum $i | sed -e "s/$i/${NAME}/" > ${NAME}.md5${i/#${NAME}/}
done
So you want to redirect output as root. It doesn't matter that you executed the command with sudo, because redirection is not part of the execution, so it's not performed by the executing user of the command, but by your current user.
The common trick is to use tee:
for i in ${NAME}*
do
md5sum $i | sed -e "s/$i/${NAME}/" | sudo tee ${NAME}.md5${i/#${NAME}/}
done
Note: I dropped the sudo from md5sum, as probably you don't need it.
Note: tee outputs in two directions: the specified file and stdout. If you want to suppress the output on stdout, redirect it to /dev/null.
You take the output of sudo md5sum $i and pipe it to a sed which is not running as root. sudo doesn't even know this sed exists.
But that's not the problem, because the sed does not need root permissions. The problem is > ${NAME}.... This redirects the output of sed to the file with this name. But the redirection is actually executed by your shell which is running as your user. And because > is a shell built-in operator, you can not prefix it with sudo.
The simple solution is to use tee. tee is a program (so you can run it with sudo) which writes it's input to the standard output and also to a file (like a T-Pipe, hence the name).
So you can just:
for i in ${NAME}*
do
md5sum $i | sed -e "s/$i/${NAME}/" | sudo tee ${NAME}.md5${i/#${NAME}/}
done
Note this will also dump all hashes to your standard output.
I'm building a little bash script to run another bash script that's found in multiple directories. Here's the code:
cd /home/mainuser/CaseStudies/
grep -R -o --include="Auto.sh" [\w] | wc -l
When I execute just that part, it finds the same file 5 times in each folder. So instead of getting 49 results, I get 245. I've written a recursive bash script before and I used it as a template for this problem:
grep -R -o --include=*.class [\w] | wc -l
This code has always worked perfectly, without any duplication. I've tried running the first code with and without the " ", I've tried -r as well. I've read through the bash documentation and I can't seem to find a way to prevent, or even why I'm getting, this duplication. Any thoughts on how to get around this?
As a separate, but related question, if I could launch Auto.sh inside of each directory so that the output of Auto.sh was dumped into that directory; without having to place Auto.sh in each folder. That would probably be much more efficient that what I'm currently doing and it would also probably fix my current duplication problem.
This is the code for Auto.sh:
#!/bin/bash
index=1
cd /home/mainuser/CaseStudies/
grep -R -o --include=*.class [\w] | wc -l
grep -R -o --include=*.class [\w] |awk '{print $3}' > out.txt
while read LINE; do
echo 'Path '$LINE > 'Outputs/ClassOut'$index'.txt'
javap -c $LINE >> 'Outputs/ClassOut'$index'.txt'
index=$((index+1))
done <out.txt
Preferably I would like to make it dump only the javap outputs for the application its currently looking at. Since those .class files could be in any number of sub-directories, I'm not sure how to make them all dump in the top folder, without executing a modified Auto.sh in the top directory of each application.
Ok, so to fix the multiple find:
grep -R -o --include="Auto.sh" [\w] | wc -l
Should be:
grep -R -l --include=Auto.sh '\w' | wc -l
The reason this was happening, was that it was looking for instances of the letter w in Auto.sh. Which occurred 5 times in the file.
However, the overall fix that doesn't require having to place Auto.sh in every directory, is something like this:
MAIN_DIR=/home/mainuser/CaseStudies/
cd $MAIN_DIR
ls -d */ > DirectoryList.txt
while read LINE; do
cd $LINE
mkdir ProjectOutputs
bash /home/mainuser/Auto.sh
cd $MAIN_DIR
done <DirectoryList.txt
That calls this Auto.sh code:
index=1
grep -R -o --include=*.class '\w' | wc -l
grep -R -o --include=*.class '\w' | awk '{print $3}' > ProjectOutputs.txt
while read LINE; do
echo 'Path '$LINE > 'ProjectOutputs/ClassOut'$index'.txt'
javap -c $LINE >> 'ProjectOutputs/ClassOut'$index'.txt'
index=$((index+1))
done <ProjectOutputs.txt
Thanks again for everyone's help!
This script looks for all users that have the string RECHERCHE inside them. I tried running it in sudo and it worked, but then stopped at line 8 (permission denied). Even when removing the sudo from the script, this issue still happens.
#!/bin/bash
#challenge : user search and permission rewriting
echo -n "Enter string to search : "
read RECHERCHE
echo $(cat /etc/passwd | grep "/home" | cut -d: -f5 | grep -i "$RECHERCHE" | sed s/,//g)
echo "Changing permissions"
export RECHERCHE
sudo ./challenge2 $(/etc/passwd) &
The second script then changes permissions of each file belonging to each user that RECHERCHE found, in the background. If you could help me figure out what this isn't doing right, it would be of great service. I
#!/bin/bash
while read line
do
if [-z "$(grep "/home" | cut -d: -f5 | grep -i "$RECHERCHE")" ]
then
user=$(cut -f: -f1)
file=$(find / -user $(user))
if [$(stat -c %a file) >= 700]
then
chmod 700 file 2>> /home/$(user)/challenge.log
fi
if [$(stat -c %a file) < 600]
then
chmod 600 file 2>> /home/$(user)/challenge.log
fi
umask 177 2>> /home/$(user)/challenge.log
fi
done
I have to idea what I'm doing.
the $(...) syntax means command substitution, that is: it will be replaced by the output of the command within the paranthesis.
since /etc/passwd is no command but just a text-file, you cannot execute it.
so if you want to pass the contents of /etc/passwd to your script, you would just call it:
./challenge2 < /etc/passwd
or, if you need special permissions to read the file, something like
sudo cat /etc/passwd | ./challenge2
also in your challenge2 script, you are using $(user) which is wrong as you really only want to expand the user variable: use curly braces for this, like ${user}
/etc/passwd?
not what you were asking, but you probably should not read /etc/passwd directly anyhow.
if you want to get a list of users, use the following command:
$ getent passwd
this will probably give you more users than those stored in /etc/passwd, as your system might use other PAM backends (ldap,...)
I am trying to run a command in a shell script but it is not working.
Out side of the script in the shell I can run the following command on the needed host. The file is created with the correct information inside.
sudo cat /etc/shadow | cut -d: -f1,8 | sed /:$/d > /tmp/expirelist.txt
When the command is run in my script I first ssh over then run the command but I get the following error.
[batch#testserver01 bin]$ checkP.sh
Testserver02
/usr/local/bin/checkP.sh: line 7: /tmp/expirelist.txt: Permission denied
Here is a part of the script. I have tried using ssh -o
#!/bin/bash
for SERVER in `cat /admin/lists/testlist`
do
echo $SERVER
ssh $SERVER sudo cat /etc/shadow | cut -d: -f1,8 | sed /:$/d > /tmp/expirelist.txt
...
What is causing the Permission denied error?
Don't use hardcoded temporary filenames -- when you do, it means that if one user (say, your development account) already ran this script and left a file named /tmp/expirelist.txt behind, no other user can run the same script.
tempfile=$(mktemp -t expirelist.XXXXXX)
ssh "$SERVER" sudo cat /etc/shadow | cut -d: -f1,8 | sed /:$/d >"$tempfile"
By using mktemp, you guarantee that each invocation will use a new, distinct, and previously-nonexisting temporary file, preventing any chance of conflict.
By the way -- if you want the file to be created on the remote system rather than the local system, you'd want to do this instead:
ssh "$SERVER" <<'EOF'
tempfile=$(mktemp -t expirelist.XXXXXX)
sudo cat /etc/shadow | cut -d: -f1,8 | sed /:$/d >"$tempfile"
EOF
I'm not sure about this, but you could be running into an issue with having the 'sudo' within your script. You could try removing the 'sudo' from the script, and running it like this:
$ sudo checkP.sh