I'm creating a script where I clone git repositories. I want my script to print "Cloning OK" when the cloning is successful and "Cloning FAILED" when it fails and ignore all command output for both occasions. This is the code I'm using:
(git clone -q "$url" && echo "$url: Cloning OK") || echo "$url: Cloning FAILED" >&2
The problem is that for successful cloning the command stays quiet but for unsuccessful cloning it doesn't. How can I make it quiet for both occasions?
output of command
Thanks in advance
You have to silence the command by sending standard output and standard error to somewhere other than the terminal. This is easiest achieved by sending both output streams to /dev/null:
(git clone -q "$url" >/dev/null 2>&1 && …
Please note that silencing this command will make your script harder to debug.
Related
I am trying to automate a process that contains a series of git commands.
I want the shell script to deal with some interactive commands, like passing the username and password to git clone url -v. I verified that if I just run git clone url -v it will show the following in order:
cloning into someRepo
asking for username
asking for password
I've tried:
echo -e 'username\n' | git clone url -v
echo -e 'username\npassword\n' | git clone url -v
git clone url -v <<< username\npassword\n
(sleep 5;echo -e 'username\n' | git clone url -v)
I thought that the first message cloning into repo will take some time. None of them is working, but all of them are showing the same message that Username for url:
Having spent lots of time in this, I know that
git clone https://$username:$password#enterpriseGithub.com/org/repo
is working, but it is UNSAFE to use since the log show the username and password explicitly.
Better practice would be to avoid user/password authentication at all (as by configuring agent-based auth, ideally backed by private keys stored on physical tokens), or set up credential storage in a keystore provided (and hopefully secured) by your operating system -- but if you just want to keep credentials off the command line, that can be done:
# Assume that we already know the credentials we want to store...
gitUsername="some"; gitPassword="values"
# Create a file containing the credentials readable only to the current user
mkdir -p "$HOME/.git-creds/https"
chmod 700 "$HOME/.git-creds"
cat >"$HOME/.git-creds/https/enterprise-github.com" <<EOF
username=$gitUsername
password=$gitPassword
EOF
# Generate a script that can retrieve stored credentials
mkdir -p -- "$HOME/bin"
cat >"$HOME/bin/git-retrieve-creds" <<'EOF'
#!/usr/bin/env bash
declare -A args=( )
while IFS= read -r line; do
case $line in
*..*) echo "ERROR: Invalid request" >&2; exit 1;;
*=*) args[${line%%=*}]=${line#*=} ;;
'') break ;;
esac
done
[[ ${args[protocol]} && ${args[host]} ]] || {
echo "Did not retrieve protocol and host" >&2; exit 1;
}
f="$HOME/.git-creds/${args[protocol]}/${args[host]}"
[[ -s $f ]] && cat -- "$f"
EOF
chmod +x "$HOME/bin/git-retrieve-creds"
# And configure git to use that
git config --global credential.helper "$HOME/bin/git-retrieve-creds"
I am running git push command in bash which generates some errors.
RESPONSE=$(git push "$target" --all | grep "error:" || true)
generates an output on the screen but variable $RESPONSE is empty
If I change the command to do this:
RESPONSE=$(git push "$target" --all 2>&1 | grep "error:" || true)
command runs silently but actually captures needed error in $RESPONSE
echo $RESPONSE
error: failed to push some refs to
'ssh://git#git.test.test.com:7999/test/test-test.git'
I really need to run this git push command in a way that it would hold the error above in $RESPONSE yet generate the entire output on the screen.
Running
RESPONSE=$(git push "$target" --all 2>&1 | tee -a log | grep "error:" || true) did not help, unless I am missing something.
One solution is to use tee; just not exactly the way you showed. Taking it step by step will perhaps make the solution easier to understand:
git push "$target" --all
will send the error you want to STDERR. That's why you added 2>&1, to redirect STDERR to STDOUT.
git push "$target" --all 2>&1
Then your pipeline (grep, etc.) is able to pick it up and eventually the variable capture is able to see it when you do
RESPONSE=$(git push "$target" --all 2>&1 | grep "error:" || true)
But because the error is no longer going to STDERR, and STDOUT is now being captured instead of sent to the screen, the output disappears.
So what you want to use tee for, is to put the output on both STDERR (for the screen) and STDOUT (for your pipeline and eventual variable capture).
RESPONSE=$(git push "$target" --all 2>&1 |tee >(cat 1>&2) | grep "error:" || true)
This will likely work as you intend, but be aware that everything you see on the screen - all output from the git command, error or otherwise - is now being passed on STDERR.
There aren't many practical reasons why this would be better than the answer about capturing to the variable and then echoing the variable (per miimote's answer), but if for some reason the non-sequential command structure seems better to you, this is a way to do it.
The first line
RESPONSE=$(git push "$target" --all | grep "error:" || true)
stores the response of the command in the var RESPONSE. So it is done in bash for any construction like VAR=$(command). If exists an error the var is empty but generates an output for the user.
If you add 2>&1, you are saying the same but if exists an error the output is the file $1, in your case the var $RESPONSE.
You could do this
RESPONSE=$(git push "$target" --all 2>&1 | grep "error:" || true); echo $RESPONSE
You can read more about command substitution and redirections
OK, so I am a total beginner with bash scripts and I am aware that the question is probably phrased a bit awkwardly, but I'll be as clear as I can!
I have written the following script to create a backup of repositories in a folder. The script is as follows:
#!/bin/bash
SVNREPO="/var/svn"
TEMP="/var/tmp"
BACKUP="/home/helix/backups"
cd $SVNREPO
if [ $# -eq 0 ]; then
for REPO in *; do
ARRAY+=($REPO)
done
else
for REPO in $#; do
ARRAY+=($REPO)
done
fi
for REPO in ${ARRAY[#]}; do
svnadmin dump $SVNREPO/$REPO -r HEAD | gzip > $TEMP/$REPO.svn.gzip sd
cp $TEMP/$REPO.svn.gzip $BACKUP/$REPO.svn.gzip
rm $TEMP/$REPO.svn.gzip
done
This script successfully produces .gzip backups of the all of the repositories in 'var/svn' when the script is called with no arguments, and creates .gzip backups of the specific repositories that are called as arguments. Great!
However, if the script is run with an argument that does not correspond to a repository that exists, then the program will crash with the error message: svnadmin: E000002: Can't open file '/var/svn/ada/format': No such file or directory. What I am trying to achieve is to catch this error and print a more user friendly output to the console. I have been trying to do this using 'trap'.
First I added the following line:
trap 'echo ERROR! The repository or repositories that you are trying to backup do not exist!' ERR
...and then I pushed the error to /dev/null at this point in the final for loop:
svnadmin dump $SVNREPO/$REPO -r HEAD 2>/dev/null | gzip > $TEMP/$REPO.svn.gzip
I pushed to the /dev/null file at the place I did because this is where the program errors out. However, the script no longer seems to work. What am I doing wrong here? Is it an issue to do with having the 2>/dev/null in the middle of a line? If so, how could I refactor this code so that it doesn't require the pipe in the middle of the line?
Many thanks for any help, I hope my question is reasonably clear! To confirm, the final non-working code is as follows:
#!/bin/bash
SVNREPO="/var/svn"
TEMP="/var/tmp"
BACKUP="/home/helix/backups"
cd $SVNREPO
if [ $# -eq 0 ]; then
for REPO in *; do
ARRAY+=($REPO)
done
else
for REPO in $#; do
ARRAY+=($REPO)
done
fi
trap 'echo ERROR! The repository or repositories that you are trying to backup do not exist!' ERR
for REPO in ${ARRAY[#]}; do
svnadmin dump $SVNREPO/$REPO -r HEAD 2>/dev/null | gzip > $TEMP/$REPO.svn.gzip sd
cp $TEMP/$REPO.svn.gzip $BACKUP/$REPO.svn.gzip
rm $TEMP/$REPO.svn.gzip
done
I do not know exactly how trap command works, but I will suggest another way that might solve your problem in another way:
First, before your for loop, add this line:
set -o pipefail
This means that when any command in a pipe fails, the last exit code ($?) will contain the error code if any failed.
On the line right after your svnadmin call, I would suggest adding this:
if [ $? -ne 0 ]; then
echo "ERROR! Received error code $? for repository '$REPO'."
continue
fi
You can of course alter the error message to your taste. The functionality should be clear: If svnadmin or bzip fails, it will print an error message and continue to next item in the for loop.
Hope this helps.
Using #cdarke suggestion of checking whether a file exists, I now have it working with the following code:
#!/bin/bash
SVNREPO="/var/svn"
TEMP="/var/tmp"
BACKUP="/home/helix/backups"
cd $SVNREPO
if [ $# -eq 0 ]; then
for REPO in *; do
ARRAY+=($REPO)
done
else
for REPO in $#; do
ARRAY+=($REPO)
done
fi
for REPO in ${ARRAY[#]}; do
if [ -f $SVNREPO/$REPO/format ]; then
vnadmin dump $SVNREPO/$REPO -r HEAD 2>/dev/null | gzip > $TEMP/$REPO.svn.gzip
cp $TEMP/$REPO.svn.gzip $BACKUP/$REPO.svn.gzip
rm $TEMP/$REPO.svn.gzip
else
echo ERROR! The repository $REPO does not exist. No backup has been made for this argument.
fi
done
I need most basic hook to prevent empty comment checkins. Googled, found sample bash script. Made it short and here is what I have:
#!/bin/sh
REPOS="$1"
TXN="$2"
# Make sure that the log message contains some text.
SVNLOOK=/usr/bin/svnlook
ICONV=/usr/bin/iconv
SVNLOOKOK=1
$SVNLOOK log -t "$TXN" "$REPOS" | \
grep "[a-zA-Z0-9]" > /dev/null || SVNLOOKOK=0
if [ $SVNLOOKOK = 0 ]; then
echo "Empty log messages are not allowed. Please provide a proper log message." >&2
exit 1
fi
# Comments should have more than 5 characters
LOGMSG=$($SVNLOOK log -t "$TXN" "$REPOS" | grep [a-zA-Z0-9] | wc -c)
if [ "$LOGMSG" -lt 6 ]; then
echo -e "Please provide a meaningful comment when committing changes." 1>&2
exit 1
fi
Now I'm testing it with Tortoise SVN and here is what I see:
Commit failed (details follow): Commit blocked by pre-commit hook
(exit code 1) with output: /home/svn/repos/apress/hooks/pre-commit:
line 11: : command not found Empty log messages are not allowed.
Please provide a proper log message. This error was generated by a
custom hook script on the Subversion server. Please contact your
server administrator for help with resolving this issue.
What is the error? svnlook is in /usr/bin
I'm very new to Linux, don't understand what happens..
To debug your script you'll have to run it manually.
To do that you'll have to get the sample values for the parameters passed to it.
Change the beginning of your script to something like
#!/bin/sh
REPOS="$1"
TXN="$2"
echo "REPOS = $REPOS, TXN = $TXN" >/tmp/svnhookparams.txt
Do a commit and check the file /tmp/svnhookparams.txt for the values.
Then do another change to the script:
#!/bin/sh
set -x
REPOS="$1"
TXN="$2"
This will enable echo of all commands run by the shell.
Now run you script directly from terminal passing to it the values you got previously.
Check the output for invalid commands or empty variable assignments.
If you have problems with that, post the output here.
$PATH is empty when running hook scripts. Thus you need to specify full paths for every external command. My guess, is that grep is not found.
I'm answering my own question.
This didn't work:
$SVNLOOK log -t "$TXN" "$REPOS" | \
grep "[a-zA-Z0-9]" > /dev/null || SVNLOOKOK=0
It had to be 1 line:
$SVNLOOK log -t "$TXN" "$REPOS" | grep "[a-zA-Z0-9]" > /dev/null || SVNLOOKOK=0
I typically run the following commands to deploy a particular app:
compass compile -e production --force
git add .
git commit -m "Some message"
git push
git push production master
How can I wrap that up into a single command?
I'd need to be able to customize the commit message. So the command might look something like:
deploy -m "Some message"
There are two possibilities:
a script, as others answered
a function, defined in your .bash_profile:
deploy() {
compass compile -e production --force &&
git add . &&
git commit -m "$#" &&
git push &&
git push production master
}
Without arguments, you'd have a third option, namely an alias:
alias deploy="compass compile -e production --force &&
git add . &&
git commit -m 'Dumb message' &&
git push &&
git push production master"
You could create a function that does what you want, and pass the commit message as argument:
function deploy() {
compass compile -e production --force
git add .
git commit "$#"
git push
git push production master
}
Put that in your .bashrc and you're good to go.
You can make a shell script. Something that looks like this (note no input validation etc):
#!/bin/sh
compass compile -e production --force
git add .
git commit -m $1
git push
git push production master
Save that to myscript.sh, chmod +x it, then do something like ./myscript.sh "Some message".
You can write a shell script for this
#!/bin/bash
compass compile -e production --force
git add .
git commit -m $1
git push
git push production master
Save this to 'deploy' and do a chmod 7xx on it. Now you can use it as ./deploy "Some message"
you could write these commands into a file named deploy.sh .
Then make it executable and run as sh deploy.sh
You could even add it to your path by exporting the path where you save the script.
everyone mentions about writing a script and this is probably the best way of doing it.
However you might someday want to use another way - merge commands with &&, for example:
cd ../ && touch abc
will create a file "abc" in a parent directory :)
It is just to let you know about such thing, for this particular scenario (and 99% of the others) please take a look at other answers :)
I would go through the effort of making the command work for more than just the current directory. One of the most versitle ways of doing this is to use getopt in a BASH script. Make sure you have getopt installed, create deploy.sh then chmod 755 deploy.sh and then do something like this:
#!/bin/bash
declare -r GETOPT=/usr/bin/getopt
declare -r ECHO='builtin echo'
declare -r COMPASS=/path/to/compass
declare -r GIT=/path/to/git
sanity() {
# Sanity check our runtime environment to make sure all needed apps are there.
for bin in $GETOPT $ECHO $COMPASS $GIT
do
if [ ! -x $bin ]
then
log error "Cannot find binary $bin"
return 1
fi
done
return 0
}
usage() {
$CAT <<!
${SCRIPTNAME}: Compile, add and commit directories
Usage: ${SCRIPTNAME} -e <env> [-v]
-p|--path=<path to add>
-c|--comment="Comment to add"
-e|--environment=<production|staging|dev>
Example:
$SCRIPTNAME -p /opt/test/env -c "This is the comment" -e production
!
}
checkopt() {
# Since getopt is used within this function, it must be called as
# checkopt "$#"
local SHORTOPT="-hp::c::e::"
local LONGOPT="help,path::,comment::,environment::"
eval set -- "`$GETOPT -u -o $SHORTOPT --long $LONGOPT -n $SCRIPTNAME -- $#`"
while true
do
case "$1" in
-h|--help)
return 1
;;
-|--path)
PATH="$2"
shift 2
;;
-c|--comment)
COMMENT=$2
shift 2
;;
-e|--environment)
ENV="$2"
shift 2
;;
--)
shift
break
;;
*)
$ECHO "what is $1?"
;;
esac
done
}
if ! sanity
then
die "Sanity check failed - Cant find proper system binaries"
fi
if checkopt $#
then
$ECHO "Running Compass Compile & Git commit sequence..."
$COMPASS compile -e $ENV --force
$GIT add $PATH
$GIT commit -m $COMMENT
$GIT push
$GIT push ENV master
else
usage
exit 1
fi
exit 0