I have a script which I call with different parameters. Depending on the value of those parameters I check out and build the 'parameter' SVN version of the project.
./deploy 3281
This command will create a 3281 directory and check out the 3281 SVN version of the project and will build it in the 3281 directory.
I need to create a key word 'HEAD' so the script will check to see the latest SVN revision number and create a folder for it (ex: 3282 ) and then checkout the head version of the project and build it there.
I find out how to get the latest revision number with svn ( svn info -r 'HEAD' --username jse http://jse#svn.ctsvpn.com/repos/Teleena/ | grep Revision | egrep -o "[0-9]+" ) and I am trying to simply implement an if like this:
#check to see if latest/head revision is called
if [ "$1" == "head" ]; then
#get latest revision number
HEADREV=$(svn info -r 'HEAD' --username jse http://jse#svn.ctsvpn.com/repos/Teleena/ | grep Revision | egrep -o "[0-9]+")
echo "=========================================="
echo "= Revision number: $HEADREV will be used ="
echo "=========================================="
#change swap the second parameter
$1=$HEADREV #<-- IS THIS CORRECT?
fi
...[rest of program here]
I want to replace the first parameter with the latest revision number and leave the rest of the script untouched. So the question is: How to I change a function's parameter value from inside the function?
You could make use of the set builtin in order to change a positional parameter.
The following snippet changes the first positional parameter, i.e. $1, to something:
set -- "something" "${#:2}"
As an example, refer to the following:
echo "Original parameters: $#"
set -- "something" "${#:2}"
echo "Modified parameters: $#"
Assuming this was placed in a script called script, and was invoked by saying bash script foo bar baz, it'd output:
Original parameters: foo bar baz
Modified parameters: something bar baz
Quoting from help set:
set: set [-abefhkmnptuvxBCHP] [-o option-name] [--] [arg ...]
Set or unset values of shell options and positional parameters.
Related
Lets say i have multiple filesnames e.g. R014-20171109-1159.log.20171109_1159.
I want to create a shell script which creates for every given date a folder and moves the files matching the date to it.
Is this possible?
For the example a folder "20171109" should be created and has the file "R014-20171109-1159.log.20171109_1159" on it.
Thanks
This is a typical application of a for-loop in bash to iterate thru files.
At the same time, this solution utilizes GNU [ shell param substitution ].
for file in /path/to/files/*\.log\.*
do
foldername=${file#*-}
foldername=${foldername%%-*}
mkdir -p "${foldername}" # -p suppress errors if folder already exists
[ $? -eq 0 ] && mv "${file}" "${foldername}" # check last cmd status and move
done
Since you want to write a shell script, use commands. To get date, use cut cmd like ex:
cat 1.txt
R014-20171109-1159.log.20171109_1159
cat 1.txt | cut -d "-" -f2
Output
20171109
is your date and create folder. This way you can loop and create as many folders as you want
Its actually quite easy(my Bash syntax might be a bit off) -
for f in /path/to/your/files*; do
## Check if the glob gets expanded to existing files.
## If not, f here will be exactly the pattern above
## and the exists test will evaluate to false.
[ -e "$f" ] && echo $f > #grep the file name for "*.log."
#and extract 8 charecters after "*.log." .
#Next check if a folder exists already with the name of 8 charecters.
#If not { create}
#else just move the file to that folder path
break
done
Main idea is from this post link. Sorry for not providing the actual code as i havent worked anytime recently on Bash
Below commands can be put in script to achieve this,
Assign a variable with current date as below ( use --date='n day ago' option if need to have an older date).
if need to get it from File name itself, get files in a loop then use cut command to get the date string,
dirVar=$(date +%Y%m%d) --> for current day,
dirVar=$(date +%Y%m%d --date='1 day ago') --> for yesterday,
dirVar=$(echo $fileName | cut -c6-13) or
dirVar=$(echo $fileName | cut -d- -f2) --> to get from $fileName
Create directory with the variable value as below, (-p : create directory if doesn't exist.)
mkdir -p ${dirVar}
Move files to directory to the directory with below line,
mv *log.${dirVar}* ${dirVar}/
I always write some magic numbers in my interactive shells and shell scripts.
For instance, If I want to list my users's names and shells, I'll write
cut --delimiter=: --fields=1,7 /etc/passwd
There exist two magic-numbers 1,7. And there are more and more magic-numbers in other circumstances.
Question
How to avoid magic-numbers in interactive shells and shell scripts?
Supplementary background
Our teacher told us using cut -d: -f1,7 /etc/passwd. But for new linux-users, they don't konw what's meaning of d,f,1,7.(not just for new linux-users，the whole system has so many configuration files that it is not easy for a person to remember every magic-numbers)
So, in interactive shells, we can use --delimiter, --fields,and the bash repl(or zsh,fish) has good tab completion to it.
How about the 1 and 7? In shell scripts, It's a good method to declare some const variables like LoginField=1 and ShellField=7 after reading the man 5 passwd. But when some one is writing in the interactive shells, it's not a good idea to open a new window and search the constants of LoginField=1,ShellField=7 and define it. how to using some thing like tab completion to simplify operations?
Use variables:
LoginField=1 ShellField=7
cut --delimiter=: --fields="$LoginField,$ShellField" /etc/passwd
Just like in other languages - by using variables. Example:
$ username_column=1
$ shell_column=7
$ cut --delimiter=: --fields="$username_column","$shell_column" /etc/passwd
The variables may be defined at the top of the script so that can be
easily modified or they can be set in an external config-like file
shared by multiple scripts.
The classic way to parse /etc/passwd is to read each column into an appropriately named variable:
while IFS=: read name passwd uid gid gecos home shell _; do
...
done < /etc/passwd
Use export:
export field_param="1,7"
(you can put it .bashrc file to have configured each time shell session is started). This export can be part of .sh script. It's a good practice to put them in the head/top of the file.
Then:
cut --delimiter=: --fields=$field_param /etc/passwd
This way you will need to edit the magic number in the only location.
Continuing from my comment, it's hard to tell exactly what you are asking. If you just want to give meaningful variable names, then do as shown in the other answers.
If however you want to be able to specify which fields are passed to cut from the command line, then you can use the positional parameters $1 and $2 to pass those values into your script.
You need to validate that two inputs are given and that both are integers. You can do that with a few simple tests, e.g.
#!/bin/bash
[ -n "$1" ] && [ -n "$2" ] || { ## validate 2 parameters given
printf "error: insufficient input\nusage: %s field1 field2\n" "${0##*/}"
exit 1
}
## validate both inputs are integer values
[ "$1" -eq "$1" >/dev/null 2>&1 ] || {
printf "error: field1 not integer value '%s'.\n" "$1"
exit 1
}
[ "$2" -eq "$2" >/dev/null 2>&1 ] || {
printf "error: field2 not integer value '%s'.\n" "$2"
exit 1
}
cut --delimiter=: --fields=$1,$2 /etc/passwd
Example Use/Output
$ bash fields.sh
error: insufficient input
usage: fields.sh field1 field2
$ bash fields.sh 1 d
error: field2 not integer value 'd'.
$ bash fields.sh 1 7
root:/bin/bash
bin:/usr/bin/nologin
daemon:/usr/bin/nologin
mail:/usr/bin/nologin
ftp:/usr/bin/nologin
http:/usr/bin/nologin
uuidd:/usr/bin/nologin
dbus:/usr/bin/nologin
nobody:/usr/bin/nologin
systemd-journal-gateway:/usr/bin/nologin
systemd-timesync:/usr/bin/nologin
systemd-network:/usr/bin/nologin
systemd-bus-proxy:/usr/bin/nologin
<snip>
Or if you choose to look at fields 1 and 3, then all you need do is pass those as the parameters, e.g.
$ bash fields.sh 1 3
root:0
bin:1
daemon:2
mail:8
ftp:14
http:33
uuidd:68
dbus:81
nobody:99
systemd-journal-gateway:191
systemd-timesync:192
systemd-network:193
systemd-bus-proxy:194
<snip>
Look things over and let me know if you have further questions.
Scraping the output of man 5 passwd for human-readable header names:
declare $(man 5 passwd |
sed -n '/^\s*·\s*/{s/^\s*·\s*//;y/ /_/;p}' |
sed -n 'p;=' | paste -d= - - )
See "how it works" below for what that does, then run:
cut --delimiter=: \
--fields=${login_name},${optional_user_command_interpreter} /etc/passwd
Which outputs the specified /etc/passwd fields.
How it works.
The man page describing /etc/passwd contains a bullet list of header names. Use GNU sed to find the bullets (·) and leading whitespace, then remove the bullets and whitespace, replace the remaining spaces with underlines; a 2nd instance of sed provides fresh line numbers, then paste the header names to the line numbers, with a = between:
man 5 passwd |
sed -n '/^\s*·\s*/{s/^\s*·\s*//;y/ /_/;p}' |
sed -n 'p;=' | paste -d= - -
Outputs:
login_name=1
optional_encrypted_password=2
numerical_user_ID=3
numerical_group_ID=4
user_name_or_comment_field=5
user_home_directory=6
optional_user_command_interpreter=7
And declare makes those active in the current shell.
Hi… Need a little help here…
I tried to emulate the DOS' dir command in Linux using bash script. Basically it's just a wrapped ls command with some parameters plus summary info. Here's the script:
#!/bin/bash
# default to current folder
if [ -z "$1" ]; then var=.;
else var="$1"; fi
# check file existence
if [ -a "$var" ]; then
# list contents with color, folder first
CMD="ls -lgG $var --color --group-directories-first"; $CMD;
# sum all files size
size=$(ls -lgGp "$var" | grep -v / | awk '{ sum += $3 }; END { print sum }')
if [ "$size" == "" ]; then size="0"; fi
# create summary
if [ -d "$var" ]; then
folder=$(find $var/* -maxdepth 0 -type d | wc -l)
file=$(find $var/* -maxdepth 0 -type f | wc -l)
echo "Found: $folder folders "
echo " $file files $size bytes"
fi
# error message
else
echo "dir: Error \"$var\": No such file or directory"
fi
The problem is when the argument contains an asterisk (*), the ls within the script acts differently compare to the direct ls command given at the prompt. Instead of return the whole files list, the script only returns the first file. See the video below to see the comparation in action. I don't know why it behaves like that.
Anyone knows how to fix it? Thank you.
Video: problem in action
UPDATE:
The problem has been solved. Thank you all for the answers. Now my script works as expected. See the video here: http://i.giphy.com/3o8dp1YLz4fIyCbOAU.gif
The asterisk * is expanded by the shell when it parses the command line. In other words, your script doesn't get a parameter containing an asterisk, it gets a list of files as arguments. Your script only works with $1, the first argument. It should work with "$#" instead.
This is because when you retrieve $1 you assume the shell does NOT expand *.
In fact, when * (or other glob) matches, it is expanded, and broken into segments by $IFS, and then passed as $1, $2, etc.
You're lucky if you simply retrieved the first file. When your first file's path contains spaces, you'll get an error because you only get the first segment before the space.
Seriously, read this and especially this. Really.
And please don't do things like
CMD=whatever you get from user input; $CMD;
You are begging for trouble. Don't execute arbitrary string from the user.
Both above answers already answered your question. So, i'm going a bit more verbose.
In your terminal is running the bash interpreter (probably). This is the program which parses your input line(s) and doing "things" based on your input.
When you enter some line the bash start doing the following workflow:
parsing and lexical analysis
expansion
brace expansion
tidle expansion
variable expansion
artithmetic and other substitutions
command substitution
word splitting
filename generation (globbing)
removing quotes
Only after all above the bash
will execute some external commands, like ls or dir.sh... etc.,
or will do so some "internal" actions for the known keywords and builtins like echo, for, if etc...
As you can see, the second last is the filename generation (globbing). So, in your case - if the test* matches some files, your bash expands the willcard characters (aka does the globbing).
So,
when you enter dir.sh test*,
and the test* matches some files
the bash does the expansion first
and after will execute the command dir.sh with already expanded filenames
e.g. the script get executed (in your case) as: dir.sh test.pas test.swift
BTW, it acts exactly with the same way for your ls example:
the bash expands the ls test* to ls test.pas test.swift
then executes the ls with the above two arguments
and the ls will print the result for the got two arguments.
with other words, the ls don't even see the test* argument - if it is possible - the bash expands the wilcard characters. (* and ?).
Now back to your script: add after the shebang the following line:
echo "the $0 got this arguments: $#"
and you will immediatelly see, the real argumemts how your script got executed.
also, in such cases is a good practice trying to execute the script in debug-mode, e.g.
bash -x dir.sh test*
and you will see, what the script does exactly.
Also, you can do the same for your current interpreter, e.g. just enter into the terminal
set -x
and try run the dir.sh test* = and you will see, how the bash will execute the dir.sh command. (to stop the debug mode, just enter set +x)
Everbody is giving you valuable advice which you should definitely should follow!
But here is the real answer to your question.
To pass unexpanded arguments to any executable you need to single quote them:
./your_script '*'
The best solution I have is to use the eval command, in this way:
#!/bin/bash
cmd="some command \"with_quetes_and_asterisk_in_it*\""
echo "$cmd"
eval $cmd
The eval command takes its arguments and evaluates them into the command as the shell does.
This solves my problem when I need to call a command with asterisk '*' in it from a script.
Hello here is a setup of what im trying to accomplish
The user has one environment properties file on their machine, it has a list of say 300 properties.
We then deploy a new build to that same system with an updated version of that file with a few more properties added. I do not want to get rid of the old env prop file i just want to add those new properties without the user having to do it. example
File A (New Environment Properties file)
DB_CONNECTION=
DB_REPO=
DB_TEST=
DB_USER=
File B (Old environment properties)
DB_REPO=
DB_USER=
I just need the DB_CONNECTION and DB_TEST added to that file, there may be more to add this is just an example.
I have tried multiple grep and diff commands but they just output the screen or replace the whole file. I don't want to do this since the user has saved values so i just need the new properties added.
Thanks in advance
I propose to source each file in its own subshell, execute set to show the complete set of environment variables, and compare both outputs. Sounds complex but is typed rather easily:
diff <(source a.sh; set) <(source b.sh; set)
The output is a typical diff output, in my case:
19d18
< DB_CONNECTION=
21d19
< DB_TEST=
87c85
< _=a.sh
---
> _=b.sh
The last two lines (_=a.sh and _=b.sh) are not interesting; they just show the last used argument (which differs of course).
Now, to add the found stuff to the file you want to patch, you can use this:
diff <(source a.sh; set) <(source b.sh; set) | grep '^<' | cut -c3- | grep -v '^_=' >> b.sh
This solution does not consider changed values, though. If the user changed a value in his old config file, the new config file will have the standard value again, this will be a difference, so it will be added to the old config file.
You might not want this, so you might want to specify further requirements on how to handle changed values.
I went with the following this will basically merge the old properties along with the values for each and add all the new properties. At the end of the script I just have the following command
sed -i 'property/d' environment.properties
Here is the merge script a co-worker helped me with
#!/bin/bash
old_file=
new_file=
while read new_line; do
# Check to see if the current line is a comment.
comment=$(echo "${new_line}" | grep '^[ \t]*[#!]')
# Print comments as-is.
if [ -n "${comment}" ]; then
echo "${new_line}"
continue
fi
# Get the property key.
key=$(echo "${new_line}" | sed -e 's/^[ \t]*\([^=:]\+\).*/\1/')
# If the line isn't a comment or a property, skip it.
if [ -z "${key}" ]; then
echo "ERROR: invalid property: ${new_line}" 1>&2
continue
fi
# Get the old value for the key.
old_line=$(grep "^[ \t]*${key}[ \t]*[=:]" ${old_file})
if [ -n "${old_line}" ]; then
# If there is an old value, used it.
echo "${old_line}" | sed -e 's/^[ \t]*//'
else
# Otherwise, use the new value.
echo "${new_line}"
echo "WARN: new property: ${key}" 1>&2
fi
done < ${new_file}
Thank you for the help
This question already has answers here:
How to programmatically determine the current checked out Git branch [duplicate]
(20 answers)
Closed 9 years ago.
I am new to shell scripting and can't figure this out. If you are unfamiliar, the command git branch returns something like
* develop
master
, where the asterisk marks the currently checked out branch. When I run the following in the terminal:
git branch | grep "*"
I get:
* develop
as expected.
However, when I run
test=$(git branch | grep "*")
or
test=`git branch | grep "*"`
And then
echo $test
, the result is just a list of files in the directory. How do we make the value of test="* develop"?
Then the next step (once we get "* develop" into a variable called test), is to get the substring. Would that just be the following?
currentBranch=${test:2}
I was playing around with that substring function and I got "bad substitution" errors a lot and don't know why.
Expanding on Noufal Ibrahim's answer, use the --short flag with git-symbolic-ref, no need to fuss with sed.
I've been using something like this in hooks and it works well:
#!/bin/bash
branch=$(git symbolic-ref --short HEAD)
echo
echo "**** Running post-commit hook from branch $branch"
echo
That outputs "**** Running post-commit hook from branch master"
Note that git-symbolic-ref only works if you're in a repository. Luckily .git/HEAD, as a leftover from Git's early days, contains the same symbolic ref. If you want to get the active branch of several git repositories, without traversing directories, you could use a bash one-liner like this:
for repo in */.git; do branch=$(cat $repo/HEAD); echo ${repo%/.git} : ${branch##*/}; done
Which outputs something like:
repo1 : master
repo2 : dev
repo3 : issue12
If you want to go further, the full ref contained in .git/HEAD is also a relative path to a file containing the SHA-1 hash of the branch's last commit.
The * is expanded, what you can do is use sed instead of grep and get the name of the branch immediately:
branch=$(git branch | sed -n -e 's/^\* \(.*\)/\1/p')
And a version using git symbolic-ref, as suggested by Noufal Ibrahim
branch=$(git symbolic-ref HEAD | sed -e 's,.*/\(.*\),\1,')
To elaborate on the expansion, (as marco already did,) the expansion happens in the echo, when you do echo $test with $test containing * master then the * is expanded according to the normal expansion rules. To suppress this one would have to quote the variable, as shown by marco: echo "$test". Alternatively, if you get rid of the asterisk before you echo it, all will be fine, e.g. echo ${test:2} will just echo master. Alternatively you could assign it anew as you already proposed:
branch=${test:2}
echo $branch
This will echo master, like you wanted.
I would use the git-symbolic-ref command in the git core. If you say git-symbolic-ref HEAD, you will get the name of the current branch.
I use this
git describe --contains --all HEAD
in my git helper scripts
example:
#!/bin/bash
branchname=$(git describe --contains --all HEAD)
git pull --rebase origin $branchname
I have that in a file called gpull in ~/scripts
Edit:
for a lot of CI environments, they'll check your code out in a "detached head" state, so then I'll use:
BRANCH=$(\
git for-each-ref \
--format='%(objectname) %(refname:short)' refs/heads \
| awk "/^$(git rev-parse HEAD)/ {print \$2}"\
)
The problem relies on:
echo $test
In fact the variable test contains a wildcard which is expanded by the shell. To avoid that just protect $test with double quotes:
echo "$test"
disable subshell glob expansion,
test=$(set -f; git branch)