I have this small bash script (simplified) which runs on Ubuntu 16.04:
tar zxvf fileNameHere.tgz <-- Untar tgz file in $SRC_DIR
files=$(ls $SRC_DIR)
echo "Extracting $files" >> $APP_LOG_DIR/update.log
mv $SRC_DIR/* $OUTPUT_DIR
shutdown -r now
I've noticed that, after rebooting, only sometimes files are not moved to target and I was wondering if that shutdown command could be the a problem. Is it necessary to call 'sync' before shutting down?
Fixed script with comments:
#!/usr/bin/env bash
# Test if SRC_DIR and OUTPUT_DIR are actual directories
if [ -d "$SRC_DIR" ] && [ -d "$OUTPUT_DIR" ]; then
# Populates the arguments array with the content
# of SRC_DIR rather than parsing the output of ls
set -- "$SRC_DIR/"*
# Prints joined file entries of the arguments array
# while stripping their leading directory path
printf 'Extracting %s\n' "${*#*/}" >> "$APP_LOG_DIR/update.log"
# Moves all the arguments array's entries (the actual
# content of the SRC_DIR) into OUTPUT_DIR
mv -- "$#" "$OUTPUT_DIR/"
shutdown -r now
fi
I need to create a new Makefile that sources the master Makefile, and then uses the variables defined within to check if the directories exist in their appropriate local branches. I've read a lot of posts on StackOverflow about checking if directories exists, but I'm stuck on how to find out if their in the appropriate branches.
#!bin/ksh
DIRLOC=/var/tmp
DIRNAMES="SchemaExtract SQL Count SchExtArchive"
for DIRNAME in ${DIRNAMES}
do
if [ -d ${DIRLOC}/${DIRNAME} ]
then
echo ${DIRLOC}/${DIRNAME} already exists
else
echo ${DIRLOC}/${DIRNAME} Creating ...
mkdir ${DIRLOC}/${DIRNAME}
chmod 755 ${DIRLOC}/${DIRNAME}
fi
done
Any help would be appreciated!
Clarification-
I want to specify in my new Makefile what git branch each directory is supposed to be in. So I need a code that reads the directories from the master Makefile, checks if they exist and if so, compare the location of the directories found with the locations that I specify in the new Makefile to determine everything is in its correct git branch.
You can use the git ls-tree command to check for a directories existence in a given branch.
As an example, consider the following repository:
# There are 3 branches.
$ git branch
branch1
branch2
* master
# master contains master_dir
$ ls
master_dir
# branch1 contains master_dir and branch1_dir
$ git checkout branch1
Switched to branch 'branch1'
$ ls
branch1_dir master_dir
# branch2 contains master_dir and branch2_dir
$ git checkout branch2
Switched to branch 'branch2'
$ ls
branch2_dir master_dir
# switch back to the master branch
$ git checkout master
Switched to branch 'master'
$ ls
master_dir
The following commands are run from the master branch.
For branch1:
$ git ls-tree -d branch1:branch1_dir
$ git ls-tree -d branch1:branch2_dir
fatal: Not a valid object name branch1:branch2_dir
For branch2:
$ git ls-tree -d branch2:branch2_dir
$ git ls-tree -d branch2:branch1_dir
fatal: Not a valid object name branch2:branch1_dir
In your shell script, you can use the return value of the command in your conditional:
$ git ls-tree -d branch1:branch1_dir 2&> /dev/null; \
> if [[ $? -eq 0 ]]; then echo "Exists"; else echo "Does not exist"; fi
Exists
$ git ls-tree -d branch1:branch2_dir 2&> /dev/null; \
> if [[ $? -eq 0 ]]; then echo "Exists"; else echo "Does not exist"; fi
Does not exist
EDIT: Example shell script using directory definitions in an external file.
$ cat branch-dirs.txt
branch1:branch1_dir
branch2:branch2_dir
branch2:non_existent_dir
$ cat check_dirs.sh
#!/bin/bash
readonly BRANCH_DIR_FILE="./branch-dirs.txt"
for dir_to_check in $(cat "$BRANCH_DIR_FILE"); do
git ls-tree -d "${dir_to_check}" 2&> /dev/null
if [[ $? -eq 0 ]]; then
echo "${dir_to_check} exists."
else
echo "${dir_to_check} does not exist."
fi
done
$ ./check_dirs.sh
branch1:branch1_dir exists.
branch2:branch2_dir exists.
branch2:non_existent_dir does not exist.
So I was browsing through and came across this post. Wouldn't this work a little better for what I need it to do in the long run since I need it to work from the top-level down?
MY_DIRNAME=../External
ifneq "$(wildcard $(MY_DIRNAME) )" ""
# if directory MY_DIRNAME exists:
INCLUDES += -I../External
else
# if it doesn't:
INCLUDES += -I$(HOME)/Code/External
endif
I am developing a bash script that adds current branch onto my Terminal prompt and shows information about the most recent commit in this folder whenever I cd into a folder that is a git repository in the terminal
Problem is that whenever I switch branches with git checkout within that repository folder the prompt does not update the current branch
this is my bash code located on my .bashrc file
cd() {
builtin cd "$#"
local status=$?
[ $status -eq 0 ] && PS1="[\e[0;32m${debian_chroot:+($debian_chroot)}\w\e[m]\e[0;35m$(parse_git_branch)\e[m \n$ "
if [ -d .git ]; then
echo -e "\nMost Recent Commit"
git show --summary;
fi
return $status
}
As documented in the Pro Git book you need the git-prompt.sh file (which should be installed as part of Git) and then in your .bashrc do something like:
. /usr/share/git-core/contrib/completion/git-prompt.sh
export GIT_PS1_SHOWDIRTYSTATE=1
export PS1='\w$(__git_ps1 " (%s)")\$ '
you have to change you ~/.bashrc and export the PS1 environment variable.
Here is an example of a ~/.bashrc:
# settings for this script
MY_DOMAIN=$(hostname -f | sed -e "s/^[^.]*\.//")
MY_FQDN=$(hostname -f)
MY_TTY=$(tty| cut -f3- -d/)
MY_USER=$(whoami)
MY_ROT="\033[31m"
MY_GRUEN="\033[32m"
MY_GELB="\033[33m"
MY_BLAU="\033[34m"
MY_LILA="\033[35m"
MY_CYAN="\033[36m"
MY_WEISS="\033[37m"
MY_FETT="\033[1m"
MY_NORMAL="\033[2m"
MY_RESET="\033[0m"
# user color
MY_U="$MY_BLAU"
case $MY_USER in
developer)
MY_U="$MY_GRUEN"
;;
root)
MY_U="$MY_ROT"
;;
esac
[ $(id -u) -eq 0 ] && MY_U="$MY_U$MY_FETT"
MY_U="\[$MY_U\]"
# host color
MY_H="$MY_ROT"
MY_H="\[$MY_H\]"
# working directory color
MY_W="\[$MY_CYAN\]"
# tty color
MY_T="\[$MY_BLAU$MY_FETT\]"
MY_R="\[$MY_RESET\]"
MY_G="\[$MY_GELB\]"
MY_GF="\[$MY_GELB$MY_FETT\]"
GIT_PS1_SHOWDIRTYSTATE=1
GIT_PS1_SHOWUNTRACKEDFILES=1
GIT_PS1_SHOWSTASHSTATE=1
GIT_PS1_SHOWUPSTREAM=verbose
export PS1="$MY_U\u$MY_R$MY_G#$MY_R$MY_H\h$MY_R$MY_G($MY_R$MY_T$MY_TTY \t$MY_R$MY_G):$MY_R$MY_W\w$MY_R\$(__git_ps1 \"$MY_GF:$MY_R \[$MY_ROT\](%s)$MY_R \")$MY_GF\$$MY_R "
which renders this:
On most linux distros you can get the branch by doing.
echo $(__git_ps1)
(develop)
If __git_ps1 unavailable, you have to source git-sh-prompt first. It may be git-prompt.sh on other distros.
source /usr/lib/git-core/git-sh-prompt
Put this one in your ~/.bash_profile file... Dynamically update your prompt with current git branch. Leave a git directory space and get a different prompt.
promptFunc() {
branch=$(git branch 2>/dev/null | grep '^*' | colrm 1 2)
if [ ! $branch ]; then
PS1=${PWD}"$ "
else
PS1="\W: "${branch}"-> "
fi
}
export PROMPT_COMMAND="promptFunc"
I am creating a bash script to extract a tar file and cd'ing into it and then it runs another script. So far this has been working pretty well with my code below, however, i ran into a case where if the extracted folder is different than the .tar file name then it would cause an issue. So my question is, how should I handle unique cases where the file name is different than then .tar filename.
e.g,) my_file.tar ---> after extraction ----> my_different_file_name
#!/bin/bash
fname=$1
echo the file you are about to extract is $fname
if [ -f $fname ]; then #if the file exists
tar -xvzf $fname #tar it
cd ${fname%.*} #the `%.*` will extract filename from filename.tgz and cd into it
echo ${fname%.*}
echo $PWD
loadIt #another script to load
fi
You could do a:
topDir=$(tar -xvzf $fname | sed "s|/.*$||" | uniq)
[ $(wc -w <<< $topDir) == 1 ] || exit 1
echo topDir=$topDir
Explanation: the first command untars vebosely (outputs all files it's untarring), and then gets all the leading directory names, and pipes them into uniq. (so basically it returns a list of all the top level directories in the tar file). The next line checks that there's exactly one entry in topDir, otherwise it exits.
At this point $topdir will be the directory you want to cd into.
Maybe you could do something like that:
cd $(tar -tf $fname | head -1)
If you don't mind moving the directory around after you extract it you can do something like this
# Create a temporary directory
$ tmpd=$(mktemp -d)
# Change to the temporary directory
$ pushd "$tmpd"
# Extract the tarball
$ tar -xf "$fname"
# Glob the directory name
$ d=(*)
# Error if we have more (or less) than one directory
$ [ "${#d}" = 0 ] || exit 1
# Explicitly use just the first directory (optional since `$d` does the same thing)
$ d=${d[0]}
# Move the extracted directory to the previous directory
$ mv "$d" "$OLDPWD"
# Change back to the starting directory
$ popd
# Remove the (now empty) temporary directory
$ rmdir "$tmpd"
# Change into the extracted directory
$ cd "$d"
# Run 'loadIt'
$ loadIt
I want to version control my web server as described in Version control for my web server, by creating a git repo out of my /var/www directory. My hope was that I would then be able to push web content from our dev server to github, pull it to our production server, and spend the rest of the day at the pool.
Apparently a kink in my plan is that Git won't respect file permissions (I haven't tried it, only reading about it now.) I guess this makes sense in that different boxes are liable to have different user/group setups. But if I wanted to force permissions to propagate, knowing my servers are configured the same, do I have any options? Or is there an easier way to approach what I'm trying to do?
Git is Version Control System, created for software development, so from the whole set of modes and permissions it stores only executable bit (for ordinary files) and symlink bit. If you want to store full permissions, you need third party tool, like git-cache-meta (mentioned by VonC), or Metastore (used by etckeeper). Or you can use IsiSetup, which IIRC uses git as backend.
See Interfaces, frontends, and tools page on Git Wiki.
The git-cache-meta mentioned in SO question "git - how to recover the file permissions git thinks the file should be?" (and the git FAQ) is the more staightforward approach.
The idea is to store in a .git_cache_meta file the permissions of the files and directories.
It is a separate file not versioned directly in the Git repo.
That is why the usage for it is:
$ git bundle create mybundle.bdl master; git-cache-meta --store
$ scp mybundle.bdl .git_cache_meta machine2:
#then on machine2:
$ git init; git pull mybundle.bdl master; git-cache-meta --apply
So you:
bundle your repo and save the associated file permissions.
copy those two files on the remote server
restore the repo there, and apply the permission
This is quite late but might help some others. I do what you want to do by adding two git hooks to my repository.
.git/hooks/pre-commit:
#!/bin/bash
#
# A hook script called by "git commit" with no arguments. The hook should
# exit with non-zero status after issuing an appropriate message if it wants
# to stop the commit.
SELF_DIR=`git rev-parse --show-toplevel`
DATABASE=$SELF_DIR/.permissions
# Clear the permissions database file
> $DATABASE
echo -n "Backing-up permissions..."
IFS_OLD=$IFS; IFS=$'\n'
for FILE in `git ls-files --full-name`
do
# Save the permissions of all the files in the index
echo $FILE";"`stat -c "%a;%U;%G" $FILE` >> $DATABASE
done
for DIRECTORY in `git ls-files --full-name | xargs -n 1 dirname | uniq`
do
# Save the permissions of all the directories in the index
echo $DIRECTORY";"`stat -c "%a;%U;%G" $DIRECTORY` >> $DATABASE
done
IFS=$IFS_OLD
# Add the permissions database file to the index
git add $DATABASE -f
echo "OK"
.git/hooks/post-checkout:
#!/bin/bash
SELF_DIR=`git rev-parse --show-toplevel`
DATABASE=$SELF_DIR/.permissions
echo -n "Restoring permissions..."
IFS_OLD=$IFS; IFS=$'\n'
while read -r LINE || [[ -n "$LINE" ]];
do
ITEM=`echo $LINE | cut -d ";" -f 1`
PERMISSIONS=`echo $LINE | cut -d ";" -f 2`
USER=`echo $LINE | cut -d ";" -f 3`
GROUP=`echo $LINE | cut -d ";" -f 4`
# Set the file/directory permissions
chmod $PERMISSIONS $ITEM
# Set the file/directory owner and groups
chown $USER:$GROUP $ITEM
done < $DATABASE
IFS=$IFS_OLD
echo "OK"
exit 0
The first hook is called when you "commit" and will read the ownership and permissions for all the files in the repository and store them in a file in the root of the repository called .permissions and then add the .permissions file to the commit.
The second hook is called when you "checkout" and will go through the list of files in the .permissions file and restore the ownership and permissions of those files.
You might need to do the commit and checkout using sudo.
Make sure the pre-commit and post-checkout scripts have execution permission.
We can improve on the other answers by changing the format of the .permissions file to be executable chmod statements, and to make use of the -printf parameter to find. Here is the simpler .git/hooks/pre-commit file:
#!/usr/bin/env bash
echo -n "Backing-up file permissions... "
cd "$(git rev-parse --show-toplevel)"
find . -printf 'chmod %m "%p"\n' > .permissions
git add .permissions
echo done.
...and here is the simplified .git/hooks/post-checkout file:
#!/usr/bin/env bash
echo -n "Restoring file permissions... "
cd "$(git rev-parse --show-toplevel)"
. .permissions
echo "done."
Remember that other tools might have already configured these scripts, so you may need to merge them together. For example, here's a post-checkout script that also includes the git-lfs commands:
#!/usr/bin/env bash
echo -n "Restoring file permissions... "
cd "$(git rev-parse --show-toplevel)"
. .permissions
echo "done."
command -v git-lfs >/dev/null 2>&1 || { echo >&2 "\nThis repository is configured for Git LFS but 'git-lfs' was not found on you
r path. If you no longer wish to use Git LFS, remove this hook by deleting .git/hooks/post-checkout.\n"; exit 2; }
git lfs post-checkout "$#"
In case you are coming into this right now, I've just been through it today and can summarize where this stands. If you did not try this yet, some details here might help.
I think #Omid Ariyan's approach is the best way. Add the pre-commit and post-checkout scripts. DON'T forget to name them exactly the way Omid does and DON'T forget to make them executable. If you forget either of those, they have no effect and you run "git commit" over and over wondering why nothing happens :) Also, if you cut and paste out of the web browser, be careful that the quotation marks and ticks are not altered.
If you run the pre-commit script once (by running a git commit), then the file .permissions will be created. You can add it to the repository and I think it is unnecessary to add it over and over at the end of the pre-commit script. But it does not hurt, I think (hope).
There are a few little issues about the directory name and the existence of spaces in the file names in Omid's scripts. The spaces were a problem here and I had some trouble with the IFS fix. For the record, this pre-commit script did work correctly for me:
#!/bin/bash
SELF_DIR=`git rev-parse --show-toplevel`
DATABASE=$SELF_DIR/.permissions
# Clear the permissions database file
> $DATABASE
echo -n "Backing-up file permissions..."
IFSold=$IFS
IFS=$'\n'
for FILE in `git ls-files`
do
# Save the permissions of all the files in the index
echo $FILE";"`stat -c "%a;%U;%G" $FILE` >> $DATABASE
done
IFS=${IFSold}
# Add the permissions database file to the index
git add $DATABASE
echo "OK"
Now, what do we get out of this?
The .permissions file is in the top level of the git repo. It has one line per file, here is the top of my example:
$ cat .permissions
.gitignore;660;pauljohn;pauljohn
05.WhatToReport/05.WhatToReport.doc;664;pauljohn;pauljohn
05.WhatToReport/05.WhatToReport.pdf;664;pauljohn;pauljohn
As you can see, we have
filepath;perms;owner;group
In the comments about this approach, one of the posters complains that it only works with same username, and that is technically true, but it is very easy to fix it. Note the post-checkout script has 2 action pieces,
# Set the file permissions
chmod $PERMISSIONS $FILE
# Set the file owner and groups
chown $USER:$GROUP $FILE
So I am only keeping the first one, that's all I need. My user name on the Web server is indeed different, but more importantly you can't run chown unless you are root. Can run "chgrp", however. It is plain enough how to put that to use.
In the first answer in this post, the one that is most widely accepted, the suggestion is so use git-cache-meta, a script that is doing the same work that the pre/post hook scripts here are doing (parsing output from git ls-files). These scripts are easier for me to understand, the git-cache-meta code is rather more elaborate. It is possible to keep git-cache-meta in the path and write pre-commit and post-checkout scripts that would use it.
Spaces in file names are a problem with both of Omid's scripts. In the post-checkout script, you'll know you have the spaces in file names if you see errors like this
$ git checkout -- upload.sh
Restoring file permissions...chmod: cannot access '04.StartingValuesInLISREL/Open': No such file or directory
chmod: cannot access 'Notebook.onetoc2': No such file or directory
chown: cannot access '04.StartingValuesInLISREL/Open': No such file or directory
chown: cannot access 'Notebook.onetoc2': No such file or directory
I'm checking on solutions for that. Here's something that seems to work, but I've only tested in one case
#!/bin/bash
SELF_DIR=`git rev-parse --show-toplevel`
DATABASE=$SELF_DIR/.permissions
echo -n "Restoring file permissions..."
IFSold=${IFS}
IFS=$
while read -r LINE || [[ -n "$LINE" ]];
do
FILE=`echo $LINE | cut -d ";" -f 1`
PERMISSIONS=`echo $LINE | cut -d ";" -f 2`
USER=`echo $LINE | cut -d ";" -f 3`
GROUP=`echo $LINE | cut -d ";" -f 4`
# Set the file permissions
chmod $PERMISSIONS $FILE
# Set the file owner and groups
chown $USER:$GROUP $FILE
done < $DATABASE
IFS=${IFSold}
echo "OK"
exit 0
Since the permissions information is one line at a time, I set IFS to $, so only line breaks are seen as new things.
I read that it is VERY IMPORTANT to set the IFS environment variable back the way it was! You can see why a shell session might go badly if you leave $ as the only separator.
In pre-commit/post-checkout an option would be to use "mtree" (FreeBSD), or "fmtree" (Ubuntu) utility which "compares a file hierarchy against a specification, creates a specification for a file hierarchy, or modifies a specification."
The default set are flags, gid, link, mode, nlink, size, time, type, and uid. This can be fitted to the specific purpose with -k switch.
I am running on FreeBSD 11.1, the freebsd jail virtualization concept makes the operating system optimal. The current version of Git I am using is 2.15.1, I also prefer to run everything on shell scripts. With that in mind I modified the suggestions above as followed:
git push: .git/hooks/pre-commit
#! /bin/sh -
#
# A hook script called by "git commit" with no arguments. The hook should
# exit with non-zero status after issuing an appropriate message if it wants
# to stop the commit.
SELF_DIR=$(git rev-parse --show-toplevel);
DATABASE=$SELF_DIR/.permissions;
# Clear the permissions database file
> $DATABASE;
printf "Backing-up file permissions...\n";
OLDIFS=$IFS;
IFS=$'\n';
for FILE in $(git ls-files);
do
# Save the permissions of all the files in the index
printf "%s;%s\n" $FILE $(stat -f "%Lp;%u;%g" $FILE) >> $DATABASE;
done
IFS=$OLDIFS;
# Add the permissions database file to the index
git add $DATABASE;
printf "OK\n";
git pull: .git/hooks/post-merge
#! /bin/sh -
SELF_DIR=$(git rev-parse --show-toplevel);
DATABASE=$SELF_DIR/.permissions;
printf "Restoring file permissions...\n";
OLDIFS=$IFS;
IFS=$'\n';
while read -r LINE || [ -n "$LINE" ];
do
FILE=$(printf "%s" $LINE | cut -d ";" -f 1);
PERMISSIONS=$(printf "%s" $LINE | cut -d ";" -f 2);
USER=$(printf "%s" $LINE | cut -d ";" -f 3);
GROUP=$(printf "%s" $LINE | cut -d ";" -f 4);
# Set the file permissions
chmod $PERMISSIONS $FILE;
# Set the file owner and groups
chown $USER:$GROUP $FILE;
done < $DATABASE
IFS=$OLDIFS
pritnf "OK\n";
exit 0;
If for some reason you need to recreate the script the .permissions file output should have the following format:
.gitignore;644;0;0
For a .gitignore file with 644 permissions given to root:wheel
Notice I had to make a few changes to the stat options.
Enjoy,
One addition to #Omid Ariyan's answer is permissions on directories. Add this after the for loop's done in his pre-commit script.
for DIR in $(find ./ -mindepth 1 -type d -not -path "./.git" -not -path "./.git/*" | sed 's#^\./##')
do
# Save the permissions of all the files in the index
echo $DIR";"`stat -c "%a;%U;%G" $DIR` >> $DATABASE
done
This will save directory permissions as well.
Another option is git-store-meta. As the author described in this superuser answer:
git-store-meta is a perl script which integrates the nice features of git-cache-meta, metastore, setgitperms, and mtimestore.
Improved version of https://stackoverflow.com/users/9932792/tammer-saleh answer:
It only updates the permissions on changed files.
It handles symlinks
It ignores empty directories (git can not handle them)
.git/hooks/pre-commit:
#!/usr/bin/env bash
echo -n "Backing-up file permissions... "
cd "$(git rev-parse --show-toplevel)"
find . -type d ! -empty -printf 'X="%p"; chmod %m "$X"; chown %U:%G "$X"\n' > .permissions
find . -type f -printf 'X="%p"; chmod %m "$X"; chown %U:%G "$X"\n' >> .permissions
find . -type l -printf 'chown -h %U:%G "%p"\n' >> .permissions
git add .permissions
echo done.
.git/hooks/post-merge:
#!/usr/bin/env bash
echo -n "Restoring file permissions... "
cd "$(git rev-parse --show-toplevel)"
git diff -U0 .permissions | grep '^\+' | grep -Ev '^\+\+\+' | cut -c 2- | /usr/bin/bash
echo "done."