Expand a possible relative path in bash - linux

As arguments to my script there are some file paths. Those can, of course, be relative (or contain ~). But for the functions I've written I need paths that are absolute, but do not have their symlinks resolved.
Is there any function for this?

MY_PATH=$(readlink -f $YOUR_ARG) will resolve relative paths like "./" and "../"
Consider this as well (source):
#!/bin/bash
dir_resolve()
{
cd "$1" 2>/dev/null || return $? # cd to desired directory; if fail, quell any error messages but return exit status
echo "`pwd -P`" # output full, link-resolved path
}
# sample usage
if abs_path="`dir_resolve \"$1\"`"
then
echo "$1 resolves to $abs_path"
echo pwd: `pwd` # function forks subshell, so working directory outside function is not affected
else
echo "Could not reach $1"
fi

http://www.linuxquestions.org/questions/programming-9/bash-script-return-full-path-and-filename-680368/page3.html has the following
function abspath {
if [[ -d "$1" ]]
then
pushd "$1" >/dev/null
pwd
popd >/dev/null
elif [[ -e "$1" ]]
then
pushd "$(dirname "$1")" >/dev/null
echo "$(pwd)/$(basename "$1")"
popd >/dev/null
else
echo "$1" does not exist! >&2
return 127
fi
}
which uses pushd/popd to get into a state where pwd is useful.

Simple one-liner:
function abs_path {
(cd "$(dirname '$1')" &>/dev/null && printf "%s/%s" "$PWD" "${1##*/}")
}
Usage:
function do_something {
local file=$(abs_path $1)
printf "Absolute path to %s: %s\n" "$1" "$file"
}
do_something $HOME/path/to/some\ where
I am still trying to figure out how I can get it to be completely oblivious to whether the path exists or not (so it can be used when creating files as well).

This does the trick for me on OS X: $(cd SOME_DIRECTORY 2> /dev/null && pwd -P)
It should work anywhere. The other solutions seemed too complicated.

If your OS supports it, use:
realpath -s "./some/dir"
And using it in a variable:
some_path="$(realpath -s "./some/dir")"
Which will expand your path. Tested on Ubuntu and CentOS, might not be available on yours. Some recommend readlink, but documentation for readlink says:
Note realpath(1) is the preferred command to use for canonicalization functionality.
In case people wonder why I quote my variables, it's to preserve spaces in paths. Like doing realpath some path will give you two different path results. But realpath "some path" will return one. Quoted parameters ftw :)
Thanks to NyanPasu64 for the heads up. You'll want to add -s if you don't want it to follow the symlinks.

Use readlink -f <relative-path>, e.g.
export FULLPATH=`readlink -f ./`

Maybe this is more readable and does not use a subshell and does not change the current dir:
dir_resolve() {
local dir=`dirname "$1"`
local file=`basename "$1"`
pushd "$dir" &>/dev/null || return $? # On error, return error code
echo "`pwd -P`/$file" # output full, link-resolved path with filename
popd &> /dev/null
}

on OS X you can use
stat -f "%N" YOUR_PATH
on linux you might have realpath executable. if not, the following might work (not only for links):
readlink -c YOUR_PATH

There's another method. You can use python embedding in bash script to resolve a relative path.
abs_path=$(python3 - <<END
from pathlib import Path
path = str(Path("$1").expanduser().resolve())
print(path)
END
)

self edit, I just noticed the OP said he's not looking for symlinks resolved:
"But for the functions I've written I need paths that are absolute, but do not have their symlinks resolved."
So guess this isn't so apropos to his question after all. :)
Since I've run into this many times over the years, and this time around I needed a pure bash portable version that I could use on OSX and linux, I went ahead and wrote one:
The living version lives here:
https://github.com/keen99/shell-functions/tree/master/resolve_path
but for the sake of SO, here's the current version (I feel it's well tested..but I'm open to feedback!)
Might not be difficult to make it work for plain bourne shell (sh), but I didn't try...I like $FUNCNAME too much. :)
#!/bin/bash
resolve_path() {
#I'm bash only, please!
# usage: resolve_path <a file or directory>
# follows symlinks and relative paths, returns a full real path
#
local owd="$PWD"
#echo "$FUNCNAME for $1" >&2
local opath="$1"
local npath=""
local obase=$(basename "$opath")
local odir=$(dirname "$opath")
if [[ -L "$opath" ]]
then
#it's a link.
#file or directory, we want to cd into it's dir
cd $odir
#then extract where the link points.
npath=$(readlink "$obase")
#have to -L BEFORE we -f, because -f includes -L :(
if [[ -L $npath ]]
then
#the link points to another symlink, so go follow that.
resolve_path "$npath"
#and finish out early, we're done.
return $?
#done
elif [[ -f $npath ]]
#the link points to a file.
then
#get the dir for the new file
nbase=$(basename $npath)
npath=$(dirname $npath)
cd "$npath"
ndir=$(pwd -P)
retval=0
#done
elif [[ -d $npath ]]
then
#the link points to a directory.
cd "$npath"
ndir=$(pwd -P)
retval=0
#done
else
echo "$FUNCNAME: ERROR: unknown condition inside link!!" >&2
echo "opath [[ $opath ]]" >&2
echo "npath [[ $npath ]]" >&2
return 1
fi
else
if ! [[ -e "$opath" ]]
then
echo "$FUNCNAME: $opath: No such file or directory" >&2
return 1
#and break early
elif [[ -d "$opath" ]]
then
cd "$opath"
ndir=$(pwd -P)
retval=0
#done
elif [[ -f "$opath" ]]
then
cd $odir
ndir=$(pwd -P)
nbase=$(basename "$opath")
retval=0
#done
else
echo "$FUNCNAME: ERROR: unknown condition outside link!!" >&2
echo "opath [[ $opath ]]" >&2
return 1
fi
fi
#now assemble our output
echo -n "$ndir"
if [[ "x${nbase:=}" != "x" ]]
then
echo "/$nbase"
else
echo
fi
#now return to where we were
cd "$owd"
return $retval
}
here's a classic example, thanks to brew:
%% ls -l `which mvn`
lrwxr-xr-x 1 draistrick 502 29 Dec 17 10:50 /usr/local/bin/mvn# -> ../Cellar/maven/3.2.3/bin/mvn
use this function and it will return the -real- path:
%% cat test.sh
#!/bin/bash
. resolve_path.inc
echo
echo "relative symlinked path:"
which mvn
echo
echo "and the real path:"
resolve_path `which mvn`
%% test.sh
relative symlinked path:
/usr/local/bin/mvn
and the real path:
/usr/local/Cellar/maven/3.2.3/libexec/bin/mvn

Do you have to use bash exclusively? I needed to do this and got fed up with differences between Linux and OS X. So I used PHP for a quick and dirty solution.
#!/usr/bin/php <-- or wherever
<?php
{
if($argc!=2)
exit();
$fname=$argv[1];
if(!file_exists($fname))
exit();
echo realpath($fname)."\n";
}
?>
I know it's not a very elegant solution but it does work.

Related

check to see if filepath exists bash linux script [duplicate]

What command checks if a directory exists or not within a Bash shell script?
To check if a directory exists:
if [ -d "$DIRECTORY" ]; then
echo "$DIRECTORY does exist."
fi
To check if a directory does not exist:
if [ ! -d "$DIRECTORY" ]; then
echo "$DIRECTORY does not exist."
fi
However, as Jon Ericson points out, subsequent commands may not work as intended if you do not take into account that a symbolic link to a directory will also pass this check.
E.g. running this:
ln -s "$ACTUAL_DIR" "$SYMLINK"
if [ -d "$SYMLINK" ]; then
rmdir "$SYMLINK"
fi
Will produce the error message:
rmdir: failed to remove `symlink': Not a directory
So symbolic links may have to be treated differently, if subsequent commands expect directories:
if [ -d "$LINK_OR_DIR" ]; then
if [ -L "$LINK_OR_DIR" ]; then
# It is a symlink!
# Symbolic link specific commands go here.
rm "$LINK_OR_DIR"
else
# It's a directory!
# Directory command goes here.
rmdir "$LINK_OR_DIR"
fi
fi
Take particular note of the double-quotes used to wrap the variables. The reason for this is explained by 8jean in another answer.
If the variables contain spaces or other unusual characters it will probably cause the script to fail.
Always wrap variables in double quotes when referencing them in a Bash script.
if [ -d "$DIRECTORY" ]; then
# Will enter here if $DIRECTORY exists, even if it contains spaces
fi
Kids these days put spaces and lots of other funny characters in their directory names. (Spaces! Back in my day, we didn't have no fancy spaces!)
One day, one of those kids will run your script with $DIRECTORY set to "My M0viez" and your script will blow up. You don't want that. So use double quotes.
Note the -d test can produce some surprising results:
$ ln -s tmp/ t
$ if [ -d t ]; then rmdir t; fi
rmdir: directory "t": Path component not a directory
File under: "When is a directory not a directory?" The answer: "When it's a symlink to a directory." A slightly more thorough test:
if [ -d t ]; then
if [ -L t ]; then
rm t
else
rmdir t
fi
fi
You can find more information in the Bash manual on Bash conditional expressions and the [ builtin command and the [[ compound commmand.
I find the double-bracket version of test makes writing logic tests more natural:
if [[ -d "${DIRECTORY}" && ! -L "${DIRECTORY}" ]] ; then
echo "It's a bona-fide directory"
fi
Shorter form:
# if $DIR is a directory, then print yes
[ -d "$DIR" ] && echo "Yes"
A simple script to test if a directory or file is present or not:
if [ -d /home/ram/dir ] # For file "if [ -f /home/rama/file ]"
then
echo "dir present"
else
echo "dir not present"
fi
A simple script to check whether the directory is present or not:
mkdir tempdir # If you want to check file use touch instead of mkdir
ret=$?
if [ "$ret" == "0" ]
then
echo "dir present"
else
echo "dir not present"
fi
The above scripts will check if the directory is present or not
$? if the last command is a success it returns "0", else a non-zero value.
Suppose tempdir is already present. Then mkdir tempdir will give an error like below:
mkdir: cannot create directory ‘tempdir’: File exists
To check if a directory exists you can use a simple if structure like this:
if [ -d directory/path to a directory ] ; then
# Things to do
else #if needed #also: elif [new condition]
# Things to do
fi
You can also do it in the negative:
if [ ! -d directory/path to a directory ] ; then
# Things to do when not an existing directory
Note: Be careful. Leave empty spaces on either side of both opening and closing braces.
With the same syntax you can use:
-e: any kind of archive
-f: file
-h: symbolic link
-r: readable file
-w: writable file
-x: executable file
-s: file size greater than zero
You can use test -d (see man test).
-d file True if file exists and is a directory.
For example:
test -d "/etc" && echo Exists || echo Does not exist
Note: The test command is same as conditional expression [ (see: man [), so it's portable across shell scripts.
[ - This is a synonym for the test builtin, but the last argument must, be a literal ], to match the opening [.
For possible options or further help, check:
help [
help test
man test or man [
Or for something completely useless:
[ -d . ] || echo "No"
Here's a very pragmatic idiom:
(cd $dir) || return # Is this a directory,
# and do we have access?
I typically wrap it in a function:
can_use_as_dir() {
(cd ${1:?pathname expected}) || return
}
Or:
assert_dir_access() {
(cd ${1:?pathname expected}) || exit
}
The nice thing about this approach is that I do not have to think of a good error message.
cd will give me a standard one line message to standard error already. It will also give more information than I will be able to provide. By performing the cd inside a subshell ( ... ), the command does not affect the current directory of the caller. If the directory exists, this subshell and the function are just a no-op.
Next is the argument that we pass to cd: ${1:?pathname expected}. This is a more elaborate form of parameter substitution which is explained in more detail below.
Tl;dr: If the string passed into this function is empty, we again exit from the subshell ( ... ) and return from the function with the given error message.
Quoting from the ksh93 man page:
${parameter:?word}
If parameter is set and is non-null then substitute its value;
otherwise, print word and exit from the shell (if not interactive).
If word is omitted then a standard message is printed.
and
If the colon : is omitted from the above expressions, then the
shell only checks whether parameter is set or not.
The phrasing here is peculiar to the shell documentation, as word may refer to any reasonable string, including whitespace.
In this particular case, I know that the standard error message 1: parameter not set is not sufficient, so I zoom in on the type of value that we expect here - the pathname of a directory.
A philosophical note:
The shell is not an object oriented language, so the message says pathname, not directory. At this level, I'd rather keep it simple - the arguments to a function are just strings.
if [ -d "$Directory" -a -w "$Directory" ]
then
#Statements
fi
The above code checks if the directory exists and if it is writable.
More features using find
Check existence of the folder within sub-directories:
found=`find -type d -name "myDirectory"`
if [ -n "$found" ]
then
# The variable 'found' contains the full path where "myDirectory" is.
# It may contain several lines if there are several folders named "myDirectory".
fi
Check existence of one or several folders based on a pattern within the current directory:
found=`find -maxdepth 1 -type d -name "my*"`
if [ -n "$found" ]
then
# The variable 'found' contains the full path where folders "my*" have been found.
fi
Both combinations. In the following example, it checks the existence of the folder in the current directory:
found=`find -maxdepth 1 -type d -name "myDirectory"`
if [ -n "$found" ]
then
# The variable 'found' is not empty => "myDirectory"` exists.
fi
DIRECTORY=/tmp
if [ -d "$DIRECTORY" ]; then
echo "Exists"
fi
Try online
Actually, you should use several tools to get a bulletproof approach:
DIR_PATH=`readlink -f "${the_stuff_you_test}"` # Get rid of symlinks and get abs path
if [[ -d "${DIR_PATH}" ]] ; Then # Now you're testing
echo "It's a dir";
fi
There isn't any need to worry about spaces and special characters as long as you use "${}".
Note that [[]] is not as portable as [], but since most people work with modern versions of Bash (since after all, most people don't even work with command line :-p), the benefit is greater than the trouble.
Have you considered just doing whatever you want to do in the if rather than looking before you leap?
I.e., if you want to check for the existence of a directory before you enter it, try just doing this:
if pushd /path/you/want/to/enter; then
# Commands you want to run in this directory
popd
fi
If the path you give to pushd exists, you'll enter it and it'll exit with 0, which means the then portion of the statement will execute. If it doesn't exist, nothing will happen (other than some output saying the directory doesn't exist, which is probably a helpful side-effect anyways for debugging).
It seems better than this, which requires repeating yourself:
if [ -d /path/you/want/to/enter ]; then
pushd /path/you/want/to/enter
# Commands you want to run in this directory
popd
fi
The same thing works with cd, mv, rm, etc... if you try them on files that don't exist, they'll exit with an error and print a message saying it doesn't exist, and your then block will be skipped. If you try them on files that do exist, the command will execute and exit with a status of 0, allowing your then block to execute.
[[ -d "$DIR" && ! -L "$DIR" ]] && echo "It's a directory and not a symbolic link"
N.B: Quoting variables is a good practice.
Explanation:
-d: check if it's a directory
-L: check if it's a symbolic link
To check more than one directory use this code:
if [ -d "$DIRECTORY1" ] && [ -d "$DIRECTORY2" ] then
# Things to do
fi
Check if the directory exists, else make one:
[ -d "$DIRECTORY" ] || mkdir $DIRECTORY
[ -d ~/Desktop/TEMPORAL/ ] && echo "DIRECTORY EXISTS" || echo "DIRECTORY DOES NOT EXIST"
Using the -e check will check for files and this includes directories.
if [ -e ${FILE_PATH_AND_NAME} ]
then
echo "The file or directory exists."
fi
This answer wrapped up as a shell script
Examples
$ is_dir ~
YES
$ is_dir /tmp
YES
$ is_dir ~/bin
YES
$ mkdir '/tmp/test me'
$ is_dir '/tmp/test me'
YES
$ is_dir /asdf/asdf
NO
# Example of calling it in another script
DIR=~/mydata
if [ $(is_dir $DIR) == "NO" ]
then
echo "Folder doesnt exist: $DIR";
exit;
fi
is_dir
function show_help()
{
IT=$(CAT <<EOF
usage: DIR
output: YES or NO, depending on whether or not the directory exists.
)
echo "$IT"
exit
}
if [ "$1" == "help" ]
then
show_help
fi
if [ -z "$1" ]
then
show_help
fi
DIR=$1
if [ -d $DIR ]; then
echo "YES";
exit;
fi
echo "NO";
As per Jonathan's comment:
If you want to create the directory and it does not exist yet, then the simplest technique is to use mkdir -p which creates the directory — and any missing directories up the path — and does not fail if the directory already exists, so you can do it all at once with:
mkdir -p /some/directory/you/want/to/exist || exit 1
if [ -d "$DIRECTORY" ]; then
# Will enter here if $DIRECTORY exists
fi
This is not completely true...
If you want to go to that directory, you also need to have the execute rights on the directory. Maybe you need to have write rights as well.
Therefore:
if [ -d "$DIRECTORY" ] && [ -x "$DIRECTORY" ] ; then
# ... to go to that directory (even if DIRECTORY is a link)
cd $DIRECTORY
pwd
fi
if [ -d "$DIRECTORY" ] && [ -w "$DIRECTORY" ] ; then
# ... to go to that directory and write something there (even if DIRECTORY is a link)
cd $DIRECTORY
touch foobar
fi
In kind of a ternary form,
[ -d "$directory" ] && echo "exist" || echo "not exist"
And with test:
test -d "$directory" && echo "exist" || echo "not exist"
file="foo"
if [[ -e "$file" ]]; then echo "File Exists"; fi;
The ls command in conjunction with -l (long listing) option returns attributes information about files and directories.
In particular the first character of ls -l output it is usually a d or a - (dash). In case of a d the one listed is a directory for sure.
The following command in just one line will tell you if the given ISDIR variable contains a path to a directory or not:
[[ $(ls -ld "$ISDIR" | cut -c1) == 'd' ]] &&
echo "YES, $ISDIR is a directory." ||
echo "Sorry, $ISDIR is not a directory"
Practical usage:
[claudio#nowhere ~]$ ISDIR="$HOME/Music"
[claudio#nowhere ~]$ ls -ld "$ISDIR"
drwxr-xr-x. 2 claudio claudio 4096 Aug 23 00:02 /home/claudio/Music
[claudio#nowhere ~]$ [[ $(ls -ld "$ISDIR" | cut -c1) == 'd' ]] &&
echo "YES, $ISDIR is a directory." ||
echo "Sorry, $ISDIR is not a directory"
YES, /home/claudio/Music is a directory.
[claudio#nowhere ~]$ touch "empty file.txt"
[claudio#nowhere ~]$ ISDIR="$HOME/empty file.txt"
[claudio#nowhere ~]$ [[ $(ls -ld "$ISDIR" | cut -c1) == 'd' ]] &&
echo "YES, $ISDIR is a directory." ||
echo "Sorry, $ISDIR is not a directoy"
Sorry, /home/claudio/empty file.txt is not a directory
There are great solutions out there, but ultimately every script will fail if you're not in the right directory. So code like this:
if [ -d "$LINK_OR_DIR" ]; then
if [ -L "$LINK_OR_DIR" ]; then
# It is a symlink!
# Symbolic link specific commands go here
rm "$LINK_OR_DIR"
else
# It's a directory!
# Directory command goes here
rmdir "$LINK_OR_DIR"
fi
fi
will execute successfully only if at the moment of execution you're in a directory that has a subdirectory that you happen to check for.
I understand the initial question like this: to verify if a directory exists irrespective of the user's position in the file system. So using the command 'find' might do the trick:
dir=" "
echo "Input directory name to search for:"
read dir
find $HOME -name $dir -type d
This solution is good because it allows the use of wildcards, a useful feature when searching for files/directories. The only problem is that, if the searched directory doesn't exist, the 'find' command will print nothing to standard output (not an elegant solution for my taste) and will have nonetheless a zero exit. Maybe someone could improve on this.
The below find can be used,
find . -type d -name dirname -prune -print
One Liner:
[[ -d $Directory ]] && echo true
(1)
[ -d Piyush_Drv1 ] && echo ""Exists"" || echo "Not Exists"
(2)
[ `find . -type d -name Piyush_Drv1 -print | wc -l` -eq 1 ] && echo Exists || echo "Not Exists"
(3)
[[ -d run_dir && ! -L run_dir ]] && echo Exists || echo "Not Exists"
If an issue is found with one of the approaches provided above:
With the ls command; the cases when a directory does not exists - an error message is shown
[[ `ls -ld SAMPLE_DIR| grep ^d | wc -l` -eq 1 ]] && echo exists || not exists
-ksh: not: not found [No such file or directory]

linux zip and exclude dir via bash/shell script

I am trying to write a bash/shell script to zip up a specific folder and ignore certain sub-dirs in that folder.
This is the folder I am trying to zip "sync_test5":
My bash script generates an ignore list (based on) and calls the zip function like this:
#!/bin/bash
SYNC_WEB_ROOT_BASE_DIR="/home/www-data/public_html"
SYNC_WEB_ROOT_BACKUP_DIR="sync_test5"
SYNC_WEB_ROOT_IGNORE_DIR="dir_to_ignore dir2_to_ignore"
ignorelist=""
if [ "$SYNC_WEB_ROOT_IGNORE_DIR" != "" ];
then
for ignoredir in $SYNC_WEB_ROOT_IGNORE_DIR
do
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/**\*"
done
fi
FILE="$SYNC_BACKUP_DIR/$DATETIMENOW.website.zip"
cd $SYNC_WEB_ROOT_BASE_DIR;
zip -r $FILE $SYNC_WEB_ROOT_BACKUP_DIR -x $ignorelist >/dev/null
echo "Done"
Now this script runs without error, however it is not ignoring/excluding the dirs I've specified.
So, I had the shell script output the command it tried to run, which was:
zip -r 12-08-2014_072810.website.zip sync_test5 -x sync_test5/dir_to_ignore/**\* sync_test5/dir2_to_ignore/**\*
Now If I run the above command directly in putty like this, it works:
So, why doesn't my shell script exclude working as intended? the command that is being executed is identical (in shell and putty directly).
Because backslash quotings in a variable after word splitting are not evaluated.
If you have a='123\4', echo $a would give
123\4
But if you do it directly like echo 123\4, you'd get
1234
Clearly the arguments you pass with the variable and without the variables are different.
You probably just meant to not quote your argument with backslash:
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/***"
Btw, what actual works is a non-evaluated glob pattern:
zip -r 12-08-2014_072810.website.zip sync_test5 -x 'sync_test5/dir_to_ignore/***' 'sync_test5/dir2_to_ignore/***'
You can verify this with
echo zip -r 12-08-2014_072810.website.zip sync_test5 -x sync_test5/dir_to_ignore/**\* sync_test5/dir2_to_ignore/**\*
And this is my suggestion:
#!/bin/bash
SYNC_WEB_ROOT_BASE_DIR="/home/www-data/public_html"
SYNC_WEB_ROOT_BACKUP_DIR="sync_test5"
SYNC_WEB_ROOT_IGNORE_DIR=("dir_to_ignore" "dir2_to_ignore")
IGNORE_LIST=()
if [[ -n $SYNC_WEB_ROOT_IGNORE_DIR ]]; then
for IGNORE_DIR in "${SYNC_WEB_ROOT_IGNORE_DIR[#]}"; do
IGNORE_LIST+=("$SYNC_WEB_ROOT_BACKUP_DIR/$IGNORE_DIR/***") ## "$SYNC_WEB_ROOT_BACKUP_DIR/$IGNORE_DIR/*" perhaps is enough?
done
fi
FILE="$SYNC_BACKUP_DIR/$DATETIMENOW.website.zip" ## Where is $SYNC_BACKUP_DIR set?
cd "$SYNC_WEB_ROOT_BASE_DIR";
zip -r "$FILE" "$SYNC_WEB_ROOT_BACKUP_DIR" -x "${IGNORE_LIST[#]}" >/dev/null
echo "Done"
This is what I ended up with:
#!/bin/bash
# This script zips a directory, excluding specified files, types and subdirectories.
# while zipping the directory it excludes hidden directories and certain file types
[[ "`/usr/bin/tty`" == "not a tty" ]] && . ~/.bash_profile
DIRECTORY=$(cd `dirname $0` && pwd)
if [[ -z $1 ]]; then
echo "Usage: managed_directory_compressor /your-directory/ zip-file-name"
else
DIRECTORY_TO_COMPRESS=${1%/}
ZIPPED_FILE="$2.zip"
COMPRESS_IGNORE_FILE=("\.git" "*.zip" "*.csv" "*.json" "gulpfile.js" "*.rb" "*.bak" "*.swp" "*.back" "*.merge" "*.txt" "*.sh" "bower_components" "node_modules")
COMPRESS_IGNORE_DIR=("bower_components" "node_modules")
IGNORE_LIST=("*/\.*" "\.* "\/\.*"")
if [[ -n $COMPRESS_IGNORE_FILE ]]; then
for IGNORE_FILES in "${COMPRESS_IGNORE_FILE[#]}"; do
IGNORE_LIST+=("$DIRECTORY_TO_COMPRESS/$IGNORE_FILES/*")
done
for IGNORE_DIR in "${COMPRESS_IGNORE_DIR[#]}"; do
IGNORE_LIST+=("$DIRECTORY_TO_COMPRESS/$IGNORE_DIR/")
done
fi
zip -r "$ZIPPED_FILE" "$DIRECTORY_TO_COMPRESS" -x "${IGNORE_LIST[#]}" # >/dev/null
# echo zip -r "$ZIPPED_FILE" "$DIRECTORY_TO_COMPRESS" -x "${IGNORE_LIST[#]}" # >/dev/null
echo $DIRECTORY_TO_COMPRESS "compressed as" $ZIPPED_FILE.
fi
After a few trial and error, I have managed to fix this problem by changing this line:
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/**\*"
to:
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/***"
Not sure why this worked, but it does :)

Symlink check - Linux Bash Script

I'm trying to create a script that searches through a directory to find symlinks that point to non-existing objects.
I have a file in a directory with a deleted symlink, but for some reason when i run the below script It says file exists.
#!/bin/bash
ls -l $1 |
if [ -d $1 ]
then
while read file
do
if test -e $1
then
echo "file exists"
else
echo "file does not exist"
fi
done
else
echo "No directory given"
fi
Thanks
Check this page. It has a test for broken links. It uses the -h operator to identify a symlink and the -e operator to check existance.
From that page:
linkchk () {
for element in $1/*; do
[ -h "$element" -a ! -e "$element" ] && echo \"$element\"
[ -d "$element" ] && linkchk $element
# Of course, '-h' tests for symbolic link, '-d' for directory.
done
}
# Send each arg that was passed to the script to the linkchk() function
#+ if it is a valid directoy. If not, then print the error message
#+ and usage info.
##################
for directory in $directorys; do
if [ -d $directory ]
then linkchk $directory
else
echo "$directory is not a directory"
echo "Usage: $0 dir1 dir2 ..."
fi
done
exit $?
You can test whether link is valid or not using:
[[ -f "$link" ]] && echo "points to a valid file"
To check if it is indeed a link use -L:
[[ -L "$link" ]] && echo "it's a link"
There seems to be a program named symlinks that does, among other things, what you're looking for.

Bash is symlinking to working directory instead of specified directory

I am working on a bash script that I am working on for a universal Linux dotfile install script. I am attempting to get the symlinking working but I have been bashing (no pun intended) my head against the wall trying to figure out why the symlinks will not work and the copying will not work. I currently have this separated into multiple files so I don't have if statements three miles long.
ultimate-install.sh
#! /bin/bash
#
# The ultimate install script for all dotfiles.
if [[ -z "$1" ]]; then
echo "Please specify the directory where all of you dotfiles are located."
exit 1
fi
# Makes sure that the directory does NOT have a trailing slash!
if [[ ${1:(-1)} == "/" ]]; then
DOTFILE_DIR=${1:0:${#1} - 1}
else
DOTFILE_DIR="$1"
fi
# TODO: Clean this mess up and make it more concise.
if [[ -z "$2" ]]; then
if [[ ! -d $HOME/.config/old_dotfiles ]]; then
mkdir "$HOME/.config/old_dotfiles"
fi
BACKUP_DIR="$HOME/.config/old_dotfiles"
else
if [[ -d "$2" ]]; then
BACKUP_DIR="$2"
else
mkdir "$2"
BACKUP_DIR="$2"
fi
fi
read DECISION
if [ $DECISION == "N" -o $DECISION == "n" ]; then
echo "Aborting installation!"
exit
fi
read DECISION
echo
if [ $DECISION == "N" -o $DECISION == "n" ]; then
source src/no-prompts.sh "$DOTFILE_DIR" "$BACKUP_DIR"
else
source src/prompts.sh "$DOTFILE_DIR" "$BACKUP_DIR"
fi
echo "Installation complete. Old dotfiles are backed up to $BACKUP_DIR."
src/no-prompts.sh
#! /bin/bash
#
# Maintained by Daniel Seymour
DOTFILE_DIR="$1"
BACKUP_DIR="$2"
TEST_DIR="/home/daniel/dotfile-test"
function no_prompt_install(){
FILE_NAME="$1"
if [ "${FILE_NAME:0:1}" == "." ]; then
ln -s "$FILE_NAME $TEST_DIR/$FILE_NAME"
else
ln -s ".$FILE_NAME $TEST_DIR/$FILE_NAME"
fi
}
# TODO: implement a check for file type and deal with unknown files.
for FILE in $DOTFILE_DIR/*; do
cp $FILE $BACKUP_DIR
no_prompt_install $FILE
done
src/prompts.sh
#! /bin/bash
#
# Maintained by Daniel Seymour
DOTFILE_DIR="$1"
BACKUP_DIR="$2"
TEST_DIR="/home/daniel/dotfile-test"
function prompt_install {
FILE_PATH=$1
FILE_NAME=${FILE_PATH##*/}
echo "Would you like to install $FILE_NAME? [Y, n]"
read DECISION
if [ $DECISION == "n" -o $DECISION == "N" ]; then
echo "Not installing."
return
else
# TODO: Clean this up into one statement.
if [ ${FILE_NAME:0:1} == "." ]; then
rm -rf "$TEST_DIR/$FILE_NAME"
ln -sn "$FILE_PATH $TEST_DIR/$FILE_NAME"
else
FILE_NAME="."$FILE_NAME
rm -rf "$TEST_DIR/$FILE_NAME"
ln -sn "$FILE_PATH $TEST_DIR/$FILE_NAME"
fi
fi
}
# TODO: implement a check for file type and deal with unknown files.
for FILE in $DOTFILE_DIR/*; do
cp $FILE $BACKUP_DIR
prompt_install $FILE
done
The above is trimmed for long echo statements that do a lot of explaining.
The basic idea of this script is to take as many as two arguments (the dotfile directory to install and if specified, the custom backup directory, $1 and $2 respectively). The script should then copy all of the files in the target directory to BACKUP_DIR and symlink all of the dotfiles in the DOTFILE_DIR to TEST_DIR. (TEST_DIR will be $HOME in the production scripts.) Great in theory, right?
The complication comes when I run the script. None of the files are copied or symlinked as they should be. Instead, I end up with NO copy (probably due to the same issue as the symlink not working) and a broken symlink in the current directory.
One last piece of information. I am executing the file from the directory that contains ultimate-install.sh (/home/daniel/Projects/Git-Repos/Ultimate-Dotfile-Install-Scripts).
So where did I go wrong?
PS Please don't comment on the TODOs. :)
Short answer
Your quoting is wrong.
ln -sn -- "$FILE_PATH" "$TEST_DIR/$FILE_NAME"
Longer answer
This does not really relate to your problem, but I want to point it out.
Do not use "" inside [[ ]], so instead of this if [[ -z "$1" ]]; then use this if [[ -z $1 ]]; then
What is the point of making sure that directory does not have a trailing slash? It has no effect! /usr/bin/ is the same directory as /usr/bin or /usr////bin or /usr////////bin//////
Do not check if a directory exists when creating directories. Use -p option! Example: mkdir -p "$HOME/.config/old_dotfiles"
Instead of if [ $DECISION == "N" -o $DECISION == "n" ]; use if [[ ${DECISION^^} == N]];
I have another great answer about bash code style HERE. Please check it out! Also read the comments, since I was explaining there exactly your issue.

How do you normalize a file path in Bash?

I want to transform /foo/bar/.. to /foo
Is there a bash command which does this?
Edit: in my practical case, the directory does exist.
if you're wanting to chomp part of a filename from the path, "dirname" and "basename" are your friends, and "realpath" is handy too.
dirname /foo/bar/baz
# /foo/bar
basename /foo/bar/baz
# baz
dirname $( dirname /foo/bar/baz )
# /foo
realpath ../foo
# ../foo: No such file or directory
realpath /tmp/../tmp/../tmp
# /tmp
realpath alternatives
If realpath is not supported by your shell, you can try
readlink -f /path/here/..
Also
readlink -m /path/there/../../
Works the same as
realpath -s /path/here/../../
in that the path doesn't need to exist to be normalized.
I don't know if there is a direct bash command to do this, but I usually do
normalDir="`cd "${dirToNormalize}";pwd`"
echo "${normalDir}"
and it works well.
Try realpath. Below is the source in its entirety, hereby donated to the public domain.
// realpath.c: display the absolute path to a file or directory.
// Adam Liss, August, 2007
// This program is provided "as-is" to the public domain, without express or
// implied warranty, for any non-profit use, provided this notice is maintained.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <libgen.h>
#include <limits.h>
static char *s_pMyName;
void usage(void);
int main(int argc, char *argv[])
{
char
sPath[PATH_MAX];
s_pMyName = strdup(basename(argv[0]));
if (argc < 2)
usage();
printf("%s\n", realpath(argv[1], sPath));
return 0;
}
void usage(void)
{
fprintf(stderr, "usage: %s PATH\n", s_pMyName);
exit(1);
}
A portable and reliable solution is to use python, which is preinstalled pretty much everywhere (including Darwin). You have two options:
abspath returns an absolute path but does not resolve symlinks:
python -c "import os,sys; print(os.path.abspath(sys.argv[1]))" path/to/file
realpath returns an absolute path and in doing so resolves symlinks, generating a canonical path:
python -c "import os,sys; print(os.path.realpath(sys.argv[1]))" path/to/file
In each case, path/to/file can be either a relative or absolute path.
Use the readlink utility from the coreutils package.
MY_PATH=$(readlink -f "$0")
Old question, but there is much simpler way if you are dealing with full path names at the shell level:
abspath="$( cd "$path" && pwd )"
As the cd happens in a subshell it does not impact the main script.
Two variations, supposing your shell built-in commands accept -L and -P, are:
abspath="$( cd -P "$path" && pwd -P )" #physical path with resolved symlinks
abspath="$( cd -L "$path" && pwd -L )" #logical path preserving symlinks
Personally, I rarely need this later approach unless I'm fascinated with symbolic links for some reason.
FYI: variation on obtaining the starting directory of a script which works even if the script changes it's current directory later on.
name0="$(basename "$0")"; #base name of script
dir0="$( cd "$( dirname "$0" )" && pwd )"; #absolute starting dir
The use of CD assures you always have the absolute directory, even if the script is run by commands such as ./script.sh which, without the cd/pwd, often gives just .. Useless if the script does a cd later on.
readlink is the bash standard for obtaining the absolute path. It also has the advantage of returning empty strings if paths or a path doesn't exist (given the flags to do so).
To get the absolute path to a directory that may or may not exist, but who's parents do exist, use:
abspath=$(readlink -f $path)
To get the absolute path to a directory that must exist along with all parents:
abspath=$(readlink -e $path)
To canonicalise the given path and follow symlinks if they happen to exist, but otherwise ignore missing directories and just return the path anyway, it's:
abspath=$(readlink -m $path)
The only downside is that readlink will follow links. If you do not want to follow links, you can use this alternative convention:
abspath=$(cd ${path%/*} && echo $PWD/${path##*/})
That will chdir to the directory part of $path and print the current directory along with the file part of $path. If it fails to chdir, you get an empty string and an error on stderr.
As Adam Liss noted realpath is not bundled with every distribution. Which is a shame, because it is the best solution. The provided source code is great, and I will probably start using it now. Here is what I have been using until now, which I share here just for completeness:
get_abs_path() {
local PARENT_DIR=$(dirname "$1")
cd "$PARENT_DIR"
local ABS_PATH="$(pwd)"/"$(basename "$1")"
cd - >/dev/null
echo "$ABS_PATH"
}
If you want it to resolve symlinks, just replace pwd with pwd -P.
My recent solution was:
pushd foo/bar/..
dir=`pwd`
popd
Based on the answer of Tim Whitcomb.
Not exactly an answer but perhaps a follow-up question (original question was not explicit):
readlink is fine if you actually want to follow symlinks. But there is also a use case for merely normalizing ./ and ../ and // sequences, which can be done purely syntactically, without canonicalizing symlinks. readlink is no good for this, and neither is realpath.
for f in $paths; do (cd $f; pwd); done
works for existing paths, but breaks for others.
A sed script would seem to be a good bet, except that you cannot iteratively replace sequences (/foo/bar/baz/../.. -> /foo/bar/.. -> /foo) without using something like Perl, which is not safe to assume on all systems, or using some ugly loop to compare the output of sed to its input.
FWIW, a one-liner using Java (JDK 6+):
jrunscript -e 'for (var i = 0; i < arguments.length; i++) {println(new java.io.File(new java.io.File(arguments[i]).toURI().normalize()))}' $paths
I'm late to the party, but this is the solution I've crafted after reading a bunch of threads like this:
resolve_dir() {
(builtin cd `dirname "${1/#~/$HOME}"`'/'`basename "${1/#~/$HOME}"` 2>/dev/null; if [ $? -eq 0 ]; then pwd; fi)
}
This will resolve the absolute path of $1, play nice with ~, keep symlinks in the path where they are, and it won't mess with your directory stack. It returns the full path or nothing if it doesn't exist. It expects $1 to be a directory and will probably fail if it's not, but that's an easy check to do yourself.
Talkative, and a bit late answer. I need to write one since I'm stuck on older RHEL4/5.
I handles absolute and relative links, and simplifies //, /./ and somedir/../ entries.
test -x /usr/bin/readlink || readlink () {
echo $(/bin/ls -l $1 | /bin/cut -d'>' -f 2)
}
test -x /usr/bin/realpath || realpath () {
local PATH=/bin:/usr/bin
local inputpath=$1
local changemade=1
while [ $changemade -ne 0 ]
do
changemade=0
local realpath=""
local token=
for token in ${inputpath//\// }
do
case $token in
""|".") # noop
;;
"..") # up one directory
changemade=1
realpath=$(dirname $realpath)
;;
*)
if [ -h $realpath/$token ]
then
changemade=1
target=`readlink $realpath/$token`
if [ "${target:0:1}" = '/' ]
then
realpath=$target
else
realpath="$realpath/$target"
fi
else
realpath="$realpath/$token"
fi
;;
esac
done
inputpath=$realpath
done
echo $realpath
}
mkdir -p /tmp/bar
(cd /tmp ; ln -s /tmp/bar foo; ln -s ../.././usr /tmp/bar/link2usr)
echo `realpath /tmp/foo`
The problem with realpath is that it is not available on BSD (or OSX for that matter). Here is a simple recipe extracted from a rather old (2009) article from Linux Journal, that is quite portable:
function normpath() {
# Remove all /./ sequences.
local path=${1//\/.\//\/}
# Remove dir/.. sequences.
while [[ $path =~ ([^/][^/]*/\.\./) ]]; do
path=${path/${BASH_REMATCH[0]}/}
done
echo $path
}
Notice this variant also does not require the path to exist.
Try our new Bash library product realpath-lib that we have placed on GitHub for free and unencumbered use. It's thoroughly documented and makes a great learning tool.
It resolves local, relative and absolute paths and doesn't have any dependencies except Bash 4+; so it should work just about anywhere. It's free, clean, simple and instructive.
You can do:
get_realpath <absolute|relative|symlink|local file path>
This function is the core of the library:
function get_realpath() {
if [[ -f "$1" ]]
then
# file *must* exist
if cd "$(echo "${1%/*}")" &>/dev/null
then
# file *may* not be local
# exception is ./file.ext
# try 'cd .; cd -;' *works!*
local tmppwd="$PWD"
cd - &>/dev/null
else
# file *must* be local
local tmppwd="$PWD"
fi
else
# file *cannot* exist
return 1 # failure
fi
# reassemble realpath
echo "$tmppwd"/"${1##*/}"
return 0 # success
}
It also contains functions to get_dirname, get_filename, get_ stemname and validate_path. Try it across platforms, and help to improve it.
Based on #Andre's answer, I might have a slightly better version, in case someone is after a loop-free, completely string-manipulation based solution. It is also useful for those who don't want to dereference any symlinks, which is the downside of using realpath or readlink -f.
It works on bash versions 3.2.25 and higher.
shopt -s extglob
normalise_path() {
local path="$1"
# get rid of /../ example: /one/../two to /two
path="${path//\/*([!\/])\/\.\./}"
# get rid of /./ and //* example: /one/.///two to /one/two
path="${path//#(\/\.\/|\/+(\/))//}"
# remove the last '/.'
echo "${path%%/.}"
}
$ normalise_path /home/codemedic/../codemedic////.config
/home/codemedic/.config
I made a builtin-only function to handle this with a focus on highest possible performance (for fun). It does not resolve symlinks, so it is basically the same as realpath -sm.
## A bash-only mimic of `realpath -sm`.
## Give it path[s] as argument[s] and it will convert them to clean absolute paths
abspath () {
${*+false} && { >&2 echo $FUNCNAME: missing operand; return 1; };
local c s p IFS='/'; ## path chunk, absolute path, input path, IFS for splitting paths into chunks
local -i r=0; ## return value
for p in "$#"; do
case "$p" in ## Check for leading backslashes, identify relative/absolute path
'') ((r|=1)); continue;;
//[!/]*) >&2 echo "paths =~ ^//[^/]* are impl-defined; not my problem"; ((r|=2)); continue;;
/*) ;;
*) p="$PWD/$p";; ## Prepend the current directory to form an absolute path
esac
s='';
for c in $p; do ## Let IFS split the path at '/'s
case $c in ### NOTE: IFS is '/'; so no quotes needed here
''|.) ;; ## Skip duplicate '/'s and '/./'s
..) s="${s%/*}";; ## Trim the previous addition to the absolute path string
*) s+=/$c;; ### NOTE: No quotes here intentionally. They make no difference, it seems
esac;
done;
echo "${s:-/}"; ## If xpg_echo is set, use `echo -E` or `printf $'%s\n'` instead
done
return $r;
}
Note: This function does not handle paths starting with //, as exactly two double slashes at the start of a path are implementation-defined behavior. However, it handles /, ///, and so on just fine.
This function seems to handle all edge cases properly, but there might still be some out there that I haven't dealt with.
Performance Note: when called with thousands of arguments, abspath runs about 10x slower than realpath -sm; when called with a single argument, abspath runs >110x faster than realpath -sm on my machine, mostly due to not needing to execute a new program every time.
If you just want to normalize a path, existed or not existed, without touching the file system, without resolving any links, and without external utils, here is a pure Bash function translated from Python's posixpath.normpath.
#!/usr/bin/env bash
# Normalize path, eliminating double slashes, etc.
# Usage: new_path="$(normpath "${old_path}")"
# Translated from Python's posixpath.normpath:
# https://github.com/python/cpython/blob/master/Lib/posixpath.py#L337
normpath() {
local IFS=/ initial_slashes='' comp comps=()
if [[ $1 == /* ]]; then
initial_slashes='/'
[[ $1 == //* && $1 != ///* ]] && initial_slashes='//'
fi
for comp in $1; do
[[ -z ${comp} || ${comp} == '.' ]] && continue
if [[ ${comp} != '..' || (-z ${initial_slashes} && ${#comps[#]} -eq 0) || (\
${#comps[#]} -gt 0 && ${comps[-1]} == '..') ]]; then
comps+=("${comp}")
elif ((${#comps[#]})); then
unset 'comps[-1]'
fi
done
comp="${initial_slashes}${comps[*]}"
printf '%s\n' "${comp:-.}"
}
Examples:
new_path="$(normpath '/foo/bar/..')"
echo "${new_path}"
# /foo
normpath "relative/path/with trailing slashs////"
# relative/path/with trailing slashs
normpath "////a/../lot/././/mess////./here/./../"
# /lot/mess
normpath ""
# .
# (empty path resolved to dot)
Personally, I cannot understand why Shell, a language often used for manipulating files, doesn't offer basic functions to deal with paths. In python, we have nice libraries like os.path or pathlib, which offers a whole bunch of tools to extract filename, extension, basename, path segments, split or join paths, to get absolute or normalized paths, to determine relations between paths, to do everything without much brain. And they take care of edge cases, and they're reliable. In Shell, to do any of these, either we call external executables, or we have to reinvent wheels with these extremely rudimentary and arcane syntaxes...
I needed a solution that would do all three:
Work on a stock Mac. realpath and readlink -f are addons
Resolve symlinks
Have error handling
None of the answers had both #1 and #2. I added #3 to save others any further yak-shaving.
#!/bin/bash
P="${1?Specify a file path}"
[ -e "$P" ] || { echo "File does not exist: $P"; exit 1; }
while [ -h "$P" ] ; do
ls="$(ls -ld "$P")"
link="$(expr "$ls" : '.*-> \(.*\)$')"
expr "$link" : '/.*' > /dev/null &&
P="$link" ||
P="$(dirname "$P")/$link"
done
echo "$(cd "$(dirname "$P")"; pwd)/$(basename "$P")"
Here is a short test case with some twisted spaces in the paths to fully exercise the quoting
mkdir -p "/tmp/test/ first path "
mkdir -p "/tmp/test/ second path "
echo "hello" > "/tmp/test/ first path / red .txt "
ln -s "/tmp/test/ first path / red .txt " "/tmp/test/ second path / green .txt "
cd "/tmp/test/ second path "
fullpath " green .txt "
cat " green .txt "
Based on loveborg's excellent python snippet, I wrote this:
#!/bin/sh
# Version of readlink that follows links to the end; good for Mac OS X
for file in "$#"; do
while [ -h "$file" ]; do
l=`readlink $file`
case "$l" in
/*) file="$l";;
*) file=`dirname "$file"`/"$l"
esac
done
#echo $file
python -c "import os,sys; print os.path.abspath(sys.argv[1])" "$file"
done
FILEPATH="file.txt"
echo $(realpath $(dirname $FILEPATH))/$(basename $FILEPATH)
This works even if the file doesn't exist. It does require the directory containing the file to exist.
I know this is an ancient question. I'm still offering an alternative. Recently I met the same issue and found no existing and portable command to do that. So I wrote the following shell script which includes a function that can do the trick.
#! /bin/sh
function normalize {
local rc=0
local ret
if [ $# -gt 0 ] ; then
# invalid
if [ "x`echo $1 | grep -E '^/\.\.'`" != "x" ] ; then
echo $1
return -1
fi
# convert to absolute path
if [ "x`echo $1 | grep -E '^\/'`" == "x" ] ; then
normalize "`pwd`/$1"
return $?
fi
ret=`echo $1 | sed 's;/\.\($\|/\);/;g' | sed 's;/[^/]*[^/.]\+[^/]*/\.\.\($\|/\);/;g'`
else
read line
normalize "$line"
return $?
fi
if [ "x`echo $ret | grep -E '/\.\.?(/|$)'`" != "x" ] ; then
ret=`normalize "$ret"`
rc=$?
fi
echo "$ret"
return $rc
}
https://gist.github.com/bestofsong/8830bdf3e5eb9461d27313c3c282868c
Since none of the presented solutions worked for me, in the case where a file does not exist, I implemented my idea.
The solution of André Anjos had the problem that paths beginning with ../../ were resolved wrongly. For example ../../a/b/ became a/b/.
function normalize_rel_path(){
local path=$1
result=""
IFS='/' read -r -a array <<< "$path"
i=0
for (( idx=${#array[#]}-1 ; idx>=0 ; idx-- )) ; do
c="${array[idx]}"
if [ -z "$c" ] || [[ "$c" == "." ]];
then
continue
fi
if [[ "$c" == ".." ]]
then
i=$((i+1))
elif [ "$i" -gt "0" ];
then
i=$((i-1))
else
if [ -z "$result" ];
then
result=$c
else
result=$c/$result
fi
fi
done
while [ "$i" -gt "0" ]; do
i=$((i-1))
result="../"$result
done
unset IFS
echo $result
}
I discovered today that you can use the stat command to resolve paths.
So for a directory like "~/Documents":
You can run this:
stat -f %N ~/Documents
To get the full path:
/Users/me/Documents
For symlinks, you can use the %Y format option:
stat -f %Y example_symlink
Which might return a result like:
/usr/local/sbin/example_symlink
The formatting options might be different on other versions of *NIX but these worked for me on OSX.
A simple solution using node.js:
#!/usr/bin/env node
process.stdout.write(require('path').resolve(process.argv[2]));

Resources