keeping track of a moving shell script - linux

I hope someone can help me out. For the past month or so I have be learning the Bash... I have a program ( a simple language study program ) that I want to be able to install and run from a script.
I have a script that will create a new folder and move itself into it. The way I am doing it at the moment is below, although I have had problems with arrays that I am using later. I was wondering if there was a cleaner way of getting the new path to file name. Any help or insight would be greatly appreciated.
#!/bin/bash
echo "# path to me ---------------> ${0} "
echo "# parent path --------------> ${0%/*} "
echo "# my name ------------------> ${0##*/} "
if [[ ! -d ${0%/*}/SomeNewFolder ]] && [[ ! -d ${0%/*}/../SomeNewFolder ]]
then
mkdir ${0%/*}/SomeNewFolder
mv ${0} ${0%/*}/SomeNewFolder/${0##*/}
fi
echo ${0%/*}
newpath=$(echo "${0%/*}/SomeNewFolder")
echo $newpath
All the best, Ben
exit

For clarity, I would probably declare named variables for your common values instead of constantly reusing the ${0} array. It's also good practice to quote variables and strings.
The only major issue I saw, was running ./script.sh would make $0 equal just the filename, so I add "./" to the beginning in that case.
#!/bin/bash -u
ME="${0}"
if [[ ! "$ME" =~ /^\// ]]; then
ME="./$ME"
fi
PARENT="${ME%/*}"
FILENAME="${ME##*/}"
FOLDER="SomeNewFolder"
NEW="$PARENT/$FOLDER"
if [[ ! -d "$NEW" ]] && [[ "${PARENT%/*}" != "$FOLDER" ]]; then
mkdir "$NEW"
mv "$ME" "$NEW"
fi
echo "$PARENT"
echo "$NEW"

Well, you could do something like this to get an absolute path:
PARENTPATH=$( cd "$( dirname "$0" )" && pwd )
NEWPATH=${PARENTPATH}/SomeNewFolder

me="$0"
newdir=SomeNewFolder
if [[ $me =~ ^/ ]] ; then
full_path="$me"
else
full_path="$PWD/$me"
fi
full_path="${full_path//\/\.\///}" # prettify
path_to_me="${full_path%/*}"
parent_dir="${path_to_me##*/}"
if [ ! "$parent_dir" = "$newdir" ] ; then
mkdir -p "$path_to_me/$newdir"
mv -f "$full_path" "$path_to_me/$newdir/"
fi
Basically similar to what lunixbochs was doing, but with a few minor alterations
lower case variable names so as not to be confused with environment variables
crudely estimates absolute path
-f and -p becuase interactivity is never cool, and why not

Installing and setting up programs is more appropriately done from a make file. Granted, it seems intimidating at first, but the basics, such as what you want, are quite simple. For your project, you would ideally have three item:
your program
your run script
your makefile i.e. your installer
This breaks apart each of these different components, making each of them easier to manage. If you tar them together, you can move the tar file to a new computer and reinstall without any changes. Bash is a wonderful tool, but an installer it is not.
Sample make script below:
.PHONY: all clean
SCRIPT=yourScriptName.sh
SUBFOLDER=someFolder
all: $(SCRIPT)
$(SCRIPT): $(SUBFOLDER)
cp $(SCRIPT) $(SUBFOLDER)
$(SUBFOLDER):
mkdir $(SUBFOLDER)
clean:
-rm -f $(SUBFOLDER)/$(SCRIPT)
-rmdir $(SUBFOLDER)
IMPORTANT! make is whitespace sensitive! Those indents are tabs not four spaces.

Related

Shell Scripting to Compress directory [duplicate]

This question already has an answer here:
Shell spacing in square brackets [duplicate]
(1 answer)
Closed 4 years ago.
$1 is file / folder that want to compressed
Output filename is the same name, plus current date and ext
if output name exist, then just give warning
Example:
./cmp.sh /home/user
It will be /home/user to /home/user_2018-03-11.tar.bz2
i already have lead, but i'm stuck
#!/bin/bash
if ["$1" == ""]; then
echo "Help : To compress file use argument with directory"
exit 0
fi
if [[ -f "$1" || -d "$1" ]]; then
tar -cvjSf $1"_"$(date '+%d-%m-%y').tar.bz2 $1
fi
but the output is _22-04-2018.tar.bz2
I see that you're using quotes to avoid the problem the underscore getting used as part of the variable name. So while $1 is a positional paramater, $1_ is a variable that you have not set in your script. You can avoid this issue by using curly braces, like ${1}. Anything inside the braces is part of the variable name, so ${1}_ works. This notation would be preferable to $1"_" which leaves a user-provided variable outside of quotes. (Of course, "$1"_ would do the job as wel.)
Also, it's probably safer to set the filename in a variable, then use that for all your needs:
#!/bin/bash
if [ -z "$1" ]; then
echo "Help : To compress file use argument with directory"
exit 0
fi
filename="${1}_$(date '+%F').tar.bz2"
if [ -e "$filename" ]; then
echo "WARNING: file exists: $filename" >&2
else
tar -cvjSf "$filename" "$#"
fi
Changes:
you need spaces around your square brackets in an if condition,
while you can test for equivalence to a null string, -z is cleaner, though you could also test for [ $# -eq 0 ], counting the parameters provided,
using $filename makes sure that your test and your tar will always use the same name, even if the script runs over midnight, and is way more readable,
variables should always be quoted.
Also, are you sure about the -S option for tar? On my system, that option extracts sparse files, and is only useful in conjunction with -x.
ALSO, I should note that as I've rewritten it, there's nothing in this script which is specific to bash, and it should be portable to POSIX shells as well (ash/dash/etc). Bash is great, but it's not universal, and if through your learning journey you can learn both, it will give you useful skills across multiple operating systems and environments.
Use -z switch to check if blank
#!/bin/bash
if [[ -z "$1" ]]; then
echo "Help : To compress file use argument with directory"
exit 0
fi
if [[ -f "$1" || -d "$1" ]]; then
tar -cvjSf $1"_"$(date '+%d-%m-%y').tar.bz2 $1
fi

extracting files that doesn't have a dir with the same name

sorry for that odd title. I didn't know how to word it the right way.
I'm trying to write a script to filter my wiki files to those got directories with the same name and the ones without. I'll elaborate further.
here is my file system:
what I need to do is print a list of those files which have directories in their name and another one of those without.
So my ultimate goal is getting:
with dirs:
Docs
Eng
Python
RHEL
To_do_list
articals
without dirs:
orphan.txt
orphan2.txt
orphan3.txt
I managed to get those files with dirs. Here is me code:
getname () {
file=$( basename "$1" )
file2=${file%%.*}
echo $file2
}
for d in Mywiki/* ; do
if [[ -f $d ]]; then
file=$(getname $d)
for x in Mywiki/* ; do
dir=$(getname $x)
if [[ -d $x ]] && [ $dir == $file ]; then
echo $dir
fi
done
fi
done
but stuck with getting those without. if this is the wrong way of doing this please clarify the right one.
any help appreciated. Thanks.
Here's a quick attempt.
for file in Mywiki/*.txt; do
nodir=${file##*/}
test -d "${file%.txt}" && printf "%s\n" "$nodir" >&3 || printf "%s\n" "$nodir"
done >with 3>without
This shamelessly uses standard output for the non-orphans. Maybe more robustly open another separate file descriptor for that.
Also notice how everything needs to be quoted unless you specifically require the shell to do whitespace tokenization and wildcard expansion on the value of a token. Here's the scoop on that.
That may not be the most efficient way of doing it, but you could take all files, remove the extension, and the check if there isn't a directory with that name.
Like this (untested code):
for file in Mywiki/* ; do
if [ -f "$d" ]; then
dirname=$(getname "$d")
if [ ! -d "Mywiki/$dirname" ]; then
echo "$file"
fi
fi
done
To List all the files in current dir
list1=`ls -p | grep -v /`
To List all the files in current dir without extension
list2=`ls -p | grep -v / | sed 's/\.[a-z]*//g'`
To List all the directories in current dir
list3=`ls -d */ | sed -e "s/\///g"`
Now you can get the desired directory listing using intersection of list2 and list3. Intersection of two lists in Bash

Bash is symlinking to working directory instead of specified directory

I am working on a bash script that I am working on for a universal Linux dotfile install script. I am attempting to get the symlinking working but I have been bashing (no pun intended) my head against the wall trying to figure out why the symlinks will not work and the copying will not work. I currently have this separated into multiple files so I don't have if statements three miles long.
ultimate-install.sh
#! /bin/bash
#
# The ultimate install script for all dotfiles.
if [[ -z "$1" ]]; then
echo "Please specify the directory where all of you dotfiles are located."
exit 1
fi
# Makes sure that the directory does NOT have a trailing slash!
if [[ ${1:(-1)} == "/" ]]; then
DOTFILE_DIR=${1:0:${#1} - 1}
else
DOTFILE_DIR="$1"
fi
# TODO: Clean this mess up and make it more concise.
if [[ -z "$2" ]]; then
if [[ ! -d $HOME/.config/old_dotfiles ]]; then
mkdir "$HOME/.config/old_dotfiles"
fi
BACKUP_DIR="$HOME/.config/old_dotfiles"
else
if [[ -d "$2" ]]; then
BACKUP_DIR="$2"
else
mkdir "$2"
BACKUP_DIR="$2"
fi
fi
read DECISION
if [ $DECISION == "N" -o $DECISION == "n" ]; then
echo "Aborting installation!"
exit
fi
read DECISION
echo
if [ $DECISION == "N" -o $DECISION == "n" ]; then
source src/no-prompts.sh "$DOTFILE_DIR" "$BACKUP_DIR"
else
source src/prompts.sh "$DOTFILE_DIR" "$BACKUP_DIR"
fi
echo "Installation complete. Old dotfiles are backed up to $BACKUP_DIR."
src/no-prompts.sh
#! /bin/bash
#
# Maintained by Daniel Seymour
DOTFILE_DIR="$1"
BACKUP_DIR="$2"
TEST_DIR="/home/daniel/dotfile-test"
function no_prompt_install(){
FILE_NAME="$1"
if [ "${FILE_NAME:0:1}" == "." ]; then
ln -s "$FILE_NAME $TEST_DIR/$FILE_NAME"
else
ln -s ".$FILE_NAME $TEST_DIR/$FILE_NAME"
fi
}
# TODO: implement a check for file type and deal with unknown files.
for FILE in $DOTFILE_DIR/*; do
cp $FILE $BACKUP_DIR
no_prompt_install $FILE
done
src/prompts.sh
#! /bin/bash
#
# Maintained by Daniel Seymour
DOTFILE_DIR="$1"
BACKUP_DIR="$2"
TEST_DIR="/home/daniel/dotfile-test"
function prompt_install {
FILE_PATH=$1
FILE_NAME=${FILE_PATH##*/}
echo "Would you like to install $FILE_NAME? [Y, n]"
read DECISION
if [ $DECISION == "n" -o $DECISION == "N" ]; then
echo "Not installing."
return
else
# TODO: Clean this up into one statement.
if [ ${FILE_NAME:0:1} == "." ]; then
rm -rf "$TEST_DIR/$FILE_NAME"
ln -sn "$FILE_PATH $TEST_DIR/$FILE_NAME"
else
FILE_NAME="."$FILE_NAME
rm -rf "$TEST_DIR/$FILE_NAME"
ln -sn "$FILE_PATH $TEST_DIR/$FILE_NAME"
fi
fi
}
# TODO: implement a check for file type and deal with unknown files.
for FILE in $DOTFILE_DIR/*; do
cp $FILE $BACKUP_DIR
prompt_install $FILE
done
The above is trimmed for long echo statements that do a lot of explaining.
The basic idea of this script is to take as many as two arguments (the dotfile directory to install and if specified, the custom backup directory, $1 and $2 respectively). The script should then copy all of the files in the target directory to BACKUP_DIR and symlink all of the dotfiles in the DOTFILE_DIR to TEST_DIR. (TEST_DIR will be $HOME in the production scripts.) Great in theory, right?
The complication comes when I run the script. None of the files are copied or symlinked as they should be. Instead, I end up with NO copy (probably due to the same issue as the symlink not working) and a broken symlink in the current directory.
One last piece of information. I am executing the file from the directory that contains ultimate-install.sh (/home/daniel/Projects/Git-Repos/Ultimate-Dotfile-Install-Scripts).
So where did I go wrong?
PS Please don't comment on the TODOs. :)
Short answer
Your quoting is wrong.
ln -sn -- "$FILE_PATH" "$TEST_DIR/$FILE_NAME"
Longer answer
This does not really relate to your problem, but I want to point it out.
Do not use "" inside [[ ]], so instead of this if [[ -z "$1" ]]; then use this if [[ -z $1 ]]; then
What is the point of making sure that directory does not have a trailing slash? It has no effect! /usr/bin/ is the same directory as /usr/bin or /usr////bin or /usr////////bin//////
Do not check if a directory exists when creating directories. Use -p option! Example: mkdir -p "$HOME/.config/old_dotfiles"
Instead of if [ $DECISION == "N" -o $DECISION == "n" ]; use if [[ ${DECISION^^} == N]];
I have another great answer about bash code style HERE. Please check it out! Also read the comments, since I was explaining there exactly your issue.

Expand a possible relative path in bash

As arguments to my script there are some file paths. Those can, of course, be relative (or contain ~). But for the functions I've written I need paths that are absolute, but do not have their symlinks resolved.
Is there any function for this?
MY_PATH=$(readlink -f $YOUR_ARG) will resolve relative paths like "./" and "../"
Consider this as well (source):
#!/bin/bash
dir_resolve()
{
cd "$1" 2>/dev/null || return $? # cd to desired directory; if fail, quell any error messages but return exit status
echo "`pwd -P`" # output full, link-resolved path
}
# sample usage
if abs_path="`dir_resolve \"$1\"`"
then
echo "$1 resolves to $abs_path"
echo pwd: `pwd` # function forks subshell, so working directory outside function is not affected
else
echo "Could not reach $1"
fi
http://www.linuxquestions.org/questions/programming-9/bash-script-return-full-path-and-filename-680368/page3.html has the following
function abspath {
if [[ -d "$1" ]]
then
pushd "$1" >/dev/null
pwd
popd >/dev/null
elif [[ -e "$1" ]]
then
pushd "$(dirname "$1")" >/dev/null
echo "$(pwd)/$(basename "$1")"
popd >/dev/null
else
echo "$1" does not exist! >&2
return 127
fi
}
which uses pushd/popd to get into a state where pwd is useful.
Simple one-liner:
function abs_path {
(cd "$(dirname '$1')" &>/dev/null && printf "%s/%s" "$PWD" "${1##*/}")
}
Usage:
function do_something {
local file=$(abs_path $1)
printf "Absolute path to %s: %s\n" "$1" "$file"
}
do_something $HOME/path/to/some\ where
I am still trying to figure out how I can get it to be completely oblivious to whether the path exists or not (so it can be used when creating files as well).
This does the trick for me on OS X: $(cd SOME_DIRECTORY 2> /dev/null && pwd -P)
It should work anywhere. The other solutions seemed too complicated.
If your OS supports it, use:
realpath -s "./some/dir"
And using it in a variable:
some_path="$(realpath -s "./some/dir")"
Which will expand your path. Tested on Ubuntu and CentOS, might not be available on yours. Some recommend readlink, but documentation for readlink says:
Note realpath(1) is the preferred command to use for canonicalization functionality.
In case people wonder why I quote my variables, it's to preserve spaces in paths. Like doing realpath some path will give you two different path results. But realpath "some path" will return one. Quoted parameters ftw :)
Thanks to NyanPasu64 for the heads up. You'll want to add -s if you don't want it to follow the symlinks.
Use readlink -f <relative-path>, e.g.
export FULLPATH=`readlink -f ./`
Maybe this is more readable and does not use a subshell and does not change the current dir:
dir_resolve() {
local dir=`dirname "$1"`
local file=`basename "$1"`
pushd "$dir" &>/dev/null || return $? # On error, return error code
echo "`pwd -P`/$file" # output full, link-resolved path with filename
popd &> /dev/null
}
on OS X you can use
stat -f "%N" YOUR_PATH
on linux you might have realpath executable. if not, the following might work (not only for links):
readlink -c YOUR_PATH
There's another method. You can use python embedding in bash script to resolve a relative path.
abs_path=$(python3 - <<END
from pathlib import Path
path = str(Path("$1").expanduser().resolve())
print(path)
END
)
self edit, I just noticed the OP said he's not looking for symlinks resolved:
"But for the functions I've written I need paths that are absolute, but do not have their symlinks resolved."
So guess this isn't so apropos to his question after all. :)
Since I've run into this many times over the years, and this time around I needed a pure bash portable version that I could use on OSX and linux, I went ahead and wrote one:
The living version lives here:
https://github.com/keen99/shell-functions/tree/master/resolve_path
but for the sake of SO, here's the current version (I feel it's well tested..but I'm open to feedback!)
Might not be difficult to make it work for plain bourne shell (sh), but I didn't try...I like $FUNCNAME too much. :)
#!/bin/bash
resolve_path() {
#I'm bash only, please!
# usage: resolve_path <a file or directory>
# follows symlinks and relative paths, returns a full real path
#
local owd="$PWD"
#echo "$FUNCNAME for $1" >&2
local opath="$1"
local npath=""
local obase=$(basename "$opath")
local odir=$(dirname "$opath")
if [[ -L "$opath" ]]
then
#it's a link.
#file or directory, we want to cd into it's dir
cd $odir
#then extract where the link points.
npath=$(readlink "$obase")
#have to -L BEFORE we -f, because -f includes -L :(
if [[ -L $npath ]]
then
#the link points to another symlink, so go follow that.
resolve_path "$npath"
#and finish out early, we're done.
return $?
#done
elif [[ -f $npath ]]
#the link points to a file.
then
#get the dir for the new file
nbase=$(basename $npath)
npath=$(dirname $npath)
cd "$npath"
ndir=$(pwd -P)
retval=0
#done
elif [[ -d $npath ]]
then
#the link points to a directory.
cd "$npath"
ndir=$(pwd -P)
retval=0
#done
else
echo "$FUNCNAME: ERROR: unknown condition inside link!!" >&2
echo "opath [[ $opath ]]" >&2
echo "npath [[ $npath ]]" >&2
return 1
fi
else
if ! [[ -e "$opath" ]]
then
echo "$FUNCNAME: $opath: No such file or directory" >&2
return 1
#and break early
elif [[ -d "$opath" ]]
then
cd "$opath"
ndir=$(pwd -P)
retval=0
#done
elif [[ -f "$opath" ]]
then
cd $odir
ndir=$(pwd -P)
nbase=$(basename "$opath")
retval=0
#done
else
echo "$FUNCNAME: ERROR: unknown condition outside link!!" >&2
echo "opath [[ $opath ]]" >&2
return 1
fi
fi
#now assemble our output
echo -n "$ndir"
if [[ "x${nbase:=}" != "x" ]]
then
echo "/$nbase"
else
echo
fi
#now return to where we were
cd "$owd"
return $retval
}
here's a classic example, thanks to brew:
%% ls -l `which mvn`
lrwxr-xr-x 1 draistrick 502 29 Dec 17 10:50 /usr/local/bin/mvn# -> ../Cellar/maven/3.2.3/bin/mvn
use this function and it will return the -real- path:
%% cat test.sh
#!/bin/bash
. resolve_path.inc
echo
echo "relative symlinked path:"
which mvn
echo
echo "and the real path:"
resolve_path `which mvn`
%% test.sh
relative symlinked path:
/usr/local/bin/mvn
and the real path:
/usr/local/Cellar/maven/3.2.3/libexec/bin/mvn
Do you have to use bash exclusively? I needed to do this and got fed up with differences between Linux and OS X. So I used PHP for a quick and dirty solution.
#!/usr/bin/php <-- or wherever
<?php
{
if($argc!=2)
exit();
$fname=$argv[1];
if(!file_exists($fname))
exit();
echo realpath($fname)."\n";
}
?>
I know it's not a very elegant solution but it does work.

How do you normalize a file path in Bash?

I want to transform /foo/bar/.. to /foo
Is there a bash command which does this?
Edit: in my practical case, the directory does exist.
if you're wanting to chomp part of a filename from the path, "dirname" and "basename" are your friends, and "realpath" is handy too.
dirname /foo/bar/baz
# /foo/bar
basename /foo/bar/baz
# baz
dirname $( dirname /foo/bar/baz )
# /foo
realpath ../foo
# ../foo: No such file or directory
realpath /tmp/../tmp/../tmp
# /tmp
realpath alternatives
If realpath is not supported by your shell, you can try
readlink -f /path/here/..
Also
readlink -m /path/there/../../
Works the same as
realpath -s /path/here/../../
in that the path doesn't need to exist to be normalized.
I don't know if there is a direct bash command to do this, but I usually do
normalDir="`cd "${dirToNormalize}";pwd`"
echo "${normalDir}"
and it works well.
Try realpath. Below is the source in its entirety, hereby donated to the public domain.
// realpath.c: display the absolute path to a file or directory.
// Adam Liss, August, 2007
// This program is provided "as-is" to the public domain, without express or
// implied warranty, for any non-profit use, provided this notice is maintained.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <libgen.h>
#include <limits.h>
static char *s_pMyName;
void usage(void);
int main(int argc, char *argv[])
{
char
sPath[PATH_MAX];
s_pMyName = strdup(basename(argv[0]));
if (argc < 2)
usage();
printf("%s\n", realpath(argv[1], sPath));
return 0;
}
void usage(void)
{
fprintf(stderr, "usage: %s PATH\n", s_pMyName);
exit(1);
}
A portable and reliable solution is to use python, which is preinstalled pretty much everywhere (including Darwin). You have two options:
abspath returns an absolute path but does not resolve symlinks:
python -c "import os,sys; print(os.path.abspath(sys.argv[1]))" path/to/file
realpath returns an absolute path and in doing so resolves symlinks, generating a canonical path:
python -c "import os,sys; print(os.path.realpath(sys.argv[1]))" path/to/file
In each case, path/to/file can be either a relative or absolute path.
Use the readlink utility from the coreutils package.
MY_PATH=$(readlink -f "$0")
Old question, but there is much simpler way if you are dealing with full path names at the shell level:
abspath="$( cd "$path" && pwd )"
As the cd happens in a subshell it does not impact the main script.
Two variations, supposing your shell built-in commands accept -L and -P, are:
abspath="$( cd -P "$path" && pwd -P )" #physical path with resolved symlinks
abspath="$( cd -L "$path" && pwd -L )" #logical path preserving symlinks
Personally, I rarely need this later approach unless I'm fascinated with symbolic links for some reason.
FYI: variation on obtaining the starting directory of a script which works even if the script changes it's current directory later on.
name0="$(basename "$0")"; #base name of script
dir0="$( cd "$( dirname "$0" )" && pwd )"; #absolute starting dir
The use of CD assures you always have the absolute directory, even if the script is run by commands such as ./script.sh which, without the cd/pwd, often gives just .. Useless if the script does a cd later on.
readlink is the bash standard for obtaining the absolute path. It also has the advantage of returning empty strings if paths or a path doesn't exist (given the flags to do so).
To get the absolute path to a directory that may or may not exist, but who's parents do exist, use:
abspath=$(readlink -f $path)
To get the absolute path to a directory that must exist along with all parents:
abspath=$(readlink -e $path)
To canonicalise the given path and follow symlinks if they happen to exist, but otherwise ignore missing directories and just return the path anyway, it's:
abspath=$(readlink -m $path)
The only downside is that readlink will follow links. If you do not want to follow links, you can use this alternative convention:
abspath=$(cd ${path%/*} && echo $PWD/${path##*/})
That will chdir to the directory part of $path and print the current directory along with the file part of $path. If it fails to chdir, you get an empty string and an error on stderr.
As Adam Liss noted realpath is not bundled with every distribution. Which is a shame, because it is the best solution. The provided source code is great, and I will probably start using it now. Here is what I have been using until now, which I share here just for completeness:
get_abs_path() {
local PARENT_DIR=$(dirname "$1")
cd "$PARENT_DIR"
local ABS_PATH="$(pwd)"/"$(basename "$1")"
cd - >/dev/null
echo "$ABS_PATH"
}
If you want it to resolve symlinks, just replace pwd with pwd -P.
My recent solution was:
pushd foo/bar/..
dir=`pwd`
popd
Based on the answer of Tim Whitcomb.
Not exactly an answer but perhaps a follow-up question (original question was not explicit):
readlink is fine if you actually want to follow symlinks. But there is also a use case for merely normalizing ./ and ../ and // sequences, which can be done purely syntactically, without canonicalizing symlinks. readlink is no good for this, and neither is realpath.
for f in $paths; do (cd $f; pwd); done
works for existing paths, but breaks for others.
A sed script would seem to be a good bet, except that you cannot iteratively replace sequences (/foo/bar/baz/../.. -> /foo/bar/.. -> /foo) without using something like Perl, which is not safe to assume on all systems, or using some ugly loop to compare the output of sed to its input.
FWIW, a one-liner using Java (JDK 6+):
jrunscript -e 'for (var i = 0; i < arguments.length; i++) {println(new java.io.File(new java.io.File(arguments[i]).toURI().normalize()))}' $paths
I'm late to the party, but this is the solution I've crafted after reading a bunch of threads like this:
resolve_dir() {
(builtin cd `dirname "${1/#~/$HOME}"`'/'`basename "${1/#~/$HOME}"` 2>/dev/null; if [ $? -eq 0 ]; then pwd; fi)
}
This will resolve the absolute path of $1, play nice with ~, keep symlinks in the path where they are, and it won't mess with your directory stack. It returns the full path or nothing if it doesn't exist. It expects $1 to be a directory and will probably fail if it's not, but that's an easy check to do yourself.
Talkative, and a bit late answer. I need to write one since I'm stuck on older RHEL4/5.
I handles absolute and relative links, and simplifies //, /./ and somedir/../ entries.
test -x /usr/bin/readlink || readlink () {
echo $(/bin/ls -l $1 | /bin/cut -d'>' -f 2)
}
test -x /usr/bin/realpath || realpath () {
local PATH=/bin:/usr/bin
local inputpath=$1
local changemade=1
while [ $changemade -ne 0 ]
do
changemade=0
local realpath=""
local token=
for token in ${inputpath//\// }
do
case $token in
""|".") # noop
;;
"..") # up one directory
changemade=1
realpath=$(dirname $realpath)
;;
*)
if [ -h $realpath/$token ]
then
changemade=1
target=`readlink $realpath/$token`
if [ "${target:0:1}" = '/' ]
then
realpath=$target
else
realpath="$realpath/$target"
fi
else
realpath="$realpath/$token"
fi
;;
esac
done
inputpath=$realpath
done
echo $realpath
}
mkdir -p /tmp/bar
(cd /tmp ; ln -s /tmp/bar foo; ln -s ../.././usr /tmp/bar/link2usr)
echo `realpath /tmp/foo`
The problem with realpath is that it is not available on BSD (or OSX for that matter). Here is a simple recipe extracted from a rather old (2009) article from Linux Journal, that is quite portable:
function normpath() {
# Remove all /./ sequences.
local path=${1//\/.\//\/}
# Remove dir/.. sequences.
while [[ $path =~ ([^/][^/]*/\.\./) ]]; do
path=${path/${BASH_REMATCH[0]}/}
done
echo $path
}
Notice this variant also does not require the path to exist.
Try our new Bash library product realpath-lib that we have placed on GitHub for free and unencumbered use. It's thoroughly documented and makes a great learning tool.
It resolves local, relative and absolute paths and doesn't have any dependencies except Bash 4+; so it should work just about anywhere. It's free, clean, simple and instructive.
You can do:
get_realpath <absolute|relative|symlink|local file path>
This function is the core of the library:
function get_realpath() {
if [[ -f "$1" ]]
then
# file *must* exist
if cd "$(echo "${1%/*}")" &>/dev/null
then
# file *may* not be local
# exception is ./file.ext
# try 'cd .; cd -;' *works!*
local tmppwd="$PWD"
cd - &>/dev/null
else
# file *must* be local
local tmppwd="$PWD"
fi
else
# file *cannot* exist
return 1 # failure
fi
# reassemble realpath
echo "$tmppwd"/"${1##*/}"
return 0 # success
}
It also contains functions to get_dirname, get_filename, get_ stemname and validate_path. Try it across platforms, and help to improve it.
Based on #Andre's answer, I might have a slightly better version, in case someone is after a loop-free, completely string-manipulation based solution. It is also useful for those who don't want to dereference any symlinks, which is the downside of using realpath or readlink -f.
It works on bash versions 3.2.25 and higher.
shopt -s extglob
normalise_path() {
local path="$1"
# get rid of /../ example: /one/../two to /two
path="${path//\/*([!\/])\/\.\./}"
# get rid of /./ and //* example: /one/.///two to /one/two
path="${path//#(\/\.\/|\/+(\/))//}"
# remove the last '/.'
echo "${path%%/.}"
}
$ normalise_path /home/codemedic/../codemedic////.config
/home/codemedic/.config
I made a builtin-only function to handle this with a focus on highest possible performance (for fun). It does not resolve symlinks, so it is basically the same as realpath -sm.
## A bash-only mimic of `realpath -sm`.
## Give it path[s] as argument[s] and it will convert them to clean absolute paths
abspath () {
${*+false} && { >&2 echo $FUNCNAME: missing operand; return 1; };
local c s p IFS='/'; ## path chunk, absolute path, input path, IFS for splitting paths into chunks
local -i r=0; ## return value
for p in "$#"; do
case "$p" in ## Check for leading backslashes, identify relative/absolute path
'') ((r|=1)); continue;;
//[!/]*) >&2 echo "paths =~ ^//[^/]* are impl-defined; not my problem"; ((r|=2)); continue;;
/*) ;;
*) p="$PWD/$p";; ## Prepend the current directory to form an absolute path
esac
s='';
for c in $p; do ## Let IFS split the path at '/'s
case $c in ### NOTE: IFS is '/'; so no quotes needed here
''|.) ;; ## Skip duplicate '/'s and '/./'s
..) s="${s%/*}";; ## Trim the previous addition to the absolute path string
*) s+=/$c;; ### NOTE: No quotes here intentionally. They make no difference, it seems
esac;
done;
echo "${s:-/}"; ## If xpg_echo is set, use `echo -E` or `printf $'%s\n'` instead
done
return $r;
}
Note: This function does not handle paths starting with //, as exactly two double slashes at the start of a path are implementation-defined behavior. However, it handles /, ///, and so on just fine.
This function seems to handle all edge cases properly, but there might still be some out there that I haven't dealt with.
Performance Note: when called with thousands of arguments, abspath runs about 10x slower than realpath -sm; when called with a single argument, abspath runs >110x faster than realpath -sm on my machine, mostly due to not needing to execute a new program every time.
If you just want to normalize a path, existed or not existed, without touching the file system, without resolving any links, and without external utils, here is a pure Bash function translated from Python's posixpath.normpath.
#!/usr/bin/env bash
# Normalize path, eliminating double slashes, etc.
# Usage: new_path="$(normpath "${old_path}")"
# Translated from Python's posixpath.normpath:
# https://github.com/python/cpython/blob/master/Lib/posixpath.py#L337
normpath() {
local IFS=/ initial_slashes='' comp comps=()
if [[ $1 == /* ]]; then
initial_slashes='/'
[[ $1 == //* && $1 != ///* ]] && initial_slashes='//'
fi
for comp in $1; do
[[ -z ${comp} || ${comp} == '.' ]] && continue
if [[ ${comp} != '..' || (-z ${initial_slashes} && ${#comps[#]} -eq 0) || (\
${#comps[#]} -gt 0 && ${comps[-1]} == '..') ]]; then
comps+=("${comp}")
elif ((${#comps[#]})); then
unset 'comps[-1]'
fi
done
comp="${initial_slashes}${comps[*]}"
printf '%s\n' "${comp:-.}"
}
Examples:
new_path="$(normpath '/foo/bar/..')"
echo "${new_path}"
# /foo
normpath "relative/path/with trailing slashs////"
# relative/path/with trailing slashs
normpath "////a/../lot/././/mess////./here/./../"
# /lot/mess
normpath ""
# .
# (empty path resolved to dot)
Personally, I cannot understand why Shell, a language often used for manipulating files, doesn't offer basic functions to deal with paths. In python, we have nice libraries like os.path or pathlib, which offers a whole bunch of tools to extract filename, extension, basename, path segments, split or join paths, to get absolute or normalized paths, to determine relations between paths, to do everything without much brain. And they take care of edge cases, and they're reliable. In Shell, to do any of these, either we call external executables, or we have to reinvent wheels with these extremely rudimentary and arcane syntaxes...
I needed a solution that would do all three:
Work on a stock Mac. realpath and readlink -f are addons
Resolve symlinks
Have error handling
None of the answers had both #1 and #2. I added #3 to save others any further yak-shaving.
#!/bin/bash
P="${1?Specify a file path}"
[ -e "$P" ] || { echo "File does not exist: $P"; exit 1; }
while [ -h "$P" ] ; do
ls="$(ls -ld "$P")"
link="$(expr "$ls" : '.*-> \(.*\)$')"
expr "$link" : '/.*' > /dev/null &&
P="$link" ||
P="$(dirname "$P")/$link"
done
echo "$(cd "$(dirname "$P")"; pwd)/$(basename "$P")"
Here is a short test case with some twisted spaces in the paths to fully exercise the quoting
mkdir -p "/tmp/test/ first path "
mkdir -p "/tmp/test/ second path "
echo "hello" > "/tmp/test/ first path / red .txt "
ln -s "/tmp/test/ first path / red .txt " "/tmp/test/ second path / green .txt "
cd "/tmp/test/ second path "
fullpath " green .txt "
cat " green .txt "
Based on loveborg's excellent python snippet, I wrote this:
#!/bin/sh
# Version of readlink that follows links to the end; good for Mac OS X
for file in "$#"; do
while [ -h "$file" ]; do
l=`readlink $file`
case "$l" in
/*) file="$l";;
*) file=`dirname "$file"`/"$l"
esac
done
#echo $file
python -c "import os,sys; print os.path.abspath(sys.argv[1])" "$file"
done
FILEPATH="file.txt"
echo $(realpath $(dirname $FILEPATH))/$(basename $FILEPATH)
This works even if the file doesn't exist. It does require the directory containing the file to exist.
I know this is an ancient question. I'm still offering an alternative. Recently I met the same issue and found no existing and portable command to do that. So I wrote the following shell script which includes a function that can do the trick.
#! /bin/sh
function normalize {
local rc=0
local ret
if [ $# -gt 0 ] ; then
# invalid
if [ "x`echo $1 | grep -E '^/\.\.'`" != "x" ] ; then
echo $1
return -1
fi
# convert to absolute path
if [ "x`echo $1 | grep -E '^\/'`" == "x" ] ; then
normalize "`pwd`/$1"
return $?
fi
ret=`echo $1 | sed 's;/\.\($\|/\);/;g' | sed 's;/[^/]*[^/.]\+[^/]*/\.\.\($\|/\);/;g'`
else
read line
normalize "$line"
return $?
fi
if [ "x`echo $ret | grep -E '/\.\.?(/|$)'`" != "x" ] ; then
ret=`normalize "$ret"`
rc=$?
fi
echo "$ret"
return $rc
}
https://gist.github.com/bestofsong/8830bdf3e5eb9461d27313c3c282868c
Since none of the presented solutions worked for me, in the case where a file does not exist, I implemented my idea.
The solution of André Anjos had the problem that paths beginning with ../../ were resolved wrongly. For example ../../a/b/ became a/b/.
function normalize_rel_path(){
local path=$1
result=""
IFS='/' read -r -a array <<< "$path"
i=0
for (( idx=${#array[#]}-1 ; idx>=0 ; idx-- )) ; do
c="${array[idx]}"
if [ -z "$c" ] || [[ "$c" == "." ]];
then
continue
fi
if [[ "$c" == ".." ]]
then
i=$((i+1))
elif [ "$i" -gt "0" ];
then
i=$((i-1))
else
if [ -z "$result" ];
then
result=$c
else
result=$c/$result
fi
fi
done
while [ "$i" -gt "0" ]; do
i=$((i-1))
result="../"$result
done
unset IFS
echo $result
}
I discovered today that you can use the stat command to resolve paths.
So for a directory like "~/Documents":
You can run this:
stat -f %N ~/Documents
To get the full path:
/Users/me/Documents
For symlinks, you can use the %Y format option:
stat -f %Y example_symlink
Which might return a result like:
/usr/local/sbin/example_symlink
The formatting options might be different on other versions of *NIX but these worked for me on OSX.
A simple solution using node.js:
#!/usr/bin/env node
process.stdout.write(require('path').resolve(process.argv[2]));

Resources