How change symlink path for many files? - linux

I was changed directory name.
In this directory thousands of files.
Some projects use this files, projects have got symlinks on it.
How to find all symlinks, which have got folder name in their address?
how to change all this symlinks to another path in automatic mode?
if 2 only bash scripting with deleting and creating new - i will do it, but may be you know more easy way?

It's a bit complicated, but it can be done with find, readlink, a check to test whether the symlink is relative or not, and sed to get rid of .. in path names (copied 1:1 from this answer).
(Note that most convenient methods (such as readlink -f) are not available due to the symlinks targets not existing anymore.)
Assuming your old path is /var/lib/old/path:
oldpath='/var/lib/old/path';
find / -type l -execdir bash -c 'p="$(readlink "{}")"; if [ "${p:0:1}" != "/" ]; then p="$(echo "$(pwd)/$p" | sed -e "s|/\./|/|g" -e ":a" -e "s|/[^/]*/\.\./|/|" -e "t a")"; fi; if [ "${p:0:'${#oldpath}'}" == "'"$oldpath"'" ]; then ...; fi;' \;
Now replace the ... from above with ln -sf (-f to override the existing link).
Assuming your new path is /usr/local/my/awesome/new/path:
oldpath='/var/lib/old/path';
newpath='/usr/local/my/awesome/new/path';
find / -type l -execdir bash -c 'p="$(readlink "{}")"; if [ "${p:0:1}" != "/" ]; then p="$(echo "$(pwd)/$p" | sed -e "s|/\./|/|g" -e ":a" -e "s|/[^/]*/\.\./|/|" -e "t a")"; fi; if [ "${p:0:'${#oldpath}'}" == "'"$oldpath"'" ]; then ln -sf "'"$newpath"'${p:'${#oldpath}'}" "{}"; fi;' \;
Note that oldpath and newpath have to be absolute paths.
Also note that this will convert all relative symlinks to absolute ones.
It would be possible to keep them relative, but only with a lot of effort.
Breaking it down
For those of you who care what that one-line-inferno actually means:
find - a cool executable
/ - where to search, in this case the system root
-type l - match symbolic links
-execdir - for every match run the following command in the directory of the matched file:
bash - well, bash
-c - execute the following string (leading and trailing ' removed):
p="$(readlink "{}")"; - starting with the most inner:
" - start a string to make sure no expansion happens
{} - placeholder for the matched file's name (feature of -execdir)
" - end the string
readlink ... - find out where the symlink points to
p="$(...)" - and store the result in $p
if [ "${p:0:1}" != "/" ]; then - if the first character of $p is / (i.e. the symlink is absolute), then...
p="$(echo "$(pwd)/$p" | sed -e "s|/\./|/|g" -e ":a" -e "s|/[^/]*/\.\./|/|" -e "t a")"; - convert the path to an absolute one:
$(pwd) - the current directory (where the matched file lies, because we're using -execdir)
/$p - append a slash and the target of the symlink to the path of the working directory
echo "$(pwd)/$p" | - pipe the above to the next command
sed ... - resolve all ..'s, see here
p="$(...)" and store the result back into $p.
fi; - end if
if [ "${p:0:'${#oldpath}'}" == "'"$oldpath"'" ]; - if $p starts with $oldpath
${p:0:'${#oldpath}'} - substring of $p, starting at position 0, with length of $oldpath:
${#oldpath} - length of variable $oldpath
'...' - required because we're inside a '-quoted string
then - then...
ln -sf - link symbolically and override existing file, with arguments:
"'"$newpath"'${p:'${#oldpath}'}" - replace the $oldpath part of $p with $newpath (actually remove as many characters from $p as $oldpath long is, and prepend $newpath to it):
" - start a string
' - end the '-string argument to bash -c
" - append a "-string to it (in which variable expansion happens), containing:
$newpath - the value of $newpath
" - end the "-string argument to bash -c
' - append a '-string to it, containing:
${p: - a substring of p, starting at:
' - end the argument to bash -c
${#oldpath} - append the length of $oldpath to it
' - append another '-string to it
} - end substring
" - end string
"{}" - the link file, whose path stays the same
fi; - end if
\; - delimiter for -execdir

Related

bash: How to transfer/copy only the file names to separate similar files?

I've some files in a folder A which are named like that:
001_file.xyz
002_file.xyz
003_file.xyz
in a separate folder B I've files like this:
001_FILE_somerandomtext.zyx
002_FILE_somerandomtext.zyx
003_FILE_somerandomtext.zyx
Now I want to rename, if possible, with just a command line in the bash all the files in folder B with the file names in folder A. The file extension must stay different.
There is exactly the same amount of files in each folder A and B and they both have the same order due to numbering.
I'm a total noob, but I hope some easy answer for the problem will show up.
Thanks in advance!
ZVLKX
*Example edited for clarification
An implementation might look a bit like this:
renameFromDir() {
useNamesFromDir=$1
forFilesFromDir=$2
for f in "$forFilesFromDir"/*; do
# Put original extension in $f_ext
f_ext=${f##*.}
# Put number in $f_num
f_num=${f##*/}; f_num=${f_num%%_*}
# look for a file in directory B with same number
set -- "$useNamesFromDir"/"${f_num}"_*.*
[[ $1 && -e $1 ]] || {
echo "Could not find file number $f_num in $dirB" >&2
continue
}
(( $# > 1 )) && {
# there's more than one file with the same number; write an error
echo "Found more than one file with number $f_num in $dirB" >&2
printf ' - %q\n' "$#" >&2
continue
}
# extract the parts of our destination filename we want to keep
destName=${1##*/} # remove everything up to the last /
destName=${destName%.*} # and past the last .
# write the command we would run to stdout
printf '%q ' mv "$f" "$forFilesFromDir/$destName.$f_ext"; printf '\n'
## or uncomment this to actually run the command
# mv "$f" "$forFilesFromDir/$destName.$f_ext"
done
}
Now, how would we test this?
mkdir -p A B
touch A/00{1,2,3}_file.xyz B/00{1,2,3}_FILE_somerandomtext.zyx
renameFromDir A B
Given that, the output is:
mv B/001_FILE_somerandomtext.zyx B/001_file.zyx
mv B/002_FILE_somerandomtext.zyx B/002_file.zyx
mv B/003_FILE_somerandomtext.zyx B/003_file.zyx
Sorry if this isn't helpful, but I had fun writing it.
This renames items in folder B to the names in folder A, preserving the extension of B.
A_DIR="./A"
A_FILE_EXT=".xyz"
B_DIR="./B"
B_FILE_EXT=".zyx"
FILES_IN_A=`find $A_DIR -type f -name *$A_FILE_EXT`
FILES_IN_B=`find $B_DIR -type f -name *$B_FILE_EXT`
for A_FILE in $FILES_IN_A
do
A_BASE_FILE=`basename $A_FILE`
A_FILE_NUMBER=(${A_BASE_FILE//_/ })
A_FILE_WITHOUT_EXTENSION=(${A_BASE_FILE//./ })
for B_FILE in $FILES_IN_B
do
B_BASE_FILE=`basename $B_FILE`
B_FILE_NUMBER=(${B_BASE_FILE//_/ })
if [ ${A_FILE_NUMBER[0]} == ${B_FILE_NUMBER[0]} ]; then
mv $B_FILE $B_DIR/$A_FILE_WITHOUT_EXTENSION$B_FILE_EXT
break
fi
done
done

for loop and files with spaces

Here is my command
for i in `find . -name '*Source*.dat'`; do cp "$i" $INBOUND/$RANDOM.dat; done;
Here are the files (just a sample):
/(12)SA1 (Admitting Diagnosis) --_TA1-1 + TA1-2/Source.dat
./(12)SA1 (Admitting Diagnosis) --_TA1-1 + TA1-2/Source_2000C.dat
./(13)SE1 (External Cause of Injury) --_ TE1-1+TE1-2/Source.dat
./(13)SE1 (External Cause of Injury) --_ TE1-1+TE1-2/Source_2000C.dat
./(13)SE1 (External Cause of Injury) --_ TE1-1+TE1-2/Source_POATest.dat
./(14)SP1(Primary)--_ TP1-1 + TP1-2/Source.dat
./(14)SP1(Primary)--_ TP1-1 + TP1-2/Source_2000C.dat
./(14)SP1(Primary)--_ TP1-1 + TP1-2/Source_ProcDateTest.dat
./(15)SP1(Primary)--_ TP1-1 + TP1-2 - SP2 -- TP2-1 + TP2-2/Source.dat
./(16)SP1(Primary)--_ TP1-1 + TP1-2 +TP1-3- SP2 -- TP2-1 + TP2-2/Source.dat
./(17)SP1(Primary)--_ TP1-1 + TP1-2 +TP1-3/Source.dat
./(18)SP1(Primary)--_ TP1-1 + TP1-2 - SP2 -- TP2-1 + TP2-2 - Copy/Source.dat
./(19)SD1 (Primary)+SD2 (Other Diagnosis)--_ TD12/Source.dat
./(19)SD1 (Primary)+SD2 (Other Diagnosis)--_ TD12/Source_2000C.dat
./(19)SD1 (Primary)+SD2 (Other Diagnosis)--_ TD12/Source_POATest.dat
./(2)SD3--_TD4 SD4--_TD4/Source.dat
./(2)SD3--_TD4 SD4--_TD4/Source2.dat
Those spaces are getting tokenized by bash and this doesn't work.
In addition, I want to append some randomness to the end of these files so they don't collide in the destination directory but that's another story.
find . -name '*Source*.dat' -exec sh -c 'cp "$1" "$2/$RANDOM.dat"' -- {} "$INBOUND" \;
Using -exec to execute commands is whitespace safe. Using sh to execute cp is necessary to get a different $RANDOM for each copy.
If all the files are at the same directory level, as in your example, you don't need find. For example,
for i in */*Source*.dat; do
cp "$i" $INBOUND/$RANDOM.dat
done
will tokenize correctly and will find the correct files provided they are all in directories which are children of the current directory.
As #chepner points out in a comment, if you have bash v4 you can use **:
for i in **/*Source*.dat; do
cp "$i" $INBOUND/$RANDOM.dat
done
which should find exactly the same files as find would, without the tokenizing issue.
How about:
find . -name '*file*' -print0 | xargs -0 -I {} cp {} $INBOUND/{}-$RANDOM.dat
xargs is a handy way of constructing an argument list and passing it to a command.
find -print0 and xargs -0 go together, and are basically an agreement between the two commands about how to terminate arguments. In this case, it means the space won't be interpreted as the end of an argument.
-I {} sets up the {} as an argument placeholder for xargs.
As for randomising the file name to avoid a collision, there are obviously lots of things you could do to generate a random string to attach. The most important part, though, is that you verify that your new file name also does not exist. You might use a loop something like this to attempt that:
$RANDOM=$(date | md5)
filename=$INBOUND/$RANDOM.dat
while [ -e $filename ]; do
$RANDOM=$(date | md5)
filename=$INBOUND/$RANDOM.dat
done
I'm not necessarily advocating for or against generating a random filename with a hash of the current time: the main point is that you want to check for existence of that file first, just in case.
There are several ways of treating files with spaces. You can use findin a pipe, while and read:
find . -name '*Source*.dat' | while read file ; do cp "$file" "$INBOUND/$RANDOM.dat"; done
try something like
while read i;do
echo "file is $i"
cp "$i" $INBOUND/$RANDOM.dat
done < <(find . -name '*Source*.dat')

How to rename multiple files in terminal (LINUX)?

I have bunch of files with no pattern in their name at all in a directory. all I know is that they are all Jpg files. How do I rename them, so that they will have some sort of sequence in their name.
I know in Windows all you do is select all the files and rename them all to a same name and Windows OS automatically adds sequence numbers to compensate for the same file name.
I want to be able to do that in Linux Fedora but I you can only do that in Terminal. Please, help. I am lost.
What is the command for doing this?
The best way to do this is to run a loop in the terminal going from picture to picture and renaming them with a number that gets bigger by one with every loop.
You can do this with:
n=1
for i in *.jpg; do
p=$(printf "%04d.jpg" ${n})
mv ${i} ${p}
let n=n+1
done
Just enter it into the terminal line by line.
If you want to put a custom name in front of the numbers, you can put it before the percent sign in the third line.
If you want to change the number of digits in the names' number, just replace the '4' in the third line (don't change the '0', though).
I will assume that:
There are no spaces or other weird control characters in the file names
All of the files in a given directory are jpeg files
That in mind, to rename all of the files to 1.jpg, 2.jpg, and so on:
N=1
for a in ./* ; do
mv $a ${N}.jpg
N=$(( $N + 1 ))
done
If there are spaces in the file names:
find . -type f | awk 'BEGIN{N=1}
{print "mv \"" $0 "\" " N ".jpg"
N++}' | sh
Should be able to rename them.
The point being, Linux/UNIX does have a lot of tools which can automate a task like this, but they have a bit of a learning curve to them
Create a script containing:
#!/bin/sh
filePrefix="$1"
sequence=1
for file in $(ls -tr *.jpg) ; do
renamedFile="$filePrefix$sequence.jpg"
echo $renamedFile
currentFile="$(echo $file)"
echo "renaming \"$currentFile\" to $renamedFile"
mv "$currentFile" "$renamedFile"
sequence=$(($sequence+1))
done
exit 0
If you named the script, say, RenameSequentially then you could issue the command:
./RenameSequentially Images-
This would rename all *.jpg files in the directory to Image-1.jpg, Image-2.jpg, etc... in order of oldest to newest... tested in OS X command shell.
I wrote a perl script a long time ago to do pretty much what you want:
#
# reseq.pl renames files to a new named sequence of filesnames
#
# Usage: reseq.pl newname [-n seq] [-p pad] fileglob
#
use strict;
my $newname = $ARGV[0];
my $seqstr = "01";
my $seq = 1;
my $pad = 2;
shift #ARGV;
if ($ARGV[0] eq "-n") {
$seqstr = $ARGV[1];
$seq = int $seqstr;
shift #ARGV;
shift #ARGV;
}
if ($ARGV[0] eq "-p") {
$pad = $ARGV[1];
shift #ARGV;
shift #ARGV;
}
my $filename;
my $suffix;
for (#ARGV) {
$filename = sprintf("${newname}_%0${pad}d", $seq);
if (($suffix) = m/.*\.(.*)/) {
$filename = "$filename.$suffix";
}
print "$_ -> $filename\n";
rename ($_, $filename);
$seq++;
}
You specify a common prefix for the files, a beginning sequence number and a padding factor.
For exmaple:
# reseq.pl abc 1 2 *.jpg
Will rename all matching files to abc_01.jpg, abc_02.jpg, abc_03.jpg...

Shell: Is there a difference between / and // in directories?

I'm making this tiny program in Shell:
#***************************************************************
# Function.
# NAME: chk_df
# Synopsis:
# Check if a local directory (dirName) exist and has a file (fileName).
#
#
# The return codes are the following:
# 99 : dirName does not exists
# 0 : dirName exists and has fileName
# 1 : dirName exists and has not fileName
#
# Parameters:
# In values: dirName <string> fileName <string>
# Out values: returnCode <int>
#
# How to use:
# chk_df dirName fileName
#***************************************************************
chk_df(){
# Check the number of arguments that could be passed.
# In this case, two, dirName, fileNAme.
if [[ ${##} != 2 ]]; then
echo "Error ...Use [Function]: chk_df <dirName> <fileName>"
echo "Ex: chk_df /foo lola.txt"
exit
fi
DIR=$1
FILE=$2
[[ ! -d $DIR ]] && return 99
[[ -d $DIR && ! -e $DIR/$FILE ]] && return 1
[[ -d $DIR && -e $DIR/$FILE ]] && return 0
}
Because I need to check if a file is in a directory, I did this (horrible?) patch $DIR/$FILE , but things like this could happen:
I) If we do: chk_df /foo lola.txt
We get: /foo/lola.txt
II) If we do: chk_df /foo/ lola.txt
We get: /foo//lola.txt [Notice the //]
In both cases the code seems to work. Why? I read that backslash acts like a space. So, could I put n backslash without unknown problems?
Could I leave it like that or it will bring problems? Is there a difference? UNIX assume it to the right way?
EXTRA QUESTION: why I can not do the returns with negative numbers? This is: return -1
/ , //, or any string of consecutive slashes have the same meaning according to the POSIX standard, with the exception that they may have a different meaning at the beginning of a path (so /foo and //foo may denote different objects). Linux does not use this exception, so any number of consecutive slashes always means the same thing as a single /.
(The exception is there to cater to the needs of other Unix-like systems that use leading // to denote a network path.)
There are no difference.
// = /
You can, in principle, use as many / separators as you want (until you start hitting PATH_MAX or some other hard limitation):
$ ls /usr/bin///////////////less
/usr/bin///////////////less
One problem you'll run into is if you ever want to test that two paths are the same[*], because /usr/bin/less and /usr/bin//less are the same path but are different strings. It can be useful to canonicalise paths before comparison.
[*] Ignoring the fact that different paths can refer to the same object.

Unable to have two-word-search in Zsh's TAB completion for Man

Problem: to have a tab completion which takes two words and calculates the best match from them for Man, and then returns the best matches
Example: The following pseudo-code should give me at least Zsh's reverse-menu-complete -command. Right now, I cannot search manuals inside manuals without zgrep.
man zsh:reverse <TAB>
where ":" is the separator which I want.
Initial Problem: Which files does the TAB completion run when I press TAB for one word in searching manuals by Zsh?
I will attempt to provide an insight to how zsh completion system works and an incomplete go at this problem.
The file that runs when you use TAB completion for man in zsh is located under the /usr/share/zsh/${ZSH_VERSION}/functions directory. The tree varies across distributions, but the file is named _man, and provides completion for man, apropos and whatis.
After _man is invoked, it works as following (rough description):
if completing for man and --local-file was specified as first flag, invoke standard files completion (_files)
construct manpath variable from a set of defaults / $MANPATH. This is where the manpages will be searched
determine if we invoked man with a section number parameter, if yes - only those sections will be searched
if the zstyle ':completion:*:manuals' separate-sections true was used, separate sections in output (don't mix between them)
invoke _man_pages to provide a list of man pages for the match
_man_pages now does a bit of magic with compfiles -p pages '' '' "$matcher" '' dummy '*'. pages is the variable with all the directories containing manpages for requested section(s). The actual globbing pattern is constructed from the implicit parameter $PREFIX and the last parameter to compfiles - * in this case. This results in /usr/share/man/man1 to be transformed into /usr/share/man/man1/foo*
The new list of glob patterns is globbed, obtaining all files which match the pattern
_man_pages then strips any suffixes from the files and adds them to the completion widget list of choices by using compadd
Now, as you can see, the list of manpages is directly determined by $PREFIX variable. In order to make zsh:foo to list only man pages of zsh* which contain the word foo, it needs to be split across : character (if any).
The following addition in _man_pages partially solve the issue (zsh 4.3.4):
Original:
_man_pages() {
local matcher pages dummy sopt
zparseopts -E M+:=matcher
if (( $#matcher )); then
matcher=( ${matcher:#-M} )
matcher="$matcher"
else
matcher=
fi
pages=( ${(M)dirs:#*$sect/} )
compfiles -p pages '' '' "$matcher" '' dummy '*'
pages=( ${^~pages}(N:t) )
(($#mrd)) && pages[$#pages+1]=($(awk $awk $mrd))
# Remove any compression suffix, then remove the minimum possible string
# beginning with .<->: that handles problem cases like files called
# `POSIX.1.5'.
[[ $OSTYPE = solaris* ]] && sopt='-s '
if ((CURRENT > 2)) ||
! zstyle -t ":completion:${curcontext}:manuals.$sect" insert-sections
then
compadd "$#" - ${pages%.(?|<->*(|.gz|.bz2|.Z))}
else
compadd "$#" -P "$sopt$sect " - ${pages%.(?|<->*(|.gz|.bz2|.Z))}
fi
}
Modified (look for ##mod comments):
_man_pages() {
local matcher pages dummy sopt
zparseopts -E M+:=matcher
if (( $#matcher )); then
matcher=( ${matcher:#-M} )
matcher="$matcher"
else
matcher=
fi
pages=( ${(M)dirs:#*$sect/} )
##mod
# split components by the ":" character
local pref_words manpage_grep orig_prefix
# save original prefix (just in case)
orig_prefix=${PREFIX}
# split $PREFIX by ':' and make it an array
pref_words=${PREFIX//:/ }
set -A pref_words ${=pref_words}
# if there are both manpage name and grep string, use both
if (( $#pref_words == 2 )); then
manpage_grep=$pref_words[2]
# PREFIX is used internally by compfiles
PREFIX=$pref_words[1]
elif (( $#pref_words == 1 )) && [[ ${PREFIX[1,1]} == ":" ]]; then
# otherwise, prefix is empty and only grep string exists
PREFIX=
manpage_grep=$pref_words[1]
fi
compfiles -p pages '' '' "$matcher" '' dummy '*'
##mod: complete, but don't strip path names
pages=( ${^~pages} )
(($#mrd)) && pages[$#pages+1]=($(awk $awk $mrd))
##mod: grep pages
# Build a list of matching pages that pass the grep
local matching_pages
typeset -a matching_pages
# manpage_grep exists and not empty
if (( ${#manpage_grep} > 0 )); then
for p in ${pages}; do
zgrep "${manpage_grep}" $p > /dev/null
if (( $? == 0 )); then
#echo "$p matched $manpage_grep"
matching_pages+=($p)
fi
done
else
# there's no manpage_grep, so all pages match
matching_pages=( ${pages} )
fi
#echo "\nmatching manpages: "${matching_pages}
pages=( ${matching_pages}(N:t) )
# keep the stripped prefix for now
#PREFIX=${orig_prefix}
# Remove any compression suffix, then remove the minimum possible string
# beginning with .<->: that handles problem cases like files called
# `POSIX.1.5'.
[[ $OSTYPE = solaris* ]] && sopt='-s '
if ((CURRENT > 2)) ||
! zstyle -t ":completion:${curcontext}:manuals.$sect" insert-sections
then
compadd "$#" - ${pages%.(?|<->*(|.gz|.bz2|.Z))}
else
compadd "$#" -P "$sopt$sect " - ${pages%.(?|<->*(|.gz|.bz2|.Z))}
fi
}
However, it's still not fully working (if you uncomment the #echo "$p matched $manpage_grep" line, you can see that it does build the list) - I suspect that somewhere internally, the completion system sees that, for instance, "zshcompctl" is not matched by prefix "zsh:foo", and does not display the resulting matches. I've tried to keep $PREFIX as it is after stripping the grep string, but it still does not want to work.
At any rate, this at least should get you started.

Resources