I'm trying to port the following robocopy command to Linux:
robocopy SrcDir DstDir *.dll *.pdb *.xml /xf Unity*.* nunit*.*
In other words, I want to:
Include all dll, pdb and xml files
Unless they start with Unity or nunit
I've read the following two threads, but can't figure the exact syntax:
Copy all files with a certain extension from all subdirectories
How to use 'cp' command to exclude a specific directory?
My best guess so far would be:
Enable shopt -s extglob
Go to the source directory cd SrcDir
Use this command: cp ((*.dll | *.pdb | *.xml) && !(Unity*.* | nunit*.*)) DstDir
But I get syntax errors inside my conditional expression, starting at *.dll.
You may use this command using extglob:
shopt -s extglob nullglob dotglob
cd "$srcDir"
cp !(#(Unity|nunit)).{dll,pdp,xml} "$dstDir"
according to below link :
BASH copy all files except one
you can use below command :
find [YOUR_SOURCE_ADDRESS] \( -iname \*.dll -o -iname \*.pdb -o -iname \*.xml \) ! \(-name "Unity*" -o -name "nunit*" \) -exec cp -t [YOUR_DEST_ADDRESS] {} +
you can change [YOUR_SOURCE_ADDRESS] and [YOUR_DEST_ADDRESS] (with your serach address and destination address for copy)
Related
I have below setup and I want to find and move files.
I have files /home/backup/abc/123.wav and /home/backup/xyz/456.wav.
Same directories exist at /usr/src/abc and /usr/src/xyz which does not have any files.
I want to find .wav files from home_dir and move them to particular dest_dir.
So 123.wav should move to /usr/src/abc and 456.wav should move to /usr/src/xyz.
I am using below command for that.
home_dir=/home/backup/
dest_dir=/usr/src/
cd $home_dir && find . -iname "*.wav" -exec mv {} $dest_dir \;
But all the .wav files(123.wav and 456.wav) moved to /usr/src and not to its respective directories(/usr/src/abc and /usr/src/xyz).
Is it possible to achieve what I want ?
Please suggest.
Use cp --parents option with find to create parent directories of each file being copied:
home_dir=/home/backup/
dest_dir=/usr/src/
cd "$home_dir"
find . -iname "*.wav" -exec cp --parents {} "$dest_dir" \; -delete
This would be a lot easier if mv had a --parents option, but unfortunately it doesn't. It's best to use mv instead of cp because cp will copy all the data unnecessarily if the source and destination directories are on the same filesystem. if you've got Bash 4 (which supports globstar) you could do it like this:
home_dir=/home/backup/
dest_dir=/usr/src/
shopt -s globstar nullglob dotglob
for src_wav in "$home_dir"/**/*.wav ; do
rel_wav=${src_wav#$home_dir/}
dst_wav=$dest_dir/$rel_wav
dst_parent=${dst_wav%/*}
[[ -d $dst_parent ]] || mkdir -p -- "$dst_parent"
mv -- "$src_wav" "$dst_wav"
done
As we know, cp -r source_dir intended_new_directory creates a copy of source directory with a new name. Now I want to do the same but want to exclude a particular file. I have found some related answers here, using tar and rsync, but in those solutions I need to create the destination directory first (using mkdir).
I honestly searched a lot, but didn't find exactly what I want.
So far the best I got is this:
tar -c --exclude=\*.dll --exclude=\*.exe sourceDir | tar -x -C destDir
(from http://www.linuxquestions.org/questions/programming-9/how-to-copy-an-entire-directory-structure-except-certain-files-385321/)
If you have binutils, you could use find to filter next cpio to copy (and create directories) :
find <sourceDir> \( ! -name *.dll \) -a \( ! -name *.exe \) | cpio -dumpv <destDir>
Try this by excluding the file using 'grep -v' ->
cp `ls | grep -v <exclude-file>` <dest-dir>
If the directory is not very large I used to write something like this:
src=path/to/source/directory
dst=path/to/destination/directory
find $src -type f | while read f ; do mkdir -p "$dst/`dirname $f`"; cp "$f" "$dst/$f" ; done
Here we list all regular files in $src, iterate over this list and for each file make a directory in $dst if it does not exist yet (-p option of mkdir), then copy the file to that directory.
The above command will copy all the files. Finally, just use
find $src -type f | grep -v whatever | while ...... # same as above
to filter out the files you don't need (e.g. \.bak$, \.orig$, or whatever files you don't want to copy).
Move all exclude file into home or other directory,copy the directory containing all remaining files to the destination folder then restore all exclude files.
#cd mydirectory
#mv exclude1 exclude2 /home/
#cp mydirectory destination_folder/
#cd /home/
#mv eclude1 exclude2 mydirectory/
I am having files like a_dbg.txt, b_dbg.txt ... in a Suse 10 system. I want to write a bash shell script which should rename these files by removing "_dbg" from them.
Google suggested me to use rename command. So I executed the command rename _dbg.txt .txt *dbg* on the CURRENT_FOLDER
My actual CURRENT_FOLDER contains the below files.
CURRENT_FOLDER/a_dbg.txt
CURRENT_FOLDER/b_dbg.txt
CURRENT_FOLDER/XX/c_dbg.txt
CURRENT_FOLDER/YY/d_dbg.txt
After executing the rename command,
CURRENT_FOLDER/a.txt
CURRENT_FOLDER/b.txt
CURRENT_FOLDER/XX/c_dbg.txt
CURRENT_FOLDER/YY/d_dbg.txt
Its not doing recursively, how to make this command to rename files in all subdirectories. Like XX and YY I will be having so many subdirectories which name is unpredictable. And also my CURRENT_FOLDER will be having some other files also.
You can use find to find all matching files recursively:
find . -iname "*dbg*" -exec rename _dbg.txt .txt '{}' \;
EDIT: what the '{}' and \; are?
The -exec argument makes find execute rename for every matching file found. '{}' will be replaced with the path name of the file. The last token, \; is there only to mark the end of the exec expression.
All that is described nicely in the man page for find:
-exec utility [argument ...] ;
True if the program named utility returns a zero value as its
exit status. Optional arguments may be passed to the utility.
The expression must be terminated by a semicolon (``;''). If you
invoke find from a shell you may need to quote the semicolon if
the shell would otherwise treat it as a control operator. If the
string ``{}'' appears anywhere in the utility name or the argu-
ments it is replaced by the pathname of the current file.
Utility will be executed from the directory from which find was
executed. Utility and arguments are not subject to the further
expansion of shell patterns and constructs.
For renaming recursively I use the following commands:
find -iname \*.* | rename -v "s/ /-/g"
small script i wrote to replace all files with .txt extension to .cpp extension under /tmp and sub directories recursively
#!/bin/bash
for file in $(find /tmp -name '*.txt')
do
mv $file $(echo "$file" | sed -r 's|.txt|.cpp|g')
done
with bash:
shopt -s globstar nullglob
rename _dbg.txt .txt **/*dbg*
find -execdir rename also works for non-suffix replacements on basenames
https://stackoverflow.com/a/16541670/895245 works directly only for suffixes, but this will work for arbitrary regex replacements on basenames:
PATH=/usr/bin find . -depth -execdir rename 's/_dbg.txt$/_.txt' '{}' \;
or to affect files only:
PATH=/usr/bin find . -type f -execdir rename 's/_dbg.txt$/_.txt' '{}' \;
-execdir first cds into the directory before executing only on the basename.
Tested on Ubuntu 20.04, find 4.7.0, rename 1.10.
Convenient and safer helper for it
find-rename-regex() (
set -eu
find_and_replace="$1"
PATH="$(echo "$PATH" | sed -E 's/(^|:)[^\/][^:]*//g')" \
find . -depth -execdir rename "${2:--n}" "s/${find_and_replace}" '{}' \;
)
GitHub upstream.
Sample usage to replace spaces ' ' with hyphens '-'.
Dry run that shows what would be renamed to what without actually doing it:
find-rename-regex ' /-/g'
Do the replace:
find-rename-regex ' /-/g' -v
Command explanation
The awesome -execdir option does a cd into the directory before executing the rename command, unlike -exec.
-depth ensure that the renaming happens first on children, and then on parents, to prevent potential problems with missing parent directories.
-execdir is required because rename does not play well with non-basename input paths, e.g. the following fails:
rename 's/findme/replaceme/g' acc/acc
The PATH hacking is required because -execdir has one very annoying drawback: find is extremely opinionated and refuses to do anything with -execdir if you have any relative paths in your PATH environment variable, e.g. ./node_modules/.bin, failing with:
find: The relative path ‘./node_modules/.bin’ is included in the PATH environment variable, which is insecure in combination with the -execdir action of find. Please remove that entry from $PATH
See also: https://askubuntu.com/questions/621132/why-using-the-execdir-action-is-insecure-for-directory-which-is-in-the-path/1109378#1109378
-execdir is a GNU find extension to POSIX. rename is Perl based and comes from the rename package.
Rename lookahead workaround
If your input paths don't come from find, or if you've had enough of the relative path annoyance, we can use some Perl lookahead to safely rename directories as in:
git ls-files | sort -r | xargs rename 's/findme(?!.*\/)\/?$/replaceme/g' '{}'
I haven't found a convenient analogue for -execdir with xargs: https://superuser.com/questions/893890/xargs-change-working-directory-to-file-path-before-executing/915686
The sort -r is required to ensure that files come after their respective directories, since longer paths come after shorter ones with the same prefix.
Tested in Ubuntu 18.10.
Script above can be written in one line:
find /tmp -name "*.txt" -exec bash -c 'mv $0 $(echo "$0" | sed -r \"s|.txt|.cpp|g\")' '{}' \;
If you just want to rename and don't mind using an external tool, then you can use rnm. The command would be:
#on current folder
rnm -dp -1 -fo -ssf '_dbg' -rs '/_dbg//' *
-dp -1 will make it recursive to all subdirectories.
-fo implies file only mode.
-ssf '_dbg' searches for files with _dbg in the filename.
-rs '/_dbg//' replaces _dbg with empty string.
You can run the above command with the path of the CURRENT_FOLDER too:
rnm -dp -1 -fo -ssf '_dbg' -rs '/_dbg//' /path/to/the/directory
You can use this below.
rename --no-act 's/\.html$/\.php/' *.html */*.html
This command worked for me. Remember first to install the perl rename package:
find -iname \*.* | grep oldname | rename -v "s/oldname/newname/g
To expand on the excellent answer #CiroSantilliПутлерКапут六四事 : do not match files in the find that we don't have to rename.
I have found this to improve performance significantly on Cygwin.
Please feel free to correct my ineffective bash coding.
FIND_STRING="ZZZZ"
REPLACE_STRING="YYYY"
FIND_PARAMS="-type d"
find-rename-regex() (
set -eu
find_and_replace="${1}/${2}/g"
echo "${find_and_replace}"
find_params="${3}"
mode="${4}"
if [ "${mode}" = 'real' ]; then
PATH="$(echo "$PATH" | sed -E 's/(^|:)[^\/][^:]*//g')" \
find . -depth -name "*${1}*" ${find_params} -execdir rename -v "s/${find_and_replace}" '{}' \;
elif [ "${mode}" = 'dryrun' ]; then
echo "${mode}"
PATH="$(echo "$PATH" | sed -E 's/(^|:)[^\/][^:]*//g')" \
find . -depth -name "*${1}*" ${find_params} -execdir rename -n "s/${find_and_replace}" '{}' \;
fi
)
find-rename-regex "${FIND_STRING}" "${REPLACE_STRING}" "${FIND_PARAMS}" "dryrun"
# find-rename-regex "${FIND_STRING}" "${REPLACE_STRING}" "${FIND_PARAMS}" "real"
In case anyone is comfortable with fd and rnr, the command is:
fd -t f -x rnr '_dbg.txt' '.txt'
rnr only command is:
rnr -f -r '_dbg.txt' '.txt' *
rnr has the benefit of being able to undo the command.
On Ubuntu (after installing rename), this simpler solution worked the best for me. This replaces space with underscore, but can be modified as needed.
find . -depth | rename -d -v -n "s/ /_/g"
The -depth flag is telling find to traverse the depth of a directory first, which is good because I want to rename the leaf nodes first.
The -d flag on rename tells it to only rename the filename component of the path. I don't know how general the behavior is but on my installation (Ubuntu 20.04), it could be the file or the directory as long as it is the leaf node of the path.
I recommend the -n (no action) flag first along with -v, so you can see what would get renamed and how.
Using the two flags together, it renames all the files in a directory first and then the directory itself. Working backwards. Which is exactly what I needed.
classic solution:
for f in $(find . -name "*dbg*"); do mv $f $(echo $f | sed 's/_dbg//'); done
I'm not exactly sure how to go about doing this, but I need to create symbolic links for certain files in one directory and place the symbolic links in another directory.
For instance, I want to link all files with the word "foo" in its name in the current directory bar1 that does not have the extension ".cc" and place the symbolic links in a directory bar2.
I was wondering if there was single line command that could accomplish this in LINUX bash.
Assuming you are in a directory that contains directories bar1 and bar2:
find bar1 -name '*foo*' -not -type d -not -name '*.cc' -exec ln -s $PWD/'{}' bar2/ \;
Try this:
cd bar1
find . -maxdepth 1 -name '*foo*' -not -name '*.cc' -exec echo ln -s $PWD/{} ../bar2 \;
Once you are satisfied with the dry run, remove echo from the command and run it for real.
This is easily handled with extended globbing:
shopt -s extglob
cd bar2
ln -s ../bar1/foo!(*.cc) .
If you really want it all on one line, just use the command separator:
shopt -s extglob; cd bar2; ln -s ../bar1/foo!(*.cc) .
The two examples are identical, but the first is much easier to read.
This technically doesn't count as a one line answer...but it can be pasted in a single instance and should do what you are looking for.
list=`ls | grep foo | grep -v .cc`;for file in $list;do ln $file /bar2/;done
I just want to rename all the *.a file to *.a.b in current directory and subdirs, how to do it in shell script?
find . -type f -name '*.a' -print0 | xargs -0 -IZZ mv ZZ ZZ.b
This should handle filenames with spaces and / or newlines. It also doesn't rename directories (the other solution doing find would). If you want it to be case-insensitive, use "-iname" instead of "-name"
Ruby(1.9+)
$ ruby -e 'Dir["**/*.a"].each{|x|File.file?x && File.rename(x,"#{x}.b")}'
In a shell script (at least Bash 4)
shopt -s globstar
shopt -s nullglob
for file in **/*.a
do
echo mv "${file}" "${file}.b"
done
To rename <files> with that, rename 's/\.a$/.a.b/' <files>. Doing so recursively will just take a bit of looping.
(or use *, */*, */*/*, */*/*/*, etc. for the files)
Try the script below:
for file in `find . -name '*.a'`; do mv $file $file.b; done