I'm trying to copy a directories contents without one file. The problem i'm having is that the file is a few folders nested and the extglob operator fails to match.
Assume the following folder structure:
I would like to copy everything from source, including subfolders and files into dest except smu.txt.
I would have thought the following would do the trick:
#!/bin/bash
shopt -s extglob
cp -vr source/!(smu.txt) dest/
But it still copies smu.txt.
I also tried the following without success:
#!/bin/bash
shopt -s extglob
shopt -s globstar
cp -vr source/!(**/smu.txt) dest/
It if smu.txt is directly under ../source it successfully ignores it, but how do I get it to ignore files within subdirectories?
Have you tried using find?
Maybe this works:
$ find -name "source/*!(smu.txt)" -exec cp -vr {} dest/\;
Related
This question already has answers here:
Bash move * to subfolder fail: cannot move to a subdirectory of itself
(5 answers)
Closed 1 year ago.
How to move content of a folder to its subfolder.
When I run this:
mv xyz/* xyz/archive
I get this notice:
mv: cannot move 'xyz/archive' to a subdirectory of itself,
'xyz/archive/archive'
Is is possible to exclude archive folder from xyz/* selection ?
You can use extended globs to exclude archive from your pattern as follows:
shopt -s extglob
mv !(archive) archive
This will move everything in the current folder, except for hidden files, to archive.
If you want to move hidden files starting with a . to the archive folder too, you need to set the dotglob option as described in bash manual under Filename Expansion:
shopt -s dotglob
Here is another method using find:
find . -maxdepth 1 -type f ! -name "archive" -exec mv -t archive {} +
Although the first method is preferred, IMO, since bash builtins are perfectly capable of what find is doing here.
Yes, it is possible to exclude archive from the selection.
Enable the extended globbing option
shopt -s extglob
Run the following command that will move everything but archive
mv !(archive) archive
It will now move everything from xyz to archive
PS: I did run the mv command when I was inside the xyz folder.
I want to mimic copying a directory structure recursively (as in cp -r or rsync -a), but only touch the copied files, i.e. make all the copied files empty.
The specific use case is for a Snakemake pipeline; Snakemake looks for existing files in order to decide whether to re-run a pipeline step, and I want to make it believe the steps have already been run while avoiding fully downloading all the files.
This is a little kludgy, but you could pipe the output of find or rsync -nv into a little bash loop with mkdir -p and touch:
find /some/dir -type f | while read FILE; do
mkdir -p $(dirname $FILE)
touch $FILE
done
I have the following structure:
FolderA
Sub1
Sub2
filexx.csv
filexx.doc
FolderB
Sub1
Sub2
fileyy.csv
fileyy.doc
I want to write a script that will move the .csv files into the folder sub1 for each parent directory (Folder A, Folder B and so on) giving me the following structure:
FolderA
Sub1
filexx.csv
Sub2
filexx.doc
FolderB
Sub1
fileyy.csv
Sub2
fileyy.doc
This is what I have till now but I get the error mv: cannot stat *.csv: No such file or directory
for f in */*/*.csv; do
mv -v "$f" */*/Sub1;
done
for f in */*/*.doc; do
mv -v "$f" */*/Sub2;
done
I am new to bash scripting so please forgive me if I have made a very obvious mistake. I know I can do this in Python as well but it will be lengthier which is why I would like a solution using linux commands.
find . -name "*.csv" -type f -execdir mv '{}' Sub1/ \;
Using find, search for all files with the extension .csv and then when we find them, execute a move command from within the directory containing the files, moving the files to directory Sub1
find . -name "*.doc" -type f -execdir mv '{}' Sub2/ \;
Follow the same principle for files with the extension .doc but this time, move the files to Sub2.
I believe you are getting this error because no file matched your wildcard. When it happens, the for loop will give $f the value of the wildcard itself. You are basically trying to move the file *.csv which does not exist.
To prevent this behavior, you can add shopt -s nullglob at the top of your script. When using this, if no file is found, your script won't enter the loop.
My advise is, make sure you run your script from the correct location when using wildcards like this. But maybe what you meant to do by writing */*/*.csv is to recursively match all the csv files. If that's what you intended to do, this is not the right way to do it.
To recursively match all csv/doc/etc files using native bash you can add shopt -s globstar to the top of your script and use **/*.csv as wildcard
#!/bin/bash
shopt -s globstar nullglob
for f in **/*.csv; do
mv "$f" Destination/ # Note that $f is surrounded by "" to handle whitespaces in filenames
done
You could also use the find (1) utility to achieve that. But if you're planning to do more processing on the files than just moving them, a for loop might be cleaner as you won't have to inline everything in the same command.
Side note : "Linux commands" as you say are actually not Linux commands, they are part of the GNU utilities (https://www.gnu.org/gnu/linux-and-gnu.en.html)
If csv files you want to move are in the top directories (from the point of view of the current directory), but not in the subdirectories of them, then simply:
#!/bin/bash
for dir in */; do
mv -v "$dir"*.csv "${dir}Sub1/"
mv -v "$dir"*.doc "${dir}Sub2/"
done
If the files in all subdirectories are wanted to be moved similarly, then:
shopt -s globstar
for file in **/*.csv; do
mv -v "$file" "${file%/*}/Sub1/"
done
for file in **/*.doc; do
mv -v "$file" "${file%/*}/Sub2/"
done
Note that, the directories Sub1 and Sub2 are relative to the directory where csv and doc files reside.
I'm trying to port the following robocopy command to Linux:
robocopy SrcDir DstDir *.dll *.pdb *.xml /xf Unity*.* nunit*.*
In other words, I want to:
Include all dll, pdb and xml files
Unless they start with Unity or nunit
I've read the following two threads, but can't figure the exact syntax:
Copy all files with a certain extension from all subdirectories
How to use 'cp' command to exclude a specific directory?
My best guess so far would be:
Enable shopt -s extglob
Go to the source directory cd SrcDir
Use this command: cp ((*.dll | *.pdb | *.xml) && !(Unity*.* | nunit*.*)) DstDir
But I get syntax errors inside my conditional expression, starting at *.dll.
You may use this command using extglob:
shopt -s extglob nullglob dotglob
cd "$srcDir"
cp !(#(Unity|nunit)).{dll,pdp,xml} "$dstDir"
according to below link :
BASH copy all files except one
you can use below command :
find [YOUR_SOURCE_ADDRESS] \( -iname \*.dll -o -iname \*.pdb -o -iname \*.xml \) ! \(-name "Unity*" -o -name "nunit*" \) -exec cp -t [YOUR_DEST_ADDRESS] {} +
you can change [YOUR_SOURCE_ADDRESS] and [YOUR_DEST_ADDRESS] (with your serach address and destination address for copy)
Where my question originated:
When running cp source/files.all destination/, all the files within source will now also exist in destination
Question:
What if I didn't want to duplicate the data from source into destination, but simply link them (with absolute path). Usually, I would run something like:
for f in $(ls source/); do ln -s $(pwd)/${f} $(pwd)/destination; done
Is there a simple command/tool that I can use (e.g. ln -a source/files.all destination/) which would create a softlink to all files in a directory, while automatically adding the absolute path as prefix. ln -r is close to what I need, but the absolute path, not the relative one?
I would use find "$PWD/source" -exec ln -s {} destination \;. The absolute path used as the first argument to find will cause {} to be replaced by an absolute path to the source file for each command.
GNU ln supports the -t option to specify the destination directory, allowing you to use a more efficient invocation of find:
find "$PWD/source" -exec ln -s -t destination {} +
The -exec ... + form requires {} to be the last argument in the command; -t lets you move the destination argument up to accommodate that requirement.
So I eventually sort of found a simple way to do this:
Simply run ln -s $(pwd -P)/source/* destination/