I'm trying to match two directories and if the file exists in the second directory, I want to move files from the first directory to a third one.
The filenames do not matching exactly, they get a "_ica" at the end of the name and a different extension.
I have tried to write a script that loops through dir1 checks if it's in dir2
and if found move to dir3:
DATA= /home/eegfilesonlyWM/*
PROCESSED= /home/eegfilesonlyWM/icaddata/*
DONE= /home/eegfilesonlyWM/done/
for f in $DATA ; do
fname=${f##*/}
fname=${fname%/}
find /home/eegfilesonlyWM/icaddata/ -iname "${fname*_ica*}" -type f -exec mv {} ./done \;
done
I would like to copy from the first directory those files that already have corresponding files in the second directory.
Thank you for any help
Maybe this will do what you want:
#!/usr/bin/env bash
#Directory paths here
DATA=./DATA
PROCESSED=./PROCESSED
DONE=./DONE
#Do the test and copy here
for f in `ls -1 $DATA`; do
#build output name
p="$PROCESSED/${f/\.xxx/}"; #xxx is the file extension of original
p="${p}_ica.yyy"; #yyy is the file extension of the processed
if [ -f $p ] ; then
cp $DATA/$f $DONE
fi
done
Related
I have the following structure:
FolderA
Sub1
Sub2
filexx.csv
filexx.doc
FolderB
Sub1
Sub2
fileyy.csv
fileyy.doc
I want to write a script that will move the .csv files into the folder sub1 for each parent directory (Folder A, Folder B and so on) giving me the following structure:
FolderA
Sub1
filexx.csv
Sub2
filexx.doc
FolderB
Sub1
fileyy.csv
Sub2
fileyy.doc
This is what I have till now but I get the error mv: cannot stat *.csv: No such file or directory
for f in */*/*.csv; do
mv -v "$f" */*/Sub1;
done
for f in */*/*.doc; do
mv -v "$f" */*/Sub2;
done
I am new to bash scripting so please forgive me if I have made a very obvious mistake. I know I can do this in Python as well but it will be lengthier which is why I would like a solution using linux commands.
find . -name "*.csv" -type f -execdir mv '{}' Sub1/ \;
Using find, search for all files with the extension .csv and then when we find them, execute a move command from within the directory containing the files, moving the files to directory Sub1
find . -name "*.doc" -type f -execdir mv '{}' Sub2/ \;
Follow the same principle for files with the extension .doc but this time, move the files to Sub2.
I believe you are getting this error because no file matched your wildcard. When it happens, the for loop will give $f the value of the wildcard itself. You are basically trying to move the file *.csv which does not exist.
To prevent this behavior, you can add shopt -s nullglob at the top of your script. When using this, if no file is found, your script won't enter the loop.
My advise is, make sure you run your script from the correct location when using wildcards like this. But maybe what you meant to do by writing */*/*.csv is to recursively match all the csv files. If that's what you intended to do, this is not the right way to do it.
To recursively match all csv/doc/etc files using native bash you can add shopt -s globstar to the top of your script and use **/*.csv as wildcard
#!/bin/bash
shopt -s globstar nullglob
for f in **/*.csv; do
mv "$f" Destination/ # Note that $f is surrounded by "" to handle whitespaces in filenames
done
You could also use the find (1) utility to achieve that. But if you're planning to do more processing on the files than just moving them, a for loop might be cleaner as you won't have to inline everything in the same command.
Side note : "Linux commands" as you say are actually not Linux commands, they are part of the GNU utilities (https://www.gnu.org/gnu/linux-and-gnu.en.html)
If csv files you want to move are in the top directories (from the point of view of the current directory), but not in the subdirectories of them, then simply:
#!/bin/bash
for dir in */; do
mv -v "$dir"*.csv "${dir}Sub1/"
mv -v "$dir"*.doc "${dir}Sub2/"
done
If the files in all subdirectories are wanted to be moved similarly, then:
shopt -s globstar
for file in **/*.csv; do
mv -v "$file" "${file%/*}/Sub1/"
done
for file in **/*.doc; do
mv -v "$file" "${file%/*}/Sub2/"
done
Note that, the directories Sub1 and Sub2 are relative to the directory where csv and doc files reside.
experts, i have many folders and inside the folder there are many sub-folders and the sub-folders contain many files.However, in all the sub-folders one file name is same that is input.ps.Now i want to rename the same input.ps with full path plus file name
so input.ps in all directories should be renamed to home_wuan_data_filess_input.ps
i tried
#!/bin/sh
for file in /home/wuan/data/filess/*.ps
do
mv $file $file_
done
But it does not do the same as i expect,i hope experts will help me.Thanks in advance.
so input.ps in all directories should be renamed to home_wuan_data_filess_input.ps
You may use this find solution:
find /home/wuan/data/filess -type f -name 'input*.ps' -exec bash -c '
for f; do fn="${f#/}"; echo mv "$f" "${fn//\//_}"; done' _ {} +
while read line;
do
fil=${line//\//_}; # Converts all / characters to _ to form the file name
fil=${fil:1}; # Remove the first -
dir=${line%/*}; # Extract the directory
echo "mv $line $dir/$fil"; # Echo the move command
# mv "$line" "$dir/$fil"; # Remove the comment to perform the actual command
done <<< "$(find /path/to/dir -name "input.ps")"
Ok, so file will end up being
/home/wuan/data/filess/input.ps
What we need here is the path, and the full, snake-cased name. Let's start by getting the path:
for f in /home/wuan/data/filess/*.ps; do
path="${f%*/}";
This will match the substring of f up until the last occurrence of /, effectively giving us the path.
Next, we want to snake_case all the things, which is even easier:
for f in /home/wuan/data/filess/*.ps; do
path="${f%*/}";
newname="${f//\//_}"
This replaces all instances of / with _, giving the name you want the new file to have. Now let's put all of this together, and move the file f to path/newname:
for f in /home/wuan/data/filess/*.ps; do
path="${f%*/}";
newname="${f//\//_}"
mv "${f}" "${path}/${newname}"
done
That should do the trick
Here's one of many sites listing some of the bash string manipulations you can use.
Sorry for the delayed update, the power in my building just cut out :)
I'm now using the bash script below to create file listing in each level 2 depth directories and output as text file in each particular folders.
But, i can only name it as "File.txt"
How to use the level 2 depth directory as the output text file name
e.g. while list the files in /users/bin/ the output file should named "bin.txt"
here's the code that i'm using.
#!/bin/bash
while IFS= read -r -d '' folder; do
echo $folder find "$folder" -type f > "$folder"/File.txt
done < <(find . -maxdepth 2 -mindepth 2 -type d -print0)
You have to account for (1) files in the present working directory (which will have no path component aside from '.'), and (2) the rest of the files in the level 2 directories. If for instance, you wanted to output your list of the files in the present directory to pwd.txt and each file in the level 2 directories to files with the respective directory names (e.g. ./dir1/file_a.txt into dir1.txt), you could do something similar to the following in the body of your loop:
tmp="${folder%/*}" ## remove filename
lvl2name="${tmp##*/}" ## remove leading path components
[[ $lvl2name = '.' ]] && lvl2name="pwd" ## handle files in pwd to pwd.txt
echo $folder find "$folder" -type f >> "$lvl2name.txt"
It is up to you to truncate any existing "$lvl2name.txt" files before each subsequent run of the script. You could make a previous call to find . -maxdepth 2 -type d ... before your enter you file loop to get the directory names or something similar.
Give it a try and let me know if you have questions.
I am on a Linux system and I am trying to rename all .jpg files in many subdirectories to sequential filenames, so all the jpeg files in each subdirectory are renamed 0001.jpg, 0002.jpg, etc. I have a 'rename' command that works in a single directory:
rename -n 's/.*/sprintf("%04d",$::iter++ +1).".jpg"/e' *.jpg
I am trying to use it like this:
for i in ls -D; do rename -n 's/.*/sprintf("%04d",$::iter++ +1).".jpg"/e' *.jpg; done
but for output I get this:
*.jpg renamed as 0001.jpg
for each subdirectory. What am I doing wrong?
You need to put the command in backticks (or use the $( ... ) bash syntax) in order
to iterate over its output. Also use the $i variable together with the *.jpg file
name pattern, e.g.
for i in `ls -D`
do
rename -n 's/.*/sprintf("%04d",$::iter++ +1).".jpg"/e' $i/*.jpg
done
however, for this scenario you want to iterate over all the subdirectories, and you are
better of using the find command:
for i in `find . -type d`; do rename ...
It seems to me you've forgot to change a current working directory so it should looks like
for i in *; do
[ -d "$i" ] || continue
pushd "$i"
# rename is here
popd
done
Let's say I have the following subdirectories
./a/, ./b/, ./c/, ...
That is, in my current working directory are these subdirectories a/, b/ and c/, and in each of these subdirectories are files. In directory a/ is the file a.in, in directory b/ is the file b.in and so forth.
I now want to copy each .in file to a .out file, that is, a.in to a.out and b.in to b.out, and I want them to reside in the directories they were copied from. So a.out will be found in directory a/.
I've tried various different approaches, such as
find ./ -name '*.in'|cp * *.out
which doesn't work because it thinks *.out is a directory. Also tried
ls -d */ | cd; cp *.in *.out
but it that would list the subdirectories, go into each one of them, but won't let cp do it's work (which still doesn't work)
The
find ./ -name '*.in'
command works fine. Is there a way to pipe arguments to an assignment operator? E.g.
find ./ -name '*.in'| assign filename=|cp filename filename.out
where assign filename= gives filename the value of each .in file. In fact, it would be even better if the assignment could get rid of the .in file extension, then instead of getting a.in.out we would get the preferred a.out
Thank you for your time.
Let the shell help you out:
find . -name '*.in' | while read old; do
new=${old%.in}.out # strips the .in and adds .out
cp "$old" "$new"
done
I just took the find command you said works and let bash read its output one filename at a time. So the bash while loop gets the filenames one at a time, does a little substitution, and a straight copy. Nice and easy (but not tested!).
Try a for loop:
for f in */*.in; do
cp $f ${f%.in}.out;
done
The glob should catch all the files one directory down that have a .in extension. In the cp command, it strips off the .in suffix and then appends a .out (see Variable Mangling in Bash with String Operators)
Alternatively, if you want to recurse into every subdirectory (not just 1 level deep) replace the glob with a find:
for f in $(find . -name '*.in'); do
cp $f ${f%.in}.out;
done
This should do the trick!
for f in `find . -type f -name "*.in"`; do cp $f `echo $f | sed 's/in$/out/g'`; done