I have a directory with several folders inside. Inside the folders I have a worksheet with the same name in all folders. I need to run a script that randomly changes the name of the worksheets so that I can throw them all in the same folder. For example: worksheet1
worksheet2
worksheet3.
today they are all called spreadsheet.csv
I have a sketch in linux.
need help please.
NN=0;
for arq in $(ls -1 *.csv);do
let NN++;
rename -n 's/'${arq}'/spreadsheet'${NN}'.csv/' ${arq};
done
(this search all files csv, but dont is recursive. Work only in one directory)
Use globstar:
n=0
shopt -s globstar nullglob || exit
for f in **/*.csv; do
until dst=spreadsheet$(( ++n )).csv; [[ ! -e ${dst} ]]; do
continue
done
mv -i -- "${f}" "${dst}"
done
Sample files:
$ find . -name "*.csv"
./spreadsheet.csv
./sub1/spreadsheet.csv
./sub1/sub2/spreadsheet.csv
One idea:
n=0
while read -r oldname
do
((++n))
newname="${oldname##*/}"
newname="${newname//.csv/-$n.csv}"
echo mv "$oldname" "$newname"
done < <(find . -name spreadsheet.csv)
This generates:
mv ./spreadsheet.csv spreadsheet-1.csv
mv ./sub1/spreadsheet.csv spreadsheet-2.csv
mv ./sub1/sub2/spreadsheet.csv spreadsheet-3.csv
Once OP is satisifed with the output, remove the echo and run the script again.
After removing the echo and running again:
$ find . -name "*.csv"
./spreadsheet-1.csv
./spreadsheet-2.csv
./spreadsheet-3.csv
Related
I'm searching for a find command to copy all wallpaper files that look like this:
3245x2324.png (All Numbers are just a placeholder)
3242x3242.jpg
I'm in my /usr/share/wallpapers folder and there are many sub folders with the files I want to copy.
There are many like "screenshot.png" and these files I don't want to copy.
My find command is like this:
find . -type f -name "*????x????.???"
If I search with this I get the files I wanted to see, but if I combine this with -exec cp:
find . -type f -name "*????x????.???" -exec cp "{}" /home/mine/Pictures/WP \;
the find command only copies 10 files and there are 77 (I counted with wc).
Does anyone know what I'm doing wrong?
You can look it up if you follow the link.
renaming with find
You can use -exec to do this. But i'm not sure you can do rename and copy in one take.Maybe with a script that got executed after every find result.
But that's only a suggestion.
One idea/approach is to copy absolute path of the file in question to the destination, but replace the / with an underscore _ since / is not allowed in file names, at least in a Unix like environment.
With find and bash, Something like.
find /usr/share/wallpapers -type f -name "????x????.???" -exec bash -c '
destination=/home/mine/Pictures/WP/
shift
for f; do
path_name=${f%/*}
file_name=${f##*/}
echo cp -vi -- "$f" "$destination${path_name//\//_}$file_name"
done' _ {} +
See understanding-the-exec-option-of-find
With globstar nullglob shell option and Associative array from the bash shell to avoid the duplicate filenames.
#!/usr/bin/env bash
shopt -s globstar nullglob
pics=(/usr/share/wallpapers/**/????x????.???)
shopt -u globstar nullglob
declare -A dups
destination=/home/mine/Pictures/WP/
for i in "${pics[#]}"; do
((!dups["${i##*/}"]++)) &&
echo cp -vi -- "$i" "$destination"
done
GNU cp(1) has the -u flag/option which might come in handy along the way.
Remove the echo if you're satisfied with the result.
Another option is to add a trailing ( ) with a number/int inside it and increment it , e.g. ????x????.???(N) where N is a number/int. Pretty much like how some gui file manager deals with duplicate file/directory names.
Something like:
#!/usr/bin/env bash
source=/usr/share/wallpapers/
destination=/home/mine/Pictures/WP/
while IFS= read -rd '' file; do
counter=1
file_name=${file##*/}
if [[ ! -e "$destination$file_name" && ! -e "$destination$file_name($counter)" ]]; then
cp -v -- "$file" "$destination$file_name"
elif [[ -e "$destination$file_name" && ! -e "$destination$file_name($counter)" ]]; then
cp -v -- "$file" "$destination$file_name($counter)"
elif [[ -e "$destination$file_name" && -e "$destination$file_name($counter)" ]]; then
while [[ -e "$destination$file_name($counter)" ]]; do
((counter++))
done
cp -v -- "$file" "$destination$file_name($counter)"
fi
done < <(find "$source" -type f -name '????x????.???' -print0)
Note that the -print0 primary is a GNU/BSD find(1) feature.
I want duplicates of the files with different name.
I am currently trying out these commands before putting them into my bash script.
$ set dir = /somewhere/states
$ find $dir -name "total.txt" -type f | xargs ls -1
/somewhere/states/florida/fixed.fl_Asite_ttl/somewhere/total.txt
/somewhere/states/hawaii/fixed.hi_Bsite_ttl/somewhere/total.txt
/somewhere/states/kentucky/fixed.ky_Asite_ttl/somewhere/total.txt
/somewhere/states/michigan/fixed.mi_Csite_ttl/somewhere/total.txt
/somewhere/states/texas/fixed.tx_Vsite_ttl/somewhere/total.txt
I know I can rename file using something like this, but it isn't exactly what I want:
$ find $dir -name "total.txt" -exec sh -c 'cp {} `dirname {}`/`basename {} `why.xls' \;
/somewhere/states/florida/fixed.fl_Asite_ttl/somewhere/total.txtwhy.xls
/somewhere/states/hawaii/fixed.hi_Bsite_ttl/somewhere/total.txtwhy.xls
/somewhere/states/kentucky/fixed.ky_Asite_ttl/somewhere/total.txtwhy.xls
/somewhere/states/michigan/fixed.mi_Csite_ttl/somewhere/total.txtwhy.xls
/somewhere/states/texas/fixed.tx_Vsite_ttl/somewhere/total.txtwhy.xls
May I know how to copy the files and have the new files in the same dir?
below are the examples.
I want to name the new files as everything behind "fixed." and before "/somewhere" and changing the file extension as well
/somewhere/states/florida/fixed.fl_Asite_ttl/somewhere/fl_Asite_ttl.xls
/somewhere/states/hawaii/fixed.hi_Bsite_ttl/somewhere/hi_Bsite_ttl.xls
/somewhere/states/kentucky/fixed.ky_Asite_ttl/somewhere/ky_Asite_ttl.xls
/somewhere/states/michigan/fixed.mi_Csite_ttl/somewhere/mi_Csite_ttl.xls
/somewhere/states/texas/fixed.tx_Vsite_ttl/somewhere/tx_Vsite_ttl.xls
Update:
/somewhere/states/florida_fixed_ttl/fixed.fl_Asite_ttl/somewhere/total.txt
Probably not the most elegant but this should work:
find . -name total.txt | while read F ; do [[ $F =~ fixed.[^/]* ]] ; N=$(echo $BASH_REMATCH | sed s/fixed\.//) ; echo "cp $F $(dirname $F)/$N.xls" ; done
If you are happy with the output just remove the last echo, i.e. this:
echo "cp $F $(dirname $F)/$N.xls"
to this:
cp "$F" "$(dirname $F)/$N.xls"
Note, if the .txt and .xls contents will always remain the same you can use ln instead of cp -- one file, two names.
I am trying to move the directories from $DIR1 to $DIR2 if $DIR2 does not have the same directory name
if [[ ! $(ls -d /$DIR2/* | grep test) ]] is what I currently have.
then
mv $DIR1/test* /$DIR2
fi
first it gives
ls: cannot access //data/lims/PROCESSING/*: No such file or directory
when $DIR2 is empty
however, it still works.
secondly
when i run the shell script twice.
it doesn't let me move the directories with the similar name.
for example
in $DIR1 i have test-1 test-2 test-3
when it runs for the first time all three directories moves to $DIR2
after that i do mkdir test-4 at $DIR1 and run the script again..
it does not let me move the test-4 because my loop thinks that test-4 is already there since I am grabbing all test
how can I go around and move test-4 ?
Firstly, you can check whether or not a directory exists using bash's built in 'True if directory exists' expression:
test="/some/path/maybe"
if [ -d "$test" ]; then
echo "$test is a directory"
fi
However, you want to test if something is not a directory. You've shown in your code that you already know how to negate the expression:
test="/some/path/maybe"
if [ ! -d "$test" ]; then
echo "$test is NOT a directory"
fi
You also seem to be using ls to get a list of files. Perhaps you want to loop over them and do something if the files are not a directory?
dir="/some/path/maybe"
for test in $(ls $dir);
do
if [ ! -d $test ]; then
echo "$test is NOT a directory."
fi
done
A good place to look for bash stuff like this is Machtelt Garrels' guide. His page on the various expressions you can use in if statements helped me a lot.
Moving directories from a source to a destination if they don't already exist in the destination:
For the sake of readability I'm going to refer to your DIR1 and DIR2 as src and dest. First, let's declare them:
src="/place/dir1/"
dest="/place/dir2/"
Note the trailing slashes. We'll append the names of folders to these paths so the trailing slashes make that simpler. You also seem to be limiting the directories you want to move by whether or not they have the word test in their name:
filter="test"
So, let's first loop through the directories in source that pass the filter; if they don't exist in dest let's move them there:
for dir in $(ls -d $src | grep $filter); do
if [ ! -d "$dest$dir" ]; then
mv "$src$dir" "$dest"
fi
done
I hope that solves your issue. But be warned, #gniourf_gniourf posted a link in the comments that should be heeded!
If you need to mv some directories to another according to some pattern, than you can use find:
find . -type d -name "test*" -exec mv -t /tmp/target {} +
Details:
-type d - will search only for directories
-name "" - set search pattern
-exec - do something with find results
-t, --target-directory=DIRECTORY move all SOURCE arguments into DIRECTORY
There are many examples of exec or xargs usage.
And if you do not want to overwrite files, than add -n option to mv command:
find . -type d -name "test*" -exec mv -n -t /tmp/target {} +
-n, --no-clobber do not overwrite an existing file
I am trying to create a small bash script that essentially looks through a directory that includes hundreds of sub directories. in SOME of these subdirectories include a textfile.txt and a htmlfile.html where the names textfile and htmlfile are variable.
I only really care about sub directories that have both the .txt and the .html, all other subdirecories can be ignored.
I then want to list all the .html files and .txt files that are in the same sub directory
this seems like a pretty simple issue to solve but I am at a loss. all I can really get working is a line of code that outputs sub directories that have either a .html file or .txt with no association with the actual sub directory they are in, and I am pretty new at bash scripting so I can't go any further
#!/bin/bash
files="$(find ~/file/ -type f -name '*.txt' -or -name '*.html')"
for file in $files
do
echo $file
done
The following find command looks checks every subdirectory and, if it has both html and txt files, it lists all of them:
find . -type d -exec env d={} bash -c 'ls "$d"/*.html &>/dev/null && ls "$d"/*.txt &>/dev/null && ls "$d/"*.{html,txt}' \;
Explanation:
find . -type d
This looks for all subdirectories of the current directory.
-exec env d={} bash -c '...' \;
This sets the environment variable d to the value of the found subdirectory and then executes the bash command that is contained within the single quotes (see below).
ls "$d"/*.html &>/dev/null && ls "$d"/*.txt &>/dev/null && ls "$d/"*.{html,txt}
This is the bash command that is executed. It consists of three statements and-ed together. The first checks to see if directory d has any html files. If so, the second statement runs and it checks to see if there are any txt files. If so, the last statement is executed and it lists all html and txt files in the directory d.
This command is safe for all file and directory names containing spaces, tabs, or other difficult characters.
You could do it by searching recursively with the globstar option:
shopt -s globstar
for file in **; do
if [[ -d $file ]]; then
for sub_file in "$file"/*; do
case "$sub_file" in
*.html)
html=1;;
*.txt)
txt=1;;
esac
done
[[ $html && $txt ]] && echo "$file"
html=""
txt=""
fi
done
You can make use of -o
#!/bin/bash
files=$(find ~/file/ -type f -name '*.txt' -o -name '*.html')
for file in $files
do
echo $file
done
#!/bin/bash
#A quick peek into a dir to see if there's at least one file that matches pattern
dir_has_file() { dir="$1"; pattern="$2";
[ -n "$(find "$dir" -maxdepth 1 -type f -name "$pattern" -print -quit)" ]
}
#Assumes there are no newline characters in the filenames, but will behave correctly with subdirectories that match *.html or *.txt
find "$1" -type d|\
while read d
do
dir_has_file "$d" '*.txt' &&
dir_has_file "$d" '*.html' &&
#Now print all the matching files
find "$d" -maxdepth 1 -type f -name '*.txt' -o -name '*.html'
done
This script takes the root directory to look into as the first argument ($1).
The test command is what you need to check for the existence of each file in each of the subdirs:
find . -type d -exec sh -c "if test -f {}/$file1 -a -f {}/$file2 ; then ls {}/*.{txt,html} ; fi" \;
where $file1 and $file2 are the two .txt and .html files you are looking for.
I have the following directory structure:
+-archive
+-a
+-data.txt
+-b
+-data.txt
+-incoming
+-a
+-data.txt
+-c
+-data.txt
How do I do the equivalent of mv incoming/* archive/ but have the contents of the files in incoming appended to those in archive rather than overwrite them?
# move to incoming/ so that we don't
# need to strip a path prefix
cd incoming
# create directories that are missing in archive
for d in `find . -type d`; do
if [ ! -d "../archive/$d" ]; then
mkdir -p "../archive/$d"
fi
done
# concatenate all files to already existing
# ones (or automatically create them)
for f in `find . -type f`; do
cat "$f" >> "../archive/$f"
done
This should find any file in incoming and concatenate it to an existing file in archive.
The important part is to be inside incoming, because else we'd had to strip the path prefix (which is possible, but in the above case unnecessary). In the above case, a value of $f typically looks like ./a/data.txt, and hence the redirection goes to ../archive/./a/data.txt.
run it on the current directory.
find ./incoming -type f | while read -r FILE
do
dest=${FILE/incoming/archive}
cat "$FILE" >> "$dest"
done
the one in incoming/c would not be appended though
Here's a version with proper quoting:
#!/bin/sh
if [ -z "$1" ]; then
# acting as parent script
find incoming -type f -exec "$0" {} \;
else
# acting as child script
for in_file; do
if [ -f "$in_file" ]; then
destfile="${in_file/incoming/archive}"
test -d "$(dirname "$destfile")" || mkdir -p "$_"
cat "$in_file" >> "$destfile" &&
rm -f "$in_file"
fi
done
fi