in my working directory I have several folders which have hundreds of files of different types (.txt, .csv, .png...). These folders are structured as this example:
myDir/
myDir/Folder_1/ ... file_1.txt, file_2.csv, file_3.txt
myDir/Folder_2/ ...
myDir/Folder_3/ ...
...
I need to move all the .csv files to a new directory but keeping the same directory tree like this:
myDir/
myDir/Folder_1/ ...
myDir/Folder_2/ ...
...
myDir/New/
myDir/New/Folder_1/ ... file_2.csv
myDir/New/Folder_2/ ...
...
I found that cp command has --parents flag that does what I want to do, but I don't want to copy these files, I only want to move them.
I searched for this question before and found this one: Bash script for moving files and their parent directory
It is quite similar to what I need to do, but I'm not able to understand it.
Check the commands the following snippet prints out. If they do what you want, then just remove the echo (before mkdir and mv).
for path in myDir/*; do
[[ -d "$path" ]] && echo mkdir -p "myDir/New/${path#myDir/}"
done
for csv in myDir/*/*.csv; do
echo mv "$csv" "myDir/New/${csv#myDir/}"
done
Related
I have several Name*.txt files in /home/user/my/path/to/my/data/, among other files with different extensions. I would like to loop over them, then use the individual file names in the code, therefore common solutions like this won't work, since the varible '$f', within each loop, stores the whole path together with the file name. I need them separately, to perform something like the "example taks" below. My attempts:
Attempt #1:
#!/bin/bash
datapath="/home/user/my/path/to/my/data/"
outpath="/home/user/my/path/to/my/outputs/"
for f in $(ls $datapath"Name*.txt"); do
echo $f
...
cp $datapath$f $outpath"example_task"${f:0:10}
done
This didn't work:
ls: cannot access /home/user/my/path/to/my/data/Name*.txt: No such file or directory.
Although running ls /home/user/my/path/to/my/data/Name*.txt on the terminal works perfectly fine. I can't understand why.
Attempt #2:
#!/bin/bash
datapath="/home/user/my/path/to/my/data/"
outpath="/home/user/my/path/to/my/outputs/"
for f in $datapath"Name*.txt"; do
echo $f
...
cp $datapath$f $outpath"example_task"${f:0:10}
done
Here, each $f contains the full list of files ls Name*.txt would normally return, and not one at a time as one would expect.
How do I do this? Any suggestions will be much appreciated.
Maybe I am on the wrong path here but this worked for me.
#!/bin/bash
datapath="/home/user/my/path/to/my/data/"
outpath="/home/user/my/path/to/my/outputs/"
cd $datapath
for f in ./*.txt; do
file=$(echo $f | cut -d '/' -f 2)
echo $file
...
cp $f $outpath"example_task"$file
done
cd
I have many files created from a simulation.
Like this: res_00001.root through res_09999.root.
I would like to create a series of folders that move in batches of 1000 files in sequence to a newly created folder based on the filename we are moving. e.g. folder1 would contain res_00001.root through res_00999.root, folder2 res_01000.root through res_01999.root, ...
I attempted to create a script but it's not working:
#!/bin/bash
N_files=$1
for (( file=0; file<$N_files; ++file )) do #state what file I am looking at
s=file%1000 INPUT=printf data/output_%04lu.root $file` OUTPUT=printf data/folder%02lu/res_%04lu.root $s # move the files
mv INPUT OUTPUT
done`
I've been banging my head against this for sometime, I appreciate any help you can provide.
Updated Answer
You can run this little script if you can't find the rename program - make backup first!
#!/bin/bash
shopt -s nullglob nocaseglob
for f in *.root; do
n=$(tr -dc '[0-9]' <<< $f)
((d=(10#$n/1000)+1))
[ ! -d folder$d ] && mkdir folder$d
echo mv "$f" folder$d/$f
done
Original Answer
Make a backup and see if this helps you on a copy of a small subset of your files:
rename --dry-run 's/[^0-9]//g; my $d=int($_/1000)+1; $_="folder$d/res_$_.root"' *root
Sample Output
'res_00001.root' would be renamed to 'folder1/res_00001.root'
'res_00002.root' would be renamed to 'folder1/res_00002.root'
'res_00003.root' would be renamed to 'folder1/res_00003.root'
'res_00004.root' would be renamed to 'folder1/res_00004.root'
'res_00005.root' would be renamed to 'folder1/res_00005.root'
...
...
'res_00997.root' would be renamed to 'folder1/res_00997.root'
'res_00998.root' would be renamed to 'folder1/res_00998.root'
'res_00999.root' would be renamed to 'folder1/res_00999.root'
'res_01000.root' would be renamed to 'folder2/res_01000.root'
'res_01001.root' would be renamed to 'folder2/res_01001.root'
'res_01002.root' would be renamed to 'folder2/res_01002.root'
'res_01003.root' would be renamed to 'folder2/res_01003.root'
...
...
If it looks good, remove the --dry-run so it actually does stuff rather than just saying what stuff it would do!
s/[^0-9]//g gets rid of anything non-numeric in the filename
my $d=int($_/1000)+1 calculates the directory name
$_="folder$d/res_$_.root" builds the output filename
This should be a relatively straightforward problem but I haven't found any answers within stackoverflow. In a given directory, I have ~1000 files that are numbered (e.g. chem-0320.inp). I would like to cp the numbered file to a correspondingly numbered directory; all copied files will be renamed with the same name. I would like to do this for a specified numbered of files (#'s 300-500 for example).
For example, I would like to copy chem-0320.inp to a directory named 320 and rename it mech.dat.
Another example: copy chem-0430.inp to a directory named 430 and rename it mech.dat.
Thanks in advance for your help!
The following script would do the work for you
for file in *.inp
do
dir=$(echo $file | sed -r 's/[^0-9]+0([0-9]+).*/\1/g')
mv $file $dir/mech.dat
done
"cd" first to right dir. Subdirs will be created there.
#!/bin/bash
lo_limit=300
hi_limit=500
for file in ./*.inp
do
dir="${file//[^0-9]/}"
dir_cut="${dir:1:3}" # leading zero cut off
if [ $dir_cut -ge $lo_limit ] && [ $dir_cut -le $hi_limit ]; then
echo "$file $dir_cut"
mkdir -p "$dir_cut"
cp "$file" "$dir_cut"/mech.dat
fi
done
I have a directory full of pictures. Lets say
/usr/pics/foo
/usr/pics/duckface.jpg
usr/pics/bar.bmp
...
I'd like to go through and rename them to
/usr/pics/pic1
/usr/pics/pic2
/usr/pics/pic3
...
Doesn't have to be sed anything I can run from a bash script will be fine. I think I can handle the regex I just don't know what to replace with.
I assume that all files in the directory are images and you want to rename all of them without worrying of extensions or file types. Then try:
for f in /usr/pics/*; do ((i++)); mv "$f" "${f%/*}/pic${i}"; done
I have a bash shell script that unzips a zip file, and manipulates the resulting files. Because of the process, I expect all the content I am interested to be within a single folder like so:
file.zip
/file
/contentFolder1
/contentFolder2
stuff1.txt
stuff2.txt
...
I've noticed users on Windows typically don't create a sub folder but instead submit an exploding zip file that looks like:
file.zip
/contentFolder1
/contentFolder2
stuff1.txt
stuff2.txt
...
How can I detect these exploding zips, so that I may handle them accordingly? Is it possible without unzipping the file first?
If you want to check, unzip -l will print the contents of the zip file without extracting them. You'll have to massage the output a bit, though, since it's printing all sorts of additional crud.
Unzip to a directory first, and then remove the extra layer if the zip is not a bomb.
tempdir=`mktemp -d`
unzip -d $tempdir file.zip
if [ $(ls $tempdir | wc -l) = 1 ]; then
mv $tempdir/* .
rmdir $tempdir
else
mv $tempdir file
fi
I wouldn't try to detect it. I'd just force unzip to do what I want. With InfoZip:
$ unzip -j -d unzip-output-dir FileFromUntrustedSource.zip
-j makes it ignore any directory structure within the file, and -d tells it to put files in a particular directory, creating it if necessary.
If there are two files with the same name but in different subdirectories, the above command will make unzip ask if you want to overwrite the first with the second. You can add -o to force it to overwrite without asking, or -f to only overwrite if the second file is newer.