Rsync - split source as multiple sets - linux

I had the source directory :
root/sourceDir1/sourceDir11
root/sourceDir1/sourceDir12
root/sourceDir1/sourceDir13
root/sourceDir1/sourceDir14
root/sourceDir2/SourceDir21
root/sourceDir2/SourceDir22
root/sourceDir2/SourceDir23
root/sourceDir2/SourceDir24
root/sourceDir2/SourceDir25
root/sourceDir2/SourceDir26
How can I use rsync command to sync these directories as specified in below:
rootTarget1/sourceDir11
rootTarget1/sourceDir12
rootTarget1/sourceDir13
rootTarget1/sourceDir14
rootTarget2/sourceDir21
rootTarget2/sourceDir22
rootTarget2/sourceDir23
rootTarget2/sourceDir24
rootTarget2/sourceDir25
rootTarget2/sourceDir26
Basically I would like to split the source directories into 2 sets, one set of dirs should sync to rootTarget1 and another set should sync to rootTarget2

try this script:
#!/bin/bash
for ((x=1;x<=2;x++));
do if [ x = 1 ] ;
then for ((y=1;y<=4;y++));
do rsync root/sourceDir${x}/SourceDir${x}${y} rootTarget${x}/Sourcedir${x}${y}" ;
done ;
else for ((y=1;y<=6;y++)) ;
do rsync root/sourceDir${x}/SourceDir${x}${y} rootTarget${x}/SourceDir${x}${y}" ;
done ;
fi;
done

Related

linux command rename dates (YYYY.MMDD) to numbers(001,002,...,066,067) sequentially

I have renamed many files by using 'rename'.
However, I find a problem with conversion dates to numbers.
The file name is 2021.0801, 2021.0802, .. etc. (Year.Month&date)
I need to change Month&date parts to numbers of 001, 002, etc.
So I need to rename
2021.0801
2021.0802
...
2021.0929
2021.0930
to
2021.001
2021.002
...
2021.0**
2021.0**
I saw I can do it when I use rename or #, ? but I could not see the specific way to solve this.
Could you please let me know the way to rename these?
p.s. I tried num=001; for i in {0801..0930}; do rename $i $num *; (($num++)); done but it showed
2021.001
2021.001001
2021.001001001001
...
Additionally, ls 2021.* shows only the files that I want to change.
Your script contains a few errors. I suggest to use https://www.shellcheck.net/ to check your scripts.
After fixing the errors, the (unfinished) result is
#!/bin/bash
num=001; for i in {0801..0930}; do rename "$i" "$num" ./*; ((num++)); done
This has 3 remaining problems.
The range {0801..0930} includes the numbers 0831, 0832, ... 0899, 0900. This can be fixed by using two ranges {0801..0831} {0901..0930}
The increment operation ((num++)) does not preserve the leading zeros. This can be fixed by conditionally adding 1 or 2 zeros.
You call rename for every combination which will check all files and probably rename only one. As you know the exact file names you can replace this with a mv command.
The final version is
num=1; for i in {0801..0831} {0901..0930}; do
if [[ num -lt 10 ]] ; then
new="00$num";
elif [[ num -lt 100 ]] ; then
new="0$num";
else
new="$num"; # this case is not necessary here
fi;
mv "2021.$i" "2021.$new";
((num++));
done
The script handles leading zeros for values of 1, 2 or 3 digits which is not needed here as all numbers are less than 100. For this case, it can be simplified as
num=1; for i in {0801..0831} {0901..0930}; do
if [[ num -lt 10 ]] ; then
new="00$num";
else
new="0$num";
fi;
mv "2021.$i" "2021.$new";
((num++));
done
The script will execute these commands:
mv 2021.0801 2021.001
mv 2021.0802 2021.002
...
mv 2021.0830 2021.030
mv 2021.0831 2021.031
mv 2021.0901 2021.032
mv 2021.0902 2021.033
...
mv 2021.0929 2021.060
mv 2021.0930 2021.061
You don't need the for loop. This single command will do it all :
rename -n 'BEGIN{our $num=1}{our $num;s/\d+$/sprintf("%03d", $num)/e; $num += 1}' 2021.*
Remove -n once the resulting renaming looks good.

Concatenate a string with an array for recursively copy file in bash

I have a concatenation problem between a string and an array
I want to copy all the files contained in the directories stored in the array, my command is in a loop (to recursively copy my files)
yes | cp -rf "./$WORK_DIR/${array[$i]}/"* $DEST_DIR
My array :
array=("My folder" "...")
I have in my array several folder names (they have spaces in their names) that I would like append to my $WORK_DIR to make it possible to copy the files for cp.
But I always have the following error
cp: impossible to evaluate './WORKDIR/my': No such files or folders
cp: impossible to evaluate 'folder/*': No such files or folders
This worked for me
#!/bin/bash
arr=("My folder" "This is a test")
i=0
while [[ ${i} -lt ${#arr[#]} ]]; do
echo ${arr[${i}]}
cp -rfv ./source/"${arr[${i}]}"/* ./dest/.
(( i++ ))
done
exit 0
I ran the script. It gave me the following output:
My folder
'./source/My folder/blah-folder' -> './dest/./blah-folder'
'./source/My folder/foo-folder' -> './dest/./foo-folder'
This is a test
'./source/This is a test/blah-this' -> './dest/./blah-this'
'./source/This is a test/foo-this' -> './dest/./foo-this'
Not sure of the exact difference, but hopefully this will help.

bash: How to transfer/copy only the file names to separate similar files?

I've some files in a folder A which are named like that:
001_file.xyz
002_file.xyz
003_file.xyz
in a separate folder B I've files like this:
001_FILE_somerandomtext.zyx
002_FILE_somerandomtext.zyx
003_FILE_somerandomtext.zyx
Now I want to rename, if possible, with just a command line in the bash all the files in folder B with the file names in folder A. The file extension must stay different.
There is exactly the same amount of files in each folder A and B and they both have the same order due to numbering.
I'm a total noob, but I hope some easy answer for the problem will show up.
Thanks in advance!
ZVLKX
*Example edited for clarification
An implementation might look a bit like this:
renameFromDir() {
useNamesFromDir=$1
forFilesFromDir=$2
for f in "$forFilesFromDir"/*; do
# Put original extension in $f_ext
f_ext=${f##*.}
# Put number in $f_num
f_num=${f##*/}; f_num=${f_num%%_*}
# look for a file in directory B with same number
set -- "$useNamesFromDir"/"${f_num}"_*.*
[[ $1 && -e $1 ]] || {
echo "Could not find file number $f_num in $dirB" >&2
continue
}
(( $# > 1 )) && {
# there's more than one file with the same number; write an error
echo "Found more than one file with number $f_num in $dirB" >&2
printf ' - %q\n' "$#" >&2
continue
}
# extract the parts of our destination filename we want to keep
destName=${1##*/} # remove everything up to the last /
destName=${destName%.*} # and past the last .
# write the command we would run to stdout
printf '%q ' mv "$f" "$forFilesFromDir/$destName.$f_ext"; printf '\n'
## or uncomment this to actually run the command
# mv "$f" "$forFilesFromDir/$destName.$f_ext"
done
}
Now, how would we test this?
mkdir -p A B
touch A/00{1,2,3}_file.xyz B/00{1,2,3}_FILE_somerandomtext.zyx
renameFromDir A B
Given that, the output is:
mv B/001_FILE_somerandomtext.zyx B/001_file.zyx
mv B/002_FILE_somerandomtext.zyx B/002_file.zyx
mv B/003_FILE_somerandomtext.zyx B/003_file.zyx
Sorry if this isn't helpful, but I had fun writing it.
This renames items in folder B to the names in folder A, preserving the extension of B.
A_DIR="./A"
A_FILE_EXT=".xyz"
B_DIR="./B"
B_FILE_EXT=".zyx"
FILES_IN_A=`find $A_DIR -type f -name *$A_FILE_EXT`
FILES_IN_B=`find $B_DIR -type f -name *$B_FILE_EXT`
for A_FILE in $FILES_IN_A
do
A_BASE_FILE=`basename $A_FILE`
A_FILE_NUMBER=(${A_BASE_FILE//_/ })
A_FILE_WITHOUT_EXTENSION=(${A_BASE_FILE//./ })
for B_FILE in $FILES_IN_B
do
B_BASE_FILE=`basename $B_FILE`
B_FILE_NUMBER=(${B_BASE_FILE//_/ })
if [ ${A_FILE_NUMBER[0]} == ${B_FILE_NUMBER[0]} ]; then
mv $B_FILE $B_DIR/$A_FILE_WITHOUT_EXTENSION$B_FILE_EXT
break
fi
done
done

Why does script not recognize file extension?

My script
#!/bin/bash
cp *.ats /home/milenko/procmt
mycd() {
cd /home/milenko/procmt
}
mycd
EXT=ats
for i in *; do
if [ "${i}" != "${i%.${EXT}}" ];then
./tsmp -ascii i
fi
done
But
milenko#milenko-HP-Compaq-6830s:~/Serra do Mel/MT06/meas_2016-07-13_20-22-00$ bash k1.sh
./tsmp: handling 1 files ************************************** total input channels: 1
the name of your file does not end with ats ... might crash soon
main (no rda) -> can not open i for input, exit
./tsmp: handling 1 files ************************************** total input channels: 1
the name of your file does not end with ats ... might crash soon
main (no rda) -> can not open i for input, exit
When I go to procmt directory and list files
milenko#milenko-HP-Compaq-6830s:~/procmt$ ls *.ats
262_V01_C00_R000_TEx_BL_2048H.ats 262_V01_C00_R086_TEx_BL_4096H.ats 262_V01_C02_R000_THx_BL_2048H.ats
262_V01_C00_R000_TEx_BL_4096H.ats 262_V01_C01_R000_TEy_BL_2048H.ats 262_V01_C03_R000_THy_BL_2048H.ats
What is wrong with my script?
If I understand correctly this should work for you:
dest='/home/milenko/procmt'
cp *.ats "$dest"
cd "$dest"
for i in *.ats; do
./tsmp -ascii "$i"
done
There is no need to loop through all files when you're only interested in .ats files. Your mycd function is just doing cd so you can avoid that as well.

How to rename multiple files in terminal (LINUX)?

I have bunch of files with no pattern in their name at all in a directory. all I know is that they are all Jpg files. How do I rename them, so that they will have some sort of sequence in their name.
I know in Windows all you do is select all the files and rename them all to a same name and Windows OS automatically adds sequence numbers to compensate for the same file name.
I want to be able to do that in Linux Fedora but I you can only do that in Terminal. Please, help. I am lost.
What is the command for doing this?
The best way to do this is to run a loop in the terminal going from picture to picture and renaming them with a number that gets bigger by one with every loop.
You can do this with:
n=1
for i in *.jpg; do
p=$(printf "%04d.jpg" ${n})
mv ${i} ${p}
let n=n+1
done
Just enter it into the terminal line by line.
If you want to put a custom name in front of the numbers, you can put it before the percent sign in the third line.
If you want to change the number of digits in the names' number, just replace the '4' in the third line (don't change the '0', though).
I will assume that:
There are no spaces or other weird control characters in the file names
All of the files in a given directory are jpeg files
That in mind, to rename all of the files to 1.jpg, 2.jpg, and so on:
N=1
for a in ./* ; do
mv $a ${N}.jpg
N=$(( $N + 1 ))
done
If there are spaces in the file names:
find . -type f | awk 'BEGIN{N=1}
{print "mv \"" $0 "\" " N ".jpg"
N++}' | sh
Should be able to rename them.
The point being, Linux/UNIX does have a lot of tools which can automate a task like this, but they have a bit of a learning curve to them
Create a script containing:
#!/bin/sh
filePrefix="$1"
sequence=1
for file in $(ls -tr *.jpg) ; do
renamedFile="$filePrefix$sequence.jpg"
echo $renamedFile
currentFile="$(echo $file)"
echo "renaming \"$currentFile\" to $renamedFile"
mv "$currentFile" "$renamedFile"
sequence=$(($sequence+1))
done
exit 0
If you named the script, say, RenameSequentially then you could issue the command:
./RenameSequentially Images-
This would rename all *.jpg files in the directory to Image-1.jpg, Image-2.jpg, etc... in order of oldest to newest... tested in OS X command shell.
I wrote a perl script a long time ago to do pretty much what you want:
#
# reseq.pl renames files to a new named sequence of filesnames
#
# Usage: reseq.pl newname [-n seq] [-p pad] fileglob
#
use strict;
my $newname = $ARGV[0];
my $seqstr = "01";
my $seq = 1;
my $pad = 2;
shift #ARGV;
if ($ARGV[0] eq "-n") {
$seqstr = $ARGV[1];
$seq = int $seqstr;
shift #ARGV;
shift #ARGV;
}
if ($ARGV[0] eq "-p") {
$pad = $ARGV[1];
shift #ARGV;
shift #ARGV;
}
my $filename;
my $suffix;
for (#ARGV) {
$filename = sprintf("${newname}_%0${pad}d", $seq);
if (($suffix) = m/.*\.(.*)/) {
$filename = "$filename.$suffix";
}
print "$_ -> $filename\n";
rename ($_, $filename);
$seq++;
}
You specify a common prefix for the files, a beginning sequence number and a padding factor.
For exmaple:
# reseq.pl abc 1 2 *.jpg
Will rename all matching files to abc_01.jpg, abc_02.jpg, abc_03.jpg...

Resources