Iterate through filelist from two directions - linux

Question

I have the following bash script
#!/bin/bash
for i in `ls /file-directory/ | grep -v static-backup | grep -v fileGroup1 | grep -v fileGroup2`
do
echo $i
rsync --delete -avz --size-only --exclude "$i/stuff1" --exclude "$i/stuff2" --exclude "$i/stuff3" --exclude "$i/stuff4" --exclude "$i/stuff5" --exclude "$i/stuff6" /file-directory/$i otherServer:/file-directory/ && echo " exit code: " $?" $i" || echo " exit code: " $?" $i"
done
The script iterates through a file directory and rsyncs its subdirectories, excluding certain fileGroups and portions of those filegroup's directories. I would like this script to spawn two rsync jobs, one that starts at the top of the directory and another that starts at the bottom. They would iterate in opposite directions and meet in the middle.
This is relatively simple to do with normal counting for loops, and wouldn't bee to hard in something like python (you could just save the number of directories as a variable, then iterate using that var). How can I do something similar in bash?

Answer 1

You can make use of & in bash to fork a process. or even loop.
I am sure there are more than one way to achieve this, but here's one way.
Pseudo code.
Get listing of files in two separate arrays such that they each contain half of listing (for even).
For odd files , second array will contain one more file.
Loop through first array and fork a process using &
Loop through second array and fork second process using &
Code
#!/bin/bash
dir_len=$(ls /file-directory/ |wc -l|awk '{print $NF}')
midpoint=$(($dir_len / 2))
array1=()
array2=()
count=0
#Loop through files and divide contents into two arrays
for i in $(ls /file-directory/)
do
count=$(($count + 1))
if [ $count -le $midpoint ] ; then
array1+=($i)
elif [[ $count -gt $midpoint && $count -le $dir_len ]] ; then
array2+=($i)
fi
done
#Loop through first array and fork
for i in "${array1[#]}"
do
echo $i
#your rsync command here
done&
#Loop through second array and fork
for j in "${array2[#]}"
do
echo $j
#your rsync command here
done &
Note:
& is optional in second for loop as u can run it in foreground since there are only two processes.
Also, you may want harden the script for undesired files or if folder is empty.
Edit.
Based upon type of files u need to rsync, refer to this link for inspiration instead of using ‘ls /folder_name/‘.
https://unix.stackexchange.com/questions/9496/looping-through-files-with-spaces-in-the-names