Change folder structure and make files available by using links - linux

Question

(Please see also the minimal example at bottom)
I have the following folder structure:
https://drive.google.com/open?id=1an6x1IRtNjOG6d9D5FlYwsUxmgaEegUY
Each folder shows at the end: the date, month and day:
e.g.
/HRIT/EPI/2004/01/14
Inside for each day, each of this sub-folders contains a lot of files (in total about 10 TB).
All available folders are:
/HRIT/EPI/YYYY/MM/DD/
/HRIT/HRV/YYYY/MM/DD/
/HRIT/VIS006/YYYY/MM/DD/
/HRIT/VIS008/YYYY/MM/DD/
/HRIT/WV_062/YYYY/MM/DD/
/HRIT/WV_073/YYYY/MM/DD/
/HRIT/IR_016/YYYY/MM/DD/
/HRIT/IR_039/YYYY/MM/DD/
/HRIT/IR_087/YYYY/MM/DD/
/HRIT/IR_097/YYYY/MM/DD/
/HRIT/IR_108/YYYY/MM/DD/
/HRIT/IR_120/YYYY/MM/DD/
/HRIT/IR_134/YYYY/MM/DD/
/HRIT/PRO/YYYY/MM/DD/
I would like to change the folder structure to:
/HRIT/YYYY/MM/DD/
All files from EPI, HRV, VIS006, VIS008, WV_062, WV_073, IR_016, IR_039, IR_087,IR_097, IR_108, IR_124, IR_134, PRO should not be copied physically but should be accessible by links in my home folder /home
-> /home/HRIT/YYYY/MM/DD/
Is something like that possible and how?
Here is an minimal example what I in principle have and what I wish as result:
I have:
/HRIT/EPI/2019/01/01/epi_1.txt
/HRIT/EPI/2019/01/01/epi_2.txt
/HRIT/HRV/2019/01/01/hrv_1.txt
/HRIT/HRV/2019/01/01/hrv_2.txt
/HRIT/VIS006/2019/01/01/vis006_1.txt
/HRIT/VIS006/2019/01/01/vis006_2.txt
As result I wish:
/home/HRIT/2019/01/01/epi_1.txt
/home/HRIT/2019/01/01/epi_2.txt
/home/HRIT/2019/01/01/hrv_1.txt
/home/HRIT/2019/01/01/hrv_2.txt
/home/HRIT/2019/01/01/vis006_1.txt
/home/HRIT/2019/01/01/vis006_2.txt
As mentioned above the files should not be copied to this new folder structure, but instead made accessible by links (because in reality I have too many files and not enough space to copy them).
!!! This is extremely simplified, since I have different years, month's and days (please see the link above).

Answer 1

Here's a small script to do this. I've made it just echo by default, you'd need to run it with --real to make it do the linking
This assumes you're running the script from within the /HRIT/ dir, and assumes that the date, then filename are the last part of the hierarchy. If there are further dirs below, this might not work for you.
DRYRUN=true
LINK="ln -s" # You can make this "ln" if you prefer hardlinks
if [[ $1 == '--real' ]]; then
DRYRUN=false
fi
for file in $(find "$PWD" -type f); do
dest=$( \
echo "$file" | awk -F/ '{ year = NF-3; mon=NF-2; day=NF-1; print "/home/HRIT/"$year"/"$mon"/"$day"/"$NF }' \
)
if [[ -f "$dest" ]]; then
echo "Skipping '$file' as destination link already exists at '$dest'"
else
if $DRYRUN; then
echo $LINK "$file" "$dest"
else
$LINK "$file" "$dest"
fi
fi
done

Answer 2

The simplest while read loop:
generate_list_of_files_for_example_with_find |
while IFS= read -r line; do
IFS='/' read -r _ hrit _ rest <<<"$line"
echo mkdir -p /home/"$hrit"/"$(dirname "$rest")"
echo ln -s /home/"$hrit/$rest" "$line"
done
Remove echos to really create links and directories.