How could I add the containing directory as a prefix to a copied file name? - linux

The issue: I have bunch of files split across multiple directories, which all have the same name (input.txt).
What I am after: I want to firstly copy all of these to a new directory, while adding the containing directory as a suffux to avoid confusion between them and prevent overwriting. This is the basis of what I am trying to do:
cp -nr /foo/bar/*/input.txt /new/path/
Where do I go from here?
To respond to the comments below, if my file structure in /old/directory contains folders:
/old/directory/1/input.txt
/old/directory/2/input.txt
/old/directory/3/input.txt
This is an example of my desired output:
/new/directory/ should contain:
1input.txt
2input.txt
3input.txt
Thanks

This will do the trick and also handle any directories that might have spaces in their names (or any other odd characters).
#!/bin/bash
tld=./old/directory
newpath=./new/directory
while IFS= read -r -d $'\0' file; do
tmp="${file#*${tld}/}"
echo cp "$file" "$newpath/${tmp//\//}"
done < <(find "$tld" -type f -name "input.txt" -print0)
Proof of Concept
$ tree ./old/directory/
./old/directory/
├── 1
│   └── input.txt
├── 2
│   └── input.txt
└── 3
├── 3a
│   └── input.txt
└── input.txt
4 directories, 4 files
$ ./mvinput.sh
cp ./old/directory/3/input.txt ./new/directory/3input.txt
cp ./old/directory/3/3a/input.txt ./new/directory/33ainput.txt
cp ./old/directory/1/input.txt ./new/directory/1input.txt
cp ./old/directory/2/input.txt ./new/directory/2input.txt

Well, the tough news is that there's not an obvious manner of doing this in one line - not one that isn't nonsensically difficult to understand anyway. There may be a way to do it with rsync, and I'm sure that someone smarter than I could do it in awk, but in my opinion, you're better off making a script, or even writing a custom binary that does this for you.
find . -name input.txt | while read line
do
cp "$line" /new/path/`echo $line | cut -d '/' -f3- | sed 's/\//_/'`
done
Note that you'll probably have to change the -f3- part of the cut command in order to select which directory name you want to start your suffix as.

One approach is to use an array to save the files, also since / is not allowed on file names an alternative is to change it to something else, like say an underscore.
#!/usr/bin/env bash
##: Just in case there are no files/directories the * glob will not expand by itself.
shopt -s nullglob
files=(foo/bar/*/input.txt)
for file in "${files[#]}"; do
new_file=${file//\//_} ##: Replace all /'s with an _ by Parameter Expansion
echo cp -v "$file" new/path/"$new_file"
done
As per OP's request here is the new answer.
#!/usr/bin/env bash
shopt -s nullglob
##: Although files=(/old/directory/{1..3}/input.txt)
##: can be a replacement, no need for nullglob
files=(/old/directory/*/input.txt)
for file in "${files[#]}"; do
tmp=${file%[0-9]*}
new_file=${file#*$tmp}
echo cp -v "$file" new/path/"${new_file//\//}"
done
Another option is to split the fields using / as the delimiter.
#!/usr/bin/env bash
##: Do not expand a literal glob * if there are no files/directories
shopt -s nullglob
##: Save the whole path and files in an array.
files=(/old/directory/*/input.txt)
for file in "${files[#]}"; do
IFS='/' read -ra path <<< "$file" ##: split via / and save in an array
tmp=${path[#]:(-2)} ##: Remain only the last 2, e.g. 1 input.txt
new_file=${tmp// } ##: Remove the space, becomes 1input.txt
echo cp -v "$file" new/path/"$new_file"
done
Remove the echo if you think that the output is correct.
It is easy to understand which directory the file came from just replace the under scores with a /

Related

Moving files to subfolders based on prefix in bash

I currently have a long list of files, which look somewhat like this:
Gmc_W_GCtl_E_Erz_Aue_Dl_281_heart_xerton
Gmc_W_GCtl_E_Erz_Aue_Dl_254_toe_taixwon
Gmc_W_GCtl_E_Erz_Homersdorf_Dl_201_head_xaubadan
Gmc_W_GCtl_E_Erz_Homersdorf_Dl_262_bone_bainan
Gmc_W_GCtl_E_Thur_Peuschen_Dl_261_blood_blodan
Gmc_W_GCtl_E_Thur_Peuschen_Dl_281_heart_xerton
The naming pattern all follow the same order, where I'm mainly seeking to group the files based on the part with "Aue", "Homersdorf", "Peuschen", and so forth (there are many others down the list), with the position of these keywords being always the same (e.g. they are all followed by Dl; they are all after the fifth underscore...etc.).
All the files are in the same folder, and I am trying to move these files into subfolders based on these keywords in bash, but I'm not quite certain how. Any help on this would be appreciated, thanks!
I am guessing you want something like this:
$ find . -type f | awk -F_ '{system("mkdir -p "$5"/"$6";mv "$0" "$5"/"$6)}'
This will move say Gmc_W_GCtl_E_Erz_Aue_Dl_281_heart_xerton into /Erz/Aue/Gmc_W_GCtl_E_Erz_Aue_Dl_281_heart_xerton.
Using the bash shell with a for loop.
#!/usr/bin/env bash
shopt -s nullglob
for file in Gmc*; do
[[ -d $file ]] && continue
IFS=_ read -ra dir <<< "$file"
echo mkdir -pv "${dir[4]}/${dir[5]}" || exit
echo mv -v "$file" "${dir[4]}/${dir[5]}" || exit
done
Place the script inside the directory in question make it executable and execute it.
Remove the echo's so it create the directories and move the files.

File renaming Linux

I have tried to rename several files on my Linux system. I usedrename 's/foo/bar/g' * All the files that I wish to change are in the current directory.
It does not change the name of the files but I think it should. Any help would be appreciated.
An easy way would to do:
mv file2rename newname
You have mentioned that you want to rename multiple files at once using rename expression. Technically you can't use only * sign for change file names. * means all files with same name. We know same file types doesn't exist with same name but you can rename some selected part from file. For an example
admin#home:~/works$ ls test*.c
test_car.c test_van.c test_dog.c
you can rename some part of these files not full name. because there cannot be exist same file name with same extention
admin#home:~/works$ rename 's/test/practice/' *.c
After executing this command every test replace with practice.
admin#home:~/works$ ls practice*.c
practice_car.c practice_van.c practice_dog.c
Rename a file mv
mv old_name new_name
The use of the mv command changes the name of the file from old_name to new_name.
Another way to rename file extentions in the current directory, for instance renaming all .txt files in .csv:
for file in $(ls .); do
mv $file ${file/.txt/.csv}
done
This will not affect files that don't have the .txt extention and it will prompt an error (should be developed further depending on your needs).
some posts points out the usage of for x in $(something); do..
please - Don't (ever, under any circumstances) use that! (see below)
Say you have a file(and, other .txt files):
"my file with a very long file - name-.txt"
and you do
for f in $(ls *.txt); do echo $f; done
(or something like that) it will output
.
..
a.sh
docs
my
file
with
a
very
long
file
-
name-.txt
(or something similar)
Instead, try the following:
#! /bin/sh
if [ $# -eq 0 ]; then
echo -n "example usage: bash $0 .txt .csv <DIR>"
echo -n "(renames all files(ending with .txt to .csv) in DIR"
exit
fi
A="$1" # OLD PREFIX (e.g .txt )
B="$2" # NEW PREFIX (e.g .csv )
DIR="$3*$A" # DIR (e.g ./ )
# for file f;
# in path $DIR;
for f in $DIR; do
## do the following:
# || here just means:
# only continue IFF(if and only if)
# the previous is-file-check's exit-status returns non-zero
[ -e "$f" ] || continue
# move file "$f" and rename it's ending $A with $B (e.g ".txt" to ".csv")
# (still, in $DIR)
mv "$f" "${f/$A/$B}"
done
### $ tree docs/
# docs/
# ├── a.txt
# ├── b.txt
# ├── c.txt
# └── d.txt
#
# 0 directories, 4 files
#
### $ bash try3.sh .txt .csv docs/
# $ tree docs/
# docs/
# ├── a.csv
# ├── b.csv
# ├── c.csv
# └── d.csv
#
# 0 directories, 4 files
##
#-------------------#
References:
Bash Pitfalls
DontReadLinesWithFor
Quotes
Bash FAQ
MAN's: ($ man "- the following")
- bash
- mv
- ls
Note: I do not mean to be offensive - so please don't take it as offense (I got the main command-idea from meniluca actually!
But since it was (for x in $(ls ..)) I decided to create a whole script, rather than just edit.

How to remove the first 2 letters of multiple file names in linux shell?

I have files with the names:
Ff6_01.png
Ff6_02.png
Ff6_03.png
...
...
FF1_01.png
FF1_02.png
FF1_03.png
I want to remove the first two letters of every file name, because then I would have a correct order of the files. Does anyone know the command in the linux shell?
You can use the syntax ${file:2} to refer to the name starting from the 3rd char.
Hence, you may do:
for file in F*png
do
mv "$file" "${file:2}"
done
In case ${file:2} did not work to you (neither rename), you can also use sed or cut:
for file in F*png
do
new_file=$(sed 's/^..//' <<< "$file") <---- cuts first two chars
new_file=$(cut -c3- <<< "$file") <---- the same
mv "$file" "$new_file"
done
Test
$ file="Ff6_01.png"
$ touch $file
$ ls
Ff6_01.png
$ mv "$file" "${file:2}"
$ ls
6_01.png

Script for renaming files with logical

Someone has very kindly help get me started on a mass rename script for renaming PDF files.
As you can see I need to add a bit of logical to stop the below happening - so something like add a unique number to a duplicate file name?
rename 's/^(.{5}).*(\..*)$/$1$2/' *
rename -n 's/^(.{5}).*(\..*)$/$1$2/' *
Annexes 123114345234525.pdf renamed as Annex.pdf
Annexes 123114432452352.pdf renamed as Annex.pdf
Hope this makes sense?
Thanks
for i in *
do
x='' # counter
j="${i:0:2}" # new name
e="${i##*.}" # ext
while [ -e "$j$x" ] # try to find other name
do
((x++)) # inc counter
done
mv "$i" "$j$x" # rename
done
before
$ ls
he.pdf hejjj.pdf hello.pdf wo.pdf workd.pdf world.pdf
after
$ ls
he.pdf he1.pdf he2.pdf wo.pdf wo1.pdf wo2.pdf
This should check whether there will be any duplicates:
rename -n [...] | grep -o ' renamed as .*' | sort | uniq -d
If you get any output of the form renamed as [...], then you have a collision.
Of course, this won't work in a couple corner cases - If your files contain newlines or the literal string renamed as, for example.
As noted in my answer on your previous question:
for f in *.pdf; do
tmp=`echo $f | sed -r 's/^(.{5}).*(\..*)$/$1$2/'`
mv -b ./"$f" ./"$tmp"
done
That will make backups of deleted or overwritten files. A better alternative would be this script:
#!/bin/bash
for f in $*; do
tar -rvf /tmp/backup.tar $f
tmp=`echo $f | sed -r 's/^(.{5}).*(\..*)$/$1$2/'`
i=1
while [ -e tmp ]; do
tmp=`echo $tmp | sed "s/\./-$i/"`
i+=1
done
mv -b ./"$f" ./"$tmp"
done
Run the script like this:
find . -exec thescript '{}' \;
The find command gives you lots of options for specifing which files to run on, works recursively, and passes all the filenames in to the script. The script backs all file up with tar (uncompressed) and then renames them.
This isn't the best script, since it isn't smart enough to avoid the manual loop and check for identical file names.

Unix: How to delete files listed in a file

I have a long text file with list of file masks I want to delete
Example:
/tmp/aaa.jpg
/var/www1/*
/var/www/qwerty.php
I need delete them. Tried rm `cat 1.txt` and it says the list is too long.
Found this command, but when I check folders from the list, some of them still have files
xargs rm <1.txt Manual rm call removes files from such folders, so no issue with permissions.
This is not very efficient, but will work if you need glob patterns (as in /var/www/*)
for f in $(cat 1.txt) ; do
rm "$f"
done
If you don't have any patterns and are sure your paths in the file do not contain whitespaces or other weird things, you can use xargs like so:
xargs rm < 1.txt
Assuming that the list of files is in the file 1.txt, then do:
xargs rm -r <1.txt
The -r option causes recursion into any directories named in 1.txt.
If any files are read-only, use the -f option to force the deletion:
xargs rm -rf <1.txt
Be cautious with input to any tool that does programmatic deletions. Make certain that the files named in the input file are really to be deleted. Be especially careful about seemingly simple typos. For example, if you enter a space between a file and its suffix, it will appear to be two separate file names:
file .txt
is actually two separate files: file and .txt.
This may not seem so dangerous, but if the typo is something like this:
myoldfiles *
Then instead of deleting all files that begin with myoldfiles, you'll end up deleting myoldfiles and all non-dot-files and directories in the current directory. Probably not what you wanted.
Use this:
while IFS= read -r file ; do rm -- "$file" ; done < delete.list
If you need glob expansion you can omit quoting $file:
IFS=""
while read -r file ; do rm -- $file ; done < delete.list
But be warned that file names can contain "problematic" content and I would use the unquoted version. Imagine this pattern in the file
*
*/*
*/*/*
This would delete quite a lot from the current directory! I would encourage you to prepare the delete list in a way that glob patterns aren't required anymore, and then use quoting like in my first example.
You could use '\n' for define the new line character as delimiter.
xargs -d '\n' rm < 1.txt
Be careful with the -rf because it can delete what you don't want to if the 1.txt contains paths with spaces. That's why the new line delimiter a bit safer.
On BSD systems, you could use -0 option to use new line characters as delimiter like this:
xargs -0 rm < 1.txt
xargs -I{} sh -c 'rm "{}"' < 1.txt should do what you want. Be careful with this command as one incorrect entry in that file could cause a lot of trouble.
This answer was edited after #tdavies pointed out that the original did not do shell expansion.
You can use this one-liner:
cat 1.txt | xargs echo rm | sh
Which does shell expansion but executes rm the minimum number of times.
Just to provide an another way, you can also simply use the following command
$ cat to_remove
/tmp/file1
/tmp/file2
/tmp/file3
$ rm $( cat to_remove )
In this particular case, due to the dangers cited in other answers, I would
Edit in e.g. Vim and :%s/\s/\\\0/g, escaping all space characters with a backslash.
Then :%s/^/rm -rf /, prepending the command. With -r you don't have to worry to have directories listed after the files contained therein, and with -f it won't complain due to missing files or duplicate entries.
Run all the commands: $ source 1.txt
cat 1.txt | xargs rm -f | bash Run the command will do the following for files only.
cat 1.txt | xargs rm -rf | bash Run the command will do the following recursive behaviour.
Here's another looping example. This one also contains an 'if-statement' as an example of checking to see if the entry is a 'file' (or a 'directory' for example):
for f in $(cat 1.txt); do if [ -f $f ]; then rm $f; fi; done
Here you can use set of folders from deletelist.txt while avoiding some patterns as well
foreach f (cat deletelist.txt)
rm -rf ls | egrep -v "needthisfile|*.cpp|*.h"
end
This will allow file names to have spaces (reproducible example).
# Select files of interest, here, only text files for ex.
find -type f -exec file {} \; > findresult.txt
grep ": ASCII text$" findresult.txt > textfiles.txt
# leave only the path to the file removing suffix and prefix
sed -i -e 's/:.*$//' textfiles.txt
sed -i -e 's/\.\///' textfiles.txt
#write a script that deletes the files in textfiles.txt
IFS_backup=$IFS
IFS=$(echo "\n\b")
for f in $(cat textfiles.txt);
do
rm "$f";
done
IFS=$IFS_backup
# save script as "some.sh" and run: sh some.sh
In case somebody prefers sed and removing without wildcard expansion:
sed -e "s/^\(.*\)$/rm -f -- \'\1\'/" deletelist.txt | /bin/sh
Reminder: use absolute pathnames in the file or make sure you are in the right directory.
And for completeness the same with awk:
awk '{printf "rm -f -- '\''%s'\''\n",$1}' deletelist.txt | /bin/sh
Wildcard expansion will work if the single quotes are remove, but this is dangerous in case the filename contains spaces. This would need to add quotes around the wildcards.

Resources