Hello from a Linux Bash newbie!
I have a list.txt containing a list of files which I want to copy to a destination($2). These are unique images but some of them have the same filename.
My plan is to loop through each line in the text file, with the copy to the destination occurring when the file is not there, and a mv rename happening when it is present.
The problem I am having is creating the pathname to check the file against. In the code below, I am taking the filename only from the pathname, and I want to add that to the destination ($2) with the "/" in between to check the file against.
When I run the program below I get "Permission Denied" at line 9 which is where I try and create the path.
for line in $(cat list.txt)
do
file=$[ basename $line ]
path=$[ $2$file ]
echo $path
if [ ! -f $path ];
then
echo cp $line $2
else
echo mv $line.DUPLICATE $2
fi
done
I am new to this so appreciate I may be missing something obvious but if anyone can offer any advice it would be much appreciated!
Submitting this since OP is new in BASH scripting no good answer has been posted yet.
DESTINATION="$2"
while read -r line; do
file="${line##*/}"
path="$2/$file"
[[ ! -f $path ]] && cp "$line" "$path" || mv "$line" "$path.DUP"
done < list.txt
Don't have logic for counting duplicates at present to keep things simple. (Which means code will take care of one dup entry) As an alternative you get uniq from list.txt beforehand to avoid the duplicate situation.
#anubhava: Your script looks good. Here is a small addition to it to work with several dupes.
It adds a numer to the $path.DUP name
UniqueMove()
{
COUNT=0
while [ -f "$1" ]
do
(( COUNT++ ))
mv -n "$1" "$2$COUNT"
done
}
while read -r line; do
file="${line##*/}"
path="$2/$file"
[[ ! -f $path ]] && cp "$line" "$path" || UniqueMove "$line" "$path.DUP"
done < list.txt
Related
sorry for that odd title. I didn't know how to word it the right way.
I'm trying to write a script to filter my wiki files to those got directories with the same name and the ones without. I'll elaborate further.
here is my file system:
what I need to do is print a list of those files which have directories in their name and another one of those without.
So my ultimate goal is getting:
with dirs:
Docs
Eng
Python
RHEL
To_do_list
articals
without dirs:
orphan.txt
orphan2.txt
orphan3.txt
I managed to get those files with dirs. Here is me code:
getname () {
file=$( basename "$1" )
file2=${file%%.*}
echo $file2
}
for d in Mywiki/* ; do
if [[ -f $d ]]; then
file=$(getname $d)
for x in Mywiki/* ; do
dir=$(getname $x)
if [[ -d $x ]] && [ $dir == $file ]; then
echo $dir
fi
done
fi
done
but stuck with getting those without. if this is the wrong way of doing this please clarify the right one.
any help appreciated. Thanks.
Here's a quick attempt.
for file in Mywiki/*.txt; do
nodir=${file##*/}
test -d "${file%.txt}" && printf "%s\n" "$nodir" >&3 || printf "%s\n" "$nodir"
done >with 3>without
This shamelessly uses standard output for the non-orphans. Maybe more robustly open another separate file descriptor for that.
Also notice how everything needs to be quoted unless you specifically require the shell to do whitespace tokenization and wildcard expansion on the value of a token. Here's the scoop on that.
That may not be the most efficient way of doing it, but you could take all files, remove the extension, and the check if there isn't a directory with that name.
Like this (untested code):
for file in Mywiki/* ; do
if [ -f "$d" ]; then
dirname=$(getname "$d")
if [ ! -d "Mywiki/$dirname" ]; then
echo "$file"
fi
fi
done
To List all the files in current dir
list1=`ls -p | grep -v /`
To List all the files in current dir without extension
list2=`ls -p | grep -v / | sed 's/\.[a-z]*//g'`
To List all the directories in current dir
list3=`ls -d */ | sed -e "s/\///g"`
Now you can get the desired directory listing using intersection of list2 and list3. Intersection of two lists in Bash
In my script, I am asking the user to input a directory and then list all the files in that specific directory. What I want to do with that is to make the display a little better in which I would be able to display a "/" if the item in the directory is another directory and if it is an executable file (not an executable directory), print with a **".
This is what I have:
echo “Directory: “
read thing
for var123 in $thing*
do
echo $var123
done
In a directory I have a few folders and a few scripts that have the execute permission. when I run the script I want to say
/folder1/subfolder1/
/folder1/subfolder2/
/folder1/file1*
/folder1/file2*
I am new to this and have no clue what I am doing. Any help will be greatly appreciated.
You might want to check and make sure the user inputs something that ends in a / first.
e.g.
[[ $thing =~ '/'$ ]] || thing="$thing/"
Also check if it exists
e.g.
[[ -d $thing ]] || exit 1
Then for checking if it's a directory use the -d test as above. To check if executable file use -x. So putting that all together, try:
#!/bin/bash
echo “Directory: “
read thing
[[ $thing =~ '/'$ ]] || thing="$thing/"
[[ -d $thing ]] || exit 1
for var123 in "$thing"*
do
if [[ -f $var123 && -x $var123 ]]; then
echo "$var123**"
elif [[ -d $var123 ]]; then
echo "$var123/"
else
echo "$var123"
fi
done
ls -F is your friend here - if you want to do it for the current directory:
ls -F
If you want to do it for all files & subfolders of the current directory:
find * -exec ls -Fd {} \;
... and for a given directory:
echo "Directory: "
read DIR
find $DIR/* -exec ls -Fd {} \;
Edit: ls -F will append a / to directories and a * to executables. If you want ** instead, just use sed to replace them:
find $DIR/* -exec ls -Fd {} \; | sed 's/\*$/&&/'
And this approach works in all shells, not just bash.
This is my first ever bash script and I am trying to iron out all of the creases and make the script run nicely. The script is to archive all of the specified .jpg files that it finds in multiple directories on a HDD/Flash drive. There are files with the same name but different content so I have used an Md5 sum to hash them.
I am getting the directory does not exist error in Geany but it runs fine from command bar missing out two of the images. I have tried everything I can think of to fix it. Is it messy code that is doing this?
#!/bin/sh
if [ ! -d "$1" ]; then
echo Directory "$1" cannot be found. Please try again.
exit
fi
if [ $# -eq 1 ]; then
echo "usage: Phar image_path archive_path"
exit
fi
if [ -d "$2" ]; then
echo "archive exists"
else
echo "the directory 'archive' does't exist. Creating directory 'archive'."
mkdir -p ~/archive
fi
find $1 -iname "IMG_[0-9][0-9][0-9][0-9].JPG" | cat > list.txt
[ -f ~/my-documents/md5.txt ] && rm md5.txt || break
while read line;
do md5sum $line | xargs >> md5.txt
done < list.txt
sort -k 1,1 -u md5.txt | cat > uniquemd5.txt
cut -d " " -f 2- uniquemd5.txt > uniquelist.txt
sort uniquelist.txt -r -o uniquelist.txt
for line in $(cat uniquelist.txt)
do
file=$(basename $line) path="$2/file"
if [ ! -f $path ];
then
cp $line $2
else
cp $line $path.JPG
fi
done
You haven't guarded against spaces in the folder and file names everywhere.
For instance:
cp $line $2
should be:
cp "$line" "$2"
You should start by eliminating these spaces as a source to your error by evaluating each variable you are referencing and adding ""'s.
If you still get the error please provide us with the arguments used and which directory that does not exist.
I received a advice for do not parse ls, like describes in this website: Don't parse ls.
I was looking for DAILY files in my directory so that's what I did then:
for f in *.DA*; do
[[ -e $f ]] || continue
for file in $f; do
echo "The file that you are working on: "$file
archiveContent=$( sed -n -e 1p $file )
echo $archiveContent
done
done
Ok, that's works well, I've two files A.DAILY and B.DAILY, with the both archives I can get what is inside it, but when I changed a little bit the loop, it doesn't iterated with all files with .DAILY extension in my directory.
for f in *.DA*; do
[[ -e $f ]] || continue
for file in $f; do
echo "The file that you are working on: "$file
archiveContent=$( sed -n -e 1p $file )
echo $archiveContent
COMPRESS $archiveContent;
done
done
when I called a function inside the loop, the loop just does for the first file, but not to the second.
Since the outer loop sets f to each file in turn, your inner loop doesn't seem to serve any purpose.
for f in *.DA*; do
[[ -e $f ]] || continue
echo "The file that you are working on: $f"
archiveContent=$( sed -n -e 1p "$f" )
echo "$archiveContent"
COMPRESS "$archiveContent"
done
This piece of bash code, shows no folder name while there exists many folders.
#!/bin/bash
for file in .; do
if [ -d $file ]; then
echo $file
fi
done
the output is only .
Can you explain why?
it reads . as an array of size one and prints it for you. use something like this instead:
for file in `ls`; do
if [ -d $file ]; then
echo $file
fi
done