I am new to linux. I am trying to copy files from one path to another path. I have a text file which has all names of files in the following pattern:
file-1.txt
file-2.pdf
file-3.ppt
....
I created a .sh file with the following code:
#!/bin/bash
file=`cat filenames.txt`;
fromPath='/root/Backup/upload/';
toPath='/root/Desktop/custom/upload/';
for i in $file;
do
filePath=$fromPath$i
#echo $filePath
if [ -e $filePath ];
then
echo $filePath
yes | cp -rf $filePath $toPath
else
echo 'no files'
fi
done
The above code is copying only the last file name from the text instead of all to the destination path.
Please help.
The proper way to loop over a set of input lines is
while read i; do
: something with "$i"
done <filenames.txt
Note the use of double quotes around "$i" and generally variable interpolation where the string contains a filename component. Unquoted values are only appropriate when you specifically require the shell to do word splitting and wildcard resolution.
Related
I have .txt files, which I am taking as $item and then I am changing the encoding with
iconv -f $currentEncoding -t $targetEncoding "$item" -o "$item.tmp"
then I am saving it again to txt file using
mv "$item.tmp" "$item.txt";
next I am trimming a few things in txt file and saving it as a csv file with
tr -d '"' < "$item.txt" > "$item.csv";
but eventually my files are getting stored with extension "*.txt.csv" - I want them to be just .csv - can anyone help me please what I am doing wrong or what could I change. Thanks
Run:
for f in *.txt.csv; do mv $f ${f/.txt./.}; done
If the variable $f contains the string item.txt.csv, the expression ${f/.txt./.} removes .txt from the file name and gives only the string item.csv.
Caution: if one of the filenames contain spaces, the for statement will not work as expected.
From what I understand, your $item already has .txt in the value.
So, if you list your files after each command, you should see intermediate files like
xyz.txt.tmp
xyz.txt.txt
xyz.txt.csv
So, when you set the item variable, just do ${item%%.*} as shown below and it should work as expected.
item=xyz.txt
item="${item%%.*}"
echo $item
xyz
I ran the following in a directory with no files:
for file in *.20191017.*;do echo ${file}; done
what it returned was this:
*.20191017.*
which is little awkward since this was just a pattern and not the filename itself.
Can anyone please help on this?
Found the reason for this anomaly (source: https://www.cyberciti.biz/faq/bash-loop-over-file/)
You can do filename expansion in loop such as work on all pdf files in current directory:
for f in *.pdf; do
echo "Removing password for pdf file - $f"
done
However, there is one problem with the above syntax. If there are no pdf files in current directory it will expand to *.pdf (i.e. f will be set to *.pdf”). To avoid this problem add the following statement before the for loop:
#!/bin/bash
# Usage: remove all utility bills pdf file password
shopt -s nullglob # expands the glob to empty string when there are no matching files in the directory.
for f in *.pdf; do
echo "Removing password for pdf file - $f"
pdftk "$f" output "output.$f" user_pw "YOURPASSWORD-HERE"
done
The for loop simply iterates over the words between in and ; (possibly expanded by bash). Here, file is just the variable name. If you want to iterate between all files that are actually present, you can, for example, add a if to check if the ${file} really exists:
for file in *.20191017.*
do
if [ -e "${file}" ]
then
echo ${file}
fi
done
Or you can use, e.g., find
find . -name '*.20191017.*' -maxdepth 1
-maxdepth 1 is to avoid recursion.
I have prepared a bash script to get only the directory (not full path) with file name where file is present. It has to be done only when file is located in sub directory.
For example:
if input is src/email/${sub_dir}/Bank_Casefeed.email, output should be ${sub_dir}/Bank_Casefeed.email.
If input is src/layouts/Bank_Casefeed.layout, output should be Bank_Casefeed.layout. I can easily get this using basename command.
src/basefolder is always constant. In some cases (after src/email(basefolder) directory), sub_directories will be there.
This script will work. I can use this script (only if module is email) to get output. but script should work even if sub directory is present in other modules. Maybe should I count the directories? if there are more than two directories (src/basefolder), script should get sub directories. Is there any better way to handle both scenarios?
#!/bin/bash
filename=`basename src/email/${sub_dir}/Bank_Casefeed.email`
echo "filename is $filename"
fulldir=`dirname src/email/${sub_dir}/Bank_Casefeed.email`
dir=`basename $fulldir`
echo "subdirectory name: $dir"
echo "concatenate $filename $dir"
Entity=$dir/$filename
echo $Entity
Using shell parameter expansion:
sub_dir='test'
files=( "src/email/${sub_dir}/Bank_Casefeed.email" "src/email/Bank_Casefeed.email" )
for f in "${files[#]}"; do
if [[ $f == *"/$sub_dir/"* ]]; then
echo "${f/*\/$sub_dir\//$sub_dir\/}"
else
basename "$f"
fi
done
test/Bank_Casefeed.email
Bank_Casefeed.email
I know there might be an easier way to do this. But I believe you can just manipulate the input string. For example:
#!/bin/bash
sub_dir='test'
DIRNAME1="src/email/${sub_dir}/Bank_Casefeed.email"
DIRNAME2="src/email/Bank_Casefeed.email"
echo $DIRNAME1 | cut -f3- -d'/'
echo $DIRNAME2 | cut -f3- -d'/'
This will remove the first two directories.
I'm trying to concatenate a bunch of files into a string so I can use them for a function.
As a test script I'm trying to do this:
#!/bin/bash
for line in $(cat list.txt)
do
x=" "
A=$A$line$x
done
echo "$A"
mv "$A" ./stuff
but I'm getting the error:
mv: cannot stat ‘x.dat y.dat z.dat ’: No such file or directory
but they are most definitely there
can I get some advice please?
This solution will handle file names with spaces too.
#!/bin/bash
mapfile -t lines < list.txt
echo "${lines[#]}"
mv "${lines[#]}" ./stuff/
It reads the entire contents of the file into an array variable, displays the content of the entire array, and finally uses those values in the mv command
Change the last line to mv $A ./stuff
That should work with files that do not have space in their names.
I have a folder that is full of .bak files and some other files also. I need to remove the extension of all .bak files in that folder. How do I make a command which will accept a folder name and then remove the extension of all .bak files in that folder ?
Thanks.
To remove a string from the end of a BASH variable, use the ${var%ending} syntax. It's one of a number of string manipulations available to you in BASH.
Use it like this:
# Run in the same directory as the files
for FILENAME in *.bak; do mv "$FILENAME" "${FILENAME%.bak}"; done
That works nicely as a one-liner, but you could also wrap it as a script to work in an arbitrary directory:
# If we're passed a parameter, cd into that directory. Otherwise, do nothing.
if [ -n "$1" ]; then
cd "$1"
fi
for FILENAME in *.bak; do mv "$FILENAME" "${FILENAME%.bak}"; done
Note that while quoting your variables is almost always a good practice, the for FILENAME in *.bak is still dangerous if any of your filenames might contain spaces. Read David W.'s answer for a more-robust solution, and this document for alternative solutions.
There are several ways to remove file suffixes:
In BASH and Kornshell, you can use the environment variable filtering. Search for ${parameter%word} in the BASH manpage for complete information. Basically, # is a left filter and % is a right filter. You can remember this because # is to the left of %.
If you use a double filter (i.e. ## or %%, you are trying to filter on the biggest match. If you have a single filter (i.e. # or %, you are trying to filter on the smallest match.
What matches is filtered out and you get the rest of the string:
file="this/is/my/file/name.txt"
echo ${file#*/} #Matches is "this/` and will print out "is/my/file/name.txt"
echo ${file##*/} #Matches "this/is/my/file/" and will print out "name.txt"
echo ${file%/*} #Matches "/name.txt" and will print out "/this/is/my/file"
echo ${file%%/*} #Matches "/is/my/file/name.txt" and will print out "this"
Notice this is a glob match and not a regular expression match!. If you want to remove a file suffix:
file_sans_ext=${file%.*}
The .* will match on the period and all characters after it. Since it is a single %, it will match on the smallest glob on the right side of the string. If the filter can't match anything, it the same as your original string.
You can verify a file suffix with something like this:
if [ "${file}" != "${file%.bak}" ]
then
echo "$file is a type '.bak' file"
else
echo "$file is not a type '.bak' file"
fi
Or you could do this:
file_suffix=$(file##*.}
echo "My file is a file '.$file_suffix'"
Note that this will remove the period of the file extension.
Next, we will loop:
find . -name "*.bak" -print0 | while read -d $'\0' file
do
echo "mv '$file' '${file%.bak}'"
done | tee find.out
The find command finds the files you specify. The -print0 separates out the names of the files with a NUL symbol -- which is one of the few characters not allowed in a file name. The -d $\0means that your input separators are NUL symbols. See how nicely thefind -print0andread -d $'\0'` together?
You should almost never use the for file in $(*.bak) method. This will fail if the files have any white space in the name.
Notice that this command doesn't actually move any files. Instead, it produces a find.out file with a list of all the file renames. You should always do something like this when you do commands that operate on massive amounts of files just to be sure everything is fine.
Once you've determined that all the commands in find.out are correct, you can run it like a shell script:
$ bash find.out
rename .bak '' *.bak
(rename is in the util-linux package)
Caveat: there is no error checking:
#!/bin/bash
cd "$1"
for i in *.bak ; do mv -f "$i" "${i%%.bak}" ; done
You can always use the find command to get all the subdirectories
for FILENAME in `find . -name "*.bak"`; do mv --force "$FILENAME" "${FILENAME%.bak}"; done