I'm trying to concatenate a bunch of files into a string so I can use them for a function.
As a test script I'm trying to do this:
#!/bin/bash
for line in $(cat list.txt)
do
x=" "
A=$A$line$x
done
echo "$A"
mv "$A" ./stuff
but I'm getting the error:
mv: cannot stat ‘x.dat y.dat z.dat ’: No such file or directory
but they are most definitely there
can I get some advice please?
This solution will handle file names with spaces too.
#!/bin/bash
mapfile -t lines < list.txt
echo "${lines[#]}"
mv "${lines[#]}" ./stuff/
It reads the entire contents of the file into an array variable, displays the content of the entire array, and finally uses those values in the mv command
Change the last line to mv $A ./stuff
That should work with files that do not have space in their names.
Related
I have an executable that takes a file and outputs a line.
I am running a loop over a directory:
for file in $DIRECTORY/*.png
do
./eval $file >> out.txt
done
The output of executable does not contain the name of the file.
I want to append the file name with each output.
EDIT1
Perhaps, I could not explain it correctly
I want the name of the file and the output of the program as well, which is processing the same file, Now I am doing following
for file in $DIRECTORY/*.png
do
echo -n $file >> out.txt
or
printf "%s" "$file" >> out.txt
./eval $file >> out.txt
done
For both new line is inserted
If I understood your question, what you want is:
get the name of the file,
...and the output or the program processing the file (in your case, eval),
...on the same line. And this last part is your problem.
Then I'd suggest composing a single line of text (using echo), comprising:
the name of the file, this is the $file part,
...followed by a separator, you may not need that but it may help further processing of the result. I used ":". You can skip this part if this is not interesting for you,
...followed by the output of the program processing the file: this is the $(...) construct
echo $file ":" $(./eval $file) >> out.txt
...and finally appending this line of text to a file, you got that part right.
please use like this
echo -n `echo ${file}|tr -d '\n'` >> out.txt
OR
newname=`echo ${file}|tr -d '\n'`
echo -n $newname >> out.txt
I have a folder that is full of .bak files and some other files also. I need to remove the extension of all .bak files in that folder. How do I make a command which will accept a folder name and then remove the extension of all .bak files in that folder ?
Thanks.
To remove a string from the end of a BASH variable, use the ${var%ending} syntax. It's one of a number of string manipulations available to you in BASH.
Use it like this:
# Run in the same directory as the files
for FILENAME in *.bak; do mv "$FILENAME" "${FILENAME%.bak}"; done
That works nicely as a one-liner, but you could also wrap it as a script to work in an arbitrary directory:
# If we're passed a parameter, cd into that directory. Otherwise, do nothing.
if [ -n "$1" ]; then
cd "$1"
fi
for FILENAME in *.bak; do mv "$FILENAME" "${FILENAME%.bak}"; done
Note that while quoting your variables is almost always a good practice, the for FILENAME in *.bak is still dangerous if any of your filenames might contain spaces. Read David W.'s answer for a more-robust solution, and this document for alternative solutions.
There are several ways to remove file suffixes:
In BASH and Kornshell, you can use the environment variable filtering. Search for ${parameter%word} in the BASH manpage for complete information. Basically, # is a left filter and % is a right filter. You can remember this because # is to the left of %.
If you use a double filter (i.e. ## or %%, you are trying to filter on the biggest match. If you have a single filter (i.e. # or %, you are trying to filter on the smallest match.
What matches is filtered out and you get the rest of the string:
file="this/is/my/file/name.txt"
echo ${file#*/} #Matches is "this/` and will print out "is/my/file/name.txt"
echo ${file##*/} #Matches "this/is/my/file/" and will print out "name.txt"
echo ${file%/*} #Matches "/name.txt" and will print out "/this/is/my/file"
echo ${file%%/*} #Matches "/is/my/file/name.txt" and will print out "this"
Notice this is a glob match and not a regular expression match!. If you want to remove a file suffix:
file_sans_ext=${file%.*}
The .* will match on the period and all characters after it. Since it is a single %, it will match on the smallest glob on the right side of the string. If the filter can't match anything, it the same as your original string.
You can verify a file suffix with something like this:
if [ "${file}" != "${file%.bak}" ]
then
echo "$file is a type '.bak' file"
else
echo "$file is not a type '.bak' file"
fi
Or you could do this:
file_suffix=$(file##*.}
echo "My file is a file '.$file_suffix'"
Note that this will remove the period of the file extension.
Next, we will loop:
find . -name "*.bak" -print0 | while read -d $'\0' file
do
echo "mv '$file' '${file%.bak}'"
done | tee find.out
The find command finds the files you specify. The -print0 separates out the names of the files with a NUL symbol -- which is one of the few characters not allowed in a file name. The -d $\0means that your input separators are NUL symbols. See how nicely thefind -print0andread -d $'\0'` together?
You should almost never use the for file in $(*.bak) method. This will fail if the files have any white space in the name.
Notice that this command doesn't actually move any files. Instead, it produces a find.out file with a list of all the file renames. You should always do something like this when you do commands that operate on massive amounts of files just to be sure everything is fine.
Once you've determined that all the commands in find.out are correct, you can run it like a shell script:
$ bash find.out
rename .bak '' *.bak
(rename is in the util-linux package)
Caveat: there is no error checking:
#!/bin/bash
cd "$1"
for i in *.bak ; do mv -f "$i" "${i%%.bak}" ; done
You can always use the find command to get all the subdirectories
for FILENAME in `find . -name "*.bak"`; do mv --force "$FILENAME" "${FILENAME%.bak}"; done
I am trying to replace the prefix of all files in the directory with another prefix (renaming).
This is my script
# Script to rename the files
#!/bin/bash
for file in $1*;
do
mv $file `echo $file | sed -e 's/^$1/$2/'`;
done
Upon executing the script with
rename.sh BIT SIT
I get the following errors
mv: `BITfile.h' and `BITFile.h' are the same file
mv: `BITDefs.cpp' and `BITDefs.cpp' are the same file
mv: `BITDefs.h' and `BITDefs.h' are the same file
Seems like sed is treating $1 and $2 as the same value, but when I print those variables on another line it shows that they are different.
As Roman Newaza says, you can use " instead of ' to tell Bash that you want variables to be expanded. However, in your case, it would be safest to write:
for file in "$1"* ; do
mv -- "$file" "$2${file#$1}"
done
so that weird characters in filenames, or in your script parameters, cannot cause any problems.
You can also use parameter expansion to replace the prefix of all files in a directory.
for file in "$1"*;
do
mv ${file} ${file/#$1/$2}
done
Use double quotes instead:
# ...
mv "$file" `echo $file | sed -e "s/^$1/$2/"`
# ...
And learn Quotes and escaping in Bash.
When you don't use double quotes, the variables won't expand.
I will rather use this
#!/bin/bash
for file in $1*;
do
mv "$file" "$1${file:${#2}}"
done
where
${file:${#2}
means substring, from length of the argument 2 to the end
I'm encountering many files with the same content and the same name on some of my servers. I need to quarantine these files for analysis so I can't just remove the duplicates. The OS is Linux (centos and ubuntu).
I enumerate the file names and locations and put them into a text file.
Then I do a for statement to move the files to quarantine.
for file in $(cat bad-stuff.txt); do mv $file /quarantine ;done
The problem is that they have the same file name and I just need to add something unique to the filename to get it to save properly. I'm sure it's something simple but I'm not good with regex. Thanks for the help.
Since you're using Linux, you can take advantage of GNU mv's --backup.
while read -r file
do
mv --backup=numbered "$file" "/quarantine"
done < "bad-stuff.txt"
Here's an example that shows how it works:
$ cat bad-stuff.txt
./c/foo
./d/foo
./a/foo
./b/foo
$ while read -r file; do mv --backup=numbered "$file" "./quarantine"; done < "bad-stuff.txt"
$ ls quarantine/
foo foo.~1~ foo.~2~ foo.~3~
$
I'd use this
for file in $(cat bad-stuff.txt); do mv $file /quarantine/$file.`date -u +%s%N`; done
You'll get everyfile with a timestamp appended (in nanoseconds).
You can create a new file name composed by the directory and the filename. Thus you can add one more argument in your original code:
for ...; do mv $file /quarantine/$(echo $file | sed 's:/:_:g') ; done
Please note that you should replace the _ with a proper character which is special enough.
I have a text file named sqlfile, with the following content:
a.sql
b.sql
c.sql
d.sql
What I want is that to store them in variables and then print using for loop.
But here I get only d.sql in the output of the script.
The script:
#!/bin/bash
while read line
do
files=`echo $line`
done < /home/abdul_old/Desktop/My_Shell_Script/sqlfile
for file in $files
do
echo $file
done
A variable can only hold one element, what you want is an array
#!/bin/bash
while read line
do
files+=( "$line" )
done < /home/abdul_old/Desktop/My_Shell_Script/sqlfile
for file in "${files[#]}"
do
echo "$file"
done
while read line
do files="$files $line"
done < /home/abdul_old/Desktop/My_Shell_Script/sqlfile
or
files=$(</home/abdul_old/Desktop/My_Shell_Script/sqlfile)
or
files=$(cat /home/abdul_old/Desktop/My_Shell_Script/sqlfile)
You're doing way too much work in your loop.
The middle alternative works with bash; the other two work with most shells. Prefer $(...) to back-quotes.
This code assumes there are no spaces in file names to mess things up. If you do use blanks in file names, you have to work marginally harder - see the array-based solution by SiegeX
I think you need to make the "files" as array. otherwise, as soon as the while finishes, "files" stores the latest "line".
try:
files=( "${files[#]}" $line )
That's right, you assifn last value to "files"
You must use for instance += instead of =
#!/bin/bash
while read line
do
files+=`echo " $line"`
done < /home/abdul_old/Desktop/My_Shell_Script/sqlfile
for file in $files
do
echo $file
done
Using read is fine but you have to set the IFS environment variable first else leading and trailing white space are removed from each line: Preserving leading white space while reading>>writing a file line by line in bash.
All you have to do is:
readarray myData < sqlfile
This will put file lines into an array called myData
Now you can access any of these lines like this:
printf "%s\n" "${myData[0]}" #outputs first line
printf "%s\n" "${myData[2]}" #outputs third line
And you can iterate over it:
for curLine in "${myData[#]}"; do
echo "$curLine"
done
Note that these lines would contain \n character as well. To remove trailing newlines you can use -t flag like this:
readarray -t myData < sqlfile
readarray is a synonym to mapfile. You can read about it in man bash