Shell Script working with multiple files [duplicate] - linux

This question already has answers here:
How to iterate over arguments in a Bash script
(9 answers)
Closed 5 years ago.
I have this code below:
#!/bin/bash
filename=$1
file_extension=$( echo $1 | cut -d. -f2 )
directory=${filename%.*}
if [[ -z $filename ]]; then
echo "You forgot to include the file name, like this:"
echo "./convert-pdf.sh my_document.pdf"
else
if [[ $file_extension = 'pdf' ]]; then
[[ ! -d $directory ]] && mkdir $directory
convert $filename -density 300 $directory/page_%04d.jpg
else
echo "ERROR! You must use ONLY PDF files!"
fi
fi
And it is working perfectly well!
I would like to create a script which I can do something like this: ./script.sh *.pdf
How can I do it? Using asterisk.
Thank you for your time!

Firstly realize that the shell will expand *.pdf to a list of arguments. This means that your shell script will never ever see the *. Instead it will get a list of arguments.
You can use a construction like the following:
#!/bin/bash
function convert() {
local filename=$1
# do your thing here
}
if (( $# < 1 )); then
# give your error message about missing arguments
fi
while (( $# > 0 )); do
convert "$1"
shift
done
What this does is first wrap your functionality in a function called convert. Then for the main code it first checks the number of arguments passed to the script, if this is less than 1 (i.e. none) you give the error that a filename should be passed. Then you go into a while loop which is executed as long as there are arguments remaining. The first argument you pass to the convert function which does what your script already does. Then the shift operation is performed, what this does is it throws away the first argument and then shifts all the remaining arguments "left" by one place, that is what was $2 now is $1, what was $3 now is $2, etc. By doing this in the while loop until the argument list is empty you go through all the arguments.
By the way, your initial assignments have a few issues:
you can't assume that the filename has an extension, your code could match a dot in some directory path instead.
your directory assignment seems to be splitting on . instead of /
your directory assignment will contain the filename if no absolute or relative path was given, i.e. only a bare filename
...
I think you should spend a bit more time on robustness

Wrap your code in a loop. That is, instead of:
filename=$1
: code goes here
use:
for filename in "$#"; do
: put your code here
done

Related

Linux bash script: Recurisely delete files if empty based on file format from user input [duplicate]

This question already has answers here:
How do I pass a wildcard parameter to a bash file
(2 answers)
Closed 5 years ago.
So as the title describes, I want to recursively delete all files which match a naming pattern given by the user, but only if the file is empty. Here is my attempt:
#!/bin/bash
_files="$1"
[ $# -eq 0 ] && { echo "Usage: $0 filename"; exit 1; }
[ ! -f "$_files" ] && { echo "Error: $0 no files were found which match given naming structure."; exit 2; }
for f in $(find -name $_files)
do
if [ -s "$f" ]
then
echo "$f has some data."
# do something as file has data
else
echo "$f is empty. Deleting file."
rm $f
fi
done
Example output:
./remove_blank.sh *.o*
./Disp_variations_higher_0.o1906168 has some data.
./remove_blank.sh *.e*
./Disp_variations_higher_15.e1906183 is empty. Deleting file.
As you can see, the code works, but only for one file at a time. Should be a relatively simple fix to get it to work, but I extremely new to bash scripting and can't seem to figure it out. Sorry for the noobish question. I did some research to find an answer but didn't find exactly what I needed. Thanks in advance for any help.
Edit
I have found two different solutions to the problem. As #David Z's suggestion, one can fix this by 1st deleting the Error checking part of the script as well as putting quotes around the $_files variable in the find function. Then the code looks like this:
#!/bin/bash
_files=$1
[ $# -eq 0 ] && { echo "Usage: $0 filename"; exit 1; }
for f in $(find -name "$_files")
do
if [ -s $f ]
then
echo "$f has some data."
# do something as file has data
else
echo "$f is empty. Deleting file."
rm $f
fi
done
Or, one can also simply change the for loop to for f in "$#", which allows the error check to be kept in the script. I am not sure which method is better but will update again if I find out.
It looks like the way you're invoking the script, the shell expands the pattern before running your script. For example, in
./remove_blank.sh *.o*
the shell converts *.o* to the list of filenames that match that pattern, and then what it actually runs is something like
./remove_blank.sh Disp_variations_higher_0.o1906168 other_file.o12345 ...
But you only check the first argument in your script, so that's the one that winds up getting deleted.
Solution: quote the pattern when you run the script.
./remove_blank.sh '*.o*'
You will also need to remove the test [ ! -f "$_files" ] ... because $_files is being set to the pattern (such as *.o*), not a filename. In fact, you might want to rename the variable to make that clear. And finally, you need to quote the variable in your find command,
... $(find -name "$_files") ...
so that the pattern makes it all the way through to find without being converted into filenames by the shell.
There are some other issues with the script but you might want to ask about those on Code Review. What I've identified here is just the minimum needed to get it working.
Incidentally, you can accomplish this whole task using find itself, which will at least give cleaner code:
find -name "$_files" -type f -empty -delete

"test: too many arguments" message because of special character * while using test command on bash to compare two strings [duplicate]

This question already has answers here:
Meaning of "[: too many arguments" error from if [] (square brackets)
(6 answers)
Closed 5 years ago.
I'm new to shell scripting and I'm having some trouble while using the "test" command and the special character * to compare two strings.
I have to write a shell script which, for every element(both files and directories) contained in the directory passed as the first argument, has to write something(for the solving of my problem it is not relevant to know what has to be written down) on the file "summary.out". What's more, there's a string passed as the second argument. Those files/directories beginning with this string must be ignored(nothing has to be written on summary.out).
Here is the code:
#!/bin/bash
TEMP=NULL
cd "$1"
for i in *
do
if test "$i" != "$2"*;then #Here is where the error comes from
if test -f "$i";then
TEMP="$i - `head -c 10 "$i"`"
elif test -d "$i";then
TEMP="$i - `ls -1 "$i" | wc -l`"
fi
echo $TEMP >> summary.out
fi
done
The error comes from the test which checks whether the current file/directory begins with the string passed as second argument, and it takes place every iteration of the for cycle. It states:"test: too many arguments"
Now, I have performed some tests which showed that the problem has nothing to do with blank spaces inside the $i or $1. The problem is linked to the fact that I use the special character * in the test(if I remove it, everything works fine).
Why can't "test" handle * ? What can I do to fix that?
* gets expanded by the shell.
In bash, you can use [[ ... ]] for conditions instead of test. They support patterns on the right hand side - * is not expanded, as double square brackets are a keyword with higher precedence.
if [[ a == * ]] ; then
echo Matches
else
echo Doesn\'t match
fi

How to use Linux pipe for Userdefined Shell Scripts

I have two user defined shell scripts:
First is Add
if [ $# -eq 3 ]
then
sum=`expr $1 + $2 `
echo $sum
else
echo "usage :$0 num1 num2"
echo "num1 and num2 are two numbers"
exit 1
fi
The next is Square
echo `expr $1 \* $1`
Can any one pleae tell me how to use Linux pipe for these shell scripts. I tried something like:
add 10 20 | square
But it is giving me the list of files in that directory.
Using a pipe will pass the output from the first command to the stdin of the second command. You want the output to be used as arguments for the second command instead. Try xargs:
add 10 20 | xargs square
Of course you have to make sure that the output of the first command is just "10" in this case.
A little more explanation: a pipe will take the output of the first command and redirect it into the standard input stream of the second command. That means you will have to use a command like "read" (as some of the other answers do) to use the information from the input stream.
But your square script doesn't read anything from the standard input: it takes an argument instead. So we want to take the output of your first command (10) and use it as the argument for your second command. The "xargs" utility does exactly that: the standard input it receives will be passed as arguments to the square command. See https://en.wikipedia.org/wiki/Xargs.
By the way, command substitution has the same effect:
square $(add 10 20)
The syntax $(add 10 20) will run the add script and replace the expression with its output. So after running the add script the line looks like this:
square 30
And, in effect, we have again turned the output from add into an argument for square.
As written, you want to use command substitution, not a pipe (since square takes command line arguments, rather than reading from standard input):
square $(add 10 20)
To modify square so that add 10 20 | square works, use the read builtin:
#!/bin/bash
read input
echo $(( $input * $input )) # No need for the external expr command
add should also write any error messages to standard error, not standard output:
if [ $# -eq 2 ]
then
sum=$(( $1 + $2 ))
echo $sum
else
echo "usage :$0 num1 num2" >&2
echo "num1 and num2 are two numbers" >&2
exit 1
fi
You can use the "read" command to read the value from STDIN, if no parameters are specified:
val="$1"
test -z "$1" && read val
echo `expr $val \* $val`

Attempting to pass two arguments to a called script for a pattern search

I'm having trouble getting a script to do what I want.
I have a script that will search a file for a pattern and print the line numbers and instances of that pattern.
I want to know how to make it print the file name first before it prints the lines found
I also want to know how to write a new script that will call this one and pass two arguments to it.
The first argument being the pattern for grep and the second the location.
If the location is a directory, it will loop and search the pattern on all files in the directory using the script.
#!/bin/bash
if [[ $# -ne 2 ]]
then
echo "error: must provide 2 arguments."
exit -1
fi
if [[ ! -e $2 ]];
then
echo "error: second argument must be a file."
exit -2
fi
echo "------ File =" $2 "------"
grep -ne "$1" "$2"
This is the script i'm using that I need the new one to call. I just got a lot of help from asking a similar question but i'm still kind of lost. I know that I can use the -d command to test for the directory and then use 'for' to loop the command, but exactly how isn't panning out for me.
I think you just want to add the -H option to grep:
-H, --with-filename
Print the file name for each match. This is the default when there is more than one file to search.
grep has an option -r which can help you avoid testing for second argument being a directory and using for loop to iterate all files of that directory.
From the man page:
-R, -r, --recursive
Recursively search subdirectories listed.
It will also print the filename.
Test:
On one file:
[JS웃:~/Temp]$ grep -r '5' t
t:5 10 15
t:10 15 20
On a directory:
[JS웃:~/Temp]$ grep -r '5' perl/
perl//hello.pl:my $age=65;
perl//practice.pl:use v5.10;
perl//practice.pl:#array = (1,2,3,4,5);
perl//temp/person5.pm:#person5.pm
perl//temp/person9.pm: my #date = (localtime)[3,4,5];
perl//text.file:This is line 5

Bash command to move only some files?

Let's say I have the following files in my current directory:
1.jpg
1original.jpg
2.jpg
2original.jpg
3.jpg
4.jpg
Is there a terminal/bash/linux command that can do something like
if the file [an integer]original.jpg exists,
then move [an integer].jpg and [an integer]original.jpg to another directory.
Executing such a command will cause 1.jpg, 1original.jpg, 2.jpg and 2original.jpg to be in their own directory.
NOTE
This doesn't have to be one command. I can be a combination of simple commands. Maybe something like copy original files to a new directory. Then do some regular expression filter on files in the newdir to get a list of file names from old directory that still need to be copied over etc..
Turning on extended glob support will allow you to write a regular-expression-like pattern. This can handle files with multi-digit integers, such as '87.jpg' and '87original.jpg'. Bash parameter expansion can then be used to strip "original" from the name of a found file to allow you to move the two related files together.
shopt -s extglob
for f in +([[:digit:]])original.jpg; do
mv $f ${f/original/} otherDirectory
done
In an extended pattern, +( x ) matches one or more of the things inside the parentheses, analogous to the regular expression x+. Here, x is any digit. Therefore, we match all files in the current directory whose name consists of 1 or more digits followed by "original.jpg".
${f/original/} is an example of bash's pattern substitution. It removes the first occurrence of the string "original" from the value of f. So if f is the string "1original.jpg", then ${f/original/} is the string "1.jpg".
well, not directly, but it's an oneliner (edit: not anymore):
for i in [0-9].jpg; do
orig=${i%.*}original.jpg
[ -f $orig ] && mv $i $orig another_dir/
done
edit: probably I should point out my solution:
for i in [0-9].jpg: execute the loop body for each jpg file with one number as filename. store whole filename in $i
orig={i%.*}original.jpg: save in $orig the possible filename for the "original file"
[ -f $orig ]: check via test(1) (the [ ... ] stuff) if the original file for $i exists. if yes, move both files to another_dir. this is done via &&: the part after it will be only executed if the test was successful.
This should work for any strictly numeric prefix, i.e. 234.jpg
for f in *original.jpg; do
pre=${f%original.jpg}
if [[ -e "$pre.jpg" && "$pre" -eq "$pre" ]] 2>/dev/null; then
mv "$f" "$pre.jpg" targetDir
fi
done
"$pre" -eq "$pre" gives an error if not integer
EDIT:
this fails if there exist original.jpg and .jpg both.
$pre is then nullstring and "$pre" -eq "$pre" is true.
The following would work and is easy to understand (replace out with the output directory, and {1..9} with the actual range of your numbers.
for x in {1..9}
do
if [ -e ${x}original.jpg ]
then
mv $x.jpg out
mv ${x}original.jpg out
fi
done
You can obviously also enter it as a single line.
You can use Regex statements to find "matches" in the files names that you are looking through. Then perform your actions on the "matches" you find.
integer=0; while [ $integer -le 9 ] ; do if [ -e ${integer}original.jpg ] ; then mv -vi ${integer}.jpg ${integer}original.jpg lol/ ; fi ; integer=$[ $integer + 1 ] ; done
Note that here, "lol" is the destination directory. You can change it to anything you like. Also, you can change the 9 in while [ $integer -le 9 ] to check integers larger than 9. Right now it starts at 0* and stops after checking 9*.
Edit: If you want to, you can replace the semicolons in my code with carriage returns and it may be easier to read. Also, you can paste the whole block into the terminal this way, even if that might not immediately be obvious.

Resources