I run the following code for concatenating files in a directory given as the argument for the script file in bash
for i in $*
do
cat $* > /home/christy/Documents/filetest/catted.txt
done
This produce the error
cat: /home/christy/Documents/filetest/catted.txt: input file is output file
I think there are at least 4 things wrong with your script....
Firstly, your loop will set the value of i to the name of each file in succession, so you would want to actually use i inside your loop, like this:
for i in $*
cat "$i" ....somewhere
done
Secondly, if you use the > redirection, each file will land exactly on top of the previous one, so you should really use the >> redirection will append the current file to the end of the previous one like this
for i in $*
do
cat "$i" >> ...somewhere
done
Thirdly, I think you should use double-quoted "$#" to get all your command-line arguments, rather than plain $*
for i in "$#"
...
Fourthly, you can achieve the exact effect I think you want with this simpler command:
cat "$#" > /home/christy/Documents/filetest/catted.txt
You can't cat a file back onto itself. That's what "input file is output file" means. Because catted.txt shows up in your list of arguments to cat, it is going to try to cat to itself. So, move catted.txt to somewhere other than the source directory.
Related
I'm trying to do a bash script that extract info from pdf documents. the first argument should be a regex or the name of a file. Es:
$ autobib shrek2001.pdf
$ autobib *.pdf
My idea is to generate a list of files matching the regex and extract information from them. My code at the moment looks like this:
for article in $(ls $1);do
pdfinfo $article
done
But doing so the loop stops at the first file. How can I loop over all the files matching my regex?
clpgr has it completely right. Change your program to look like this:
for article in "$#" ;do
pdfinfo $article
done
The reason your program only does the first file is that the shell command gets globbed. That is, when you issue the command autobib *.pdf, you are really issuing this command: autobib 1.pdf 2.pdf 3.pdf (well, I'm making up some file names since I don't know what's in the directory. But the point is, your program will have $1 set to 1.pdf so you'll be executing this code $( ls 1.pdf ) which would only return 1.pdf.
Truth is, your program may have worked (depending on the file names in the directory) if you executed this way: autobib "*.pdf". In this example, the "*.pdf" is not globbed by the shell because it is quoted. Now, your program's $1 variable will have the value *.pdf.
That said, "$#" is soooooo much better than $( ls $1 ). "$#" will actually preserve spaces in the arguments.
I would like to know how to access the contents of a variety of txt files by passing arguments into shell scripts. I'll have different files and I'm expecting to execute with this command:
./script.sh FileA.txt
What should I put into my shell script so that I can access and manipulate the contents of the files?
I tried this but it outputs 0:
echo "$#"
I also tried these, but both output nothing:
for i in $1
do
echo "$i"
done
echo "$1"
To sum up the contents see this link to understand bash arguments more https://tecadmin.net/tutorial/bash-scripting/bash-command-arguments/ . Also as #Barmar said, to iterate through a list of arguments of unknown quantity use for i in "$#" .
edit
and as #Barmar said, $1 is simply the name of the argument. So echoing $1 will just echo the name.
I don't understand your question fully. Lets assume you have list of file names in a text file "FileA.txt".
And you wanted to run some commands for each file in the "FileA.txt" file.
Can you try below:
for i in `cat $1`
do
echo $i
done
I am having a problem where cmd1 works, but not cmd2 in my Bash script ending in .sh. I have made the Bash script executable.
Additionally, I can execute cmd2 just fine from my Bash terminal. I have tried to make a minimally reproducible example, but my larger goal is to run a complicated executable with command line arguments and pass output to a file that may or may not exist (rather than displaying the output in the terminal).
Replacing > with >> also gives the same error in the script, but not the terminal.
My Bash script:
#!/bin/bash
cmd1="cat test.txt"
cmd2="cat test.txt > a"
echo $cmd1
$cmd1
echo $cmd2
$cmd2
test.txt has the words "dog" and "cat" on two separate lines without quotes.
Short answer: see BashFAQ #50: I'm trying to put a command in a variable, but the complex cases always fail!.
Long answer: the shell expands variable references (like $cmd1) toward the end of the process of parsing a command line, after it's done parsing redirects (like > a is supposed to be) and quotes and escapes and... In fact, the only thing it does with the expanded value is word splitting (e.g. treating cat test.txt > a as "cat" followed by "test.txt", ">", and finally "a", rather than a single string) and wildcard expansion (e.g. if $cmd expanded to cat *.txt, it'd replace the *.txt part with a list of matching files). (And it skips word splitting and wildcard expansion if the variable is in double-quotes.)
Partly as a result of this, the best way to store commands in variables is: don't. That's not what they're for; variables are for data, not commands. What you should do instead, though, depends on why you were storing the command in a variable.
If there's no real reason to store the command in a variable, then just use the command directly. For conditional redirects, just use a standard if statement:
if [ -f a ]; then
cat test.txt > a
else
cat test.txt
fi
If you need to define the command at one point, and use it later; or want to use the same command over and over without having to write it out in full each time, use a function:
cmd2() {
cat test.txt > a
}
cmd2
It sounds like you may need to be able to define the command differently depending on some condition, you can actually do that with a function as well:
if [ -f a ]; then
cmd() {
cat test.txt > a
}
else
cmd() {
cat test.txt
}
fi
cmd
Alternately, you can wrap the command (without redirect) in a function, then use a conditional to control whether it redirects:
cmd() {
cat test.txt
}
if [ -f a ]; then
cmd > a
else
cmd
fi
It's also possible to wrap a conditional redirect into a function itself, then pipe output to it:
maybe_redirect_to() {
if [ -f "$1" ]; then
cat > "$1"
else
cat
fi
}
cat test.txt | maybe_redirect_to a
(This creates an extra cat process that isn't really doing anything useful, but if it makes the script cleaner, I'd consider that worth it. In this particular case, you could minimize the stray cats by using maybe_redirect_to a < test.txt.)
As a last resort, you can store the command string in a variable, and use eval to parse it. eval basically re-runs the shell parsing process from the beginning, meaning that it'll recognize things like redirects in the string. But eval has a well-deserved reputation as a bug magnet, because it's easy for it to treat parts of the string you thought were just data as command syntax, which can cause some really weird (& dangerous) bugs.
If you must use eval, at least double-quote the variable reference, so it runs through the parsing process just once, rather than sort-of-once-and-a-half as it would unquoted. Here's an example of what I mean:
cmd3="echo '5 * 3 = 15'"
eval "$cmd3"
# prints: 5 * 3 = 15
eval $cmd3
# prints: 5 [list of files in the current directory] 3 = 15
# ...unless there are any files with shell metacharacters in their names, in
# which case something more complicated might happen.
BashFAQ #50 discusses some other possible reasons and solutions. Note that the array approach will not work here, since arrays also get expanded after redirects are parsed.
If you pop an 'eval' in front of $cmd2 it should work as expected:
#!/bin/bash
cmd2="cat test.txt > a"
eval $cmd2
If you're not sure about the operation of a script you could always use the debug mode to see if you can determine the error.
bash -x scriptname
This will run the command and display the output of variable evaluations. Hopefully this will reveal any issues with syntax.
I am trying to write a small shell script that makes the multiple copies of a file. I am able to take the file name as input but not the number of copies. Here is what I wrote. But I am unable to pass the NUMBER variable to for loop.
echo -n "Enter filename: "
read FILENAME
echo -n "Number of copies to be made: "
read NUMBER
for i in {2..$NUMBER}
do
cp -f $FILENAME ${FILENAME%%.*}"_"$i.csv
done
Unfortunately it doesn't work like that. Bash performs brace expansion before parameter expansion, so your brace will be expanded before $NUMBER is evaluated. See also the Bash Pitfall #33, which explains the issue.
One way to do this, using your code, would be:
for i in $(eval echo {2..$NUMBER})
do
# ...
done
Or, even shorter:
for i in $(seq 2 $NUMBER)
# ...
(thanks, Glenn Jackman!)
Note that typically, variables should be quoted. This is especially important for file names. What if your file is called foo bar? Then your cp -f would copy foo and bar since the arguments are split by whitespace.
So, do something like this:
cp -f "$FILENAME" "${FILENAME%%.*}_${i}.csv"
While it might not matter if your files don't contain whitespace, quoting variables is something you should do automatically to prevent any surprises in the future.
I'm trying to automate a proces which I have to do over and over again in which I have to parse the output from a shell function, look for 5 different things, and then put them on a file
I know I can match patterns with grep however I don't know how to store the result on a variable so I can use it after :(
I also have to parse this very same output to get the other 5 values
I have no idea on how to use the same output for the 5 grep's i need to do and then store it to 5 different variables for after use
I know i have to create a nice and tidy .sh but I don't know how to do this
Currently im trying this
#!/bin/bash
data=$(cat file)
lol=$(echo data|grep red)
echo $lol
not working , any ideas?
you should show some examples of what you want to do next time..
assuming you shell function is called func1
func1(){
echo "things i want to get are here"
}
func1 | grep -E "things|want|are|here|get" > outputfile.txt
Update:
your code
#!/bin/bash
data=$(cat file)
lol=$(echo data|grep red)
echo $lol
practically just means this
lol=$(grep "red" file)
or
lol=$(awk '/red/' file)
also, if you are considering using bash, this is one way you can do it
while read -r myline
do
case "$myline" in
*"red"* ) echo "$myline" >> output.txt
esac
done <file
You can use the following syntax:
VAR=$(grep foo bar)
or alternatively:
VAR=`grep foo bar`
The easiest thing to do would be to redirect the output of the function to a file. You can then run multiple greps on it and only delete the file once you are done with it.
To save the output, you want to use command substitution. This runs a command and then converts the output into command line parameter. Combined with variable assignment you get:
variable=$(grep expression file)
Your second line is wrong. Change it to this:
lol=$(echo "$data"|grep red)
use egrep istead of grep.
variable=$(egrep "exp1|exp2|exp3|exp4|exp5" file)