I'm trying to automate a proces which I have to do over and over again in which I have to parse the output from a shell function, look for 5 different things, and then put them on a file
I know I can match patterns with grep however I don't know how to store the result on a variable so I can use it after :(
I also have to parse this very same output to get the other 5 values
I have no idea on how to use the same output for the 5 grep's i need to do and then store it to 5 different variables for after use
I know i have to create a nice and tidy .sh but I don't know how to do this
Currently im trying this
#!/bin/bash
data=$(cat file)
lol=$(echo data|grep red)
echo $lol
not working , any ideas?
you should show some examples of what you want to do next time..
assuming you shell function is called func1
func1(){
echo "things i want to get are here"
}
func1 | grep -E "things|want|are|here|get" > outputfile.txt
Update:
your code
#!/bin/bash
data=$(cat file)
lol=$(echo data|grep red)
echo $lol
practically just means this
lol=$(grep "red" file)
or
lol=$(awk '/red/' file)
also, if you are considering using bash, this is one way you can do it
while read -r myline
do
case "$myline" in
*"red"* ) echo "$myline" >> output.txt
esac
done <file
You can use the following syntax:
VAR=$(grep foo bar)
or alternatively:
VAR=`grep foo bar`
The easiest thing to do would be to redirect the output of the function to a file. You can then run multiple greps on it and only delete the file once you are done with it.
To save the output, you want to use command substitution. This runs a command and then converts the output into command line parameter. Combined with variable assignment you get:
variable=$(grep expression file)
Your second line is wrong. Change it to this:
lol=$(echo "$data"|grep red)
use egrep istead of grep.
variable=$(egrep "exp1|exp2|exp3|exp4|exp5" file)
Related
I have a bash script that employs the read command to read arguments to commands interactively, for example yes/no options. Is there a way to call this script in a non-interactive script passing default option values as arguments?
It's not just one option that I have to pass to the interactive script.
Many ways
pipe your input
echo "yes
no
maybe" | your_program
redirect from a file
your_program < answers.txt
use a here document (this can be very readable)
your_program << ANSWERS
yes
no
maybe
ANSWERS
use a here string
your_program <<< $'yes\nno\nmaybe\n'
For more complex tasks there is expect ( http://en.wikipedia.org/wiki/Expect ).
It basically simulates a user, you can code a script how to react to specific program outputs and related stuff.
This also works in cases like ssh that prohibits piping passwords to it.
You can put the data in a file and re-direct it like this:
$ cat file.sh
#!/bin/bash
read x
read y
echo $x
echo $y
Data for the script:
$ cat data.txt
2
3
Executing the script:
$ file.sh < data.txt
2
3
Just want to add one more way. Found it elsewhere, and is quite simple.
Say I want to pass yes for all the prompts at command line for a command "execute_command", Then I would simply pipe yes to it.
yes | execute_command
This will use yes as the answer to all yes/no prompts.
You can also use printf to pipe the input to your script.
var=val
printf "yes\nno\nmaybe\n$var\n" | ./your_script.sh
I am having a problem where cmd1 works, but not cmd2 in my Bash script ending in .sh. I have made the Bash script executable.
Additionally, I can execute cmd2 just fine from my Bash terminal. I have tried to make a minimally reproducible example, but my larger goal is to run a complicated executable with command line arguments and pass output to a file that may or may not exist (rather than displaying the output in the terminal).
Replacing > with >> also gives the same error in the script, but not the terminal.
My Bash script:
#!/bin/bash
cmd1="cat test.txt"
cmd2="cat test.txt > a"
echo $cmd1
$cmd1
echo $cmd2
$cmd2
test.txt has the words "dog" and "cat" on two separate lines without quotes.
Short answer: see BashFAQ #50: I'm trying to put a command in a variable, but the complex cases always fail!.
Long answer: the shell expands variable references (like $cmd1) toward the end of the process of parsing a command line, after it's done parsing redirects (like > a is supposed to be) and quotes and escapes and... In fact, the only thing it does with the expanded value is word splitting (e.g. treating cat test.txt > a as "cat" followed by "test.txt", ">", and finally "a", rather than a single string) and wildcard expansion (e.g. if $cmd expanded to cat *.txt, it'd replace the *.txt part with a list of matching files). (And it skips word splitting and wildcard expansion if the variable is in double-quotes.)
Partly as a result of this, the best way to store commands in variables is: don't. That's not what they're for; variables are for data, not commands. What you should do instead, though, depends on why you were storing the command in a variable.
If there's no real reason to store the command in a variable, then just use the command directly. For conditional redirects, just use a standard if statement:
if [ -f a ]; then
cat test.txt > a
else
cat test.txt
fi
If you need to define the command at one point, and use it later; or want to use the same command over and over without having to write it out in full each time, use a function:
cmd2() {
cat test.txt > a
}
cmd2
It sounds like you may need to be able to define the command differently depending on some condition, you can actually do that with a function as well:
if [ -f a ]; then
cmd() {
cat test.txt > a
}
else
cmd() {
cat test.txt
}
fi
cmd
Alternately, you can wrap the command (without redirect) in a function, then use a conditional to control whether it redirects:
cmd() {
cat test.txt
}
if [ -f a ]; then
cmd > a
else
cmd
fi
It's also possible to wrap a conditional redirect into a function itself, then pipe output to it:
maybe_redirect_to() {
if [ -f "$1" ]; then
cat > "$1"
else
cat
fi
}
cat test.txt | maybe_redirect_to a
(This creates an extra cat process that isn't really doing anything useful, but if it makes the script cleaner, I'd consider that worth it. In this particular case, you could minimize the stray cats by using maybe_redirect_to a < test.txt.)
As a last resort, you can store the command string in a variable, and use eval to parse it. eval basically re-runs the shell parsing process from the beginning, meaning that it'll recognize things like redirects in the string. But eval has a well-deserved reputation as a bug magnet, because it's easy for it to treat parts of the string you thought were just data as command syntax, which can cause some really weird (& dangerous) bugs.
If you must use eval, at least double-quote the variable reference, so it runs through the parsing process just once, rather than sort-of-once-and-a-half as it would unquoted. Here's an example of what I mean:
cmd3="echo '5 * 3 = 15'"
eval "$cmd3"
# prints: 5 * 3 = 15
eval $cmd3
# prints: 5 [list of files in the current directory] 3 = 15
# ...unless there are any files with shell metacharacters in their names, in
# which case something more complicated might happen.
BashFAQ #50 discusses some other possible reasons and solutions. Note that the array approach will not work here, since arrays also get expanded after redirects are parsed.
If you pop an 'eval' in front of $cmd2 it should work as expected:
#!/bin/bash
cmd2="cat test.txt > a"
eval $cmd2
If you're not sure about the operation of a script you could always use the debug mode to see if you can determine the error.
bash -x scriptname
This will run the command and display the output of variable evaluations. Hopefully this will reveal any issues with syntax.
I run the following code for concatenating files in a directory given as the argument for the script file in bash
for i in $*
do
cat $* > /home/christy/Documents/filetest/catted.txt
done
This produce the error
cat: /home/christy/Documents/filetest/catted.txt: input file is output file
I think there are at least 4 things wrong with your script....
Firstly, your loop will set the value of i to the name of each file in succession, so you would want to actually use i inside your loop, like this:
for i in $*
cat "$i" ....somewhere
done
Secondly, if you use the > redirection, each file will land exactly on top of the previous one, so you should really use the >> redirection will append the current file to the end of the previous one like this
for i in $*
do
cat "$i" >> ...somewhere
done
Thirdly, I think you should use double-quoted "$#" to get all your command-line arguments, rather than plain $*
for i in "$#"
...
Fourthly, you can achieve the exact effect I think you want with this simpler command:
cat "$#" > /home/christy/Documents/filetest/catted.txt
You can't cat a file back onto itself. That's what "input file is output file" means. Because catted.txt shows up in your list of arguments to cat, it is going to try to cat to itself. So, move catted.txt to somewhere other than the source directory.
Normally when one wants to look at specific output lines from running something, one can do something like:
./a.out | grep IHaveThisString
but what if IHaveThisString is something which changes every time so you need to first run it, watch the output to catch what IHaveThisString is on that particular run, and then grep it out? I can just dump to file and later grep but is it possible to do something like background it and then bring it to foreground and bringing it back but now piped to some grep? Something akin to:
./a.out
Ctrl-Z
fg | grep NowIKnowThisString
just wondering..
No, it is only in your screen buffer if you didn't save it in some other way.
Short form: You can do this, but you need to know that you need to do it ahead-of-time; it's not something that can be put into place interactively after-the-fact.
Write your script to determine what the string is. We'd need a more detailed example of the output format to give a better example of usage, but here's one for the trivial case where the entire first line is the filter target:
run_my_command | { read string_to_filter_for; fgrep -e "$string_to_filter_for" }
Replace the read string_to_filter_for with as many commands as necessary to read enough input to determine what the target string is; this could be a loop if necessary.
For instance, let's say that the output contains the following:
Session id: foobar
and thereafter, you want to grep for lines containing foobar.
...then you can pipe through the following script:
re='Session id: (.*)'
while read; do
if [[ $REPLY =~ $re ]] ; then
target=${BASH_REMATCH[1]}
break
else
# if you want to print the preamble; leave this out otherwise
printf '%s\n' "$REPLY"
fi
done
[[ $target ]] && grep -F -e "$target"
If you want to manually specify the filter target, this can be done by having the loop check for a file being created with filter contents, and using that when starting up grep afterwards.
That is a little bit strange what you need, but you can do it tis way:
you must go into script session first;
then you use shell how usually;
then you start and interrupt you program;
then run grep over typescript file.
Example:
$ script
$ ./a.out
Ctrl-Z
$ fg
$ grep NowIKnowThisString typescript
You could use a stream editor such as sed instead of grep. Here's an example of what I mean:
$ cat list
Name to look for: Mike
Dora 1
John 2
Mike 3
Helen 4
Here we find the name to look for in the fist line and want to grep for it. Now piping the command to sed:
$ cat list | sed -ne '1{s/Name to look for: //;h}' \
> -e ':r;n;G;/^.*\(.\+\).*\n\1$/P;s/\n.*//;br'
Mike 3
Note: sed itself can take file as a parameter, but you're not working with text files, so that's how you'd use it.
Of course, you'd need to modify the command for your case.
I want to write several lines (5 or more) to a file I'm going to create in script. I can do this by echo >> filename. But I would like to know what the best way to do this?
You can use a here document:
cat <<EOF >> outputfile
some lines
of text
EOF
I usually use the so-called "here-document" Dennis suggested. An alternative is:
(echo first line; echo second line) >> outputfile
This should have comparable performance in bash, as (....) starts a subshell, but echo is 'inlined' - bash does not run /bin/echo, but does the echo by itself.
It might even be faster because it involves no exec().
This style is even more useful if you want to use output from another command somewhere in the text.