Pipe output to bash function - linux

I have as simple function in a bash script and I would like to pipe stdout to it as an input.
jc_hms(){
printf "$1"
}
I'd like to use it in this manner.
var=`echo "teststring" | jc_hms`
Of course I used redundant functions echo and printf to simplify the question, but you get the idea. Right now I get a "not found" error, which I assume means my parameter delimiting is wrong (the "$1" part). Any suggestions?
Originally the jc_hms function was used like this:
echo `jc_hms "teststring"` > //dev/tts/0
but I'd like to store the results in a variable for further processing first, before sending it to the serial port.
EDIT:
So to clarify, I am NOT trying to print stuff to the serial port, I'd like to interface to my bash functions should the "|" pipe character, and I am wondering if this is possible.
EDIT: Alright, here's the full function.
jc_hms(){
hr=$(($1 / 3600))
min=$(($1 / 60))
sec=$(($1 % 60))
printf "$hs:%02d:%02d" $min $sec
}
I'm using the function to form a string which come this line of code
songplaytime=`echo $songtime | awk '{print S1 }'`
printstring="`jc_hms $songplaytime`" #store resulting string in printstring
Where $songtime is a string expressed as "playtime totaltime" delimited by a space.
I wish I can just do this in one line, and pipe it after the awk
printstring=`echo $songtime | awk '{print S1 }' | jc_hms`
like so.

To answer your actual question, when a shell function is on the receiving end of a pipe, standard input is inherited by all commands in the function, but only commands that actually read form their standard input consume any data. For commands that run one after the other, later commands can only see what isn't consumed by previous commands. When two commands run in parallel, which commands see which data depends on how the OS schedules the commands.
Since printf is the first and only command in your function, standard input is effectively ignored. There are several ways around that, including using the read built-in to read standard input into a variable which can be passed to printf:
jc_hms () {
read foo
hr=$(($foo / 3600))
min=$(($foo / 60))
sec=$(($foo % 60))
printf "%d:%02d:%02d" "$hr" "$min" "$sec"
}
However, since your need for a pipeline seems to depend on your perceived need to use awk, let me suggest the following alternative:
printstring=$( jc_hms $songtime )
Since songtime consists of a space-separated pair of numbers, the shell performs word-splitting on the value of songtime, and jc_hms sees two separate parameters. This requires no change in the definition of jc_hms, and no need to pipe anything into it via standard input.
If you still have a different reason for jc_hms to read standard input, please let us know.

You can't pipe stuff directly to a bash function like that, however you can use read to pull it in instead:
jc_hms() {
while read -r data; do
printf "%s" "$data"
done
}
should be what you want

1) I know this is a pretty old post
2) I like most of the answers here
However, I found this post because I needed to something similar. While everyone agrees stdin is what needs to be used, what the answers here are missing is the actual usage of the /dev/stdin file.
Using the read builtin forces this function to be used with piped input, so it can no longer be used in a typical way. I think utilizing /dev/stdin is a superior way of solving this problem, so I wanted to add my 2 cents for completeness.
My solution:
jc_hms() {
declare -i i=${1:-$(</dev/stdin)};
declare hr=$(($i/3600)) min=$(($i/60%60)) sec=$(($i%60));
printf "%02d:%02d:%02d\n" $hr $min $sec;
}
In action:
user#hostname:pwd$ jc_hms 7800
02:10:00
user#hostname:pwd$ echo 7800 | jc_hms
02:10:00
I hope this may help someone.
Happy hacking!

Or, you can also do it in a simple way.
jc_hms() {
cat
}
Though all answers so far have disregarded the fact that this was not what OP wanted (he stated the function is simplified)

I like user.friendly's answer using the Bash built-in conditional unset substitution syntax.
Here's a slight tweak to make his answer more generic, such as for cases with an indeterminate parameter count:
function myfunc() {
declare MY_INPUT=${*:-$(</dev/stdin)}
for PARAM in $MY_INPUT; do
# do what needs to be done on each input value
done
}

Hmmmm....
songplaytime=`echo $songtime | awk '{print S1 }'`
printstring="`jc_hms $songplaytime`" #store resulting string in printstring
if you're calling awk anyway, why not use it?
printstring=`TZ=UTC gawk -vT=$songplaytime 'BEGIN{print strftime("%T",T)}'`
I'm assuming you're using Gnu's Awk, which is the best one and also free; this will work in common linux distros which aren't necessarily using the most recent gawk. The most recent versions of gawk will let you specify UTC as a third parameter to the strftime() function.

The proposed solutions require content on stdin or read to be only conditionally called. Otherwise the function will wait for content from the console and require an Enter or Ctrl+D before continuing.
A workaround is to use read with a timeout. e.g. read -t <seconds>
function test ()
{
# ...
# process any parameters
# ...
read -t 0.001 piped
if [[ "${piped:-}" ]]; then
echo $piped
fi
}
Note, -t 0 did not work for me.
You might have to use a different value for the time-out.
Too small a value might result in bugs and a too large time-out delays the script.

seems nothing works, but there are work arounds
mentioned work around xargs ref function
$ FUNCS=$(functions hi); seq 3 | xargs -I{} zsh -c "eval $FUNCS; hi {}"
then this doesn't work either because your function could reference another function. so I ended up writing some function that accepts pipe inputs, like this:
somefunc() {
while read -r data; do
printf "%s" "$data"
done
}

Related

How to use awk for filtering(perl automation)

This is my txt file
type=0
vcpu_count=10
maste=0
h=0
p=0
memory=23.59
num=2
I want to get the vcpu_count and memory values and store it in some array through perl(automating script) .
awk -F'=' '/vcpu_count/{printf "\n",$1}' .vmConfig.txt
i am using this command just to test on terminal.but am getting a blank line. How do i do it. I need to get these two values and check for condition
If you are using Perl anyway, just use Perl for this too.
my %array;
open ($config, "<", ".vmConfig.txt") or die "$0: Could not open .vmConfig.txt: $!\n";
while (<$config>) {
next unless /^\s*(vcpu_count|memory)\s*=\s*(.*?)\s*\n/;
$array{$1} = $2;
}
close($config);
If you don't want the result to be an associative array (aka hash), refactoring should be relatively easy.
Following awk may help you on same.
Solution 1st:
awk '/vcpu_count/{print;next} /memory/{print}' Input_file
Output will be as follows:
vcpu_count=10
memory=23.59
Solution 2nd:
In case you want to print the values on a single line using printf then following may help you on same:
awk '/vcpu_count/{val=$0;next} /memory/{printf("%s AND %s\n",val,$0)}' Input_file
Output will be as follows:
vcpu_count=10 AND memory=23.59
when you use awk -F'=' '/vcpu_count/{printf "\n",$1}' .vmConfig.txt there are a couple of mistakes. Firstly, printf "\n" will only ever print a new line, as you have found. You need to add a format specifier - something like printf "%s\n", $2 will treat field 2 as a string and add it into the printed string. Checking out man printf at the command line will explain a bit more,.
Secondly, as I changed there, when you used $1 you were using the first field, which is the key in this case (while $0 is the whole line.)
Triplees solution is probably the most appropriate, but if there is a particular reason to start awk to perform this before perl, the following may help.
As you have done, it splits on =, but then outputs as csv, which you can change as appropriate. Even if input lines are not always in same order, will output in predictable order on single line
awk 'BEGIN {
FS="=";
OFS="," # tabs, etc if wanted, delete for spaces.
}
/vcpu_count/ {cpu=$2}
/memory/ {mem=$2}
END { print cpu, mem }'
This gives
10,23.59

using awk and bash for monitoring exec output to log

I am looking for some help with awk and bash commands,
my project have an embedded (so very limited) hardware,
i need to run a specific command called "digitalio show"
the command output is:
Input=0x50ff <-- last char only change
Output=0x7f
OR
Input=0x50fd <-- last char only change
Output=0x7f
i need to extract the input parameter and convert it into either Active or Passive and log them to a file with timestamp.
the log file should look like this:
YYMMDDhhmmss;Active
YYMMDDhhmmss;Passive
YYMMDDhhmmss;Active
YYMMDDhhmmss;Passive
while logging only changes
The command "digitalio show" is an embedded specific command that give the I/O state at the time of the execution, so i basically need to log every change in the I/O into a file using a minimal tools i have in the embedded H/W.
i can run the command for every 500msec, but if i will log all the outputs i can finish the flash very quickly, so i need only log changes.
in the end this will run as a background deamon.
Thanks !
Rotem.
As far as I understand, a single run of digitalio show command outputs two lines in the following format:
Input=HEX_NUMBER
Output=0x7f
where HEX_NUMBER is either 0x50ff, or 0x50fd. Suppose, the former stands for "Active", the latter for "Passive".
Running the command once per 500 milliseconds requires keeping the state. The most obvious implementation is a loop with a sleep.
However, sleep implementations vary. Some of them support a floating point argument (fractional seconds), and some don't. For example, the GNU implementation accepts arbitrary floating point numbers, but the standard UNIX implementation guarantees to suspend execution for at least the integral number of seconds. There are many alternatives, though. For instance, usleep from killproc accepts microseconds. Alternatively, you can write your own utility.
Let's pick the usleep command. Then the Bash script may look like the following:
#!/bin/bash -
last_state=
while true ; do
i=$(digitalio show | awk -F= '/Input=0x[a-zA-Z0-9]+/ {print $2}')
if test "$i" = "0x50ff" ; then
state="Active"
else
state="Passive"
fi
if test "$state" != "$last_state" ; then
printf '%s;%s\n' $(date '+%Y%m%d%H%M%S') "$state"
fi
last_state="$state"
usleep 500000
done
Sample output
20161019103534;Active
20161019103555;Passive
The script launches digitalio show command in an infinite loop, then extracts the hex part from Input lines with awk.
The $state variable is assigned to whether "Active", or "Passive" depending on the value of hex string.
The $last_state variable keeps the value of $state in the last iteration. If $state is not equal to $last_state, then the state is printed to the standard output in the specific format.

Store for loop results as a variable in bash

I have a loop similar to the following:
for time in ${seconds_list}; do
echo "scale=2; (${cur_time}-${time})/3600" | bc
done
Of course, I could "echo" the results to a file and be done with it, but I think a more elegant approach would be to store all the for-loop results in one variable, that I could use at a later time.
The variable containing all results would have to look something like this:
var='30.25
16.15
64.40
29.80'
Is there an easy way in which I can achieve this?
It's really easy, you can just redirect the output of the whole loop to a variable (if you want to use just one variable as stated):
VARIABLE=$(for time in ...; do ...; done)
your example:
var=$(for time in ${seconds_list}; do
echo "scale=2; (${cur_time}-${time})/3600" | bc
done)
Just enclosing your code into $().
Better to use a BASH array to store your results:
results=()
for time in ${seconds_list}; do
results+=($(bc -l <<< "scale=2; ($cur_time-$time)/3600"))
done
# print the results:
printf "%s\n" "${results[#]}"

returning values in a bash function

I'm working with a growing bash script and within this script I have a number of functions. One of these functions is supposed to return a variables value, but I am running into some issues with the syntax. Below is an example of the code.
ShowTags() {
local tag=0
read tag
echo "$tag"
}
selected_tag=$(ShowTags)
echo "$selected_tag"
pulled this code from a Linux Journal article, but the problem is it doesn't seem to work, or perhaps it does and im missing something. Essentially whenever the function is called the script hangs up and does not output anything, I need to CTRL+C to drop back to CLI.
The article in question is below.
http://www.linuxjournal.com/content/return-values-bash-functions
So my question is this the proper way to return a value? Is there a better or more dependable way of doing this? And if there is please give me an example so I can figure this out without using global variables.
EDIT:
The behavior of this is really getting to me now. I am using the following script.
ShowTags() {
echo "hi"
local tag=0
read tag
echo "$tag"
}
selected_tag=$(ShowTags)
echo "$selected_tag
Basically what happens is bash will act as if the read command is taking place before the echo tag at the top of the function. As soon as I pass something to read though it will run the top echo, and complete the rest of the script. I am not sure why this is happening. This is exactly what is happening in my main script.
Change echo "hi" to echo "hi" >/dev/tty.
The reason you're not seeing it immediately is that $(ShowTags) captures all the standard output of the function, and that gets assigned to selected_tag. So you don't see any of it until you echo that variable.
By redirecting the prompt to /dev/tty, it's always displayed immediately on the terminal, not sent to the function's stdout, so it doesn't get captured by the command substitution.
You are trying to define a function with Name { ... ]. You have to use name() { ... }:
ShowTags() { # add ()
local tag=0
read tag
echo "$tag"
} # End with }
selected_tag=$(ShowTags)
echo "$selected_tag"
It now lets the user type in a string and have it written back:
$ bash myscript
hello world # <- my input
hello world # script's output
You can add a prompt with read -p "Enter tag: " tag to make it more obvious when to write your input.
As #thatotherguy pointed out, your function declaration syntax is off; but I suspect that's a transcription error, as if it was wrong in the script you'd get different problems. I think what's going on is that the read tag command in the function is trying to read a value from standard input (by default that's the terminal), and pausing until you type something in. I'm not sure what it's intended to do, but as written I'd expect it to pause indefinitely until something's typed in.
Solution: either type something in, or use something other than read. You could also add a prompt (read -p "Enter a tag: " tag) to make it more clear what's going on.
BTW, I have a couple of objections to the linux journal article you linked. These aren't relevant to your script, but things you should be aware of.
First, the function keyword is a nonstandard bashism, and I recommend against using it. myfunc() ... is sufficient to introduce a function definition.
Second, and more serious, the article recommends using eval in an unsafe way. Actually, it's really hard to use eval safely (see BashFAQ #48). You can improve it a great deal just by changing the quoting, and even more by not using eval at all:
eval $__resultvar="'$myresult'" # BAD, can evaluate parts of $myresult as executable code
eval $__resultvar='"$myresult"' # better, is only vulnerable to executing $__resultvar
declare $__resultvar="$myresult" # better still
See BashFAQ #6 for more options and discussion.

Bash script key/value pair regardless of bash version

I am writing a curl bash script to test webservices. I will have file_1 which would contain the URL paths
/path/to/url/1/{dynamic_path}.xml
/path/to/url/2/list.xml?{query_param}
Since the values in between {} is dynamic, I am creating a separate file, which will have values for these params. the input would be in key-value pair i.e.,
dynamic_path=123
query_param=shipment
By combining two files, the input should become
/path/to/url/1/123.xml
/path/to/url/2/list.xml?shipment
This is the background of my problem. Now my questions
I am doing it in bash script, and the approach I am using is first reading the file with parameters and parse it based on '=' and store it in key/value pair. so it will be easy to replace i.e., for each url I will find the substring between {} and whatever the text it comes with, I will use it as the key to fetch the value from the array
My approach sounds okay (at least to me) BUT, I just realized that
declare -A input_map is only supported in bashscript higher than 4.0. Now, I am not 100% sure what would be the target environment for my script, since it could run in multiple department.
Is there anything better you could suggest ? Any other approach ? Any other design ?
P.S:
This is the first time i am working on bash script.
Here's a risky way to do it: Assuming the values are in a file named "values"
. values
eval "$( sed 's/^/echo "/; s/{/${/; s/$/"/' file_1 )"
Basically, stick a dollar sign in front of the braces and transform each line into an echo statement.
More effort, with awk:
awk '
NR==FNR {split($0, a, /=/); v[a[1]]=a[2]; next}
(i=index($0, "{")) && (j=index($0,"}")) {
key=substr($0,i+1, j-i-1)
print substr($0, 1, i-1) v[key] substr($0, j+1)
}
' values file_1
There are many ways to do this. You seem to think of putting all inputs in a hashmap, and then iterate over that hashmap. In shell scripting it's more common and practical to process things as a stream using pipelines.
For example, your inputs could be in a csv file:
123,shipment
345,order
Then you could process this file like this:
while IFS=, read path param; do
sed -e "s/{dynamic_path}/$path/" -e "s/{query_param}/$param/" file_1
done < input.csv
The output will be:
/path/to/url/1/123.xml
/path/to/url/2/list.xml?shipment
/path/to/url/1/345.xml
/path/to/url/2/list.xml?order
But this is just an example, there can be so many other ways.
You should definitely start by writing a proof of concept and test it on your deployment server. This example should work in old versions of bash too.

Resources