How can I get the current network interface throughput statistics on Linux/UNIX? [closed] - linux

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 8 years ago.
Improve this question
Tools such as MRTG provide network throughput / bandwidth graphs for the current network utilisation on specific interfaces, such as eth0. How can I return that information at the command line on Linux/UNIX?
Preferably this would be without installing anything other than what is available on the system as standard.

iftop does for network usage what top(1) does for CPU usage -- http://www.ex-parrot.com/~pdw/iftop/
I don't know how "standard" iftop is, but I was able to install it with yum install iftop on Fedora.

Got sar? Likely yes if youre using RHEL/CentOS.
No need for priv, dorky binaries, hacky scripts, libpcap, etc. Win.
$ sar -n DEV 1 3
Linux 2.6.18-194.el5 (localhost.localdomain) 10/27/2010
02:40:56 PM IFACE rxpck/s txpck/s rxbyt/s txbyt/s rxcmp/s txcmp/s rxmcst/s
02:40:57 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00
02:40:57 PM eth0 10700.00 1705.05 15860765.66 124250.51 0.00 0.00 0.00
02:40:57 PM eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00
02:40:57 PM IFACE rxpck/s txpck/s rxbyt/s txbyt/s rxcmp/s txcmp/s rxmcst/s
02:40:58 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00
02:40:58 PM eth0 8051.00 1438.00 11849206.00 105356.00 0.00 0.00 0.00
02:40:58 PM eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00
02:40:58 PM IFACE rxpck/s txpck/s rxbyt/s txbyt/s rxcmp/s txcmp/s rxmcst/s
02:40:59 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00
02:40:59 PM eth0 6093.00 1135.00 8970988.00 82942.00 0.00 0.00 0.00
02:40:59 PM eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average: IFACE rxpck/s txpck/s rxbyt/s txbyt/s rxcmp/s txcmp/s rxmcst/s
Average: lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average: eth0 8273.24 1425.08 12214833.44 104115.72 0.00 0.00 0.00
Average: eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00

I wrote this dumb script a long time ago, it depends on nothing but Perl and Linux≥2.6:
#!/usr/bin/perl
use strict;
use warnings;
use POSIX qw(strftime);
use Time::HiRes qw(gettimeofday usleep);
my $dev = #ARGV ? shift : 'eth0';
my $dir = "/sys/class/net/$dev/statistics";
my %stats = do {
opendir +(my $dh), $dir;
local #_ = readdir $dh;
closedir $dh;
map +($_, []), grep !/^\.\.?$/, #_;
};
if (-t STDOUT) {
while (1) {
print "\033[H\033[J", run();
my ($time, $us) = gettimeofday();
my ($sec, $min, $hour) = localtime $time;
{
local $| = 1;
printf '%-31.31s: %02d:%02d:%02d.%06d%8s%8s%8s%8s',
$dev, $hour, $min, $sec, $us, qw(1s 5s 15s 60s)
}
usleep($us ? 1000000 - $us : 1000000);
}
}
else {print run()}
sub run {
map {
chomp (my ($stat) = slurp("$dir/$_"));
my $line = sprintf '%-31.31s:%16.16s', $_, $stat;
$line .= sprintf '%8.8s', int (($stat - $stats{$_}->[0]) / 1)
if #{$stats{$_}} > 0;
$line .= sprintf '%8.8s', int (($stat - $stats{$_}->[4]) / 5)
if #{$stats{$_}} > 4;
$line .= sprintf '%8.8s', int (($stat - $stats{$_}->[14]) / 15)
if #{$stats{$_}} > 14;
$line .= sprintf '%8.8s', int (($stat - $stats{$_}->[59]) / 60)
if #{$stats{$_}} > 59;
unshift #{$stats{$_}}, $stat;
pop #{$stats{$_}} if #{$stats{$_}} > 60;
"$line\n";
} sort keys %stats;
}
sub slurp {
local #ARGV = #_;
local #_ = <>;
#_;
}
It just reads from /sys/class/net/$dev/statistics every second, and prints out the current numbers and the average rate of change:
$ ./net_stats.pl eth0
rx_bytes : 74457040115259 4369093 4797875 4206554 364088
rx_packets : 91215713193 23120 23502 23234 17616
...
tx_bytes : 90798990376725 8117924 7047762 7472650 319330
tx_packets : 93139479736 23401 22953 23216 23171
...
eth0 : 15:22:09.002216 1s 5s 15s 60s
^ current reading ^-------- averages ---------^

nload is a great tool for monitoring bandwidth in real time and easily installed in Ubuntu or Debian with sudo apt-get install nload.
Device eth0 [10.10.10.5] (1/2):
=====================================================================================
Incoming:
. ...|
# ####|
.. |#| ... #####. .. Curr: 2.07 MBit/s
###.### #### #######|. . ## | Avg: 1.41 MBit/s
########|#########################. ### Min: 1.12 kBit/s
........ ################################### .### Max: 4.49 MBit/s
.##########. |###################################|##### Ttl: 1.94 GByte
Outgoing:
########## ########### ###########################
########## ########### ###########################
##########. ########### .###########################
########### ########### #############################
########### ###########..#############################
############ ##########################################
############ ##########################################
############ ########################################## Curr: 63.88 MBit/s
############ ########################################## Avg: 32.04 MBit/s
############ ########################################## Min: 0.00 Bit/s
############ ########################################## Max: 93.23 MBit/s
############## ########################################## Ttl: 2.49 GByte
Another excellent tool is iftop, also easily apt-get'able:
191Mb 381Mb 572Mb 763Mb 954Mb
└────────────┴──────────┴─────────────────────┴───────────┴──────────────────────
box4.local => box-2.local 91.0Mb 27.0Mb 15.1Mb
<= 1.59Mb 761kb 452kb
box4.local => box.local 560b 26.8kb 27.7kb
<= 880b 31.3kb 32.1kb
box4.local => userify.com 0b 11.4kb 8.01kb
<= 1.17kb 2.39kb 1.75kb
box4.local => b.resolvers.Level3.net 0b 58b 168b
<= 0b 83b 288b
box4.local => stackoverflow.com 0b 42b 21b
<= 0b 42b 21b
box4.local => 224.0.0.251 0b 0b 179b
<= 0b 0b 0b
224.0.0.251 => box-2.local 0b 0b 0b
<= 0b 0b 36b
224.0.0.251 => box.local 0b 0b 0b
<= 0b 0b 35b
─────────────────────────────────────────────────────────────────────────────────
TX: cum: 37.9MB peak: 91.0Mb rates: 91.0Mb 27.1Mb 15.2Mb
RX: 1.19MB 1.89Mb 1.59Mb 795kb 486kb
TOTAL: 39.1MB 92.6Mb 92.6Mb 27.9Mb 15.6Mb
Don't forget about the classic and powerful sar and netstat utilities on older *nix!

You could parse /proc/net/dev.

dstat - Combines vmstat, iostat, ifstat, netstat information and more
iftop - Amazing network bandwidth utility to analyse what is really happening on your eth
netio - Measures the net throughput of a network via TCP/IP
inq - CLI troubleshooting utility that displays info on storage, typically Symmetrix. By default, INQ returns the device name, Symmetrix ID, Symmetrix LUN, and capacity.
send_arp - Sends out an arp broadcast on the specified network device (defaults to eth0), reporting an old and new IP address mapping to a MAC address.
EtherApe - is a graphical network monitor for Unix modeled after etherman. Featuring link layer, IP and TCP modes, it displays network activity graphically.
iptraf - An IP traffic monitor that shows information on the IP traffic passing over your network.
More details:
http://felipeferreira.net/?p=1194

You can parse the output of ifconfig

I got another quick'n'dirty bash script for that:
#!/bin/bash
IF=$1
if [ -z "$IF" ]; then
IF=`ls -1 /sys/class/net/ | head -1`
fi
RXPREV=-1
TXPREV=-1
echo "Listening $IF..."
while [ 1 == 1 ] ; do
RX=`cat /sys/class/net/${IF}/statistics/rx_bytes`
TX=`cat /sys/class/net/${IF}/statistics/tx_bytes`
if [ $RXPREV -ne -1 ] ; then
let BWRX=$RX-$RXPREV
let BWTX=$TX-$TXPREV
echo "Received: $BWRX B/s Sent: $BWTX B/s"
fi
RXPREV=$RX
TXPREV=$TX
sleep 1
done
It's considering that sleep 1 will actually last exactly one second, which is not true, but good enough for a rough bandwidth assessment.
Thanks to #ephemient for the /sys/class/net/<interface>! :)

Besides iftop and iptraf, also check:
bwm-ng (Bandwidth Monitor Next Generation)
and/or
cbm (Color Bandwidth Meter)
ref: http://www.powercram.com/2010/01/bandwidth-monitoring-tools-for-ubuntu.html

If you want just to get the value, you can use simple shell oneliner like this:
S=10; F=/sys/class/net/eth0/statistics/rx_bytes; X=`cat $F`; sleep $S; Y=`cat $F`; BPS="$(((Y-X)/S))"; echo $BPS
It will show you the average "received bytes per second" for period of 10 seconds (you can change period by changing S=10 parameter, and you can measure transmitted BPS instead of received BPS by using tx_bytes instead of rx_bytes). Don't forget to change eth0 to network device you want to monitor.
Of course, you are not limited to displaying the average rate (as mentioned in other answers, there are other tools that will show you much nicer output), but this solution is easily scriptable to do other things.
For example, the following shell script (split into multiple lines for readability) will execute offlineimap process only when 5-minute average transmit speed drops below 10kBPS (presumably, when some other bandwidth-consuming process finishes):
#!/bin/sh
S=300; F=/sys/class/net/eth0/statistics/tx_bytes
BPS=999999
while [ $BPS -gt 10000 ]
do
X=`cat $F`; sleep $S; Y=`cat $F`; BPS="$(((Y-X)/S))";
echo BPS is currently $BPS
done
offlineimap
Note that /sys/class/... is Linux specific (which is ok as submitter did choose linux tag), and needs non-archaic kernel. Shell code itself is /bin/sh compatible (so not only bash, but dash and other /bin/sh implementations will work) and /bin/sh is something that is really always installed.

I like iptraf but you probably have to install it and it seems to not being maintained actively anymore.

I find dstat to be quite good. Has to be installed though. Gives you way more information than you need. Netstat will give you packet rates but not bandwith also. netstat -s

You can use iperf to benchmark network performance (maximum possible throughput).
See following links for details:
http://en.wikipedia.org/wiki/Iperf
https://iperf.fr/
https://code.google.com/p/iperf/

I couldn't get the parse ifconfig script to work for me on an AMI so got this to work measuring received traffic averaged over 10 seconds
date && rxstart=`ifconfig eth0 | grep bytes | awk '{print $2}' | cut -d : -f 2` && sleep 10 && rxend=`ifconfig eth0 | grep bytes | awk '{print $2}' | cut -d : -f 2` && difference=`expr $rxend - $rxstart` && echo "Received `expr $difference / 10` bytes per sec"
Sorry, it's ever so cheap and nasty but it worked!

ifconfig -a
ip -d link
ls -l /sys/class/net/ (physical and virtual devices)
route -n
If you want the output of (ifconfig -a) in json format you can use this (python)

Related

Bash alias cpu usage

I've tried this command but I have a percentage error calculator:
alias cpu="mpstat | awk '\$12 ~ /[0-9.]+/ { print 100 - $12\"%\" }'"
Thank you for help
Change it to this:
alias cpu="mpstat | awk '\$12 ~ /[0-9.]+/ { print 100 - \$12\"%\" }'"
\ was missing after 100 -.
-> mpstat
Linux 3.2.0-69-virtual (myhost) 01/06/2017 _x86_64_ (8 CPU)
10:18:16 PM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %idle
10:18:16 PM all 12.06 7.00 7.96 0.02 0.00 0.24 0.22 0.00 72.49
-> cpu
27.51%

perf : How to check processess running on particular cpu

Is there any option in perf to look into processes running on a particular cpu /core, and how much percentage of that core is taken by each process.
Reference links would be helpful.
perf is intended to do a profiling which is not good fit for your case. You may try to do sampling /proc/sched_debug (if it is compiled in your kernel). For example you may check which process is currently running on CPU:
egrep '^R|cpu#' /proc/sched_debug
cpu#0, 917.276 MHz
R egrep 2614 37730.177313 ...
cpu#1, 917.276 MHz
R bash 2023 218715.010833 ...
By using his PID as a key, you may check how many CPU time in milliseconds it consumed:
grep se.sum_exec_runtime /proc/2023/sched
se.sum_exec_runtime : 279346.058986
However, as #BrenoLeitão mentioned, SystemTap is quite useful for your script. Here is script for your task.
global cputimes;
global cmdline;
global oncpu;
global NS_PER_SEC = 1000000000;
probe scheduler.cpu_on {
oncpu[pid()] = local_clock_ns();
}
probe scheduler.cpu_off {
if(oncpu[pid()] == 0)
next;
cmdline[pid()] = cmdline_str();
cputimes[pid(), cpu()] <<< local_clock_ns() - oncpu[pid()];
delete oncpu[pid()];
}
probe timer.s(1) {
printf("%6s %3s %6s %s\n", "PID", "CPU", "PCT", "CMDLINE");
foreach([pid+, cpu] in cputimes) {
cpupct = #sum(cputimes[pid, cpu]) * 10000 / NS_PER_SEC;
printf("%6d %3d %3d.%02d %s\n", pid, cpu,
cpupct / 100, cpupct % 100, cmdline[pid]);
}
delete cputimes;
}
It traces moments when process is running on CPU and stops execution on that (due to migration or sleeping) by attaching to scheduler.cpu_on and scheduler.cpu_off probes. Second probe calculates time difference between these events and saves it to cputimes aggregation along with process command line arguments.
timer.s(1) fires once per second -- it walks over aggregation and calculates percentage. Here is sample output for Centos 7 with bash running infinite loop:
0 0 100.16
30 1 0.00
51 0 0.00
380 0 0.02 /usr/bin/python -Es /usr/sbin/tuned -l -P
2016 0 0.08 sshd: root#pts/0 "" "" "" ""
2023 1 100.11 -bash
2630 0 0.04 /usr/libexec/systemtap/stapio -R stap_3020c9e7ba76838179be68cd2390a10c_2630 -F3
I understand that perf is not the proper way to do it, although you can limit perf per CPU, as using perf record -C <cpulist> or even perf stat -c <cpulist>.
The close you are going to see is the context-switch event, but, this is not going to provide you the application names at all.
I think you are going to need something more powerful, as systemtap.

time command output on an already running process

I have a process that spawns some other processes,
I want to use the time command on a specific process and get the same output as the time command.
Is that possible and how?
I want to use the time command on a specific process and get the same output as the time command.
Probably it is enough just to use pidstat to get user and sys time:
$ pidstat -p 30122 1 4
Linux 2.6.32-431.el6.x86_64 (hostname) 05/15/2014 _x86_64_ (8 CPU)
04:42:28 PM PID %usr %system %guest %CPU CPU Command
04:42:29 PM 30122 706.00 16.00 0.00 722.00 3 has_serverd
04:42:30 PM 30122 714.00 12.00 0.00 726.00 3 has_serverd
04:42:31 PM 30122 714.00 14.00 0.00 728.00 3 has_serverd
04:42:32 PM 30122 708.00 16.00 0.00 724.00 3 has_serverd
Average: 30122 710.50 14.50 0.00 725.00 - has_serverd
If not then according to strace time uses wait4 system call (http://linux.die.net/man/2/wait4) to get information about a process from the kernel. The same info returns getrusage but you cannot call it for an arbitrary process according to its documentation (http://linux.die.net/man/2/getrusage).
So, I do not know any command that will give the same output. However it is feasible to create a bash script that gets PID of the specific process and outputs something like time outpus then
This script does these steps:
1) Get the number of clock ticks per second
getconf CLK_TCK
I assume it is 100 and 1 tick is equal to 10 milliseconds.
2) Then in loop do the same sequence of commands while exists the directory /proc/YOUR-PID:
while [ -e "/proc/YOUR-PID" ];
do
read USER_TIME SYS_TIME REAL_TIME <<< $(cat /proc/PID/stat | awk '{print $14, $15, $22;}')
sleep 0.1
end loop
Some explanation - according to man proc :
user time: ($14) - utime - Amount of time that this process has been scheduled in user mode, measured in clock ticks
sys time: ($15) - stime - Amount of time that this process has been scheduled in kernel mode, measured in clock ticks
starttime ($22) - The time in jiffies the process started after system boot.
3) When the process is finished get finish time
read FINISH_TIME <<< $(cat '/proc/self/stat' | awk '{print $22;}')
And then output:
the real time = ($FINISH_TIME-$REAL_TIME) * 10 - in milliseconds
user time: ($USER_TIME/(getconf CLK_TCK)) * 10 - in milliseconds
sys time: ($SYS_TIME/(getconf CLK_TCK)) * 10 - in milliseconds
I think it should give roughly the same result as time. One possible problem I see is if the process exists for a very short period of time.
This is my implementation of time:
#!/bin/bash
# Uses herestrings
print_res_jeffies()
{
let "TIME_M=$2/60000"
let "TIME_S=($2-$TIME_M*60000)/1000"
let "TIME_MS=$2-$TIME_M*60000-$TIME_S*1000"
printf "%s\t%dm%d.%03dms\n" $1 $TIME_M $TIME_S $TIME_MS
}
print_res_ticks()
{
let "TIME_M=$2/6000"
let "TIME_S=($2-$TIME_M*6000)/100"
let "TIME_MS=($2-$TIME_M*6000-$TIME_S*100)*10"
printf "%s\t%dm%d.%03dms\n" $1 $TIME_M $TIME_S $TIME_MS
}
if [ $(getconf CLK_TCK) != 100 ]; then
exit 1;
fi
if [ $# != 1 ]; then
exit 1;
fi
PROC_DIR="/proc/"$1
if [ ! -e $PROC_DIR ]; then
exit 1
fi
USER_TIME=0
SYS_TIME=0
START_TIME=0
while [ -e $PROC_DIR ]; do
read TMP_USER_TIME TMP_SYS_TIME TMP_START_TIME <<< $(cat $PROC_DIR/stat | awk '{print $14, $15, $22;}')
if [ -e $PROC_DIR ]; then
USER_TIME=$TMP_USER_TIME
SYS_TIME=$TMP_SYS_TIME
START_TIME=$TMP_START_TIME
sleep 0.1
else
break
fi
done
read FINISH_TIME <<< $(cat '/proc/self/stat' | awk '{print $22;}')
let "REAL_TIME=($FINISH_TIME - $START_TIME)*10"
print_res_jeffies 'real' $REAL_TIME
print_res_ticks 'user' $USER_TIME
print_res_ticks 'sys' $SYS_TIME
And this is an example that compares my implementation of time and real time:
>time ./sys_intensive > /dev/null
Alarm clock
real 0m10.004s
user 0m9.883s
sys 0m0.034s
In another terminal window I run my_time.sh and give it PID:
>./my_time.sh `pidof sys_intensive`
real 0m10.010ms
user 0m9.780ms
sys 0m0.030ms

How to find user memory usage in linux

How i can see memory usage by user in linux centos 6
For example:
USER USAGE
root 40370
admin 247372
user2 30570
user3 967373
This one-liner worked for me on at least four different Linux systems with different distros and versions. It also worked on FreeBSD 10.
ps hax -o rss,user | awk '{a[$2]+=$1;}END{for(i in a)print i" "int(a[i]/1024+0.5);}' | sort -rnk2
About the implementation, there are no shell loop constructs here; this uses an associative array in awk to do the grouping & summation.
Here's sample output from one of my servers that is running a decent sized MySQL, Tomcat, and Apache. Figures are in MB.
mysql 1566
joshua 1186
tomcat 353
root 28
wwwrun 12
vbox 1
messagebus 1
avahi 1
statd 0
nagios 0
Caveat: like most similar solutions, this is only considering the resident set (RSS), so it doesn't count any shared memory segments.
EDIT: A more human-readable version.
echo "USER RSS PROCS" ; echo "-------------------- -------- -----" ; ps hax -o rss,user | awk '{rss[$2]+=$1;procs[$2]+=1;}END{for(user in rss) printf "%-20s %8.0f %5.0f\n", user, rss[user]/1024, procs[user];}' | sort -rnk2
And the output:
USER RSS PROCS
-------------------- -------- -----
mysql 1521 1
joshua 1120 28
tomcat 379 1
root 19 107
wwwrun 10 10
vbox 1 3
statd 1 1
nagios 1 1
messagebus 1 1
avahi 1 1
Per-user memory usage in percent using standard tools:
for _user in $(ps haux | awk '{print $1}' | sort -u)
do
ps haux | awk -v user=${_user} '$1 ~ user { sum += $4} END { print user, sum; }'
done
or for more precision:
TOTAL=$(free | awk '/Mem:/ { print $2 }')
for _user in $(ps haux | awk '{print $1}' | sort -u)
do
ps hux -U ${_user} | awk -v user=${_user} -v total=$TOTAL '{ sum += $6 } END { printf "%s %.2f\n", user, sum / total * 100; }'
done
The first version just sums up the memory percentage for each process as reported by ps. The second version sums up the memory in bytes instead and calculates the total percentage afterwards, thus leading to a higher precision.
If your system supports, try to install and use smem:
smem -u
User Count Swap USS PSS RSS
gdm 1 0 308 323 820
nobody 1 0 912 932 2240
root 76 0 969016 1010829 1347768
or
smem -u -t -k
User Count Swap USS PSS RSS
gdm 1 0 308.0K 323.0K 820.0K
nobody 1 0 892.0K 912.0K 2.2M
root 76 0 937.6M 978.5M 1.3G
ameskaas 46 0 1.2G 1.2G 1.5G
124 0 2.1G 2.2G 2.8G
In Ubuntu, smem can be installed by typing
sudo apt install smem
This will return the total ram usage by users in GBs, reverse sorted
sudo ps --no-headers -eo user,rss | awk '{arr[$1]+=$2}; END {for (i in arr) {print i,arr[i]/1024/1024}}' | sort -nk2 -r
You can use the following Python script to find per-user memory usage using only sys and os module.
import sys
import os
# Get list of all users present in the system
allUsers = os.popen('cut -d: -f1 /etc/passwd').read().split('\n')[:-1]
for users in allUsers:
# Check if the home directory exists for the user
if os.path.exists('/home/' + str(users)):
# Print the current usage of the user
print(os.system('du -sh /home/' + str(users)))

Best way to divide in bash using pipes?

I'm just looking for an easy way to divide a number (or provide other math functions). Let's say I have the following command:
find . -name '*.mp4' | wc -l
How can I take the result of wc -l and divide it by 3?
The examples I've seen don't deal with re-directed out/in.
Using bc:
$ bc -l <<< "scale=2;$(find . -name '*.mp4' | wc -l)/3"
2.33
In contrast, the bash shell only performs integer arithmetic.
Awk is also very powerful:
$ find . -name '*.mp4' | wc -l | awk '{print $1/3}'
2.33333
You don't even need wc if using awk:
$ find . -name '*.mp4' | awk 'END {print NR/3}'
2.33333
Edit 2018-02-22: Adding shell connector
There is more than 1 way:
Depending on precision required and number of calcul to be done! See shell connector further!
Using bc (binary calculator)
find . -type f -name '*.mp4' -printf \\n | wc -l | xargs printf "%d/3\n" | bc -l
6243.33333333333333333333
or
echo $(find . -name '*.mp4' -printf \\n | wc -l)/3|bc -l
6243.33333333333333333333
or using bash, result in integer only:
echo $(($(find . -name '*.mp4' -printf \\n| wc -l)/3))
6243
Using bash interger builtin math processor
res=000$((($(find . -type f -name '*.mp4' -printf "1+")0)*1000/3))
printf -v res "%.2f" ${res:0:${#res}-3}.${res:${#res}-3}
echo $res
6243.33
Pure bash
With recent 64bits bash, you could even use #glennjackman's ideas of using globstar, but computing pseudo floating could be done by:
shopt -s globstar
files=(**/*.mp4)
shopt -u globstar
res=$[${#files[*]}000/3]
printf -v res "%.2f" ${res:0:${#res}-3}.${res:${#res}-3}
echo $res
6243.33
There is no fork and $res contain a two digit rounded floating value.
Nota: Care about symlinks when using globstar and **!
Introducing shell connector
If you plan to do a lot of calculs, require high precision and use bash, you could use long running bc sub process:
mkfifo /tmp/mybcfifo
exec 5> >(exec bc -l >/tmp/mybcfifo)
exec 6</tmp/mybcfifo
rm /tmp/mybcfifo
then now:
echo >&5 '12/34'
read -u 6 result
echo $result
.35294117647058823529
This subprocess stay open and useable:
ps --sid $(ps ho sid $$) fw
PID TTY STAT TIME COMMAND
18027 pts/9 Ss 0:00 bash
18258 pts/9 S 0:00 \_ bc -l
18789 pts/9 R+ 0:00 \_ ps --sid 18027 fw
Computing $PI:
echo >&5 '4*a(1)'
read -u 6 PI
echo $PI
3.14159265358979323844
To terminate sub process:
exec 6<&-
exec 5>&-
Little demo, about The best way to divide in bash using pipes!
Computing range {1..157} / 42 ( I will let you google for answer to the ultimate question of life, the universe, and everything ;)
... and print 13 result by lines in order to reduce output:
printf -v form "%s" "%5.3f "{,}{,}{,,};form+="%5.3f\n";
By regular way
testBc(){
for ((i=1; i<157; i++)) ;do
echo $(bc -l <<<"$i/42");
done
}
By using long running bc sub process:
testLongBc(){
mkfifo /tmp/mybcfifo;
exec 5> >(exec bc -l >/tmp/mybcfifo);
exec 6< /tmp/mybcfifo;
rm /tmp/mybcfifo;
for ((i=1; i<157; i++)) ;do
echo "$i/42" 1>&5;
read -u 6 result;
echo $result;
done;
exec 6>&-;
exec 5>&-
}
Let's see without:
time printf "$form" $(testBc)
0.024 0.048 0.071 0.095 0.119 0.143 0.167 0.190 0.214 0.238 0.262 0.286 0.310
0.333 0.357 0.381 0.405 0.429 0.452 0.476 0.500 0.524 0.548 0.571 0.595 0.619
0.643 0.667 0.690 0.714 0.738 0.762 0.786 0.810 0.833 0.857 0.881 0.905 0.929
0.952 0.976 1.000 1.024 1.048 1.071 1.095 1.119 1.143 1.167 1.190 1.214 1.238
1.262 1.286 1.310 1.333 1.357 1.381 1.405 1.429 1.452 1.476 1.500 1.524 1.548
1.571 1.595 1.619 1.643 1.667 1.690 1.714 1.738 1.762 1.786 1.810 1.833 1.857
1.881 1.905 1.929 1.952 1.976 2.000 2.024 2.048 2.071 2.095 2.119 2.143 2.167
2.190 2.214 2.238 2.262 2.286 2.310 2.333 2.357 2.381 2.405 2.429 2.452 2.476
2.500 2.524 2.548 2.571 2.595 2.619 2.643 2.667 2.690 2.714 2.738 2.762 2.786
2.810 2.833 2.857 2.881 2.905 2.929 2.952 2.976 3.000 3.024 3.048 3.071 3.095
3.119 3.143 3.167 3.190 3.214 3.238 3.262 3.286 3.310 3.333 3.357 3.381 3.405
3.429 3.452 3.476 3.500 3.524 3.548 3.571 3.595 3.619 3.643 3.667 3.690 3.714
real 0m10.113s
user 0m0.900s
sys 0m1.290s
Wow! Ten seconds on my raspberry-pi!!
Then with:
time printf "$form" $(testLongBc)
0.024 0.048 0.071 0.095 0.119 0.143 0.167 0.190 0.214 0.238 0.262 0.286 0.310
0.333 0.357 0.381 0.405 0.429 0.452 0.476 0.500 0.524 0.548 0.571 0.595 0.619
0.643 0.667 0.690 0.714 0.738 0.762 0.786 0.810 0.833 0.857 0.881 0.905 0.929
0.952 0.976 1.000 1.024 1.048 1.071 1.095 1.119 1.143 1.167 1.190 1.214 1.238
1.262 1.286 1.310 1.333 1.357 1.381 1.405 1.429 1.452 1.476 1.500 1.524 1.548
1.571 1.595 1.619 1.643 1.667 1.690 1.714 1.738 1.762 1.786 1.810 1.833 1.857
1.881 1.905 1.929 1.952 1.976 2.000 2.024 2.048 2.071 2.095 2.119 2.143 2.167
2.190 2.214 2.238 2.262 2.286 2.310 2.333 2.357 2.381 2.405 2.429 2.452 2.476
2.500 2.524 2.548 2.571 2.595 2.619 2.643 2.667 2.690 2.714 2.738 2.762 2.786
2.810 2.833 2.857 2.881 2.905 2.929 2.952 2.976 3.000 3.024 3.048 3.071 3.095
3.119 3.143 3.167 3.190 3.214 3.238 3.262 3.286 3.310 3.333 3.357 3.381 3.405
3.429 3.452 3.476 3.500 3.524 3.548 3.571 3.595 3.619 3.643 3.667 3.690 3.714
real 0m0.670s
user 0m0.190s
sys 0m0.070s
Less than one second!!
Hopefully, results are same, but execution time is very different!
My shell connector
I've published a connector function: Connector-bash on GitHub.com
and shell_connector.sh on my own site.
source shell_connector.sh
newConnector /usr/bin/bc -l 0 0
myBc 1764/42 result
echo $result
42.00000000000000000000
find . -name '*.mp4' | wc -l | xargs -I{} expr {} / 2
Best used if you have multiple outputs you'd like to pipe through xargs. Use{} as a placeholder for the expression term.
Depending on your bash version, you don't even need find for this simple task:
shopt -s nullglob globstar
files=( **/*.mp4 )
dc -e "3 k ${#files[#]} 3 / p"
This method will correctly handle the bizarre edgecase of filenames containing newlines.

Resources