This question already has answers here:
Pipe only STDERR through a filter
(7 answers)
Closed 8 years ago.
I've been trying to use the following command on my server
dd if=/dev/zero bs=1M count=1024 | md5sum
The output:
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 2.92245 s, 367 MB/s
cd573cfaace07e7949bc0c46028904ff -
How do I let it show the speed (367 MB/s) as output only? The status is printed to stderr.
I'm currently using awk but it showed the md5 hash.
Helps are appreciated :)
First, a function to simulate your command
simulation() {
echo "1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 2.92245 s, 367 MB/s" >&2
echo "cd573cfaace07e7949bc0c46028904ff -"
}
$ simulation >/dev/null
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 2.92245 s, 367 MB/s
$ simulation 2>/dev/null
cd573cfaace07e7949bc0c46028904ff -
Then, the solution: redirecting stderr to a process substitution that displays the desired output back to stderr, capturing stdout in a variable.
$ md5sum=$( simulation 2> >(sed -n '/MB\/s/ {s/.*, //p; q}' >&2) )
367 MB/s
$ echo $md5sum
cd573cfaace07e7949bc0c46028904ff -
Related
Inside the docker container I test the
dd if=/dev/zero of=/tmp/test2.img bs=512 count=1000 2> >( grep copied )
line execution. I drill into container using 2 ways.
1) the classical one is docker exec -it 2b65c84ddce2 /bin/sh
the execution the line inside the contained inherinted from the alpine I'm greeting
/bin/sh: syntax error: unexpected redirection beacuse of something near >(
2) when I enter the container into the bash executor like docker exec -it 2b65c84ddce2 /bin/bash
dd if=/dev/zero of=/tmp/test2.img bs=512 count=1000 2> >( grep copied ) returns no output
dd if=/dev/zero of=/tmp/test2.img bs=512 count=1000 returns 2 lines output only, while the expectation is 3:
1+0 records in
1+0 records out
At the host level the same dd command is returning 3 lines like this:
dd if=/dev/zero of=/tmp/test2.img bs=512 count=1000
1000+0 records in
1000+0 records out
512000 bytes (512 kB, 500 KiB) copied, 0.0109968 s, 46.6 MB/s
and with redirection the output is the last line:
dd if=/dev/zero of=/tmp/test2.img bs=512 count=1000 2> >( grep copied )
512000 bytes (512 kB, 500 KiB) copied, 0.0076261 s, 67.1 MB/s
So how can I get the last line of dd output from the inside of the docker container?
PS.
The redirecting stderr to stdout doesn't help in general:
/ # dd if=/dev/zero of=/tmp/test2.img bs=512 count=1000 2>&1 | grep copied
/ # dd if=/dev/zero of=/tmp/test2.img bs=512 count=1000 2>&1
1000+0 records in
1000+0 records out
while at the host system it works
$ dd if=/dev/zero of=/tmp/test2.img bs=512 count=1000 2>&1 | grep copied
512000 bytes (512 kB, 500 KiB) copied, 0.00896706 s, 57.1 MB/s
host:
dd --v
dd (coreutils) 8.30
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Paul Rubin, David MacKenzie, and Stuart Kemp.
container:
/ # dd --v
BusyBox v1.31.1 () multi-call binary.
Usage: dd [if=FILE] [of=FILE] [ibs=N obs=N/bs=N] [count=N] [skip=N] [seek=N]
[conv=notrunc|noerror|sync|fsync]
[iflag=skip_bytes|fullblock] [oflag=seek_bytes|append]
Copy a file with converting and formatting
if=FILE Read from FILE instead of stdin
of=FILE Write to FILE instead of stdout
bs=N Read and write N bytes at a time
ibs=N Read N bytes at a time
obs=N Write N bytes at a time
count=N Copy only N input blocks
skip=N Skip N input blocks
seek=N Skip N output blocks
conv=notrunc Don't truncate output file
conv=noerror Continue after read errors
conv=sync Pad blocks with zeros
conv=fsync Physically write data out before finishing
conv=swab Swap every pair of bytes
iflag=skip_bytes skip=N is in bytes
iflag=fullblock Read full blocks
oflag=seek_bytes seek=N is in bytes
oflag=append Open output file in append mode
status=noxfer Suppress rate output
status=none Suppress all output
N may be suffixed by c (1), w (2), b (512), kB (1000), k (1024), MB, M, GB, G
they are in fact, different
For anyone searching this question:
the DD used was being used with BusyBox. The third line is an optional output which is defined when compiling BusyBox from Source. The pre compiled versions have this disabled
ENABLE_FEATURE_DD_THIRD_STATUS_LINE must be defined.
see https://git.busybox.net/busybox/tree/coreutils/dd.c line 166.
#if ENABLE_FEATURE_DD_THIRD_STATUS_LINE
# if ENABLE_FEATURE_DD_STATUS
if (G.flags & FLAG_STATUS_NOXFER) /* status=noxfer active? */
return;
//TODO: should status=none make dd stop reacting to USR1 entirely?
//So far we react to it (we print the stats),
//status=none only suppresses final, non-USR1 generated status message.
# endif
fprintf(stderr, "%llu bytes (%sB) copied, ",
G.total_bytes,
/* show fractional digit, use suffixes */
make_human_readable_str(G.total_bytes, 1, 0)
);
/* Corner cases:
* ./busybox dd </dev/null >/dev/null
* ./busybox dd bs=1M count=2000 </dev/zero >/dev/null
* (echo DONE) | ./busybox dd >/dev/null
* (sleep 1; echo DONE) | ./busybox dd >/dev/null
*/
seconds = (now_us - G.begin_time_us) / 1000000.0;
bytes_sec = G.total_bytes / seconds;
fprintf(stderr, "%f seconds, %sB/s\n",
seconds,
/* show fractional digit, use suffixes */
make_human_readable_str(bytes_sec, 1, 0)
);
#endif
}
I use pv -f in piping like in 'command | pv -f | command2'. pv shows a progress bar like this
83.6MB 0:00:03 [27.9MB/s] [ <=> ]
When pv exits it shows more information. How can I show only the progress bar?
500+0 records in28.3MB/s] [ <=> ]
500+0 records out
524288000 bytes (524 MB) copied, 17.6912 s, 29.6 MB/s
500MB 0:00:17 [28.3MB/s] [ <=> ]
1024000+0 records in
1024000+0 records out
524288000 bytes (524 MB) copied, 17.6956 s, 29.6 MB/s
Edit:
My test case was
dd if=/dev/zero bs=100k count=80000 | pv -f | dd of=/dev/null
So I tried this:
root#kali:~/Desktop/fmk# binwalk upgrade-2.4.0.bin
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
512 0x200 LZMA compressed data, properties: 0x6D, dictionary size: 8388608 bytes, uncompressed size: 2805816 bytes
927576 0xE2758 Squashfs filesystem, little endian, version 4.0, compression:xz, size: 12316692 bytes, 2963 inodes, blocksize: 262144 bytes, created: 2015-08-04 02:40:49
And then I used the following dd:
sudo dd if=upgrade-2.4.0.bin of=pineapple.squashfs bs=1 count=12316692
And I can't unsquashfs pineapple.squashfs.
Can't find a SQUASHFS superblock on pineapple.squashfs
You have to set the offset where the squashfs is
Usage: dd [OPERAND]...
or: dd OPTION
Copy a file, converting and formatting according to the operands.
bs=BYTES read and write up to BYTES bytes at a time
cbs=BYTES convert BYTES bytes at a time
conv=CONVS convert the file as per the comma separated symbol list
count=N copy only N input blocks
ibs=BYTES read up to BYTES bytes at a time (default: 512)
if=FILE read from FILE instead of stdin
iflag=FLAGS read as per the comma separated symbol list
obs=BYTES write BYTES bytes at a time (default: 512)
of=FILE write to FILE instead of stdout
oflag=FLAGS write as per the comma separated symbol list
seek=N skip N obs-sized blocks at start of output
skip=N skip N ibs-sized blocks at start of input
status=LEVEL The LEVEL of information to print to stderr;
'none' suppresses everything but error messages,
'noxfer' suppresses the final transfer statistics,
'progress' shows periodic transfer statistics
...
So, to extract the filesystem
dd if=upgrade-2.4.0.bin of=pineapple.squashfs bs=1 skip=927576
I did it with:
binwalk -Me upgrade-2.4.0.bin
I execute a curl [url] command in a Linux shell script. I want to get this command's exit code and its output at the same time without using a temporary file.
Is there any method to do that?
I'm assuming the problem is that you have an intermediary command that's supplanting the exit code of the last command.
To get around this, just store the exit code and stdout in variables:
OUTPUT=$(curl example.org)
EXIT_CODE=$?
then you can simply output these either in the same line:
echo "$EXIT_CODE: $OUTPUT"
or call them separately as needed.
(I don't have enough reputation points to comment on user559633's answer.)
Apparently this won't work if you send STDOUT to a local variable:
test.sh:
#!/bin/bash
function test1 () {
OUTPUT=$( ping -c 1 -W 1 blah.org )
EXIT_CODE=$?
echo "$EXIT_CODE: $OUTPUT"
}
function test2 () {
local OUTPUT=$( ping -c 1 -W 1 blah.org )
EXIT_CODE=$?
echo "$EXIT_CODE: $OUTPUT"
}
test1
test2
Output:
# ./test.sh
1: PING blah.org (205.150.150.140) 56(84) bytes of data.
--- blah.org ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms
0: PING blah.org (205.150.150.140) 56(84) bytes of data.
--- blah.org ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms
Note the exitcode from test1 is 1, but for test2, it's 0.
EDIT: It seems that separating the local declaration from the assignment takes care of this:
#!/bin/bash
function test1 () {
OUTPUT=$( ping -c 1 -W 1 blah.org )
EXIT_CODE=$?
echo "$EXIT_CODE: $OUTPUT"
}
function test2 () {
local OUTPUT
OUTPUT=$( ping -c 1 -W 1 blah.org )
EXIT_CODE=$?
echo "$EXIT_CODE: $OUTPUT"
}
test1
test2
Output:
1: PING blah.org (205.150.150.140) 56(84) bytes of data.
--- blah.org ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms
1: PING blah.org (205.150.150.140) 56(84) bytes of data.
--- blah.org ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms
There's no standard way to get the exit status of a command within a pipe.
However, if you're using (or able to use) Bash as your shell, the PIPESTATUS built-in array variable holds the results of the most recent pipeline executed. (Note that a single command counts as a pipeline here).
Example:
true | false | false | false; echo "${PIPESTATUS[0]}" "${PIPESTATUS[1]}"
Output:
0 1
I am alright at writing Linux scripts but could use some advice. I know the problem is sort of vague, so if you can provide any help whatsoever I will appreciate it!
The following issue is for personal growth, and because I am writing some network tools for fun/learning. No homework involved (I'm a senior in college, none of my classes require this stuff!)
I am using tshark to get information about packet captures. This is what it looks like:
rachel#Ubuntu-1:~/PCAP$ tshark -r LargeTorrent.pcap -q -z io,phs
===================================================================
Protocol Hierarchy Statistics
Filter:
eth frames:4309 bytes:3984321
ip frames:4119 bytes:3969006
icmp frames:1316 bytes:1308988
udp frames:1408 bytes:1350786
data frames:1368 bytes:1346228
dns frames:16 bytes:1176
nbns frames:14 bytes:1300
http frames:8 bytes:1596
nbdgm frames:2 bytes:486
smb frames:2 bytes:486
mailslot frames:2 bytes:486
browser frames:2 bytes:486
tcp frames:1395 bytes:1309232
data frames:1300 bytes:1294800
http frames:6 bytes:3763
data-text-lines frames:2 bytes:324
xml frames:2 bytes:3205
tcp.segments frames:1 bytes:787
nbss frames:34 bytes:5863
smb frames:17 bytes:3047
pipe frames:4 bytes:686
lanman frames:4 bytes:686
smb2 frames:13 bytes:2444
bittorrent frames:10 bytes:1709
tcp.segments frames:2 bytes:433
bittorrent frames:2 bytes:433
bittorrent frames:1 bytes:258
bittorrent frames:2 bytes:221
bittorrent frames:2 bytes:221
arp frames:146 bytes:8760
ipv6 frames:44 bytes:6555
udp frames:40 bytes:6211
dns frames:18 bytes:1711
dhcpv6 frames:14 bytes:2114
http frames:6 bytes:1014
data frames:2 bytes:1372
icmpv6 frames:4 bytes:344
===================================================================
What I would like for it to look like:
rachel#Ubuntu-1:~/PCAP$ tshark -r LargeTorrent.pcap -q -z io,phs
===================================================================
Protocol Hierarchy Statistics
Filter:
Protocol Bytes
=====================================
eth 984321
ip 3969006
icmp 1308988
udp 1350786
data 1346228
dns 1176
nbns 1300
http 1596
nbdgm 486
smb 486
mailslot 486
browser 486
tcp 1309232
data 1294800
http 3763
data-text-lines 324
xml 3205
tcp.segments 787
nbss 5863
smb 3047
pipe 686
lanman 686
smb2 2444
bittorrent 1709
tcp.segments 433
bittorrent 433
bittorrent 258
bittorrent 221
bittorrent 221
arp 8760
ipv6 6555
udp 6211
dns 1711
dhcpv6 2114
http 1014
data 1372
icmpv6 344
===================================================================
Edit: I am going to add the original question for the purpose of making sense of the (great) answer that was provided.
Originally, I wanted to only print statistics for "leaves" because eth, ip, etc. are all parents and their statistics are not necessary for my purposes. In addition, instead of having a god-awful block of text with only spaces to show hierarchy, I wanted to erase all the statistics for parents, and show them as breadcrumbs behind the child.
Example:
eth frames:4309 bytes:3984321
ip frames:4119 bytes:3969006
icmp frames:1316 bytes:1308988
udp frames:1408 bytes:1350786
data frames:1368 bytes:1346228
dns frames:16 bytes:1176
Should become
eth:ip:icmp - 1308988 bytes
eth:ip:udp:data - 1346228 bytes
eth:ip:udp:dns - 1176 bytes
To preserve the hierarchy and avoid printing useless statistics.
Anyway, the approved answer by Etan solved this perfectly! And for those of you who are on my level who are unsure of how to proceed after this answer, this will help you finish up:
Save the given script as a filename.awk file
Save the block of text you want to manipulate as a filename.txt file
Call awk -f filename.awk filename.txt
Optionally pipe the output to a file ( awk -f filename.awk filename.txt >> output.txt )
The output I originally thought you wanted could be achieved with this awk script. (I think this can probably be done cleaner but this seems to work well enough.)
function entry() {
# Don't want to print empty entries.
if (ind[0]) {
printf "%s", ind[0]
for (i = 1; i <= ls; i++) {
printf ":%s", ind[i]
}
split(b, a, /:/)
printf " - %s %s\n", a[2], a[1]
}
}
# Found our data marker. Note that and print the current line.
$1 == "Filter:" {d=1; print; next}
# Print lines until we see our data marker.
!d {print; next}
# Print empty lines.
!NF {print; next}
# Save our trailing line for later.
/===/ {suf=$0; next}
{
# Save our previous indentation level.
ls = s
# Find our new indentation level (by where the first field starts).
s = (match($0, /[^[:space:]]/)-1) / 2
# If the current line is at or below the last indent level print the last line.
if (s <= ls) {
entry()
}
# Save the current line's byte count.
b=$NF
# Save the current line's field name.
ind[s] = $1
}
END {
# Print a final line if we had one.
entry()
# Print the suffix line if we have one.
if (suf) {
print suf
}
}
Which, on the sample input, gets you this output.
===================================================================
Protocol Hierarchy Statistics
Filter:
eth:ip:icmp - 1308988 bytes
eth:ip:udp:data - 1346228 bytes
eth:ip:udp:dns - 1176 bytes
eth:ip:udp:nbns - 1300 bytes
eth:ip:udp:http - 1596 bytes
eth:ip:udp:nbdgm:smb:mailslot:browser - 486 bytes
eth:ip:tcp:data - 1294800 bytes
eth:ip:tcp:http:data-text-lines - 324 bytes
eth:ip:tcp:http:xml:tcp.segments - 787 bytes
eth:ip:tcp:nbss:smb:pipe:lanman - 686 bytes
eth:ip:tcp:nbss:smb2 - 2444 bytes
eth:ip:tcp:bittorrent:tcp.segments:bittorrent:bittorrent - 258 bytes
eth:ip:tcp:bittorrent:bittorrent:bittorrent - 221 bytes
eth:arp - 8760 bytes
eth:ipv6:udp:dns - 1711 bytes
eth:ipv6:udp:dhcpv6 - 2114 bytes
eth:ipv6:udp:http - 1014 bytes
eth:ipv6:udp:data - 1372 bytes
eth:ipv6:icmpv6:data - 344 bytes
===================================================================
Output like what you edited to indicate you want is probably more easily handled with sed though.
/Filter:/a \
Protocol Bytes \
=====================================
s/frames:[^ ]*//
s/ b/b/
s/bytes:\([^ ]*\)/\1/
Which ends up with output.
===================================================================
Protocol Hierarchy Statistics
Filter:
Protocol Bytes
=====================================
eth 3984321
ip 3969006
icmp 1308988
udp 1350786
data 1346228
dns 1176
nbns 1300
http 1596
nbdgm 486
smb 486
mailslot 486
browser 486
tcp 1309232
data 1294800
http 3763
data-text-lines 324
xml 3205
tcp.segments 787
nbss 5863
smb 3047
pipe 686
lanman 686
smb2 2444
bittorrent 1709
tcp.segments 433
bittorrent 433
bittorrent 258
bittorrent 221
bittorrent 221
arp 8760
ipv6 6555
udp 6211
dns 1711
dhcpv6 2114
http 1014
data 1372
icmpv6 344
===================================================================
A simple script with sed will work as well.
$ printf "\n==========================================================\n"; printf "Protocol Hierarchy Statistics\nFilter:\n\n";printf "\nProtocol\t\t\t\t Bytes\n================================================\n" && sed -e 's/\(frames[:].*bytes[:]\)\(.*$\)/\2/' dat/tshark.txt | tail -n+4 | head -n-1 && printf "================================================\n"
broken down into script form (where dat/tshark.txt is the filename holding the tshark output):
printf "\n==========================================================\n"
printf "Protocol Hierarchy Statistics\nFilter:\n\n"
printf "\nProtocol\t\t\t\t Bytes\n================================================\n"
sed -e 's/\(frames[:].*bytes[:]\)\(.*$\)/\2/' dat/tshark.txt | tail -n+4 | head -n-1
printf "================================================\n"
Output
==========================================================
Protocol Hierarchy Statistics
Filter:
Protocol Bytes
================================================
eth 3984321
ip 3969006
icmp 1308988
udp 1350786
data 1346228
dns 1176
nbns 1300
http 1596
nbdgm 486
smb 486
mailslot 486
browser 486
tcp 1309232
data 1294800
http 3763
data-text-lines 324
xml 3205
tcp.segments 787
nbss 5863
smb 3047
pipe 686
lanman 686
smb2 2444
bittorrent 1709
tcp.segments 433
bittorrent 433
bittorrent 258
bittorrent 221
bittorrent 221
arp 8760
ipv6 6555
udp 6211
dns 1711
dhcpv6 2114
http 1014
data 1372
icmpv6 344
================================================
Formatting
Following on from your comment on how to align the bytes info given the variable length of the protocol tags, you can make use of printf to format the output as you have indicated. Like Ethan, I started working on your original question that had the tags consolidated. My initial approach was to read the different levels into different associative arrays that could be combined into what you initially specified. Doing so, I had to produce the output lined up using printf. Here is the first attempt I made working with the first 4-levels of your tshark data:
declare -i ln=0
declare -A l1 l2 l3 l4
## read each line in file and assing to associative arrays for each level
while read -r line; do
ln=${#line} # base level on length of line read
[ $ln -gt 66 ] && continue;
[ $ln -eq 66 ] && { iface="${line%% *}"; l1[${iface}]="${line##* }"; }
[ $ln -eq 64 ] && { proto="${iface}:${line%% *}"; l2[${proto}]="${line##* }"; }
[ $ln -eq 62 ] && { ptype="${proto}:${line%% *}"; l3[${ptype}]="${line##* }"; }
[ $ln -le 60 ] && { data="${ptype}:${line%% *}"; l4[${data}]="${line##* }"; }
done < "$1"
## output a summary of the file
printf "\n4-level deep summary of file '%s':\n\n" "$1"
for i in "${!l1[#]}"; do
for j in "${!l2[#]}"; do
printf " %-32s %s\n" "$j" "${l2[$j]}"
for k in "${!l3[#]}"; do
printf " %-32s %s\n" "$k" "${l3[$k]}"
for l in "${!l4[#]}"; do
[ "${l%:*}" == "$k" ] && printf " %-32s %s\n" "$l" "${l4[$l]}"
done
done
done
done
The output it produced was for example:
eth:ip frames:4119 bytes:3969006
eth:ip:udp frames:1408 bytes:1350786
eth:ip:udp:data frames:1368 bytes:1346228
eth:ip:udp:nbdgm frames:2 bytes:486
eth:ip:udp:nbns frames:14 bytes:1300
You can look at the various printf statements in the code above and see how the alignment is handled. Let me know if you have further questions.
I'm a little surprised that tshark doesn't have a JSON or machine-readable way to get the -z io,phs info, when it has so many ways to extract packet info.
I tried playing with some of the above, but bash seems to have changed over the years (or has different defaults depending on the environment). I am also not sure which shell or version of it was used to produce the above.
The line lengths/output from tshark have also changed: My debugging showed different line lengths, so the trick above using line lengths, e.g. [ $ln -gt 66 ] didn't work for me.
It seems that read -r strips out leading/trailing whitespaces. If you actually want it, you need IFS= to make it give you the spaces:
## read each line in file
while IFS= read -r line ; do
...
done
The "nested" levels associative arrays is clever, but hard to work with - it shows what rabbit holes you can go down with bash - although now when iterating through it, bash produces it in "hash" order and not the order they were added.
Since I actually needed the data in the rest of my script, the nested arrays made it particularly fiddly to deal with. Fine for printf purposes where you just print the line, but what if you actually want to get the frames count for each item and do then do something with it.
Here was my attempt that simplified it a bit. I implemented it as a bash function which gets a few other bits of info from the sample file:
TSHARK=/usr/bin/tshark
CAPINFOS=/usr/bin/capinfos
declare -A fcount
declare -A bcount
declare -A capinfo
function loadcapinfo
{
local sample=$1
local statstofile=$2
local bytes
local frames
local key
if [ ! -f "$sample" ] ; then
echo "FATAL: loadcapinfo: file does not exist: $sample"
exit 1
fi
capinfo[start_time_epoch]=$($CAPINFOS -Tr -Sa $sample | cut -f2)
capinfo[start_time]=$($CAPINFOS -Tr -a $sample | cut -f2)
capinfo[end_time_epoch]=$($CAPINFOS -Tr -Se $sample | cut -f2)
capinfo[end_time]=$($CAPINFOS -Tr -e $sample | cut -f2)
capinfo[size]=$($CAPINFOS -Tr -s $sample | cut -f2)
declare -i ln=0
while IFS= read -r line ; do
ln=${#line} # base level on length of line read
[ $ln -le 1 ] && continue;
pat=".*frames:([0-9]+)\s+bytes:([0-9]+)"
pat_1="^(\w+)"
pat_2="^\s{2}(\w+)"
pat_3="^\s{4}(\w+)"
pat_4="^\s{6}(\w+)"
ethertype="ethertype"
[[ $line =~ $pat ]] && { frames=${BASH_REMATCH[1]}; bytes=${BASH_REMATCH[2]}; } || continue;
[[ $line =~ $pat_1 ]] && { encap="${BASH_REMATCH[1]}:${ethertype}"; key="${encap}"; }
[[ $line =~ $pat_2 ]] && { proto=${BASH_REMATCH[1]}; key="${encap}:${proto}"; }
[[ $line =~ $pat_3 ]] && { ptype=${BASH_REMATCH[1]}; key="${encap}:${proto}:${ptype}"; }
[[ $line =~ $pat_4 ]] && { data=${BASH_REMATCH[1]}; key="${encap}:${proto}:${ptype}:${data}"; }
[ "$proto" = "llc" ] && { key=${key/eth:ethertype:llc/eth:llc} ; }
fcount[${key}]=${frames:=0}
bcount[${key}]=${bytes:=0}
if [ -n "$statstofile" ] ; then
echo "${capinfo[start_time_epoch]},${key},${frames},${bytes}" >> $statstofile
fi
done < <($TSHARK -qr $sample -z io,phs)
unset fcount[0]
}
Now, after this in the script, we can do:
loadcapinfo /my/sample/file.pcap /tmp/stats.txt
Optionally write the counts to a file, /tmp/stats.txt
This uses one associative array for each count, and puts other info into capinfo so now we can do things like:
echo "IPv4 Packet Count is: ${fcount[eth:ethertype:ip]}"
echo "IPv6 Packet Count is: ${fcount[eth:ethertype:ipv6]}"
echo "ARP Count is: ${fcount[eth:ethertype:arp]}"
echo "STP Count is: ${fcount[eth:llc:stp]}"
echo "Start time: ${capinfo[start_time]}"
echo "End time: ${capinfo[end_time]}"
echo "File size: ${capinfo[size]}"
I made the keys match Wireshark's frame.protocols field, which inserts some "pseudo protocol" for most things called "ethertype". This way, if you want to then iterate through the associative array to find the packet(s) in the pcap file, you can use the information to find packets with a given protocol.
tshark -r /my/sample/file.pcap -Y "frame.protocols == eth:ethertype:ip:udp:snmp" -Tfields -e frame.number -e eth.src_resolved -e eth.dst_resolved -e ip.src -e ip.dst -e frame.protocols
for i in "${!fcount[#]}"; do
tshark -r /my/sample/file.pcap -Y "frame.protocols == $i" -Tfields -e frame.number -e eth.src_resolved -e eth.dst_resolved -e ip.src -e ip.dst -e frame.protocols > /tmp/$i.txt
done