Node request output differing from curl output - node.js

I'm able to make a request using curl, get the response and parse it into protobuf. However, on trying to do the same with node's request, I'm unable to parse the body argument into protobuf, even when I've converted it into a buffer.
Here's the hex dump of what the request should have output
00000000: 1088 8080 80f0 fca4 f02c a206 00a2 0600 .........,......
00000010: a206 5408 a7b5 a9a4 e22a 10ff 8ac8 b9e3 ..T......*......
00000020: 2a1a 121a 100a 0e09 2130 0000 0000 0000 *.......!0......
00000030: 107b 18d4 011a 121a 100a 0e09 2130 0000 .{..........!0..
00000040: 0000 0000 107b 18d4 011a 121a 100a 0e09 .....{..........
00000050: 2130 0000 0000 0000 107b 18d4 011a 081a !0.......{......
00000060: 0612 0408 0110 15a2 0600 a206 00 .............
This is what dumping the created buffer onto disk and using xxd to dump it into hex
00000000: 10ef bfbd efbf bdef bfbd efbf bdef bfbd ................
00000010: efbf bdef bfbd efbf bd2c efbf bd06 00ef .........,......
00000020: bfbd 0600 efbf bd06 5408 efbf bdef bfbd ........T.......
00000030: efbf bdef bfbd efbf bd2a 10ef bfbd efbf .........*......
00000040: bdef bfbd efbf bdef bfbd 2a1a 121a 100a ..........*.....
00000050: 0e09 2130 0000 0000 0000 107b 18ef bfbd ..!0.......{....
00000060: 011a 121a 100a 0e09 2130 0000 0000 0000 ........!0......
00000070: 107b 18ef bfbd 011a 121a 100a 0e09 2130 .{............!0
00000080: 0000 0000 0000 107b 18ef bfbd 011a 081a .......{........
00000090: 0612 0408 0110 15ef bfbd 0600 efbf bd06 ................
There might be a small difference due to change in timestamps (my server sends that back within the protobuf), but both of the files should have been parsable.

Okay, so request assumes your response to be a unicode string, strips non-unicode characters and mutilates it unless you add encoding: null
to requestSettings.
Adding the said param to the settings fixes it.

Related

imagemagick gm node buffer file conversion

I'm currently using ImageMagick and gm to process images from a buffer. My problem is that I cannot control what file types are put in the buffer but wish to have everything turned to jpg.
Not sure how to do that using the buffer since I'm not giving it an output file path with extension.
gm(buf).command('convert').in('-auto-orient','-resize','500x','-quality','92','-strip','-quality','100','jpg').toBuffer((err, buffer) => err ? reject(err) : resolve(buffer));
I don't know anything about the node.js bindings of ImageMagick or GraphicsMagick, but I do know that if you do this in Terminal, you will force a JPEG output, so maybe you can adapt that...
# Make a 1x1 black image and write to stdout as JPEG. Dump result with 'xxd'
gm convert xc:black jpg:- | xxd
00000000: ffd8 ffe0 0010 4a46 4946 0001 0101 0048 ......JFIF.....H
00000010: 0048 0000 ffdb 0043 0008 0606 0706 0508 .H.....C........
00000020: 0707 0709 0908 0a0c 140d 0c0b 0b0c 1912 ................
00000030: 130f 141d 1a1f 1e1d 1a1c 1c20 242e 2720 ........... $.'
00000040: 222c 231c 1c28 3729 2c30 3134 3434 1f27 ",#..(7),01444.'
00000050: 393d 3832 3c2e 3334 32ff c000 0b08 0001 9=82<.342.......
00000060: 0001 0101 1100 ffc4 0014 0001 0000 0000 ................
00000070: 0000 0000 0000 0000 0000 0008 ffc4 0014 ................
00000080: 1001 0000 0000 0000 0000 0000 0000 0000 ................
00000090: 0000 ffda 0008 0101 0000 3f00 3fbf ffd9 ..........?.?...
It is just the same with ImageMagick and, say, PNG output:
magick xc:black png:-

Unix -Delete even bytes from a file

I want to find and delete all even bytes from a file given in command line. Is there any command for this situation?
I think this does what you want. It dumps the file as continuous plain hex, then reads two bytes, saving them for later and then two more bytes. Then it outputs the bytes it saved and pipes the whole lot back into xxd to "reconstruct":
xxd -p INPUT | sed -E 's/(..)../\1/g' | xxd -r -p > OUTPUT
So, if I dump a PNG image like this:
xxd image.png
00000000: 8950 4e47 0d0a 1a0a 0000 000d 4948 4452 .PNG........IHDR
00000010: 0000 0064 0000 0064 0203 0000 000d 8c7d ...d...d.......}
00000020: c700 0000 0467 414d 4100 00b1 8f0b fc61 .....gAMA......a
00000030: 0500 0000 2063 4852 4d00 007a 2600 0080 .... cHRM..z&...
00000040: 8400 00fa 0000 0080 e800 0075 3000 00ea ...........u0...
00000050: 6000 003a 9800 0017 709c ba51 3c00 0000 `..:....p..Q<...
00000060: 0950 4c54 4500 0000 ff00 00ff ffff 6719 .PLTE.........g.
00000070: 641e 0000 0001 624b 4744 0266 0b7c 6400 d.....bKGD.f.|d.
00000080: 0000 0774 494d 4507 e404 1c0a 0820 506e ...tIME...... Pn
00000090: 92c5 0000 0025 4944 4154 48c7 6360 1805 .....%IDATH.c`..
000000a0: a300 0c58 4361 2064 5466 5466 5466 5466 ...XCa dTfTfTfTf
000000b0: 5466 5406 8fcc 2818 3100 00e8 0fcb 7f57 TfT...(.1......W
000000c0: afef dd00 0000 2574 4558 7464 6174 653a ......%tEXtdate:
000000d0: 6372 6561 7465 0032 3032 302d 3034 2d32 create.2020-04-2
000000e0: 3854 3130 3a30 383a 3332 2b30 303a 3030 8T10:08:32+00:00
000000f0: d513 d3d0 0000 0025 7445 5874 6461 7465 .......%tEXtdate
00000100: 3a6d 6f64 6966 7900 3230 3230 2d30 342d :modify.2020-04-
00000110: 3238 5431 303a 3038 3a33 322b 3030 3a30 28T10:08:32+00:0
00000120: 30a4 4e6b 6c00 0000 0049 454e 44ae 4260 0.Nkl....IEND.B`
00000130: 82
and then put that through my filter and display it again:
xxd -p image.png | sed -E 's/(..)../\1/g' | xxd -r -pp | xxd
00000000: 894e 0d1a 0000 4944 0000 0000 0200 008c .N....ID........
00000010: c700 0441 4100 8ffc 0500 2048 4d00 2600 ...AA..... HM.&.
00000020: 8400 0000 e800 3000 6000 9800 70ba 3c00 ......0.`...p.<.
00000030: 094c 4500 ff00 ff67 6400 0062 4702 0b64 .LE....gd..bG..d
00000040: 0007 4945 e41c 0850 9200 0049 4148 6318 ..IE...P...IAHc.
00000050: a30c 4320 5454 5454 5454 8f28 3100 0f7f ..C TTTTTT.(1...
00000060: afdd 0025 4574 6165 6365 7400 3030 302d ...%Etaecet.000-
00000070: 3831 3a38 332b 3030 d5d3 0000 7458 6474 81:83+00....tXdt
00000080: 3a6f 6979 3232 2d34 3254 3030 3a32 303a :oiy22-42T00:20:
00000090: 304e 6c00 0045 4442 82 0Nl..EDB.
Or in Perl
#!/usr/bin/perl
use strict;
use warnings;
open my $ifh, '<', $ARGV[0] or die "Need an input filename";
binmode $ifh;
open my $ofh, '>', $ARGV[1] or die "Need an output filename";
binmode $ofh;
my $x;
while (read $ifh,$x,2) {
my $num = pack 'C',ord($x); # or ord(substr $x,1) for odd bytes
print $ofh $num;
}
I am not sure if there such shell/bash built in command. seems like you need to write you own bash script for doing so.

Why TCP FIN/ACK packets have payload data but length is 0?

I use tcpdump to capture some data, but found that the FIN and ACK packets have some payload while the length is 0. Can anyone explain that? When I use Wireshark to see the pcap file, all is right. Why is that?
20:56:05.174314 IP 10.0.2.15.20281 > 192.168.4.80.21224: Flags [.], ack 1721, win 33232, length 0
0x0000: 0004 0001 0006 0800 2793 4e00 0000 0800 ........'.N.....
0x0010: 4500 0028 417a 4000 4006 284f 0a00 020f E..(Az#.#.(O....
0x0020: c0a8 0450 4f39 52e8 b35c bf82 f4d8 b0ba ...PO9R..\......
0x0030: 5010 81d0 d121 0000 4745 5420 2f75 7365 P....!..GET./use
0x0040: 7273 2f31 3f75 7365 rs/1?use
20:56:15.179096 IP 10.0.2.15.20281 > 192.168.4.80.21224: Flags [F.], seq 649, ack 1721, win 33232, length 0
0x0000: 0004 0001 0006 0800 2793 4e00 0000 0800 ........'.N.....
0x0010: 4500 0028 417b 4000 4006 284e 0a00 020f E..(A{#.#.(N....
0x0020: c0a8 0450 4f39 52e8 b35c bf82 f4d8 b0ba ...PO9R..\......
0x0030: 5011 81d0 d121 0000 0000 0000 0000 2e31 P....!.........1
0x0040: 2035 3030 2049 6e74 .500.Int
20:56:15.179528 IP 192.168.4.80.21224 > 10.0.2.15.20281: Flags [.], ack 650, win 65535, length 0
0x0000: 0000 0001 0006 5254 0012 3502 0000 0800 ......RT..5.....
0x0010: 4500 0028 73f3 0000 4006 35d6 c0a8 0450 E..(s...#.5....P
0x0020: 0a00 020f 52e8 4f39 f4d8 b0ba b35c bf83 ....R.O9.....\..
0x0030: 5010 ffff 2438 0000 0000 0000 0000 2e31 P...$8.........1
0x0040: 2035 3030 2049 6e74 6572 6e61 6c20 .500.Internal.
20:56:15.181826 IP 192.168.4.80.21224 > 10.0.2.15.20281: Flags [F.], seq 1721, ack 650, win 65535, length 0
0x0000: 0000 0001 0006 5254 0012 3502 0000 0800 ......RT..5.....
0x0010: 4500 0028 73f5 0000 4006 35d4 c0a8 0450 E..(s...#.5....P
0x0020: 0a00 020f 52e8 4f39 f4d8 b0ba b35c bf83 ....R.O9.....\..
0x0030: 5011 ffff 2437 0000 0000 0000 0000 7365 P...$7........se
0x0040: 7273 2f31 3f75 7365 725f 6964 3d35 rs/1?user_id=5
20:56:15.181884 IP 10.0.2.15.20281 > 192.168.4.80.21224: Flags [.], ack 1722, win 33232, length 0
0x0000: 0004 0001 0006 0800 2793 4e00 0000 0800 ........'.N.....
0x0010: 4500 0028 eaf7 4000 4006 7ed1 0a00 020f E..(..#.#.~.....
0x0020: c0a8 0450 4f39 52e8 b35c bf83 f4d8 b0bb ...PO9R..\......
0x0030: 5010 81d0 a266 0000 4745 5420 2f75 7365 P....f..GET./use
0x0040: 7273 2f31 3f75 7365 rs/1?use

Get information from a file shell script linux grep command

I need help to extract information from a file outuput from a network traffic with tcpdump command
tcpdump -Xvv -i eth0 > capture.txt
Given a field of any Ethernet headers, IP and TCP, and a value, indicate the source and destination IP machines that were reported under this condition (without repeating them in the output).
The content of the file:
09:26:13.245546 IP (tos 0x0, ttl 1, id 3439, offset 0, flags [none], proto UDP (17), length 1018)
10.0.0.226.58935 > 239.255.255.250.3702: UDP, ack 555, win 6584, length 990
0x0000: 4500 03fa 0d6f 0000 0111 ada8 0a00 00e2 E....o..........
0x0010: efff fffa e637 0e76 03e6 7ec0 3c3f 786d .....7.v..~.<?xm
0x0020: 6c20 7665 7273 696f 6e3d 2231 2e30 2220 l.version="1.0".
0x0030: 656e 636f 6469 6e67 3d22 7574 662d 3822 encoding="utf-8"
0x0040: 3f3e 3c73 6f61 703a 456e 7665 ?><soap:Enve
09:26:13.339173 IP6 (hlim 1, next-header UDP (17) payload length: 998) fe80::21e9:f54b:9ae7:6383.58936 > ff02::c.3702: UDP, length 990
0x0000: 6000 0000 03e6 1101 fe80 0000 0000 0000 `...............
0x0010: 21e9 f54b 9ae7 6383 ff02 0000 0000 0000 !..K..c.........
0x0020: 0000 0000 0000 000c e638 0e76 03e6 666c .........8.v..fl
0x0030: 3c3f 786d 6c20 7665 7273 696f 6e3d 2231 <?xml.version="1
0x0040: 2e30 2220 656e 636f 6469 6e67 .0".encoding
09:26:13.407313 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.3.118 tell 10.0.1.215, length 46
0x0000: 0001 0800 0604 0001 0009 0fcb 0a0c 0a00 ................
0x0010: 01d7 0000 0000 0000 0a00 0376 0000 0000 ...........v....
0x0020: 0000 0000 0000 0000 0000 d9c4 62a8 ............b.
09:26:13.525954 IP (tos 0x0, ttl 128, id 3441, offset 0, flags [none], proto UDP (17), length 161)
10.0.0.226.59131 > 239.255.255.250.1900: UDP, length 133
0x0000: 4500 00a1 0d71 0000 0111 b0ff 0a00 00e2 E....q..........
0x0010: efff fffa e6fb 076c 008d 6fa6 4d2d 5345 .......l..o.M-SE
0x0020: 4152 4348 202a 2048 5454 502f 312e 310d ARCH.*.HTTP/1.1.
0x0030: 0a48 6f73 743a 3233 392e 3235 352e 3235 .Host:239.255.25
0x0040: 352e 3235 303a 3139 3030 0d0a 5.250:1900..
09:26:13.557002 IP (tos 0x0, ttl 1, id 3442, offset 0, flags [none], proto UDP (17), length 161)
10.0.0.226.59131 > 239.255.255.250.1900: UDP, length 133
0x0000: 4500 00a1 0d72 0000 0111 b0fe 0a00 00e2 E....r..........
0x0010: efff fffa e6fb 076c 008d 6fa6 4d2d 5345 .......l..o.M-SE
0x0020: 4152 4348 202a 2048 5454 502f 312e 310d ARCH.*.HTTP/1.1.
0x0030: 0a48 6f73 743a 3233 392e 3235 352e 3235 .Host:239.255.25
0x0040: 352e 3235 303a 3139 3030 0d0a 5.250:1900..
09:26:13.642734 IP (tos 0x0, ttl 1, id 21767, offset 0, flags [none], proto UDP (17), length 684)
10.0.0.237.58882 > 239.255.255.250.3702: UDP, length 656
0x0000: 4500 02ac 5507 0000 0111 6753 0a00 00ed E...U.....gS....
0x0010: efff fffa e602 0e76 0298 5568 3c3f 786d .......v..Uh<?xm
0x0020: 6c20 7665 7273 696f 6e3d 2231 2e30 2220 l.version="1.0".
0x0030: 656e 636f 6469 6e67 3d22 7574 662d 3822 encoding="utf-8"
0x0040: 3f3e 3c73 6f61 703a 456e 7665 ?><soap:Enve
09:26:13.642960 IP6 (hlim 1, next-header UDP (17) payload length: 664) fe80::b8a2:bd0:4e0b:1bb5.58883 > ff02::c.3702: UDP, length 656
0x0000: 6000 0000 0298 1101 fe80 0000 0000 0000 `...............
0x0010: b8a2 0bd0 4e0b 1bb5 ff02 0000 0000 0000 ....N...........
0x0020: 0000 0000 0000 000c e603 0e76 0298 248c ...........v..$.
0x0030: 3c3f 786d 6c20 7665 7273 696f 6e3d 2231 <?xml.version="
09:26:13.642999 IP (tos 0x0, ttl 64, id 21767, offset 0, flags [none], proto UDP (17), length 684)
10.0.0.237.58882 > 239.255.255.250.3702: UDP, length 656
0x0000: 4500 02ac 5507 0000 0111 6753 0a00 00ed E...U.....gS....
0x0010: efff fffa e602 0e76 0298 5568 3c3f 786d .......v..Uh<?xm
0x0020: 6c20 7665 7273 696f 6e3d 2231 2e30 2220 l.version="1.0".
0x0030: 656e 636f 6469 6e67 3d22 7574 662d 3822 encoding="utf-8"
0x0040: 3f3e 3c73 6f61 703a 456e 7665 ?><soap:Enve
For example is the header is: ttl 1
The result must be:
Source: 10.0.0.226.58935 --- Destination: 239.255.255.250.3702 - 1 Time
Source: 10.0.0.237.58882 --- Destination: 239.255.255.250.3702 - 2 Times
Other way: is the header is: ack or win: for example: ack 555
Source: 10.0.0.226.58935 --- Destination: 239.255.255.250.3702 - 1 Time
Using awk matching ack 555:
$ awk -F'[:>]' '/ack 555/{u["Source: "$1"--- Destination:"$2]++}END{for(k in u)print k,u[k]" - time"(u[k]>1?"s":"")}' file
Source: 10.0.0.226.58935 --- Destination: 239.255.255.250.3702 1 - time
Matching ttl 1:
$ awk -F'[>:]' '/ttl 1,/{getline;u["Source: "$1"--- Destination:"$2]++}END{for(k in u)print k,u[k]" - time"(u[k]>1?"s":"")}' file
Source: 10.0.0.237.58882 --- Destination: 239.255.255.250.3702 1 - time
Source: 10.0.0.226.59131 --- Destination: 239.255.255.250.1900 1 - time
Source: 10.0.0.226.58935 --- Destination: 239.255.255.250.3702 1 - time
You example doesn't match your expected output however.

Shell script linux substract parameter grep

I need help to extract coincidences from a file.
I capture network traffic with tcpdump command
tcpdump -Xvv -i eth0 > captureFile.txt
Given any field of IP headers, TCP and Ethernet specify all values ​​found in the captured traffic and count how many times that value for that field. For example if suppose TTL = 128 TTL = 64 then indicate how many packets have that field with each of these values​​.
The content of the file:
09:26:13.245546 IP (tos 0x0, ttl 1, id 3439, offset 0, flags [none], proto UDP (17), length 1018)
10.0.0.226.58935 > 239.255.255.250.3702: UDP, length 990
0x0000: 4500 03fa 0d6f 0000 0111 ada8 0a00 00e2 E....o..........
0x0010: efff fffa e637 0e76 03e6 7ec0 3c3f 786d .....7.v..~.<?xm
0x0020: 6c20 7665 7273 696f 6e3d 2231 2e30 2220 l.version="1.0".
0x0030: 656e 636f 6469 6e67 3d22 7574 662d 3822 encoding="utf-8"
0x0040: 3f3e 3c73 6f61 703a 456e 7665 ?><soap:Enve
09:26:13.339173 IP6 (hlim 1, next-header UDP (17) payload length: 998) fe80::21e9:f54b:9ae7:6383.58936 > ff02::c.3702: UDP, length 990
0x0000: 6000 0000 03e6 1101 fe80 0000 0000 0000 `...............
0x0010: 21e9 f54b 9ae7 6383 ff02 0000 0000 0000 !..K..c.........
0x0020: 0000 0000 0000 000c e638 0e76 03e6 666c .........8.v..fl
0x0030: 3c3f 786d 6c20 7665 7273 696f 6e3d 2231 <?xml.version="1
0x0040: 2e30 2220 656e 636f 6469 6e67 .0".encoding
09:26:13.407313 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.3.118 tell 10.0.1.215, length 46
0x0000: 0001 0800 0604 0001 0009 0fcb 0a0c 0a00 ................
0x0010: 01d7 0000 0000 0000 0a00 0376 0000 0000 ...........v....
0x0020: 0000 0000 0000 0000 0000 d9c4 62a8 ............b.
09:26:13.525954 IP (tos 0x0, ttl 128, id 3441, offset 0, flags [none], proto UDP (17), length 161)
10.0.0.226.59131 > 239.255.255.250.1900: UDP, length 133
0x0000: 4500 00a1 0d71 0000 0111 b0ff 0a00 00e2 E....q..........
0x0010: efff fffa e6fb 076c 008d 6fa6 4d2d 5345 .......l..o.M-SE
0x0020: 4152 4348 202a 2048 5454 502f 312e 310d ARCH.*.HTTP/1.1.
0x0030: 0a48 6f73 743a 3233 392e 3235 352e 3235 .Host:239.255.25
0x0040: 352e 3235 303a 3139 3030 0d0a 5.250:1900..
09:26:13.557002 IP (tos 0x0, ttl 1, id 3442, offset 0, flags [none], proto UDP (17), length 161)
10.0.0.226.59131 > 239.255.255.250.1900: UDP, length 133
0x0000: 4500 00a1 0d72 0000 0111 b0fe 0a00 00e2 E....r..........
0x0010: efff fffa e6fb 076c 008d 6fa6 4d2d 5345 .......l..o.M-SE
0x0020: 4152 4348 202a 2048 5454 502f 312e 310d ARCH.*.HTTP/1.1.
0x0030: 0a48 6f73 743a 3233 392e 3235 352e 3235 .Host:239.255.25
0x0040: 352e 3235 303a 3139 3030 0d0a 5.250:1900..
09:26:13.642734 IP (tos 0x0, ttl 1, id 21767, offset 0, flags [none], proto UDP (17), length 684)
10.0.0.237.58882 > 239.255.255.250.3702: UDP, length 656
0x0000: 4500 02ac 5507 0000 0111 6753 0a00 00ed E...U.....gS....
0x0010: efff fffa e602 0e76 0298 5568 3c3f 786d .......v..Uh<?xm
0x0020: 6c20 7665 7273 696f 6e3d 2231 2e30 2220 l.version="1.0".
0x0030: 656e 636f 6469 6e67 3d22 7574 662d 3822 encoding="utf-8"
0x0040: 3f3e 3c73 6f61 703a 456e 7665 ?><soap:Enve
09:26:13.642960 IP6 (hlim 1, next-header UDP (17) payload length: 664) fe80::b8a2:bd0:4e0b:1bb5.58883 > ff02::c.3702: UDP, length 656
0x0000: 6000 0000 0298 1101 fe80 0000 0000 0000 `...............
0x0010: b8a2 0bd0 4e0b 1bb5 ff02 0000 0000 0000 ....N...........
0x0020: 0000 0000 0000 000c e603 0e76 0298 248c ...........v..$.
0x0030: 3c3f 786d 6c20 7665 7273 696f 6e3d 2231 <?xml.version="
09:26:13.642999 IP (tos 0x0, ttl 64, id 21767, offset 0, flags [none], proto UDP (17), length 684)
10.0.0.237.58882 > 239.255.255.250.3702: UDP, length 656
0x0000: 4500 02ac 5507 0000 0111 6753 0a00 00ed E...U.....gS....
0x0010: efff fffa e602 0e76 0298 5568 3c3f 786d .......v..Uh<?xm
0x0020: 6c20 7665 7273 696f 6e3d 2231 2e30 2220 l.version="1.0".
0x0030: 656e 636f 6469 6e67 3d22 7574 662d 3822 encoding="utf-8"
0x0040: 3f3e 3c73 6f61 703a 456e 7665 ?><soap:Enve
The result must be:
ttl 64 - 1 time
ttl 128 - 1 time
ttl 1 - 3 times
I think this would be exactly same as your expected output.
grep -ioP 'ttl \d+' file|awk '{a[$0]++}END{for(x in a)print x" - "a[x]" times"}'
output would be:
ttl 1 - 3 times
ttl 64 - 1 times
ttl 128 - 1 times
well not exactly same, since I didn't check time and times.. do you really need it? it could be done easily..
EDIT
as OP asks, output time/times depends on the count:
grep -ioP 'ttl \d+' file|awk '{a[$0]++}END{for(x in a)print x" - "a[x]" time"(a[x]>1?"s":"")}'
output:
ttl 1 - 3 times
ttl 64 - 1 time
ttl 128 - 1 time
It's a bit long and I'm sure it can be refactored quite a lot but it works if you don't|can't have perl installed:
grep ttl captureFile.txt | awk '{print $5,$6}' | sed 's/,//' | sort | uniq -c | awk '{print $2,$3,"-",$1,"times"}'
Two approaches:
If you have perl,
captureFile.txt |
perl -ne '/ttl (\d+),/ and $TTL{$1}++;
END { for my $ttl (keys %TTL) {print "* ttl $1 - $TTL{$ttl} time\n"}}'
Should do it. But I think uniq -c may also work with grep...
captureFile.txt | egrep -o 'ttl ([0-9]+)' | uniq -c
And to get the exact output format you asked for, just add this after uniq -c
| awk '{print "* ttl "$3" - "$1" time"}'
grep "ttl [0-9]*" captureFile.txt -o
Would get only the relevant parts of the text file.
grep "ttl [0-9]*" captureFile.txt -o |
awk 'NF{ count[ toupper( $0 ) ]++}
END{ for ( name in count ) { print "*" name " - " count[ name ] " times"
};
}'
Would get the formatting you wanted.
Simple awk script:
$ awk -F, '/ttl/{u[$2]++}END{for(k in u)print k" - "u[k]" time"(u[k]>1?"s":"")}'
ttl 128 - 1 time
ttl 64 - 1 time
ttl 1 - 3 times
No need to waste sub-process.

Resources