I need to be able to search all tcp streams that contain a particular string, not just a particular packet. Something like:
tcp.stream contains "string"
I need to do this in order to filter out all streams containing a certain string to get exactly what I'm looking for. My end goal filter would look something like this:
!(tcp.stream contains "string I do not want")
You have two choices:
Option 1 - Display Filter:
Try the following display filter tcp and frame contains "xxxxxx"
Option 2 - Ctrl+F:
Find (Ctrl +F)
Find by String
Search in packet Bytes
Related
I'm new to GSM AT Commands and I'm using ZTE MF190S USB 3G USB Modem with Modem Nodejs Module also tried with MS Hyperterminal.
The problem I'm facing that when I receive SMS using AT+CMGL="ALL" from Whatsapp (as example) I got in sender number field 81084326797126204 which is an invalid phone number and should be "Whatsapp" instead although the manufacture's software is showing message from Whatsapp normally.
Note: I tried to decode it as a Hexadecimal string but not done.
> AT+CSCS="GSM"
> AT+CPMS="ME"
> AT+CMGL="ALL"
//Result
+CMGL: 0,"REC UNREAD","81084326797126204",,"20/05/30,14:53:55+08"
FEFF000000000000000000000000000000000000020
As described in this link
+CMGL: index,message_status,address,[address_text],[service_center_time_stamp][,address_type,sms_message_body_length]sms_message_body[+CMGL:
...]
The address Field
The third field of the information response of the +CMGL AT command,
address, is a string that contains the address/phone number stored in
the SMS message header. If the SMS message read is an incoming SMS
message, the address field contains the originator address. If the SMS
message read is an outgoing SMS message, the address field contains
the recipient address. Usually the address field value is a phone
number formatted using the typical ISDN / telephony numbering plan
(ITU E.164/E.163). For example, "+85291234567".
The address_text Field
The fourth field of the information response of the +CMGL AT command,
address_text, is a string that contains the text associated to address
in the phonebook. For example, if the text "Alice" is associated to
the phone number "91234567" in the phonebook, address_text will be
"Alice". The AT command +CSCS (command name in text: Select TE
Character Set) can be used to specify the character set for displaying
address_text. Note that address_text is an optional field. Some
GSM/GPRS modems and mobile phones (examples: most Nokia products,
including my Nokia 6021) leave this field empty.
How can I identify the message is from Whatsapp or any other services providers names?
You wrote
The fourth field of the information response of the +CMGL AT command, address_text, is a string that contains the text associated to address in the phonebook
If your device used to recognize that number, it probably was in one of device's phonebooks, and maybe it got deleted.
What you can try to do is to store again the number using AT+CPBW command:
AT+CPBW=[<index>][,<number>[,<type>[,<text>]]]
where
index is the record position in the phonebook. If omitted the record is stored in the first free position
number is the number to be stored in phonebook, in string format
type is the number type; 129 for national scheme, 145 for international scheme
text is the name of the entry in the phonebook
So, in your case:
AT+CPBW=,"81084326797126204",129,"WhatsApp"
And after it you should see the name appearing in +CGML list. You might need a device reboot after the PB write in order to see the change making effect.
The change is done in the current phonebook, usually the SIM card. It can be queried with command AT+CPBS? (it also allows to change it; see the guide for further explanation).
I suggest exploring the content of every phonebook, in order to discover any device (and SIM) default contents. The command to read phonebook entries is **AT+CPBR**. Providing
AT+CPBR=1,N
all index between 1 and N are shown. If you set N to the size of the phonebook, you can list all the entries in it.
I have pcap file which contains many DNS request and responses and i want to find the max value of ttl field from all of these packets for example:
If my pcap packets are the following:
DNS response ttl 1045
DNS response ttl 202
DNS response ttl 45
DNS response ttl 162
DNS response ttl 398
I want to find out how to recieve the value 1045 or even the packet itself.
It's all new to me so please try to explain carefully.
thanks for the helpers
To find the maximum TTL among packets from your pcap file, you could add a new TTL column and sort by this column.
To do this, you can right click on one of the column's name (e.g., Source), go to Column Preferences..., click the + sign at the bottom of the new window, and complete the new row that appeared with a title and dns.resp.ttl as the Fields option.
If you go back to the main Wireshark window, you should have a new column, which you can use to sort packets.
You can also accomplish this by using command-line tools, which I find to be faster and simpler, and depending on your needs, can also be scripted. For example:
tshark -r file.pcap -Y dns.resp.ttl -T fields -e dns.resp.ttl -E aggregator=/s | sort -nr | head -1
This command:
Utilizes the Wireshark command-line companion capture tool tshark to read the given file, filtering only for those packets containing a dns.resp.ttl field and then writing only that field to stdout, which is then piped to sort
sort is then instructed to conduct a reverse numeric sort (so highest-to-lowest value instead of the default lowest-to-highest) and pipe that output to head
head -1 will then display only the 1st line of output (instead of the default 10 lines), which will be the largest value ... probably*.
Refer to the tshark man page for more details about the options I used, such as -Y and -e, and to the sort and head man pages for more details about those commands.
*You should know that it's possible for some DNS packets to contain more than one occurrence of the dns.resp.ttl field, so this command may not always give you the largest overall value if the largest value happens to be contained within a packet with multiple occurrences of that field and where it isn't the first occurrence. This is also true for the Wireshark solution though. In other words, when you sort the column from high-to-low, the largest value may not necessarily be the first one if a packet contains multiple occurrences of the field because the sort only takes into account the value of the first occurrence.
I am having trouble creating a particular type of visualization in Kibana. My events in Kibana are statistics on communications between two ip address. Two of the fields are lists of ports used by the particular ip address. An example of the fields would be:
ip1 = 192.168.101.2
ip2 = 192.168.101.3
ip2Ports = 80,443
ip1Ports = 80,57000,0
I would like to have a top count of all the values such as
port count
80 2
57000 1
443 1
I have been able to parse ip2Ports to be ip2Ports_List.column1, ip2Ports_List.column2, ect, but I can only choose one term with term aggregation in the visualization. I can split the chart, but that leads to separate counts for each field. If I go by the original ip2Ports field, it is just aggregated as the string such as, "80,443".
Is it even possible to create a top count visualization of fields with multiple values? If so, how would I do so. If not, is there a way to restructure my data so I can do it? Thank you!
My issue stemmed from the format of the values being sent in by Logstash. I had thought that the 'ip2Ports_List.column1' format, which was a result from using the csv filter, was part of an array. It wasn't. After analyzing it, 'ip2Ports_List.column1' didn't seem to be much different from a new field.
Elastic needed an array to give me the visualization I wanted. I wasn't sure what the best way to produce it was, so I just ended up using the ruby filter. This is what the code ended up looking like:
ruby {
code => "fields = event.get('portsIp').split(',')
event.set('portsIpArray',fields)"
}
Where 'portsIp' looked something like "80,443". Splitting it turned 'portsIp' into a Ruby array. I just set that array as the value for a new event field, 'portsIpArray'.
From there when I tried visualize the 'portsIpArray' field, it looked exactly how I wanted it to, treating each port as separate value, and still associating each port with the same event/field.
Extra:
Also something I discovered is if you're writing your code like I was, directly in the Logstash conf file, Logstash doesn't like it if you use double quotes within the double quoted code. In hindsight it makes sense, but it doesn't give a clear error so it's difficult to figure out.
So, say I have an event coming into Logstash as a multiline object (there are many events that all basically match the pattern below):
Starting script at 2015-11-12 15:06 EST
Found result a at 127.0.0.1
Found result b at 127.0.0.1
Found result c at 0.0.0.0
Script ended at 2015-11-12 15:07 EST
How would I go about matching this in such a way as to store each of the "Found ..." lines separately?
My current config file is something like:
filter {
grok {
break_on_match => false
match => {
"message" => [
"Starting script at ${TIMESTAMP_ISO8601:run_time}",
"Found result %{GREEDYDATA:result} at ${IP:result_ip}"
]
}
}
}
As it stands, this only captures one of the "Found result..." lines. (That is, it matches them all, but only stores one of them - there's only one result variable output.) I'd like to individually capture them, and store them as an... well, anything, so long as they're all there.
Is there a way to capture multiple of the same pattern and store all of the resultant capture data distinctly, while keeping the whole multiline event together so that I can tie it to header data such as the script start time?
I think you can use the split filter to achieve what you want. It allows you to split one event into several parts. The indivdual parts are all copies of the original event as far as I remember. You have to play with the terminator parameter which controls when the message is split into parts.
Check out the docs at: https://www.elastic.co/guide/en/logstash/current/plugins-filters-split.html#plugins-filters-split-target
I am using Logstash (with Kibana as the UI). I would like to extract some fields from my logs so that I can filter by them on the LHS of the UI.
A sample line from my log looks like this:
2013-07-04 00:27:16.341 -0700 [Comp40_db40_3720_18_25] client_login=C-316fff97-5a19-44f1-9d87-003ae0e36ac9 ip_address=192.168.4.1
In my logstash conf file, I put this:
filter {
grok {
type => "mylog"
pattern => "(?<CLIENT_NAME>Comp\d+_db\d+_\d+_\d+_\d+)"
}
}
Ideally, I would like to extract Comp40_db40_3720_18_25 (the number of digits can vary, but will always be at least 1 in each section separated by _) and client_login (can also be client_logout). Then, I can search for CLIENT_NAME=Comp40... CLIENT_NAME=Comp55, etc.
Am I missing something in my config to make this a field that I can use in Kibana?
Thanks!
If you are having any difficulty getting the pattern to match correctly, using the Grok Debugger is a great solution.
For your given problem you could just separate out your search data into another variable, and save the additional varying digits in another (trash) variable.
For example:
(?<SEARCH_FIELD>Comp\d+)%{GREEDYDATA:trash_variable}]
(Please use the Grok Debugger on the above pattern)