HI everyone first of all thank you for visiting my question
I am working with a new IDS, OSSIM, It's database requires a host's:
Name, CIDR, and Description in a .csv format for uploading through a web UI.
Version 4.x.x: "Netname";"CIDRs(CIDR1,CIDR2,...
)";"Description";"Asset value"*;"Net ID"
Currently I have the full list of hosts in a .txt file like so,
Department1 129.252.136.128 129.252.136.255 contact1#email.com,contact2#email.com,contact3#email.com
Department2 129.252.154.64 129.252.154.127 contact1#email.com
If anyone has any Idea how to get the IP range converted into CIDR notation then the file into a .csv format I would greatly appreciate it.
For CIDR1 and CIDR2 in their binary representation, compare CIDR1 and CIDR2 bit-by-bit and set a bit in the target netmask until the bits stopped matching.
Example:
CIDR1 = 192.168.127.0 = 11000000101010000111111100000000
CIDR2 = 192.168.127.32 = 11000000101010000111111100100000
Netmask = 255.255.255.192 = 11111111111111111111111111000000
EDIT
In order to automate this, as per your comment, you ought to use a language that has easy access to IP functions, e.g. php has ip2long function.ip2long as well as CSV handling functions fgetcsv function.fgetcsv
Related
Is there a way to have excel read text and decipher whether it does or doesn’t have certain character/letters?
Here is my example sheet
I am looking for something that deciphers using
these guidelines. 1. If entry has a / then output
URL. 2. If entry is not a URL and has only numbers
and special characters then output IP. 3. If entry is
not a URL or IP and has more than 1
dots/periods/decimals then output HOST. If entry
is not a URL, IP, or HOST (or only has 1
dot/period/decimal) then output FQDN.
Here is an example of what I'm looking for
I have tried using these below:
=IF(LEN(A1)-LEN(SUBSTITUTE(A1,"/“,””))=1,"URL",IF(LEN(A1)-LEN(SUBSTITUTE(A1,”.”,""))=1,"FQDN"‚IF(LEN(A1)-LEN(SUBSTITUTE(A1,".",”"))>1,"HOST")))
That works for reading URL, HOST, and FQDN;
however, it reads IP's as HOST's.
I have also used
=IF(OR(ISNUMBER(SEARCH({"A","B","C",”D","E","F”,"G","H","I","J","K",”L”,"M",”N","O","P","Q”,"R","S","T","U","V","W",”X","Y","Z"},A1))),””,"IP")
That works for reading if an entry contains letters and if not it outputs IP.
Is there a way to combine these or simplify what I am trying to do?
Thanks!
This produces the desired output for your sample (at least)
=IF(COUNTIF(A1,"*/*"),"URL",IF(ISNUMBER(VALUE(SUBSTITUTE(A1,".",""))),"IP",IF(LEN(A1)-LEN(SUBSTITUTE(A1,".",""))>1,"HOST","FQDN")))
A possible solution (tested with O365) :
=IFS(ISNUMBER(VALUE(LEFT(A1:A5)))=TRUE,"IP",LEN(A1:A5)-LEN(SUBSTITUTE(A1:A5,".",""))>1,"HOST",LEN(A1:A5)-LEN(SUBSTITUTE(A1:A5,"/",""))=1,"URL",LEN(A1:A5)-LEN(SUBSTITUTE(A1:A5,".",""))=1,"FQDN")
Classical way (in B1) :
=IF(ISERROR(SEARCH("/",A1))=FALSE,"URL",IF(ISERROR(VALUE(LEFT(A1)))=FALSE,"IP",IF(LEN(A1)-LEN(SUBSTITUTE(A1,".",""))>1,"HOST",IF(ISBLANK(A1)=TRUE,"","FQDN"))))
Output :
Using Terraform to configure vSphere vms, I'd like to be able to provide an IP address (and gateway and netmask) in the tfvars file, but have the vm default to using DHCP if the values are not provided. I know it will use DHCP if the 'vsphere_virtual_machine' resources' 'customize' block contains an empty 'network_interface' block. I was hoping that be giving a default value of "" to the settings in the variables.tf file I could set values if present and use DHCP if not, but I get an error stating:
Error: module.vm.vsphere_virtual_machine.node:
clone.0.customize.0.network_interface.0.ipv4_netmask: cannot parse ''
as int: strconv.ParseInt: parsing "": invalid syntax
So putting in a blank string won't parse, and it won't just leave the whole network_interface blank if the values are blank.
I can't use COUNT on a subresource, so the only thing I've come up with so far is to put two entire, nearly identical, 'vsphere_virtual_machine' resources into my module and then put COUNT statements on both so only one gets created, depending on whether the network settings are provided or not, but man, does that seem ugly...?
I think you are in luck. I've been waiting for this exact same problem to be solved since almost a year now.
Lo and behold, Terraform v0.12.0-alpha1:
They now support dynamic block definitions instead of just static ones
Enjoy, while I'm gonna throw away a couple of hundreds of lines worth of hacks just like the one you mentioned...
I am having trouble creating a particular type of visualization in Kibana. My events in Kibana are statistics on communications between two ip address. Two of the fields are lists of ports used by the particular ip address. An example of the fields would be:
ip1 = 192.168.101.2
ip2 = 192.168.101.3
ip2Ports = 80,443
ip1Ports = 80,57000,0
I would like to have a top count of all the values such as
port count
80 2
57000 1
443 1
I have been able to parse ip2Ports to be ip2Ports_List.column1, ip2Ports_List.column2, ect, but I can only choose one term with term aggregation in the visualization. I can split the chart, but that leads to separate counts for each field. If I go by the original ip2Ports field, it is just aggregated as the string such as, "80,443".
Is it even possible to create a top count visualization of fields with multiple values? If so, how would I do so. If not, is there a way to restructure my data so I can do it? Thank you!
My issue stemmed from the format of the values being sent in by Logstash. I had thought that the 'ip2Ports_List.column1' format, which was a result from using the csv filter, was part of an array. It wasn't. After analyzing it, 'ip2Ports_List.column1' didn't seem to be much different from a new field.
Elastic needed an array to give me the visualization I wanted. I wasn't sure what the best way to produce it was, so I just ended up using the ruby filter. This is what the code ended up looking like:
ruby {
code => "fields = event.get('portsIp').split(',')
event.set('portsIpArray',fields)"
}
Where 'portsIp' looked something like "80,443". Splitting it turned 'portsIp' into a Ruby array. I just set that array as the value for a new event field, 'portsIpArray'.
From there when I tried visualize the 'portsIpArray' field, it looked exactly how I wanted it to, treating each port as separate value, and still associating each port with the same event/field.
Extra:
Also something I discovered is if you're writing your code like I was, directly in the Logstash conf file, Logstash doesn't like it if you use double quotes within the double quoted code. In hindsight it makes sense, but it doesn't give a clear error so it's difficult to figure out.
I am looking for a fast way to check if IP addresses are part of a list of CIDR notated IP ranges. I've seen examples before use netaddr like:
from netaddr import IPNetwork, IPAddress
for CIDR in CIDRLIST:
if IPAddress(row[0]) in IPNetwork(CIDR):
print('success')
However this solution is way too slow for my problem (800 IP ranges in CIDR and 500.000 IP adresses).
What could be a way to do this faster? I've read about using pytries, but I am not certain this is the solution.
Patricia/Radix tree/tries seem to be the answer. I found them by searching for algorithms for looking up routing tables.
There is a python implementation here.
A little later: I now have this working fine in Ruby:
require 'rpatricia'
require 'uoainfoblox'
ib = UoAIinfoblox.new ({'user' => 'xxxxx', 'password' => 'yyyy', 'host' => 'ipam.auckland.ac.nz'})
pt = Patricia.new
ib.get_networks('*roaming_network=true').each do |net, info |
pt.add(net)
end
puts "'130.216.66.65 #{ pt.include?('130.216.66.65')}"
puts "130.216.5.128 #{pt.include?('130.216.5.128') }"
Infoblox is an IP Management system and UoAInfoblox is a wrapper around their web api. So here I get a list of the roaming networks add them into a patricia tree and then check two IP addresses (that I know the status of).
Edit: I have just found out from a friend who uses python and who teaches networking in our CS department that he used the python radix module in his research scripts. I know he was processing very large amounts of data from a /8 darkenet for CAIDA.
Over at dmarc.org, it's suggested that IN TXT records of DNS could be written in a special form within the zone file to make them not overflow the lines in your text editor.
The DMARC policy record might look like this when retrieved using a
common command-line tool:
% dig +short TXT _dmarc.example.com.
"v=DMARC1\; p=none\; rua=mailto:dmarc-feedback#example.com"
To publish such a record, the DNS administrator for the Domain Owner
creates an entry like the following in the appropriate zone file
(following the conventional zone file format):
; DMARC record for the domain example.com
_dmarc IN TXT ( "v=DMARC1; p=none; "
"rua=mailto:dmarc-feedback#example.com" )
I've tried following the example in my actual zone file with NSD; however, when I then query the domain, I actually get the results wrapped up on multiple lines, too.
% dig +short TXT _dmarc.example.su
"v=DMARC1\; " "p=reject\; " "rua=mailto:rua-dmarc#example.su"
Is this expected? Is this likely to break some software that's supposed to parse these TXT records to get the DMARC / SPF / DKIM / etc?
The individual components of a TXT record may only contain up to 255 characters each, since they're transmitted on the wire in <length><data ...> format.
Any code that's potentially expecting to take more than 255 characters SHOULD be able to coalesce multiple components into a single character array.
In master file format the braces surrounding the strings indicate that multiple components are to be included in a single TXT record - without them this would have created two separate TXT records, and the relative order of the two records would be undefined and subject to change.
It's hard to tell since it will depend on the final implementation of the DMARC checker. However, even detailed in the DMARC document, a DMARC record is no such big that you could overflow your editor.
In my case, using the last opendmarc package on Ubuntu Trusty (14.04) along with Postfix, made the daemon crash when processing some weird/malformed DMARC DNS records (but not exactly the case you mention).
I would simply add the one-line approach and play it safe, not just because of a possible break of the checker software, but even worse would be getting your mail rejected because policy doesn't seem to be alligned when it actually is!
So I'd just add something like this:
_dmarc.example.su IN TXT "v=DMARC1; p=reject; rua=mailto:rua-dmarc#example.su"