I am new to Graylog pipeline rules. I want to write a pipeline to skip the non-existing keys in the look up table from indexing.
we have 20 Ip addresses as Key and 4 regions as values. we are receiving more than 100 IP addresses from source. I want to index only 10 existing IP addresses in the look up table.
CSV file:
"IP_Address","region"
"IP1","region1"
"IP2","region4"
"IP3","region3"
"IP4","region1"
"IP5","region2"
"IP6","region4"
"IP7","region1"
I have tried by using many functions as below.
Option 1: rule "look up IP address"
when
is_null(lookup_value("IP_Addr_table", IP_Address))
then
drop_message();
end
Option 2: Tried with two stages
stage: 0
rule "lookup region"
when
has_field("region")
then
let device_region = lookup_value("IP_addr_table", to_string($message.IP_Address));
end
Stage: 1
rule "look up region"
when
has_field("device_region") -- I want to check the variable value here.
then
end
Also, can anyone provide a good place to learn pipelines and how to use rules other than Graylog documentation
Just for reference - the same question is asked in the Graylog Community:
https://community.graylog.org/t/pipeline-rule-to-stop-indexing-for-non-lookup-values/9580
My answer will be given in that place.
Related
The following query fails with 2 ParserFailure errors, both on line 5. At least that's where the query builder shows the red curly line.
The intention of this query is probably obvious to the Azure KQL initiates, but I'll explain nonetheless just to make sure it's clear.
This query should list all NSGs that do not have either one of the rules named "AllowThis" or "AllowThat".
Resources
| where type == "microsoft.network/networksecuritygroups"
| where isnotempty(properties.securityRules)
| where not(properties.securityRules
| where (tolower(tostring(properties.securityRules.ruleName)) =~ "allowthis|allowthat"))
| project NSGName = name
| order by NSGName asc
It would even be nicer if the table shows the actual missing rule(s) for the listed NSGs, but I have no idea where to start with that.
Does anyone have a working version of this type of query? Having to go through a lot of NSGs manually can't be the answer.
I have tried multiple variations of the query, but I couldn't find a single working version.
Below are my findings and observations from the query posted in question.
Lines 1 to 3 looks good and will give you list of NSG resources which has values for "securityRules" field.
For line number 4
| where not(properties.securityRules)
I am not sure what are you trying to achieve in this step. The not() takes bool values as mentioned in the documentation.
For line number 5
| where (tolower(tostring(properties.securityRules.ruleName)) =~ "allowthis|allowthat")
There is no need to use tolower() when you are using =~ as this supports case-insensitive match. Also under "securityRules" in NSG json object there is no field named as "ruleName", however there is a field "name". Please find the document for the same - Link. You can use the same documentation to check for the fields available to query NSG resource data.
When you are trying to write condition for "AllowThis" or "AllowThat" in Azure Resource Graph Explorer you should use the syntax properties.securityRules.name == "allowthis" or properties.securityRules.name == "allowthat"
If you write anything within quotes it will be taken as single string. Hence in your query "allowthis|allowthat" will be considered as a single string.
I am trying to "beautify" the data I receive from some windows logs on Graylog. My idea is to change the windows log ID from a number to the actual definition for that ID. For example: I receive a log with ID 4625, I want to show in my widget "An account failed to log on".
To do that, I am using a pipeline and a lookup table, which reads the IDs and the respective definitions in natural language from a .csv that I've uploaded on the server.
This is the rule that I wrote for my pipeline, that doesn't seem to work:
rule "eventid_windows_rule"
when
has_field("winlogbeat_winlog_event_id")
then
let winlogbeat_winlog_italiano = lookup("winlogbeat_winlog_event_id", to_string($message.winlogbeat_winlog_event_id));
set_field("winlogbeat_winlog_italiano", winlogbeat_winlog_italiano);
end
I think my problem is specifically in this rule, because Graylog allows to test the lookup tables, and if I manually write an ID, the lookup table finds the respective description.
I solved the issue myself, this is the correct code for the rule:
rule "eventid_windows_rule"
when
has_field("winlogbeat_winlog_event_id")
then
let winlogbeat_winlog_italiano = lookup("eventid_widget_windows_lookup", $message.winlogbeat_winlog_event_id);
set_field("winlogbeat_winlog_italiano", winlogbeat_winlog_italiano);
end
This rule checks if the log has the field "winlogbeat_winlog_event_id", then it generates the new field "winlogbeat_winlog_italiano", associates the numeric value of "winlogbeat_winlog_event_id" with the description in natural language thanks to the .csv that I've created, then puts the description in the field "winlogbeat_winlog_italiano".
Using Terraform to configure vSphere vms, I'd like to be able to provide an IP address (and gateway and netmask) in the tfvars file, but have the vm default to using DHCP if the values are not provided. I know it will use DHCP if the 'vsphere_virtual_machine' resources' 'customize' block contains an empty 'network_interface' block. I was hoping that be giving a default value of "" to the settings in the variables.tf file I could set values if present and use DHCP if not, but I get an error stating:
Error: module.vm.vsphere_virtual_machine.node:
clone.0.customize.0.network_interface.0.ipv4_netmask: cannot parse ''
as int: strconv.ParseInt: parsing "": invalid syntax
So putting in a blank string won't parse, and it won't just leave the whole network_interface blank if the values are blank.
I can't use COUNT on a subresource, so the only thing I've come up with so far is to put two entire, nearly identical, 'vsphere_virtual_machine' resources into my module and then put COUNT statements on both so only one gets created, depending on whether the network settings are provided or not, but man, does that seem ugly...?
I think you are in luck. I've been waiting for this exact same problem to be solved since almost a year now.
Lo and behold, Terraform v0.12.0-alpha1:
They now support dynamic block definitions instead of just static ones
Enjoy, while I'm gonna throw away a couple of hundreds of lines worth of hacks just like the one you mentioned...
For example:
If Marathon is running a task named /cassandra, Mesos-DNS assigns it a DNS name - cassandra.marathon.mesos.
Now I have a task named /monit/promdash. How can I find its DNS name?
Already tried:
monit_promdash.marathon.mesos, promdash_monit.marathon.mesos (and with - instead of _), monit.marathon.mesos, promdash.marathon.mesos, ...)
There's a HTTP interface. Couldn't find how to list all DNS names either...
Thanks,
Marathon reverses the hierarchical names, concatenates them with - and this is the app name then, so in your case it would be promdash-monit.marathon.mesos. Try it out.
At the bottom of the Mesos-DNS naming documentation we provide some more details about how these FQHN are constructed and you can also check out a complete end-to-end example I've put together, using two levels of hierarchies.
HI everyone first of all thank you for visiting my question
I am working with a new IDS, OSSIM, It's database requires a host's:
Name, CIDR, and Description in a .csv format for uploading through a web UI.
Version 4.x.x: "Netname";"CIDRs(CIDR1,CIDR2,...
)";"Description";"Asset value"*;"Net ID"
Currently I have the full list of hosts in a .txt file like so,
Department1 129.252.136.128 129.252.136.255 contact1#email.com,contact2#email.com,contact3#email.com
Department2 129.252.154.64 129.252.154.127 contact1#email.com
If anyone has any Idea how to get the IP range converted into CIDR notation then the file into a .csv format I would greatly appreciate it.
For CIDR1 and CIDR2 in their binary representation, compare CIDR1 and CIDR2 bit-by-bit and set a bit in the target netmask until the bits stopped matching.
Example:
CIDR1 = 192.168.127.0 = 11000000101010000111111100000000
CIDR2 = 192.168.127.32 = 11000000101010000111111100100000
Netmask = 255.255.255.192 = 11111111111111111111111111000000
EDIT
In order to automate this, as per your comment, you ought to use a language that has easy access to IP functions, e.g. php has ip2long function.ip2long as well as CSV handling functions fgetcsv function.fgetcsv