Need help in configuring email alert in logstash 1.5.4 - logstash

I was using logstash 1.4.2 and this email alert was working fine at that time. Now my system is upgraded to logstash 1.5.4 and logstash is failing because of older settings as below:
email {
from => "{{Mailfrom}}"
match => [ "ERROR ALERT", "LOGLEVEL,ERROR" ]
subject => "%{matchName}"
to => "{{Rcptto}}"
via => "smtp"
htmlbody => "<h2>%{matchName}</h2><br/><br/><div
align='center'>%{message}</div>"
options => [
"smtpIporHost", "{{smtpIporHost}}",
"port", "{{smtpPort}}",
"domain", "{{mailDomain}}",
"userName", "{{Mailfrom}}",
"password", "{{MailFromPassword}}",
"authenticationType", "plain",
"starttls", "true"
]
}
Now I am getting error as "You are using a deprecated config setting \"match\" set in email. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. If you have any questions about this, please visit the #logstash channel on freenode irc."
Can anyone help me here, as how should i configure it according to ERROR condition in 1.5.4?
Thanks in advance!

Correct configuration:
if ([LOGLEVEL] =~ /ERROR/) {
email {
from => "{{Mailfrom}}"
subject => "ERROR ALERT"
to => "{{Rcptto}}"
via => "smtp"
htmlbody => "<h2>ERROR ALERT</h2><br/><br/><div align='center'>%{message}</div>"
options => [
"smtpIporHost", "{{smtpIporHost}}",
"port", "{{smtpPort}}",
"domain", "{{mailDomain}}",
"userName", "{{Mailfrom}}",
"password", "{{MailFromPassword}}",
"authenticationType", "plain",
"starttls", "true"
]
}
}
Explanation:
According to the deprecated documentation of logstash output email the syntax of match => is:
{ "match name", "field.in.event,value.expected, , operand(and/or),field.in.event,value.expected, , or...", "match name", "..." }
In your given example the field is LOGLEVEL and the expected value is ERROR.
From the new documentation of logstash output email:
This setting is deprecated in favor of Logstash’s "conditionals"
feature If you were using this setting previously, please use
conditionals instead.
The corresponding conditional that checks if field LOGLEVEL contains ERROR:
if ([LOGLEVEL] =~ /ERROR/) { }

Related

DocuSign Api creating envelope in Production

I have a problem with creating Envelope through the Api.
The authorization is performed without errors.
$options = new \DocuSign\eSign\Api\AuthenticationApi\LoginOptions();
$loginInformation = $authenticationApi->login($options);
I get this in response:
LoginInformation {
#container: array:2 [
"api_password" => null
"login_accounts" => array:1 [
0 => LoginAccount {
#container: array:11 [
"account_id" => "xxxxx"
"account_id_guid" => null
"base_url" => "https://na2.docusign.net/restapi/v2/accounts/xxxxx"
"email" => "xxxxx#xxxxxx.com"
"is_default" => "true"
"login_account_settings" => null
"login_user_settings" => null
"name" => "xxxxxxx, LLC"
"site_description" => ""
"user_id" => "xxxxxxxxxxxxxxxxxxx"
"user_name" => "xxxxxx Contracts Team"
]
}
]
]
}
But when I’m trying to create the Envelope this way:
$envelopeApi->createEnvelope($this->config->getAccountId(), $envelop_definition, $options);
I get this error: «[401] Error connecting to the API (https://www.docusign.net/restapi/v2/accounts/xxxxx/envelopes)”
This error notifies me that I’m not authorized. This code works properly in Sandbox.
I think that the problem is in settings of account - maybe there is a lack of special rights...
You need to use the right production platform. The right production platform is determined by the DocuSign Account ID.
See step 1 in the Post Go Live document.

How to capture repeated pattern in logstash(5.4.0) grok?

I would appreciate if someone can help me out with logstash grok.
Given a log like below ,
IN 192.168.11.2 IN 192.168.11.3
My goal is to put the ip address into array using grok. List of ip is dynamic and possible to extend more than 2.
e.g
tmp = [
"192.168.11.2", "192.168.11.3"
]
However, if I use a filter like below it ends up in single field.
filter {
grok {
match => { "message" => "(?<tmp>(IN %{IPV4}(\s)?)*)" }
}
}
Result,
"path" => "/tmp/sample.csv",
"#timestamp" => 2017-08-24T05:00:08.093Z,
"tmp" => "IN 192.168.11.2 IN 192.168.11.3",
"#version" => "1",
"host" => "host.ywlocal.net",
"message" => "IN 192.168.11.2 IN 192.168.11.3"
Would this be possible?
You can use the ruby filter for more advanced parsing:
filter {
ruby {
code => "event.set('ips') = event.get('message').scan(/\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\/)"
}
}
Regexp is not 100% correct to match ip address but should work for your needs

How to test if jmx plugin is working in logstash

I am new to the whole ElasticSearch framework and have downloaded an installed the logstash-input-jmx plugin and now I need to test my configuration but I can't find in any of the LogStash documentation exactly how to test the plugin. All they have in the plugin documentation in GitHub is a sentence down at the bottom that says to start LogStash and test your plugin, they don't tell you exactly how to accomplish that. As a matter of fact that seems to be the standard blurb for all of the plugins which isn't very helpful if you're coming in without any knowledge of the framework.
Here are some details for my configuration if that helps:
logstash.conf
input {
jmx
{
path => "file://*machinename*/D$/LS/logstash-5.1.1/config/jmx"
polling_frequency => 15
type => "jmx"
}
}
filter {
it [type] == "jmx" {
if ("Memory.HeapMemoryUsage" in [metric_path] or "Memory.NonHeapMemoryUsage" in [metric_path]) {
ruby {
code => "event['memoryUsage'] = event['metric_value_number'] * 100"
add_tag => [ "memoryUsage" ]
}
}
}
}
jmx.conf:
{
"host" : *ip address of machine*,
"port" : *jmx listener port*,
"queries" : [
"object_name" : "java.lang:type=Memory",
"object_alias" : "Memory"
]
}
TIA,
Bill
Figured it out by doing a complete uninstall/reinstall of the framework and found a very good tutorial on Ivan Krizsan's blog (https://www.ivankrizsan.se/2015/09/27/jmx-monitoring-with-the-elk-stack/) that was instrumental in helping me get the plug-in up and running.

Logstash. Get fields by position number

Background
I have the scheme: logs from my app go through rsyslog to central log server, then to Logstash and Elasticsearch.
Logs from app is a pure JSON, but rsyslog adds to log "timestamp", "app name" and "server name" fileds. And log becomes to this:
timestamp app-name server-name [JSON]
Question
How can I remove first three fields with Logstash filters?
Can I get fields by position numbers (like in awk) and do something like:
filter {
somefilter_name {
remove_field => $1, $2, $3
}
}
Or maybe my vision is totally wrong and I must do this in another way?
Thank you!
Use grok{} to match them (they may be useful on their own!) and put the remainder of the event back into the [message] field:
Given input like:
2015-06-16 13:37:30 myApp myServer { "jsonField": "jsonValue" }
And this config:
grok {
pattern => "%{TIMESTAMP_ISO8601:timestamp} %{WORD:app} %{WORD:server} %{GREEDYDATA:message}"
overwrite => [ "message" ]
}
json {
source => "message"
}
Will produce this document:
{
"message" => "{ \"jsonField\": \"jsonValue\" }",
"#version" => "1",
"#timestamp" => "2015-06-16T20:38:55.658Z",
"host" => "0.0.0.0",
"timestamp" => "2015-06-16 13:37:30",
"app" => "myApp",
"server" => "myServer",
"jsonField" => "jsonValue"
}

Logstash email alert

I'm trying to configure logstash to send mail when someone login my server. But it seems doesn't work. This is my config file in /etc/logstash/conf.d/email.conf
My file:
input {
file {
type => "syslog"
path => "/var/log/auth.log"
}
}
filter {
if [type] == "syslog" {
grok {
pattern => [ "%{SYSLOGBASE} Failed password for %{USERNAME:user} from % {IPORHOST:host} port %{POSINT:port} %{WORD:protocol}" ]
add_tag => [ "auth_failure" ]
}
}
}
output {
email {
tags => [ "auth_failure" ]
to => "<admin#gmail.com>"
from => "<alert#abc.com>"
options => [ "smtpIporHost", "smtp.abc.com",
"port", "25",
"domail", "abc.com",
"userName", "alert#abc.com",
"password", "mypassword",
"authenticationType", "plain",
"debug", "true"
]
subject => "Error"
via => "smtp"
body => "Here is the event line %{#message}"
htmlbody => "<h2>%{matchName}</h2><br/><br/><h3>Full Event</h3><br/><br/><div align='center'>%{#message}</div>"
}
}
My logstash file /var/log/logstash/logstash.log
{:timestamp=>"2015-03-10T11:46:41.152000+0700", :message=>"Using milestone 1 output plugin 'email'. This plugin should work, but would benefit from use by folks like you. Please let us know if you find bugs or have suggestions on how to improve this plugin. For more information on plugin milestones, see http://logstash.net/docs/1.4.1/plugin-milestones", :level=>:warn}
any body please help!
You're not using the correct syntax in your grok filter. It should look like this:
grok {
match => ["message", "..."]
}
Other minor comments:
Using tags => ["auth_failure"] for conditional filtering is deprecated. Prefer if "auth_failture" in [tags].
In the email body you're referring to the message with #message. That's deprecated too and the field is named plain message.

Resources