I've got a problem with running logstash. My conf looks like this:
input {
udp {
port => 1514
type => docker
}
}
filter {
grok {
match => {
"message" => "<%{NUMBER}>%{DATA}(?:\s+)%{DATA:hostname}(?:\s+)%{DATA:imageName}(?:\s+)%{DATA:containerName}(?:\s*\[%{NUMBER}\]:) (\s+(?<logDate>%{YEAR}-%{MONTHNUM}-%{MONTHDAY}\s+%{HOUR}:%{MINUTE}:%{SECOND}) %{LOGLEVEL:logLevel}(?:\s*);* %{DATA:logAdditionalInfo};*)?%{GREEDYDATA:logMsg}"
}
keep_empty_captures => false
remove_field => ["message"]
}
}
output {
if [type] == "gelf" {
elasticsearch {
index => "phpteam-%{+YYYY.MM.dd}"
}
} else {
elasticsearch { }
}
}
The configuration is correct, but after running it /var/log/logstash/logstash.log shows the following output:
{:timestamp=>"2016-06-22T11:43:03.105000+0200", :message=>"SIGTERM
received. Shutting down the pipeline.", :level=>:warn}
{:timestamp=>"2016-06-22T11:43:03.532000+0200", :message=>"UDP
listener died", :exception=>#,
:backtrace=>["org/jruby/RubyIO.java:3682:in select'",
"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-udp-2.0.3/lib/logstash/inputs/udp.rb:77:in
udp_listener'",
"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-udp-2.0.3/lib/logstash/inputs/udp.rb:50:in
run'",
"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/pipeline.rb:206:in
inputworker'",
"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/pipeline.rb:199:in
`start_input'"], :level=>:warn}
The only thing I found to workaround this error is to edit those .rb files, but sadly I have no idea how to do it. Could you help me somehow?
Thanks in advance.
I found solution that is not perfect, but works, so maybe it will help somebody.
After installing the whole instance on new server everything works fine.
Everything crashed after upgrade'ing logstash/elasticsearch/kibana, so maybe there's something wrong with configuration files, but I couldn't figure out which ones.
Related
How can I print out a command's output?
let mut telnet_scan_command = Command::new("telnet");
telnet_scan_command.arg("0.0.0.0");
match telnet_scan_command.output() {
Ok(o) => {
println!("{}", String::from_utf8_unchecked(o.stdout))
}
Err(e) => {
println!("Something Went wrong Log: {} ", e)
}
The program runs perfectly fine but the output should be:
Trying 0.0.0.0...
telnet: Unable to connect to remote host: Connection refused
The actual output is:
Trying 0.0.0.0...
How can I get the rest of the output?
Using logstash 2.3.4-1 on centos 7 with kafka-input plugin I sometimes get
{:timestamp=>"2016-09-07T13:41:46.437000+0000", :message=>#0, :events_consumed=>822, :worker_count=>1, :inflight_count=>0, :worker_states=>[{:status=>"dead", :alive=>false, :index=>0, :inflight_count=>0}], :output_info=>[{:type=>"http", :config=>{"http_method"=>"post", "url"=>"${APP_URL}/", "headers"=>["AUTHORIZATION", "Basic ${CREDS}"], "ALLOW_ENV"=>true}, :is_multi_worker=>false, :events_received=>0, :workers=>"", headers=>{..}, codec=>"UTF-8">, workers=>1, request_timeout=>60, socket_timeout=>10, connect_timeout=>10, follow_redirects=>true, pool_max=>50, pool_max_per_route=>25, keepalive=>true, automatic_retries=>1, retry_non_idempotent=>false, validate_after_inactivity=>200, ssl_certificate_validation=>true, keystore_type=>"JKS", truststore_type=>"JKS", cookies=>true, verify_ssl=>true, format=>"json">]>, :busy_workers=>1}, {:type=>"stdout", :config=>{"ALLOW_ENV"=>true}, :is_multi_worker=>false, :events_received=>0, :workers=>"\n">, workers=>1>]>, :busy_workers=>0}], :thread_info=>[], :stalling_threads_info=>[]}>, :level=>:warn}
this is the config
input {
kafka {
bootstrap_servers => "${KAFKA_ADDRESS}"
topics => ["${LOGSTASH_KAFKA_TOPIC}"]
}
}
filter {
ruby {
code =>
"require 'json'
require 'base64'
def good_event?(event_metadata)
event_metadata['key1']['key2'].start_with?('good')
rescue
true
end
def has_url?(event_data)
event_data['line'] && event_data['line'].any? { |i| i['url'] && !i['url'].blank? }
rescue
false
end
event_payload = JSON.parse(event.to_hash['message'])['payload']
event.cancel unless good_event?(event_payload['event_metadata'])
event.cancel unless has_url?(event_payload['event_data'])
"
}
}
output {
http {
http_method => 'post'
url => '${APP_URL}/'
headers => ["AUTHORIZATION", "Basic ${CREDS}"]
}
stdout { }
}
Which is odd, since it is written to logstash.log and not logstash.err
What does this error mean and how can I avoid it? (only restarting logstash solves it, until the next time it happens)
According to this github issue your ruby code could be causing the issue. Basically any ruby exception will cause the filter worker to die. Without seeing your ruby code, it's impossible to debug further, but you could try wrapping your ruby code in an exception handler and logging the exception somewhere (at least until logstash is updated to log it).
I am using Logstash 2.2.4.
My current configuration
input {
file {
path => "/data/events/*/*.txt"
start_position => "beginning"
codec => "json"
ignore_older => 0
max_open_files => 46000
}
}
filter {
if [type] not in ["A", "B", "C"] {
drop {}
}
}
output {
HTTP {
http_method => "post"
workers => 3
url => "http://xxx.amazonaws.com/event"
}
}
In an input folder, I have about 25000 static (never updatable) txt files.
I configured --pipeline-workers to 16. In described configuration LS process running 1255 threads and opens about 2,560,685 file descriptors.
After some investigation, I found that LS keeping open files descriptor for all the files in the input folder and HTTP output traffic became very slow.
My question is why LS does not close file descriptor of already processed (transferred) files or implementing kind of input files pagination?
Maybe someone meets the same problem? If yes, please share your solution.
Thanks.
I want to execute an external program via std::process::Command::spawn. Furthermore I want to know the reason why spawning the process failed: is it because the given program name doesn't exist/is not in PATH or because of some different error?
Example code of what I want to achieve:
match Command::new("rustc").spawn() {
Ok(_) => println!("Was spawned :)"),
Err(e) => {
if /* ??? */ {
println!("`rustc` was not found! Check your PATH!")
} else {
println!("Some strange error occurred :(");
}
},
}
When I try to execute a program that isn't on my system, I get:
Error { repr: Os { code: 2, message: "No such file or directory" } }
But I don't want to rely on that. Is there a way to determine if a program exists in PATH?
You can use e.kind() to find what ErrorKind the error was.
match Command::new("rustc").spawn() {
Ok(_) => println!("Was spawned :)"),
Err(e) => {
if let NotFound = e.kind() {
println!("`rustc` was not found! Check your PATH!")
} else {
println!("Some strange error occurred :(");
}
},
}
Edit: I didn't find any explicit documentation about what error kinds can be returned, so I looked up the source code. It seems the error is returned straight from the OS. The relevant code seems to be in src/libstd/sys/[unix/windows/..]/process.rs. A snippet from the Unix version:
One more edit: On a second thought, I'm not sure if the licenses actually allows posting parts of Rust sources here, so you can see it on github
Which just returns Error::from_raw_os_err(...). The Windows version seemed more complicated, and I couldn't immediately find where it even returns errors from. Either way, it seems you're at the mercy of your operating system regarding that. At least I found the following test in src/libstd/process.rs:
Same as above: github
That seems to guarantee that an ErrorKind::NotFound should be returned at least when the binary is not found. It makes sense to assume that the OS wouldn't give a NotFound error in other cases, but who knows. If you want to be absolutely sure that the program really was not found, you'll have to search the directories in $PATH manually. Something like:
use std::env;
use std::fs;
fn is_program_in_path(program: &str) -> bool {
if let Ok(path) = env::var("PATH") {
for p in path.split(":") {
let p_str = format!("{}/{}", p, program);
if fs::metadata(p_str).is_ok() {
return true;
}
}
}
false
}
fn main() {
let program = "rustca"; // shouldn't be found
if is_program_in_path(program) {
println!("Yes.");
} else {
println!("No.");
}
}
I have ganglia set up on a cluster of servers, all of which have gmond, one of which has gmetad, and one which has log stash and elasticsearch. I’d like to use Logstash’s ganglia input plugin to collect data directly from the monitoring daemons, but I’ve been unsuccessful so far. My logstash logs always show:
{:timestamp=>"2015-07-14T14:33:25.192000+0000", :message=>"ganglia udp listener died", :address=>"10.1.10.178:8664", :exception=>#, :backtrace=>["org/jruby/ext/socket/RubyUDPSocket.java:160:in bind'", "/opt/logstash/lib/logstash/inputs/ganglia.rb:61:inudp_listener'", "/opt/logstash/lib/logstash/inputs/ganglia.rb:39:in run'", "/opt/logstash/lib/logstash/pipeline.rb:163:ininputworker'", "/opt/logstash/lib/logstash/pipeline.rb:157:in `start_input'"], :level=>:warn}
Here's the input config I've been testing with:
input {
ganglia {
host => "10.1.10.178" #ip of logstash node
port => 8666
type => "ganglia_test"
}
}
and I have this in gmond.conf on one of the gmond nodes
udp_send_channel {
host = 10.1.10.178 #logstash node
port = 8666
bind_hostname = yes
}
I've found this problem too. It looks like there's a bug in the Ganglia listener since about version 1.2 (I know it used to work in 1.1..)
I managed to work around the problem by adding an explicit 'UDP' listener. This seems to satisfy logstash and allows the Ganglia listener to keep running.
e.g.
input {
udp {
port => "1112"
type => "dummy"
}
ganglia {
port => "8666"
type => "ganglia"
}
}