I have followed this guide to configure ElasticSearch 2.3.0, Logstash 2.3.0, Kibana 4.5.0 to get logs from my pfsense 2.3:
http://pfelk.3ilson.com/
My problem is that I use pfsense 2.3 and this tutorial is for pfsense 2.2. Logs are received and showed correctly by Kibana, but the format and the Available Fields are not all that I need to do a dashboard.
Where can I find a grok compatible with pfsense 2.3?
Thank you
Finally I found out that the problema was my pfSense. I tried with another one and now eveything Works fine!
Related
Using Elasticsearch 7.3 on CentOS 7 and Java version 11, I need to parse and find specific Error/Exception in Java Application log files, wondering if I should use Filebeat or Metricbeat to parse and ship some text patterns in log files to Elastic?
Sample code to show how to parse/ship specific error pattern is very helpful.
Any help is appreciated.
Filebeat is for logs.
To make your life easier, I'd use a structured log format. We have just released https://github.com/elastic/java-ecs-logging which ties right into Elasticsearch and also provides the right Filebeat configuration.
Hi everyone!
I have "ELK" (6.4.2) working perfectly with filebeat, metricbeat, packetbeat and winlogbeat in CentOS 7 x86_64 (Kernel 3.10.0-862.11.6.el7.x86_64).
I'm trying to integrate zipkin + elk (see https://logz.io/blog/zipkin-elk/), but Elasticsearch does not create indices with Kibana.
When trying to create the indices in Kibana, the process does not end. (Follow logs below).
I suspect the zipkin connection drivers are not compatible with elk 6.4.2. Has anyone had the same problem and has a "light at the end of the tunnel"?
Tks for all!
Java version:
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
Zipkin startup:
java -DSTORAGE_TYPE=elasticsearch -DES_HOSTS=http://localhost:9200 -jar /opt/zipkin.io/bin/zipkin.jar
Error log in Elasticsearch:
[2018-10-24T11:31:59,933][WARN ][o.e.d.i.m.MapperService ] Setting index.mapper.dynamic is deprecated since indices may not have more than one type anymore.
[2018-10-24T11:31:59,936][WARN ][o.e.d.i.m.MapperService ] [_default_] mapping is deprecated since it is not useful anymore now that indexes cannot have more than one type
[2018-10-24T11:31:59,954][WARN ][o.e.d.i.m.MapperService ] Setting index.mapper.dynamic is deprecated since indices may not have more than one type anymore.
[2018-10-24T11:32:00,033][WARN ][o.e.d.c.m.MetaDataCreateIndexService] index or alias name [zipkin:span-2018-10-24] containing ':' is deprecated and will not be supported in Elasticsearch 7.0+
[2018-10-24T11:32:00,245][WARN ][o.e.d.i.m.MapperService ] Setting index.mapper.dynamic is deprecated since indices may not have more than one type anymore.
[2018-10-24T11:33:47,717][WARN ][o.e.d.a.a.i.t.p.PutIndexTemplateRequest] Deprecated field [template] used, replaced by [index_patterns]
Here is the related issue:
we also mentioned recently that for data to appear, applications need to be
sending traces
https://github.com/openzipkin/zipkin#quick-start
you can tell also by hitting the /metrics endpoint and look at stats named
collector
https://github.com/openzipkin/zipkin/issues/1939
I opened a issue on the zipkin github, a theme already being treated as a bug.
Initial thread:
https://github.com/openzipkin/zipkin/issues/2218#issuecomment-432876510
Bug track:
https://github.com/openzipkin/zipkin/issues/2219
Tks for all!
I used nutch1.10 but i cannot find the appropriate version of elasticsearch for indexing.I used elasticsearch 2.4 but the log of elasticsearch displays not support.so does 2.1.0 ?I want to konw what is the appropriate version?
According to https://github.com/apache/nutch/blob/branch-1.10/src/plugin/indexer-elastic/ivy.xml#L39 the correct version should be elasticsearch 1.4.1.
We are using Elasticsearch 2.3.4 and Kibana 4.5.3 in our application. We would like to add authentication to our Kibana server. That means, when running the kibana server from browser, it should prompt for user name and password.
We are looking open source software's/plugins to be added to Kibana and Elastic server.
Thanks for your help in advance.
Kibana work really good with the plugin Shield.
shield plugin is not an open source project but instead you can use searchguard or elasticfence.
I am struggling with getting ISP info using Logstash GeoIP plugin with Logstash Version: 2.4.
I also tried logstash 5.0 Beta but no luck in that too.
can someone please help me in this