I am creating a dashboard in Grafana with data from PNP4Nagios for problem resolution. One of the criterias is if there is a connection to a certain service. I have a plugin that verifies this properly. The answer is either connected or not conncted.
Is it possible to generate an output that PNP4Nagios will understand the output so I can add it to my Dashboard?
Was looking for a status plugin for grafana when I found this question.
Pnp4nagios only understands performance data, so as stated by pzkpfw, you need to add that in your check script by adding a pipe after your message and a label=value. Then if you want to display up/down or ok/warning/critical, there's the vonage status panel.
Related
I have a Shodan account and am trying to get it to scan an IP and report the results. Unfortunately, the method reported in the documentation for doing this doesn't seem to work. Here's what I've been doing, using the Shodan CLI. All of these commands are being issued using the same API key.
Used the shodan scan submit command to initiate a scan of the
desired IP.
Used shodan scan list to monitor the status of the scan I submitted. From the list entry, I can get the scan ID and its status. Wait until the scan status is "DONE"
According to the Shodan API documentation, the way to retrieve my results is by using shodan download <download_file_name> scan:<my_scan_id>. However, when I send that command I am informed it is downloading 0 results.
Searching the database with shodan search scan:<my_scan_id> also shows zero results.
I've looked through the documentation and there doesn't seem to be another way of getting results without a dedicated data pipe, which I can't since I'm on the $50 lifetime level. So what's going on? Has the API changed? Does it take time for the results of on-demand scans to be incorporated into the database?
Thanks in advance to anyone who can offer some insight on this.
So after a late night with Shodan's API, I think I've figured this one out. There does not appear to be a way to download your scan results after the fact. They appear on the command line that launched the scan once it concludes. The only way to save them is to use scan submit --filename <your_file_name> <your_ip>. This also means you need to make sure the process that submitted the scan is still around to receive the result.
I can't account for the documentation saying you can use shodan download with your scan ID, but I've tried to multiple times from the Python API and the Shodan CLI and it doesn't seem to work. Unless someone comes along to tell me differently, I'm assuming that functionality is not available.
I installed Nagios Core and NCPA on a Mac. Implemented a few checks via custom plugins to understand how to use it. I am trying to understand the following:
Protocol that Nagios server actually use to communicate with NCPA agent and how exactly does NCPA return the result back to Nagios. Does it ssh into Nagios server and writes a file that server processes?
From application monitoring standpoint how can it be leveraged? Is it just to monitor that application is up and running (I read its not just for that it can do more but couldn't find any place where I could see how its actually implemented) or is there a restful API as well that we invoke from with in our application to send custom notification to Nagios server. I understand it might require some configuration at Nagios server end as well.
I came across Pager Duty and Sematext articles i.e PagerDuty Integration and SemaText Nagios Alert Integration where they have integrated their solution with Nagios I am trying to do something similar. Adding integration support for Nagios so that a user can utilise our applications UI to configure alerts/notification. For e.g. if a condition is met then alert or notify Nagios server to show a notification on its dashboard.
Can we generate an alert from within a spark streaming application based on a variable e.g. if its value is above a threshold or some condition is met send an alert to Nagios Server to display as notification on Nagios Dashboard. I came across a link where we can monitor status of a spark application but didn't find anything for something within a spark application.
I tried looking for answers to above questions but couldn't find anything useful or complete as such online. I would really appreciate if someone could help me understand above.
Nagios is highly configurable, and can communicate across many protocols. NCPA can return JSON or XML data. The most common agentless protocol is probably SNMP. If you can read Python, look directly at the /usr/local/nagios/libexec/check_ncpa.py file to see what's up.
Nagios can check whether a system is running a service, how much resources it is consuming, etc... There is a restful API.
Nagios offers an application with a more advanced graphical interface called Nagios XI. Perhaps that is what you are after.
I bet you probably could, yeah. It might take some development work to get the systems to communicate though.
I'm trying to get a few metrics from a Cassandra node that has a Cassandra Exporter running on it (https://github.com/criteo/cassandra_exporter/). I don't want to go into the details, but using Prometheus is not an option at this time.
I'd like to access the data with HTTP requests or something similar. With a simple HTTP Get I can access all the cached information, but I would like to do more sophisticated operations on this, such as filtering for certain messages. Is there a way to do this? I could not find any information on this. Or do I have to get the entire log and then do filtering operations on my local machine?
I'm using the jmx-exporter tag because cassandra-exporter used to be a fork of it and I couldn't find a more fitting tag.
I would suggest to use telegraf + jolokia.
It is easy to setup and it will expose the metrics via HTTP.
I wrote a post about it (in my case I saved the result into InfluxDb and used it in Grafana), it might be useful:
cassandra-performance-monitoring-by-using-jolokia-agent-telegraf-influxdb-and-grafana
Using Prometheus exporters without the Prometheus server itself is a perfectly valid approach if you don't care about historical data and just want to get an immediate snapshot of metrics (state of the system) or make a recording of some short period manually.
One of the instruments you might look at is Metricat application (https://metricat.dev/), it allows you to have filters by metrics and make recordings of how metrics change in time during period of your interest.
I'd like to get the following system: once an event occurs in Cloud Foundry, it is loaded to elasticsearch. Using logstash would be fine, but I explored its input plugin and couldn't find anything that I could use. What is the best solution for this scenario? At the moment I can think of writing a script that would continuously pull the data using CF api and load it to elasticsearch. Is there a better way of doing it?
I can think of two solutions:
Create a "drain" (e.g., via the drain CLI) for the app you
would like to see events for and drain it to your ELK deployment.
This should forward each event (formatted as rfc 5425 syslog) to
elastic search.
If you are using the Loggregator Firehose to write data into elastic
search (e.g., via firehose-to-syslog) then you will get events
(as log messages). This has the downside of everything ends up in
your ELK deployment.
I am working on a solution for centralized log file aggregation from our CentOs 6.x servers. After installing Elasticsearch/Logstash/Kibana (ELK) stack I came across an Rsyslog omelasticsearch plugin which can send messages from Rsyslog to Elasticsearch in logstash format and started asking myself why I need Logstash.
Logstash has a lot of different input plugins including the one accepting Rsyslog messages. Is there a reason why I would use Logstash for my use case where I need to gather the content of logs files from multiple servers? Also, is there a benefit of sending messages from Rsyslog to Logstash instead of sending them directly to Elasticsearch?
I would use Logstash in the middle if there's something I need from it that rsyslog doesn't have. For example, getting GeoIP from an IP address.
If, on the other hand, I would need to get syslog or file contents indexed in Elasticsearch, I'd use rsyslog directly. It can do buffering (disk+memory), filtering, you can choose how the document will look like (you can put the textual severity instead of the number, for example), and it can parse unstructured data. But the main advantage is performance, on which rsyslog is focused on. Here's a presentation with some numbers (and tips and tricks) on Logstash, rsyslog and Elasticsearch:
http://blog.sematext.com/2015/05/18/tuning-elasticsearch-indexing-pipeline-for-logs/
I would recommend logstash. That would be easier to setup, more examples and they are tested to fit together.
Also, there are some benefits, in logstash you can filter and modify your logs.
You can extend logs with useful data: server name, timestamp, ...
Cast types, string to int, etc. (useful for correct Elastic index)
Filter out logs by some rules
Moreover, you can setup batch size to optimize saving to elastic.
Another feature, if something went wrong and there are crazy amount of logs per second that elastic can not process, you can setup logstash that it would save some queue of events or drop events that can not be saved.
If you go straight from the server to elasticsearch, you can get the basic documents in (assuming the source is json, etc). For me, the power of logstash is to add value to the logs by applying business logic to modify and extend the logs.
Here's one example: syslog provides a priority level (0-7). I don't want to have a pie chart where the values are 0-7, so I make a new field that contains the pretty names ("emerg", "debug", etc) that can be used for display.
Just one example...
Neither are a viable option if you really want to rely on the system to operate under load and be highly available.
We found that using rsyslog to send to a centralized location, archive it using redis of kafka and then using logstash to do its magic and ship to Elasticsearch is the best option.
Read our blog about it here - http://logz.io/blog/deploy-elk-production/
(Disclaimer - I am the VP product for logz.io and we offer ELK as a service)