We have implemented a monitoring solution in node.js, which does some basic checks for database integrity and API up-time. We want to expand this system to collect basic system stats of our Linux servers like CPU and disc usage. Some of these servers are behind a firewall which is out of our control, with only some very basic ports open (ssh,ftp,http,https).
How can I gather the system information of these servers in node.js. Are there monitoring systems which expose these information through a (secured) RESTful API?
I've had a lot of success with this ssh client written in javascript:
https://github.com/mscdex/ssh2
So there tons of available solutions for monitoring system stats: Nagios, Zabbix, Scout, Cacti. There are even some hosted ones like ServerDensity.
All of these systems should cover the top-level stats: CPU, RAM, Disk IO & Network. They all have a plug-in infrastructure so that you can send custom stats (API uptime, DB availability) and send them along with the regular stats.
If you're running on a cloud infrastructure somewhere, many of these provide information "out of the box", generally in your account dashboard (see guys like Joyent or Azure).
Big question here is "what else do you need"?
Use NRPE from Nagios as a client on the box you want to monitor. It's fairly simple to set up and it's API is documentented. http://exchange.nagios.org/directory/Addons/Monitoring-Agents/NRPE--2D-Nagios-Remote-Plugin-Executor/details
Related
how to monitor Performance counters of HP Web Tour sample application apache server installed locally in system using jvisualvm or any other utility.
Looking into Monitoring Your Server and 9 Key Apache Web Server Performance Metrics to Monitor it appears that:
You need to keep an eye on Apache error log
You need to consider Apache specific metrics like requests per second, bytes per second, bytes per request. You should be also able to extract these metrics from your performance testing tool, normally they must report these kind of stats.
You need to consider infrastructure metrics like CPU, RAM, Disk, Network, Swap usage on the machine where you're running this sample application. The majority of operating system come with built-in monitoring tools i.e. Windows Performance Monitor or number of command-line utilities for Linux or a 3rd-party cross-platform monitoring solution like PerfMon or Zabbix
I work with an ARM based embedded system with a Linux kernel and a fairly large filesystem image(~1GB). The kernel and filesystem are under my control, so I can add modules and re-build if necessary.
The system has Node.js and on top of this Node-Red for an IOT application. I want to leverage the simple server capability of Node-Red to serve a web page showing graphically the system performance statistics.
I am considering building collectd for the target system and using it to write performance data to the filesystem. Then, I will use Node-Red/Node.js to present this information as a web page. This approach seems straightforward enough to be doable.
My question is: is there an alternative, established way of implementing such a remote system performance viewer? Or can anyone suggest a lightweight performance monitor and a method of displaying the statistics graphically on a web page?
I started node-spm for that logging to Sematext Logsene and writing custom metrics to SPM. I collected also process and OS metrics. It's in an early stage and I will do in the next few weeks more on that.
Is there a way to monitor realtime IP traffic coming into my server and seeing how much bandwith and RAM is being used?
Every once in a while it seems like I get a DOS type of attack where my website becomes unresponsive, and I can't do anthing until I request a hard re-boot from my hosting company. I would like to be able to see which IP addresses are currently listed on my server at the time the server becomes unresponsive due to exhausted RAM usage this way I can block these IPs from accessing my website again in the future.
Thank you!
There are a lot of tools that you can use:
Command line tools: iptraf, iftop, jnettop
Web interface tools Ntop(It runs as a Daemon and you can see a lot of graphs and summaries about all traffic in your server)
The old Nagios (Good and Robust but you may need some time to familiarize yourself with)
You can find here a good article on best monitoring tools in the Unix/Linux world : Linux performance monitoring tools.
Is there a way to enable Performance Counters to monitor Node.js application performance in Windows Azure?
I haven't experimented with it myself yet, but there is node-perfmon which is a wrapper around typeperf. That says it allows you to write performance counters, as well as simple memory / cpu monitoring. Is this the sort of monitoring you were looking for?
Just adding more to above answers..
For application stats monitoring on Node.js you can use Hummingbird. It supports status over http so you can integrate the code in your node.js app add one port to get the monitoring data over HTTP. No need to use Azure Storage Diagnostics and all info in real time in same machine. It's still in pre-alfa, but is handling with few tasks really well.
http://projects.nuttnet.net/hummingbird/
I know about the node.js "monitor" plugin which is the best for Linux machines for system specific performance and also use HTTP to provide system specific data. I am not sure if that can be ported to Windows Server but if can that is one great choice. Read more about monitor usage here:
http://www.sys-con.com/node/2275314
You may want to also look at these, they aren't directly using perfmon, but allow you to monitor the performance of your Node.js server:
http://search.npmjs.org/#/Probes.js
http://search.npmjs.org/#/nodetime
The NPM registry is a great tool for finding Node.js packages.
Ok so I have an idea I want to peruse but before I do I need to understand a few things fully.
Firstly the way I think im going to go ahead with this system is to have 3 Server which are described below:
The First Server will be my web Front End, this is the server that will be listening for connection and responding to clients, this server will have 8 cores and 16GB Ram.
The Second Server will be the Database Server, pretty self explanatory really, connect to the host and set / get data.
The Third Server will be my storage server, this will be where downloadable files are stored.
My first questions is:
On my front end server, I have 8 cores, what's the best way to scale node so that the load is distributed across the cores?
My second question is:
Is there a system out there I can drop into my application framework that will allow me to talk to the other cores and pass messages around to save I/O.
and final question:
Is there any system I can use to help move the content from my storage server to the request on the front-end server with as little overhead as possible, speed is a concern here as we would have 500+ clients downloading and uploading concurrently at peak times.
I have finally convinced my employer that node.js is extremely fast and its the latest in programming technology, and we should invest in a platform for our Intranet system, but he has requested detailed documentation on how this could be scaled across the current hardware we have available.
On my front end server, I have 8
cores, what's the best way to scale
node so that the load is distributed
across the cores?
Try to look at node.js cluster module which is a multi-core server manager.
Firstly, I wouldn't describe the setup you propose as 'scaling', it's more like 'spreading'. You only have one app server serving the requests. If you add more app servers in the future, then you will have a scaling problem then.
I understand that node.js is single-threaded, which implies that it can only use a single core. Not my area of expertise on how to/if you can scale it, will leave that part to someone else.
I would suggest NFS mounting a directory on the storage server to the app server. NFS has relatively low overhead. Then you can access the files as if they were local.
Concerning your first question: use cluster (we already use it in a production system, works like a charm).
When it comes to worker messaging, i cannot really help you out. But your best bet is cluster too. Maybe there will be some functionality that provides "inter-core" messaging accross all cluster workers in the future (don't know the roadmap of cluster, but it seems like an idea).
For your third requirement, i'd use a low-overhead protocol like NFS or (if you can go really crazy when it comes to infrastructure) a high-speed SAN backend.
Another advice: use MongoDB as your database backend. You can start with low-end hardware and scale up your database instance with ease using MongoDB's sharding/replication set features (if that is some kind of requirement).