I have a NodeJs application that listens to messages via subscribe on a Redis server. It collects the messages for a period of 5 Seconds and then pushes them out to the connected clients, the code looks something like this:
io.sockets.on('connection', function (socket) {
nClients++;
console.log("Number of clients connected " + nClients);
socket.on('disconnect', function () {
nClients--;
console.log("Number of clients remaining " + nClients);
});
});
Receiving messages to send out to the clients
cli_sub.on("message",function(channel,message) {
oo = JSON.parse(message);
ablv_last_message[oo[0]["base"]+"_"+oo[0]["alt"]] = message;
});
setInterval(function() {
Object.keys(ablv_last_message).forEach( function(key) {
io.sockets.emit('ablv', ablv_last_message[key]);
});
ablv_last_message = [];
}, 5000);
SOLUTION FOUND (at least I think so): Node didn't crash because it reached some internal memory limits, it looks as if it crashed because my VPS ran out of memory, it was a 2GB VPS running one or two other processes too. After upgrading it to 4GB, Node runs smoothly, yes always around 1.6 to 2.0 GB but I believe its the GC who does its work here.
It is better you try some tools for finding leaks in node.js.
Tools for Finding Leaks
Jimb Esser’s node-mtrace, which uses the
GCC mtrace utility to profile heap usage.
Dave Pacheco’s node-heap-dump takes a snapshot of the V8 heap and serializes the whole thing out in a huge JSON file. It includes tools to traverse and investigate
the resulting snapshot in JavaScript.
Danny Coates’s v8-profiler and node-inspector provide Node bindings for the V8 profiler and a Node debugging interface using the WebKit Web Inspector.
Felix Gnass’s fork of the same that un-disables the retainers graph
Felix Geisendörfer’s Node Memory Leak Tutorial is a short and sweet explanation of how to use the v8-profiler and node-debugger, and is presently the state-of-the-art for most Node.js memory leak debugging.
Joyent’s SmartOS platform, which furnishes an arsenal of tools at your disposal for debugging Node.js memory leaks
From Tracking Down Memory Leaks in Node.js – A Node.JS Holiday Season.
And another blog
It looks to me that you keep adding keys to the global ablv_last_message object and never clean it.
You may use Object.getOwnPropertyNames rather than Object.keys
Related
On a cold start (after deploying or after 3hrs) the function to request a document from Firestore takes an incredible amount of time which is different to when it's used rapidly.
Cold Start:
Function execution took 4593 ms, finished with status code: 200
Rapid fire (me sending using the same function over and over):
Function execution took 437 ms, finished with status code: 200
My code for getting the documents is quite simple:
function getWorkspaceDocument(teamSpaceId) {
return new Promise((resolve, reject) => {
var teamRef = db.instance.collection('teams').doc(teamSpaceId);
teamRef.get().then(doc => {
if (doc.exists) {
resolve(doc.data());
return;
}
else {
reject(new Error("Document cant be found"));
return;
}
}).catch(error => {
reject(new Error("Document cant be found"));
});
});
}
I'm trying to make a Slack bot and the slow returns on Firebase Firestore throw time outs in Slacks API. Is there a way on Firebase to stop cold starts from happening and letting it persist through out?
If the cloud function needs to start a new instance your cold start time seems normal. This is one drawback of a serverless function.
I think there is a problem with your implementation. Could you show more details?
Here is a nice little video about this topic:
https://youtu.be/v3eG9xpzNXM
firebaser here
We actually just released a new preferRest API that should considerably improve the cold start times for Cloud Functions that use Firestore. The documentation for it is not very complete, but you can enable the feature with:
import { initializeFirestore }
from 'firebase-admin/firestore';
const app = initializeApp();
const firestore = initializeFirestore(app,
{ preferRest: true }); // 👈
firestore.collection(...);
With preferRest: true the Firestore Admin SDK uses the REST transport layer by default, and it then only loads and uses the gRPC libraries when it encounters an operation that needs them.
Since the gRPC libraries are quite big and the only operation that requires gRPC is creating a snapshot listener, this should reduce the cold start times for most Cloud Functions implementations significantly.
I haven't had a chance to test this myself yet and there are still some known issues, so YMMV and I'd love to hear specifics on what performance change you see from this.
Also see:
the written release note about this option
the Release Notes video where this is mentioned
Another thing I would suggest checking is the amount of memory allocated to a particular function. Each level selected increases non only the RAM, but the CPU frequency as well (and the costs, be careful and don't forget about the pricing calculator!). There is a direct dependency between the package size of your function and the cold-start (source: https://mikhail.io/serverless/coldstarts/gcp/).
I can see that you are using the Firestore admin package, which is not considered to be lightweight (source: https://github.com/firebase/firebase-admin-node/issues/238). Thus, 128MB configuration might not be enough.
For our project increasing the RAM from 128MB to 512MB decreased the cold boot 10x from 20 seconds to 2.5seconds on average. Be sure not to overlook this in case you have several dependencies (7 in our case).
I am using sharp image processing module to resize the image and render it on UI.
app.get('/api/preview-small/:filename',(req,res)=>{
let filename = req.params.filename;
sharp('files/' + filename)
.resize(200, 200, {
fit: sharp.fit.inside,
withoutEnlargement: true
})
.toFormat('jpeg')
.toBuffer()
.then(function(outputBuffer) {
res.writeHead('200',{"Content-Type":"image/jpeg"});
res.write(outputBuffer);
res.end();
});
});
I am running above code on a single board computer Rock64 with 1 GB ram. When I run a Linux htop command and monitor the memory utilization, I could see the memory usage is adding up exponentially from 10% to 60% after every call to the nodejs app and it never comes down.
CPU USAGE
Though it does not give any issue running the application, my only concern is memory usage does not come down, even when the app is not running and I am not sure if this will crash the application eventually if this application runs continuously.
or if I move a similar code snippet to the cloud will it keep occupying memory even when it's not running?
Anyone who is using sharp module facing the similar issue or is this a known issue with node.js. Do we have a way to flush out/clear out the memory or will node do garbage collection?
Any help is appreciated. Thanks
sharp has some memory debugging stuff built in:
http://sharp.dimens.io/en/stable/api-utility/#cache
You can control the libvips cache, and get stats about resource usage.
The node version has a very strong effect on memory behaviour. This has been discussed a lot on the sharp issue tracker, see for example:
https://github.com/lovell/sharp/issues/429
Or perhaps:
https://github.com/lovell/sharp/issues/778
I'm trying to increase the SO_RCVBUF of a datagram socket in order to run a high-bandwidth UDP stream. With the default settings I am dropping packets even when doing no processing. Looking around this seems to be an issue related to the default buffer sizes when creating sockets in windows. I don't seem to have an issue with packets dropping in Linux so I am fairly certain it is the buffer size problem.
The issue is that the native node.js dgram class does not expose setsockopt for the socket so I can't change SO_RCVBUF.
I have found a potential solution through node-ffi, as outlined in this response, however when I try to load the FFI library I get a win32 error regarding the dll. From my understanding, entering null as the library name should just load the current process, where node.js is residing. Is this correct? The default example from ffi is below but gives the same win32 error 127 result. Should I be pointing it at a different dll, perhaps libuv?
const ffi = require('ffi');
var current = ffi.Library(null, {
'atoi': ['int', ['string']]
});
If it helps I am running Windows 10, and the code is running on a node child process forked from the electron main process.
Node: 7.9.0
Electron 1.7.4
We're currently in the process of updating from node 0.10 to node 4.1.2 and we're seeing some weird patterns. The number of connections to our postgres database doubles1 and we're seeing the same pattern with requests to external services2. We are running a clustered app running the native cluster API and the number of workers is the same for both versions.
I'm failing to understand why upgrading the runtime language would apparently change application behaviour by doubling requests to external services.
One of the interesting things I've noticed with 0.12 and 4.x is the change in garbage collection. I've not used the pg module before so I don't know internally how it maintains it's pools of if it would be affected by memory or garbage collection. If you haven't defined default memory setting for node you could try giving that a shot and see if you see any other results.
node --max_old_space_size <some sane value in MB>
I ran into something similar, but I was getting double file writes. I don't know your exact case, but I've seen a scenario where requests could almost exactly double.
in the update to 4.1.2, process.send and child.send has gone from synchronous to asynchronous.
I found an issue like this:
var child = fork('./request.js');
var test = {};
child.send(small request);
child.send(large request);
child.on('response', function (val) {
console.log('small request came back: ' + val);
test = val;
});
if(!test){
//retry request
} ...
So where as previously the blocking sends has allowed this code to work, the non-blocking version assumes an error has occurred and retries. No error actually occurred, so double the requests come in.
I'm attempting to create a flame graph for a Node app that's causing some issues, and while I am able to profile it using Xcode and get its CPU trace, the Node perf map isn't dumping to, for example, /tmp/perf-30001.map, when I exit it uncleanly (unfortunately, the issue I'm running into isn't allowing me to exit the Node app cleanly). I'm running the app with the --perf-basic-prof flag.
Is there any way to get Node to dump the memory map either continuously or on any kind of exit?
The map file is written continuously, be sure to use at least node 0.12 and to disable kptr_restrict sudo sysctl kernel/kptr_restrict=0.
And if you want a memory dump at exit you can later open in v8 debugger :
var heapdump = require('heapdump');
process.on('exit', function() {
heapdump.writeSnapshot(Date.now() + '.heapsnapshot');
});