I'm running some file management tasks through a node script. The node-windows package is included to allow me to run this script as a windows service. I encountered a serious error this morning when I realized that the service had started a duplicate instance of the same script. This is very bad, it corrupted 24-hrs worth of data because both scripts were trying to process the same data sets and ended up shredding them. I've never seen the windows service allow something like this. Has anyone else had this problem or have any idea what is causing it?
See my comment about node-windows instances.
The real problem, which is data corruption, doesn't have anything to do with node-windows. The node script should have fault tolerance for this. More specifically, it should be implementing file locking, which is a standard practice to prevent this exact scenario.
There are a couple of file locking modules available. lockfile is what npm uses. There is also another project called proper-lockfile, which solves the problem in a slightly different (more Windows-friendly) way.
Related
i have to do a reptitive task in nodeJS and i've seen there is existing package like this one.
https://www.npmjs.com/package/node-cron
and the actual platform where i'm hosted propose inside cronjob.
https://www.netlify.com/docs/webhooks/
so my question is when it's more interessant to use the platform or a package.
thanks.
From the URL posted i didn't see any method of setting up a cron job using webhooks. Unless you were thinking of setting up a webhook that listens for a post which is sent using a linux cron job or the like?
Regardless, the actual question about using a platform or a package. They have pros and cons, but based purely on your question I would go with the platform.
If you choose to use a package you will have to write the code to call the package (which you need to test, maintain and run). You need to ensure that the node process is always up and running, if it dies or exits that it is re-spawned, that if the operating system reboots the node process gets kicked off again. All these problems are can be easily solved (PM2 for instance) but the fact is you need to think of the problems and solve them yourself or the cron job might not run when you want it to.
When using the platform you know that it is well tested, that it will work as documented, and that it will be resilient to failure modes that you might not be aware of.
I'm receiving SIGSEGV quite randomly when running an express app with PM2. The strange thing is the server runs quite well for the past few weeks. It does not print any error message except:
App [XXX] with id [7] and pid [27757], exited with code [255] via signal [SIGSEGV]
After implementing the "segfault-handler" module, I started to receive some stack traces. It seems the app encounters a few different segmentation fault:
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10330)[0x7fd211f87330]
node(_ZN2v88internal9HashTableINS0_15ObjectHashTableENS0_20ObjectHashTableShapeENS0_6HandleINS0_6ObjectEEEE18FindInsertionEntryEj+0x40)[0xc0b680]
node(_ZN2v88internal15ObjectHashTable3PutENS0_6HandleIS1_EENS2_INS0_6ObjectEEES5_i+0x124)[0xc0c0a4]
node(_ZN2v88internal7Runtime17WeakCollectionSetENS0_6HandleINS0_16JSWeakCollectionEEENS2_INS0_6ObjectEEES6_i+0x59)[0xc7d639]
node(_ZN2v88internal25Runtime_WeakCollectionSetEiPPNS0_6ObjectEPNS0_7IsolateE+0x11d)[0xc7d89d]
[0x2acdd80963b]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10330)[0x7f0fc311c330]
node(_ZN2v88internal32IncrementalMarkingMarkingVisitor26VisitFixedArrayIncrementalEPNS0_3MapEPNS0_10HeapObjectE+0x376)[0xad8a16]
node(_ZN2v88internal18IncrementalMarking4StepElNS1_16CompletionActionENS1_18ForceMarkingActionENS1_21ForceCompletionActionE+0x2c1)[0xad6181]
node(_ZN2v88internal8NewSpace15SlowAllocateRawEiNS0_19AllocationAlignmentE+0x74)[0xb05244]
node(_ZN2v88internal4Heap11AllocateRawEiNS0_15AllocationSpaceES2_NS0_19AllocationAlignmentE+0x1b9)[0xa678c9]
node(_ZN2v88internal4Heap20AllocateFillerObjectEibNS0_15AllocationSpaceE+0x19)[0xab00b9]
node(_ZN2v88internal7Factory15NewFillerObjectEibNS0_15AllocationSpaceE+0x2d)[0xa67d1d]
node(_ZN2v88internal29Runtime_AllocateInTargetSpaceEiPPNS0_6ObjectEPNS0_7IsolateE+0x5e)[0xc99e8e]
[0x249862c06355]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10330)[0x7fbebabd2330]
node(_ZN2v88internal9HashTableINS0_15ObjectHashTableENS0_20ObjectHashTableShapeENS0_6HandleINS0_6ObjectEEEE18FindInsertionEntryEj+0x40)[0xc0b680]
node(_ZN2v88internal15ObjectHashTable3PutENS0_6HandleIS1_EENS2_INS0_6ObjectEEES5_i+0x124)[0xc0c0a4]
node(_ZN2v88internal7Runtime17WeakCollectionSetENS0_6HandleINS0_16JSWeakCollectionEEENS2_INS0_6ObjectEEES6_i+0x59)[0xc7d639]
node(_ZN2v88internal25Runtime_WeakCollectionSetEiPPNS0_6ObjectEPNS0_7IsolateE+0x11d)[0xc7d89d]
[0x125b9620963b]
I know there is little information here. Can anyone please tell me a good way to start diagnosing? I've checked the PM2 log, mongoDB log but no luck.
Thanks!
Mars
Since the stack trace is different every time and not very illuminating, all you can do is try things. The first main suspects will be things that use native code because it's not that likely that plain Javascript is causing a segFault. It is probably native code that is somehow corrupting memory or not properly interacting with the garbage collector in node.js.
So, the things to look for are the interaction between your current version of node.js and the things you have that use native code (such as mongoDB). Here are things to try:
Identify all modules that use native code and temporarily remove any that you can live without.
Upgrade both node.js and mongoDB to recent versions in case you have some interaction between their specific versions that is causing the problem. If you can't upgrade node.js to a recent stable version, then make absolutely sure that all the modules you are running are certified to be stable with the version of node.js that you do have.
Restart your server just in case there's anything goofed up in the OS that is contributing to the problem.
Start with a clean database or run some sort of database check on your database in order to verify that there is no corruption there.
Whenever you update your DB scheme, make sure you have a strategy for moving the prior database forward (it looks like in MongoDB you can just make sure you assign a default value to new scheme elements).
Gather new info after making changes and repeat the process, trying to only change one thing at a time so that if it fixes the issue you will know exactly which item it was that fixed it.
Something like that can happen when you copy the code with node_modules that included binary modules compiled for a different architecture than the one you're trying to run it on.
Try either removing node_modules and running npm install from scratch, or you can try running npm rebuild without removing node_modules.
Since I first knew of Docker, I thought it might be the solution for several problems we are usually facing at the lab. I work as a Data Analyst for a small Biology research group. I am using Snakemake for defining the -usually big and quite complex- workflows for our analyses.
From Snakemake, I usually call small scripts in R, Python, or even Command Line Applications such as aligners or annotation tools. In this scenario, it is not uncommon to suffer from dependency hell, hence I was thinking about wrapping some of the tools in Docker containers.
At this moment I am stuck at a point where I do not know if I have chosen technology badly, or if I am not able to properly assimilate all the information about Docker.
The problem is related to the fact that you have to run the Docker tools as root, which is something I would not like to do at all, since the initial idea was to make the dockerized applications available to every researcher willing to use them.
In AskUbuntu, the most voted answer proposes to add the final user to the docker group, but it seems that this is not good for security. In the security articles at Docker, on the other hand, they explain that running the tools as root is good for your security. I have found similar questions at SO, but related to the environment inside the container.
Ok, I have no problem with this, but as every moderate-complexity example I happen to find, it seems it is more oriented towards web-applications development, where the system could initially start the container once and then forget about it.
Things I am considering right now:
Configuring the Docker daemon as a TLS-enabled, TCP remote service, and provide the corresponding users with certificates. Would there be any overhead in running the applications? Security issues?
Create images that only make available the application to the host by sharing a /usr/local/bin/ volume or similar. Is this secure? How can you create a daemonized container that does not need to execute anything? The only example I have found implies creating an infinite loop.
The nucleotid.es page seem to do something similar to what I want, but I have not found any reference to security issues. Maybe they are running all the containers inside a virtual machine, where they do not have to worry about these issues, due to the fact that they do not need to expose the dockerized applications to more people.
Sorry about my verbosity. I just wanted to write down the mental process (possibly flawed, I know, I know) where I am stuck. To sum up:
Is there any possibility to create a dockerized command line application which does not need to be run using sudo, is available for several people in the same server, and which is not intended to run in a daemonized fashion?
Thank you in advance.
Regards.
If users will be able to execute docker run then will be able to control host system just because they could map files from host to container and in container they always could be root if they could use docker run or docker exec. So users should not be able to execute docker directly. I think easiest solution here to create scripts which run docker and these scripts could either have suid flag or users could have sudo access to them.
In the past couple of years NodeJS became a major player in the server landscape - and I really find it hard to believe that there is no decent way to have nodejs run as a service on a linux box. On Windows we have iisnode - but for non Windows environments the forever package is suggested as the way to go - instead of a real solution.
Is there maybe a servicized version of nodejs out there that I could not locate?
There isn't a "servicized" version of Node.js available in the sense you are thinking. Keeping your Node application running (for example in the event of a fatal error) is up to you entirely.
As suggested in the first comment, this is fairly subjective, but really there are two big packages (and one or two alternative methods) for making a service out of your Node application. As you've mentioned, forever is a popular choice. If you've never taken a look at pm2, I suggest doing so, as it offers some services that forever does not. Alternatively, you could search for information on supervisord, which I've had success with in the past. Finally, daemonizing Node with upstart is something to look at if the others don't fit well for you.
according to many articles, the best way to handle uncaught exceptions in a node.js app is let the process crash and than restart it. This avoids to have our application in a unstable state.
I think it can be done with an external process like a watchdog (sometimes called angel process).
What is the best way to do this in a linux system? At first glance a bash script checking the node process state every n seconds could be a possible solution.
Thank in advance, any suggestion will be really appreciated.
Node.js Best Practice Exception Handling
http://shapeshed.com/uncaught-exceptions-in-node/
http://debuggable.com/posts/node-js-dealing-with-uncaught-exceptions:4c933d54-1428-443c-928d-4e1ecbdd56cb
There are a handful of solutions like the watchdog/angel process you mention. The one you most often hear thrown around is "forever". It's available via npm.
To install, just: npm install -g forever
Then when you want to run your app: forever app.js instead of node app.js
I have successfully used supervisord, I think it fits your purpose very well. It's not node-specific and you can use it virtually with anything that won't detach from the console.
And with the events feature, it gives a nice flexible tool for all kinds of scenarios.