Child Process exits abruptly while executing in node.js - node.js

So I have been working on a project and it requires me to convert the office files into PDFs and subsequently images. I've written and integrated everything into one single node.js script, but for some reason the script keeps on bypassing the synchronous child process creation. Here is the code:
down.download(parsed_url);
var f_name=obj.doc;
var ext=f_name.slice(f_name.length-4);
var w_path="C:\\Users\\Akshay\\Desktop\\conv_Scripts\\word_pdf.ps1";
var e_path="C:\\Users\\Akshay\\Desktop\\conv_Scripts\\excel_pdf.ps1";
var p_path="C:\\Users\\Akshay\\Desktop\\conv_Scripts\\power_pdf.ps1";
var file_name=f_name.slice(0,f_name.length-5);
console.log(ext);
console.log(f_name);
console.log(file_name);
if(ext==="docx"){
word.wordpdf(w_path);
}
else if(ext==="xlsx"){
excel.excelpdf(e_path);}
else if(ext==="pptx"){
ppt.pptpdf(p_path);
console.log("Done converting to PD");
}
else if(ext==".pdf"){
img.img(f_name);
}
else{
console.log("Can't convert to PDF");
}
crawlpdf.crawlpdf(file_name,function(collect){
collect.forEach(function(col){
img.img(col);
console.log('Done!');
});
the wordpdf,excelpdf and pptpdf functions are same in their structure. I'll write down the wordpdf module's code here:
var spawn=require('child_process').spawnSync,
child;
exports.wordpdf=function(filepath){
child=spawn("powershell.exe",[filepath]);
};
The trouble is that when I execute the script,it shows me "Done converting to PD" (since the downloaded file was a ppt) but I do not find any pdf of the downloaded file .The .ps1 scripts in the path are already tested and there is no issue with them. If you could shed some light it would be really a massive help to me.
Thanks.

Alright, In case anyone else runs into the same issue as this, here's the explanation for what went wrong:
The message coming from the RabbitMQ messsage queue is small.Hence when the message arrives it is rapidly consumed,leading to race conditions in the code. For better and more accurate performance try parsing and writing the message to a file and then use async module to do the necessary steps.

Related

Best way to copy a directory from an external drive to a local folder with electronjs?

Just wondering if anyone has ever attempted to copy a directory from an external drive (connected via USB) to a local folder.
I am using ElectronJS so I can use my JavaScript, HTML/CSS skills to create a desktop application without utilising a C language. (i.e. C# or C++) With ElectronJS there's a lot less to worry about.
Here is the list of things I've tried so far:
basic fs.copyFile (using copyFile intially and will then loop round the directory to copy all files)
var fs = require('fs');
window.test = () => {
fs.moveSync("targetFile","destDir", function(err) {
if(err){
console.log(err);
}else{
console.log("copy complete")
}
});
}
fs.moveSync is not a function even though Visual Studio Code brought up moveSync as a suggestion when I entered fs. (ctrl + space)
using child_process functions to copy files using the command line.
Code is:
var process = require('child_process')
window.test = function(){
process.exec('ipconfig', function(err, stdout, stderr){
if(err){
console.log(err);
}else{
console.log(stdout)
}
})
}
Then bundled with browserify. Bundle.js is then imported into the html file and the test function is called on the click of a button. I'm aware the command is ipconfig for now, this was merely used to see if a command could be executed. It appears it could because I was getting process.exec is not defined.
use the node-hid node module to read and trasfer data from the external drive.
The exposed functions within this module were also reported as being undefined. And I thought about the use case longer I thought a simple copy process would suffice because external drive can be accessed like any other folder in the file explorer.
Unfortunately, all of the above have failed and I've spent the most part of the day looking for alternative modules and/or solutions.
Thanks in advance because any help to achieve this would be much appreciated.
Thanks
Patrick
The npm package fs-extra should solve your problem.
It has the move function, which
Moves a file or directory, even across devices
Ended up adding this to my preload.js for:
window.require = require;
It will work for now but is due to be depreciated.
I'll use this for now and make other updates when I have to.

Forever Node.js Script Hangs Up on Loop

I have made a Node.js script which checks for new entries in a MySQL database and uses socket.io to send data to the client's web browser. The script is meant to check for new entries approximately every 2 seconds. I am using Forever to keep the script running as this is hosted on a VPS.
I believe what's happening is that the for loop is looping infinitely (more on why I think that's the issue below). There are no error messages in the Forever generated log file and the script is "running" even when it's started to hang up. Specifically, the part of the script that hangs up is the script stops accepting browser requests at port 8888 and doesn't serve the client-side socket.io js files. I've done some troubleshooting and identified a few key components that may be causing this issue, but at the end of the day, I'm not sure why it's happening and can't seem to find a work around.
Here is the relevant part of the code:
http.listen(8888,function(){
console.log("Listening on 8888");
});
function checkEntry() {
pool.getConnection(function(err,connection) {
connection.query("SELECT * FROM `data_alert` WHERE processtime > " + (Math.floor(new Date() / 1000) - 172800) + " AND pushed IS NULL", function (err, rows) {
connection.release();
if (!err) {
if(Object.keys(rows).length > 0) {
var x;
for(x = 0; x < Object.keys(rows).length; x++) {
connection.query("UPDATE `data_alert` SET pushed = 1 WHERE id = " + rows[x]['id'],function() {
connection.release();
io.emit('refresh feed', 'refresh');
});
}
}
}
});
});
setTimeout(function() { checkEntry();var d = new Date();console.log(d.getTime()); },1000);
}
checkEntry();
Just a few interesting things I've discovered while trouble shooting...
This only happens when I run the script on Forever. Work's completely fine if I use shell and just leave my terminal open.
It starts to happen after 5-30 minutes of running the script, it does not immediately hang up on the first execution of the checkEntry function.
I originally tried this with setInterval instead of setTimeout, the issue has remained exactly the same.
If I remove the setInterval/setTimeout function and run the checkEntry function only once, it does not hang up.
If I take out the javascript for loop in the checkEntry function, the hang ups stop (but obviously, that for loop controls necessary functionality so I have to at least find another way of using it).
I've also tried using a for-in loop for the rows object and the performance is exactly the same.
Any ideas would be immensely helpful at this point. I started working with Node.js just recently so there may be a glaringly obvious reason that I'm missing here.
Thank you.
So I just wanted to come back to this and address what the issue was. It took me quite some time to figure out and it can only be explained by my own inexperience. There is a section to my script where my code contained the following:
app.get("/", (request, response) => {
// Some code to log things to the console here.
});
The issue was that I was not sending a response. The new code looks as follows and has resolved my hang up issues:
app.get("/", (request, response) => {
// Some code to log things to the console here.
response.send("OK");
});
The issue had nothing to do with the part of the code I presented in the initial question.

event that the unzip process is finished

I use the following code which works
but I want to know when the API is done the extracting
and finish the process, is there anyway to do it with this lib?
it based on yauzl
https://www.npmjs.com/package/extract-zip
var extract = require('extract-zip')
extract(source, {dir: target}, function (err) {
// extraction is complete. make sure to handle the err
})
I didnt find any event that the process done, but maybe I miss something ?
This module doesn't fire events. However, the callback function is called whenever the unzip process has finished (or an error occurred). This is the only way you are informed the process has finished. You can put your logic that needs to run after the process has finished there.
var extract = require('extract-zip')
extract(source, {dir: target}, function (err) {
// extraction is complete. make sure to handle the err
// If you are here the process of unzipping is done (or an error occurred)
})
If you really do want events, you can take a look at the underlying yauzl package as it uses events and streams extensively.

Nightmare doesn't run twice in a row - NodeJS

EDIT
I have noticed the removal of the .end() function appears to solve the issue, but after reading the Nightmare docs on the use of .end() it says: Completes any queue operations, disconnect and close the electron process.
Now while this does solve the problem, am I now just opening more and more electron processes each time the route is called, which will eventually cause the server to run out of memory, or is this a safe way to fix the issue?
ORIGINAL TEXT
Please consider the following problem:
I am developing a Node based service that will allow the user to request screenshot of a particular URL.
For this I am using Nightmare to visit the URL, wait 2 seconds, take a screenshot, which is saved to the disk, convert it to base64, delete the image and then return the base64 string.
console.log('Nightmare starts');
nightmare
.goto(url)
.wait(2000)
.screenshot(filename)
.end()
.then(function (result)
{
fs.exists(filename, function(exists)
{
if (exists)
{
data = fs.readFileSync(filename);
var base64 = data.toString('base64')
fs.unlink(filename);
var output = {'message':'success','map_image':base64};
res.send(output);
}
});
})
.catch(function (error)
{
console.error('Search failed:', error);
});
console.log("Nightmare Finished");
The above code works just fine, the first time it runs. However any subsequent calls to this just consoles "Nightmare starts" and "Nightmare Finished" instantly with the actual code in-between not running. I don't appear to have any errors display, nothing is caught if I wrap it in a try/catch. The node requires a reboot to allow it to happen again.
Something worth noting is that I am running on a headless ubuntu machine, as electron (one of the nightmare dependencies) appears to need a GUI, I am using xvfb to launch the node using the following command:
xvfb-run --auto-servernum --server-num=1 node server.js
I'm assuming this may be an issue with some resource not being released correctly on the first run, but any assistance would be appreciated.
Also open to any constructive criticism of my code, very new to Node and i'm sure i'm not writing in the most optimal way (sync file loading etc)
It appears that you are simply misplacing where you are creating the nightmare instances. Cannot help much without some more code snippet and information.
Way 1
Create nightmare instance every time and close them after you are done with your task. It will require some time to boot up the instance, but it will also lessen the memory load. Not to mention you can have multiple nightmare instances for different users.
Way 2
Don't end and re-use same nightmare instance. Have multiple nightmare instances and queue the call for screenshot. The websites will load fast and it won't take time to boot up an instance, but you will have longer wait time for longer queue.

Node js module mkdirp only creates half the directories

I'm trying to use mkdirp for a project, but when I feed it a var with my dir path I want created, it only creates the first half of it. I've installed the module locally with npm. I'm using Node v0.10.20 on a Raspberry Pi.
This is how it looks:
var filePath = "upload/home/pi/app/temp";
mkdirp(filePath, function(error) {
if(error) {
console.log(error);
} else {
...
}
});
I don't get an error creating the path, but it only creates "upload/home/pi", however if I run my script again, it creates the rest of the directory structure. Upload is a
directory in the current working directory which is the user home.
I emailed the author of the module who suggested that it could be because I'm using a flash drive as my medium, which in turn lies about when IO operations are complete, which I guess confuses node.js to think it has successfully written the path to disk. How should I tackle my problem? I guess I can do a check on if the directory was created, and loop that until it has, but that feels like the wrong thing to do. Any suggestions welcome.
Thanks.
Try doing this synchronously:
var filePath = "upload/home/pi/app/temp";
mkdirp(filePath)

Resources