node.js. Problems saving requested zip file to disk - node.js

I'm having trouble saving a remote zip file to disk with with node.
I'm using request library to make requests. I want to request a zip file,
if the request is succesful write it to disk. I can't get a good combination of
correct error handling and writing the file.
I want to do the following :
request.get('https://example.com/example.zip', {
'auth': { 'bearer': accessToken },
}, function(error, response, body) {
// shortcircuit with notification if unsuccessful request
if (error) { return handleError() }
// I want to save to file only if no errors
// obviously this doesn't work because body is not a stream
// but this is where I want to handle it.
body.pipe(fs.createWriteStream('./output.zip'));
});
I know I can pipe the request directly as follows but I can't get decent error handling. The on error callback doesn't fire for 404s, and if I catch the request and throw an error if !response.ok the empty output file is still written to disk
request.get('https://example.com/example.zip', {
'auth': { 'bearer': accessToken },
})
.on('error', handleError)
.pipe(fs.createWriteStream('./output.zip'));

Instead of using body.pipe(), use response.pipe().
request.get('https://example.com/example.zip', {
auth: {
bearer: accessToken
}
}, (err, res, body) => {
if (res.statusCode !== 200) { // really should check 2xx instead
return handleError();
}
res.pipe(fs.createWriteStream('./output.zip');
});
The downside here though is that the request module is going to buffer the full response. Easy fix... don't use the request module. http.get() is fine and is a drop-in replacement.
Also, I highly recommend checking out the request-promise module which has an option for failing on 404.

Related

ERR_HTTP_HEADERS_SENT caused by response to POST request after refresh node.js Express. res.send inside fs.watch callback

On my webpage user can enter text and press send. this causes the server to append the message to a json object stored in a file. when this file is altered it then sends the new json to the client.
app.post("/recieve",function(req,res){
watcher = fs.watch(__dirname+"/msgs/msg.json", (eventName, filename) => {
watcher.close();
fs.readFile(__dirname+"/msgs/msg.json", (err,data) => {
return res.send(data);
});
});
})
here is the client side
async function recieveMSG(){
$.ajax({
url: "recieve",
type: "POST",
contentType: "text; charset=utf-8"
}).done(function(data){
$("#msgbox").html("<br>"+data+"<br>");
recieveMSG();
});
}
recieveMSG();
As shown in the code above, the client sends a POST request to the server. Next after the json file is changed the server responds to the POST request with the json. I know this may be the completely wrong way to do it, but I want to know why res.send(data) is being called twice on the same res object.
It seems after the first refresh the recieve POST request just doesnot do anything
app.post("/recieve",async function(req,res){
try{
watcher.close();
}
catch(e){
console.log("WatcherUndefined --first execution");
}
watcher = fs.watch(__dirname+"/msgs/msg.json", (eventName, filename) => {
watcher.close();
fs.readFile(__dirname+"/msgs/msg.json", (err,data) => {
return res.send(data);
});
});
})
The problem was that the watcher wasn't getting closed after the client refreshed/disconnected. After the client refreshed the res object generated by their stale request is unusable. I believe that the watcher's callback was never redefined with the new res object (after refresh). I do not know if my assumption is correct, and would like to hear other's thoughts on this as I am new to nodejs.

How to handle a C# ByteArrayContent in NodeJS

I have a simple C# web api service that returns a pdf document as bytearraycontent in the response of an IHttpActionResult. I can easily handle this as a blob on the client (js in browser) and set it as a the src of an iframe. Everything displays correctly.
Due to some changes in requirements, I'm now implementing a simple appserver in nodejs using express; this will serve as a proxy server. From my client, I'm able to make the call to this express server, which in turn calls my C# service. I get 200 http OK status code, but seem to not be able to the handle pdf byte array in node. I tried Buffer methods, readstreams etc... Nothing worked.
Can you guys throw some light on this? Below is the code:-
httpRequester.get(`${constants.serviceTarget}/api/DocGen/Download/Id/true`, (error, response, body) => {
if(error) {
console.log(error)
} else {
console.log(Buffer.isBuffer(body)) // this is always false
const stream = streamifier.createReadStream(response.body)
console.log("Created stream")
res.setHeader("Access-Control-Allow-Origin", "*")
res.setHeader("Content-Type", "application/pdf");
res.setHeader("Content-Disposition", "inline; filename=document.pdf");
res.writeHead(response.statusCode)
stream.pipe(res)
console.log("Sent stream to response")
console.log("***********************************")
}
})
So turns out I was doing too much here. Since I'm using nodejs request module to make the GET call, all I need to then do is pipe the response to the client; like below:-
httpRequester.get(`${constants.serviceTarget}/api/Pandadoc/DownloadDocument/${opportunityId}/true`, (error, response, body) => {
if(error) {
console.log(error)
}
}).pipe(res)
I got the idea from: https://github.com/expressjs/express/issues/2910

Problem with redirect using 'fetch' from frontend page

I am using the following JS in a webpage to send information to a Node.js server upon 'clicking' on an image in the webpage, I am having trouble with the 'redirect' once the 'fetch' is executed:
fetch('/members/pages/callup', {
method: 'post',
headers: {
'Accept': 'application/json, text/plain, */*',
'Content-Type': 'application/json'
},
body: JSON.stringify({name: splits[1], presence: available, str: 'Some string: &=&'})
})
.then(function(res) {res.json()})
.then(function(res) {
if(res.response) {
redirect: window.location.replace("/members/pages/" + splits[1]);
} else {
alert("Error in the response");
}
})
.catch(function(err) {
alert("Error in the fetch call..." + err);
})
The fetch seems to properly send the 'body' data to the server. However I am getting the following error: "Error in the fetch call...TypeError: Cannot read property 'response' of undefined"...
The server performs a database call using the information sent by the frontend, and I thought all I needed to do was to send a "200 (OK)" response back...here is the server code:
app.post('/member/pages/callup', jsonParser, function (req, res) {
console.log("I RECEIVED FROM CLIENT THE FOLLOWING:");
console.log(req.body); //works fine, prints output from frontend 'fetch' to console...
db.lookupMember(req.body.name)
.then(function(foundUser) {
console.log('Async success!', foundUser); //works fine, prints database info to console...
if (typeof foundUser != "undefined") {
res.sendStatus(200); //trying this to 'reply' back to 'fetch' in frontend...is this not correct?
} //'foundUser' is NOT'undefined'...
})
.catch(function(error) {
console.log('UNABLE TO RETRIEVE MEMBER INFORMATION FROM THE DATABASE...' + error);
res.redirect('/'); //route to splash page...
});
})
Any suggestions appreciated, this has gone from a minor irritant to a major problem. I thank you in advance.
There are few issues in the code. If fixed, code should work fine.
You forgot to return res.json() from the function at one place. Make it return res.json() and it will work fine (Inside fetch, 1st then). Due to not returning, res is undefined which is giving the error
You are trying to use res.response but res is not send as a proper json from node server. This will fail at res.json(). You should be doing something like res.send({response: true})
After the if loop in server there is syntax error. It needs to be redirect = instead of redirect:. Also redirect is not declared anywhere which. (Note: you might not need redirect variable here, simply window.lo... should also work)
Note: Have updated the original answer after having the discussion with OP

Finding errors with request

I have a script that posts to an endpoint, like so using the node.js request module https://github.com/request/request
const options = {
url: path,
formData: {
name: name,
bundle: fs.createReadStream(path)
}
}
request.post(options, function(err, httpResponse, body) {
if (err) {
console.log('Error!')
} else {
console.log('Success!')
}
})
And I'm trying to catch when the post fails and doesn't work. I tried purposely uploading something and got a 400 response back, but it still came back with success. Is there a more appropriate way to handle error catching with the request module?
The request library doesn't populate the error argument of the request callback unless there is an actual error in the transmission or some other runtime issue. See this issue on the GitHub: 404 error does not cause callback to fail #2196.
Currently request does not handle the HTTP errors. You can wrap the
callback and add your own logic there.
To check for HTTP errors, check the statusCode property of the response argument:
request.post(options, function (err, httpResponse, body) {
if (err || httpResponse.statusCode >= 400) {
return console.error("Something went wrong");
}
console.log('Success!')
});

How do I consume SkyBiometry's faces/detect by POSTing a multipart jpeg request?

I want to write a request to the SkyBiometry API, and pass the faces/detect endpoint multiple files from my filesystem. The documentation states that the images should be a MIME multi-part message, where each argument is a separate chunk of form data.
I'm pretty new to using express and programming and general, and I was wondering whether what I'm doing is correct.
request.post(
{
url: 'http://api.skybiometry.com/fc/faces/detect.json?api_key=MYKEY',
form: {
files: [
fs.createReadStream(__dirname + '/images/img1.jpg'),
fs.createReadStream(__dirname + '/images/img2.jpg')
]
}
}, function (error, response, body) {
if (error !== null) {
console.log(error);
}
});
I'm getting a 402 - MISSING ARGUMENTS error, and it says I'm missing either urls or files. Does anyone know what I'm doing wrong?

Resources