Log request that caused error in node modules - node.js

I run a NodeJS server with two new error types in the logs:
[2021-05-21T09:11:33.891Z] SyntaxError: Unexpected token h in JSON at position 0
at JSON.parse (<anonymous>)
at createStrictSyntaxError (~/server/node_modules/body-parser/lib/types/json.js:158:10)
at parse (~/server/node_modules/body-parser/lib/types/json.js:83:15)
at ~/server/node_modules/body-parser/lib/read.js:121:18
at invokeCallback (~/server/node_modules/raw-body/index.js:224:16)
at done (~/server/node_modules/raw-body/index.js:213:7)
at IncomingMessage.onEnd (~/server/node_modules/raw-body/index.js:273:7)
at IncomingMessage.emit (events.js:323:22)
at IncomingMessage.EventEmitter.emit (domain.js:482:12)
at endReadableNT (_stream_readable.js:1204:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
The stacktrace shows only node_modules paths, not where in my code this error may have started. The stdout logs do not show what could have originated this error around that time either.
The server code that handles JSON objects is:
// Use JSON parser (required to parse POST forms)
app.use((req, res, next) => {
bodyParser.json()(req, res, next);
});
app.use(bodyParser.urlencoded({extended: true}));
I added logging inside this function in case I have the same error in the future.
In general, how can I log information about the request that caused an error in the node modules?
update with client-side code
This error originated from a user and I am unable to replicate it. The client-side code sending JSON data is:
// `id` indicates the ID of the video
var body = {
percent: percent,
videoId: id,
eventLabel: eventLabel
}
async function view() {
return await fetch("/viewership", {
method: "POST",
credentials: "include",
headers: {
Accept: "application/json",
"Content-Type": "application/json"
},
body: JSON.stringify(body)
});
};
The Network tab of the Chrome debugger shows this request payload:
{percent: 0, videoId: ..., eventLabel: "play"}

Well, here's what we know.
The code is processing an IncomingMessage (and incoming http request)
The error comes from the body-parser module
The error comes from JSON.parse() on what is apparently supposed to be a JSON body
The error appears to come from this particular section of code in the body-parser module.
That code is this:
if (strict) {
var first = firstchar(body)
if (first !== '{' && first !== '[') {
debug('strict violation')
throw createStrictSyntaxError(body, first)
}
}
So, it is apparently failing to find a leading { or [ on the JSON and is instead finding an h.
We can deduce from that information that an incoming http request (probably a POST) is supposed to have a JSON body, but the data is not legal JSON.
Your first point of debugging is to see exactly the JSON body data is in the request. If this request is coming from a browser, you can look in the Chrome network tab of the debugger and see exactly what the browser is sending your server.
So, this is most likely a client-caused error. Either the content-type is set wrongly to JSON when the data is not JSON or the client is supposed to be sending JSON, but is not sending proper JSON.
If you can show us the client-side code for this, we may be able to spot the error in that code.
Do you know a way to log any request that throws an error, e.g. for other bad requests in the future unrelated to JSON?
When the body-parser gets bad JSON, it calls the Express error handler with the exception. If you go the the "Writing Error Handlers" on this Express doc page, it will show you how to catch these errors and handle them with some error page back to the client and as much logging as you want.

Related

Json.parse throws: "SyntaxError: Unexpected end of JSON input"

Hello fellow stackoverflow members,
The Problem
as described in the titel I have a problem parsing an string to JSON.
The string I want to parse is send by the client to the server with a POST Method. The request is handled by this function on the server:
async function handleReservierung (req: Http.IncomingMessage, res: Http.ServerResponse): Promise<void> {
res.setHeader("Content-Type", "application/json");
res.setHeader("Access-Control-Allow-Headers", "content-type");
res.setHeader("Access-Control-Allow-Origin", "*");
let body: string = "";
req.on("data", chunk => {
body += chunk.toString();
});
req.on("end", () => {
console.log(JSON.parse(body));
res.end(body);
});
}
I've read about incoming POST requests and how to handle them and saw this solution with the incoming chunks added up in the variable body. So far so good.
I thought probably the chunks were not added correctly but as I console.log(body) it would give me a perfectly valid string:
{"ids":["600c4eff6c54c802cc1cac4f","600c4f576c54c802cc1cac51"],"name":"Someons Name"}
like so tested in a JSON validator.
I built an Interface
interface Reservierungen {
ids: number[];
name?: string;
}
which is the one used on the client Side to fill out all the data and then its getting send in the Body of the request with JSON.stringify(body)
When it now ty to JSON.parse(body) on the server it throws this error:
SyntaxError: Unexpected end of JSON input
at JSON.parse (<anonymous>)
at IncomingMessage.<anonymous> (C:\Users\felix\Documents\GIS\GIS-WiSe2020_2021\server\index.js:72:26)
at IncomingMessage.emit (events.js:327:22)
at endReadableNT (_stream_readable.js:1221:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
Why do I need to Parse the JSON?
I want to get the data from the JSON Object to update my MongoDB. I thought I could just parse the JSON string send in the body and use it to update the DB. After being stuck for quite a while I am kindly asking for you advice now :)
Important to know
Due to the limitations of our uni project: not useing any library like express or jquery I am asking for nativ typescript solutions.
I assume something is wrong with the string but I can't figure out what it is. I already tried to trim of the whitespaces which did not help either.
I hope I gave you all the information needed.
Thank you in advance.
I figured it out.
The problem was that I did copy the fetch from MDN and enabled CORS on the fetch() statement .
With CORS enabled there comes a preflight Request (CORS Details) which has an OPTIONS header. My Typescript server wasent configured to accept CORS requests (so no way to handle OPTIONS header) and did try to use the udefined body of the preflight Request.
I disabled the CORS header on the fetch() statement. Which resulted in a working JSON.parse()

Why does Firebase Cloud-Function send back an error object instead of the actual data

I'm testing some google cloud functions locally by making a small test application in angular and a Cloud-Function. I am trying to read the response back from the Cloud-Function when I make a post request. The cloud function then returns an error with the data inside of it instead of just the actual data. Curiously if I don't subscribe to the server's response, then I never see an error in the console, meaning it must be a setting on the cloud function itself. However, I copied and pasted the cloud function code straight from the documentation. Please help.
TL;DR: Http Cloud-Function returning an error instead of data, why?
Here is the code for the cloud function:
exports.helloWorld = functions.https.onRequest((request, response) => {
response.header('Access-Control-Allow-Origin', '*');
response.status(200).send('poop');
});
Here is the code for the test post:
test() {
var req = this.http
.post(
'http://localhost:5000/merchantapi-b7b17/us-central1/helloWorld',
JSON.stringify({ poop: 'poop' })
)
.subscribe(_val => {
console.log(_val);
});
}
Here is the error I get on the console:
HttpErrorResponse
error: {error: SyntaxError: JSON Parse error: Unexpected identifier "poop", text: "poop"}
headers: HttpHeaders {normalizedNames: Map, lazyUpdate: null, lazyInit: function}
message: "Http failure during parsing for http://localhost:5000/merchantapi-b7b17/us-central1/helloWorld"
name: "HttpErrorResponse"
ok: false
status: 200
statusText: "OK"
url: "http://localhost:5000/merchantapi-b7b17/us-central1/helloWorld"
HttpErrorResponse Prototype
defaultErrorLogger — core.js:6014
handleError — core.js:6066
next — core.js:40558
(anonymous function) — core.js:35336
__tryOrUnsub — Subscriber.js:185
next — Subscriber.js:124
_next — Subscriber.js:72
next — Subscriber.js:49
next — Subject.js:39
emit — core.js:35298
run — zone-evergreen.js:124
onHandleError — core.js:39735
runTask — zone-evergreen.js:171
invokeTask — zone-evergreen.js:465
timer — zone-evergreen.js:2650
defaultErrorLogger — core.js:6014
As you can see the error contains the data, and I can use that, but I'd like to understand why its sending an error and how to squash it.
By default modern HttpClient of Angular expects a JSON response and the string 'poop' is not a valid JSON.
To fix you need to send valid JSON back to the client:
exports.helloWorld = functions.https.onRequest((request, response) => {
response.header('Access-Control-Allow-Origin', '*');
response.status(200).send({'message':'poop'});
});
Valid json:
{
"message": "poop"
}

Error trying to render new page after fetch request in Express

Problem
I have set up an HTTP fetch request from the client side with some JSON data (a dataURL from a canvas), and I want to be able to access it from the server side and then render a new page that displays that dataURL. This is an Express only app (no client-side frameworks).
However, I keep getting an error. It would seem to me that there's something wrong with the JSON data or the way I handle it, but look at the code.
In Safari, it's Unhandled Promise Rejection: SyntaxError: The string did not match the expected pattern.
In Chrome, it's Uncaught (in promise) SyntaxError: Unexpected token < in JSON at position 0
In Firefox, it's SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data
Code
Client Side
I have an HTML canvas that I've converted into a dataURL and saved in a JSON object. That object gets sent to sendData.
If I get rid of the line response.json(), then I don't get the error. But I want to render a new page with the dataURL displayed in a paragraph tag.
function sendData(input) {
fetch('/', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(input)
})
.then((response) => {
response.json()
})
}
//Saves dataURL as string
function saveData(input){
var dataURL = input.toDataURL()
data = { imageData: dataURL}
sendData(data)
}
Server Side
If I get rid of the line res.render('images'), I don't get the error, but, once again, I can't render the new page then. Also, this might be another issue, but Express is already not rendering the new page. Logging the req.body.imageData works.
exports.submit_image = function (req, res, next) {
console.log(req.body.imageData)
res.render('images')
}
Routes
And just in case this helps, here's the index.js in the routes directory
router.get('/', index.get_index)
router.post('/', index.submit_image)
How can I resolve this issue?
Node.js automatically handles rejected promises by logging them. You can suppress it using the following code. Take a look at this page for more information about promises.
window.addEventListener('unhandledrejection', event => {
event.preventDefault()
}, false)

React Native/Node API: Can't set headers after they are sent

I am having an issue with my react native app interacting with my Node API.
When I try to post data to update some values in the database through node (which then goes to a stored procedure), I get a 500 - Can't set headers after they are sent in Node.
Everything I can find says that this could be due to sending responses twice. I don't think this is the case here. I've tested in postman and things work fine, it returns a status of 200 with the correct returned data.
I am trying to post data to the API like so:
const myHeaders = new Headers();
myHeaders.append('Content-Type', 'application/json');
fetch(`http://localhost:3000/user/preferences`, {
method: 'POST',
headers: myHeaders,
mode: 'cors',
cache: 'default',
body: JSON.stringify({
id: '1',
minage: this.state.minageValue,
maxage: this.state.maxageValue
})
})
.then(response => response.json())
.then(body => console.log(body))
.catch(err => console.log(err));
I am receiving it on the API side and passing the data to a stored procedure with this:
function updatePreferences(req, res, next) {
console.log(req);
var userid = req.body[0].id;
var minage = req.body[0].minage;
var maxage = req.body[0].maxage;
db.func('___spsavepreferences', [userid, minage, maxage])
.then(function(data){
res.status(200)
.json({
status: 'success',
preferences: data
});
});
}
Any ideas?
EDIT:
Don't know if this tells me anything, but I have logged the error message to the console:
Error: Can't set headers after they are sent.
at validateHeader (_http_outgoing.js:504:11)
at ServerResponse.setHeader (_http_outgoing.js:511:3)
at ServerResponse.header (/Users/user/Projects/Project_node/node_modules/express/lib/response.js:730:10)
POST /user/preferences 500 410.809 ms - 189
at ServerResponse.send (/Users/user/Projects/Project_node/node_modules/express/lib/response.js:170:12)
at done (/Users/user/Projects/Project_node/node_modules/express/lib/response.js:967:10)
at Object.exports.renderFile (/Users/user/Projects/Project_node/node_modules/jade/lib/index.js:374:12)
at View.exports.__express [as engine] (/Users/user/Projects/Project_node/node_modules/jade/lib/index.js:417:11)
at View.render (/Users/user/Projects/Project_node/node_modules/express/lib/view.js:128:8)
at tryRender (/Users/user/Projects/Project_node/node_modules/express/lib/application.js:640:10)
at Function.render (/Users/user/Projects/Project_node/node_modules/express/lib/application.js:592:3)
at ServerResponse.render (/Users/user/Projects/Project_node/node_modules/express/lib/response.js:971:7)
at /Users/user/Projects/Project_node/app.js:51:7
at Layer.handle_error (/Users/user/Projects/Project_node/node_modules/express/lib/router/layer.js:71:5)
at trim_prefix (/Users/user/Projects/Project_node/node_modules/express/lib/router/index.js:315:13)
at /Users/user/Projects/Project_node/node_modules/express/lib/router/index.js:284:7
at Function.process_params (/Users/user/Projects/Project_node/node_modules/express/lib/router/index.js:335:12)
EDIT 2: Got it. I needed to recieve the values as req.body.id, req.body.minage, etc... not as req.body[0].id.
Weird to get an error like that but that solved the error.
If you’re seeing different behavior in postman, that’s because postman just does the POST directly while the browser instead does a CORS preflight OPTIONS request before trying the POST.
So it sounds like the request that’s causing the “Can't set headers after they are sent” you’re seeing is that OPTIONS request, not the POST request.
You can confirm that by sending a OPTIONS request with postman, and making sure to also supply an Origin request header as part of the request. With curl you can do that like this:
curl -X OPTIONS -i -H 'Origin: http://x.y' http://localhost:3000/user/preferences
That will likely produce the same 500 error you’re seeing from the browser request.
As far as how to fix it, I think there’s no way anybody else here could tell you without seeing a lot more of your server side code. But what it would seem to come down to is, somewhere in your code you have one or more places with logic for handling OPTIONS requests, and somewhere in that OPTIONS-handling code, it’s trying to send headers after they’ve already been sent.
To track down where that is, it seems like you want to look through the server logs on the server side and find any message that’s getting logged when that 500 failure occurs.
And by the way, in this case the thing that triggers the browser to do a COR preflight OPTIONS request to begin with is the myHeaders.append('Content-Type', 'application/json') part.

Workaround from client side for invalid HTTP response headers that contain both Transfer-Encoding and Content-Length?Can see response using Postman

I'm calling an API that returns 5 keys in the header including Transfer-encoding:chunked and Content-Length values, in the header.
According to stackoverflow question, and this below, this is illegal:
"...Node.js server returns a parse error with code HPE_UNEXPECTED_CONTENT_LENGTH when consuming the endpoint, because the response headers contains both Transfer-encoding:chunked and Content-Length values.
This is considered has an error as specified in RFC 7230 section 3.3.3.3 :
If a message is received with both a Transfer-Encoding and a Content-Length header field, the Transfer-Encoding overrides the Content-Length. Such a message might indicate an attempt to perform request smuggling (Section 9.5) or response splitting (Section 9.4) and ought to be handled as an error. A sender MUST remove the received Content-Length field prior to forwarding such a message downstream." (1)
I'm using node.js and npm-request module to send the request and try and parse the response. The function being called is
function createPostAPI(req, res, userid, detailText, title, location, custImpact){
var myJSONObject = {
"userId" : userid,
"apiKey" : "{API-Key}",
"detail" : {
"detailText" : detailText
},
"info" : {
"title":title
"location" : location,
"priority" : 0,
"customerImpact" : custImpact
}
}
request1({
url: "http://theAPIurl",
method: "POST",
body:JSON.stringify(myJSONObject)
json: true, // <--Very important!!!
body: myJSONObject
}, function (error, response, body){
console.log(error);
console.log(response);
console.log(body);
});
}
This is the error i get as expected :
{ Error: Parse Error
at Error (native)
at Socket.socketOnData (_http_client.js:362:20)
at emitOne (events.js:96:13)
at Socket.emit (events.js:188:7)
at readableAddChunk (_stream_readable.js:176:18)
at Socket.Readable.push (_stream_readable.js:134:10)
at TCP.onread (net.js:547:20) bytesParsed: 194, code: 'HPE_UNEXPECTED_CONTENT_LENGTH' }
However, I can see that the API call is working, since I can visually verify that it's working! Also using Postman I can see the response that I need!
Postman Response:
The header with the 2 headers (content-length and transfer-encoding):
The response body that I need that i can see being returned:
So finally my question is, is there anything i can do from my side (client) to avoid the error and read that http response from the server (even though the format is wrong)? How come I can see the body in Postman? Any ideas would be truly appreciated!
Using node v6.10.3
Reference:
(1) https://jira.spring.io/browse/SPR-15212

Resources