How do I make sure that any invalid or unwanted request that isn't following sockets protocol gets rejected and my socket aren't closed.
Consider a simple example :
var net = require('net');
net.createServer(function(socket) {
socket.on('data', function(data) {
var parsedData = JSON.parse(data.toString());
});
}).listen(5555);
server.listen(1337, '127.0.0.1');
and try visiting that port from browser , and again look at terminal. It produces following error :
undefined:1
GET / HTTP/1.1
^
SyntaxError: Unexpected token G
at Object.parse (native)
at Socket.<anonymous> (/home/agauniyal/projects/watch/watch.js:115:35)
at Socket.emit (events.js:107:17)
at readableAddChunk (_stream_readable.js:163:16)
at Socket.Readable.push (_stream_readable.js:126:10)
at TCP.onread (net.js:529:20)
Now I fully agree that visiting that port is a mistake of mine , but closing a socket on different format rather than rejecting that request isn't the ideal way to do it. And what if someone tries to access a webserver at that port only realizing later that there wasn't any at 8080.
So how should I make sure socket isn't closed and rather rejects that request?
EDIT : This is possibily happening because I'm calling JSON.Parse(data.toString()) method on data being received from socket and http headers aren't being parsed by that method.
Entire problem rose because I was getting socket response , converting it to string and then parsing it as JSON. Now in my case , I've either to gurantee that incoming data is valid JSON string or find that a browser is sending request to that socket.
For a real-quick-fix , I am extracting first word from incoming data and checking it to be 'GET' ( which a browser sends ) , and if it evaluates to true , just doing nothing else proceeding as I was earlier. Catching JSON.parse error is better solution too.
Related
I'm currently trying to access the JokeAPI using Node's module HTTPS, where below is my code:
const https = require('https');
let url = 'https://v2.jokeapi.dev/joke/Any';
https.get(url, function (res) {
console.log(res);
})
However, somehow I keep getting this error:
node:events:498
throw er; // Unhandled 'error' event
^
Error: self signed certificate in certificate chain
at TLSSocket.onConnectSecure (node:_tls_wrap:1530:34)
at TLSSocket.emit (node:events:520:28)
at TLSSocket._finishInit (node:_tls_wrap:944:8)
at TLSWrap.ssl.onhandshakedone (node:_tls_wrap:725:12)
Emitted 'error' event on ClientRequest instance at:
at TLSSocket.socketErrorListener (node:_http_client:442:9)
at TLSSocket.emit (node:events:520:28)
at emitErrorNT (node:internal/streams/destroy:157:8)
at emitErrorCloseNT (node:internal/streams/destroy:122:3)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
code: 'SELF_SIGNED_CERT_IN_CHAIN'
Is it because I don't have a server set up? I'm a beginner with JS and Node, so any help would be very appreciated. Thank you!
Edit: I actually added 'process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = '0';' to the top of the code and it worked, but now I'm getting an 'undefined' for a response body. And I'm not sure why that is happening :(
My system does not show the self signed certificate in certificate chain error, but it sounds like you have a work-around for that.
However, now I'm getting an 'undefined' for a response body. I'm not too sure what's causing this.
https.get() just sends the request and reads the headers. It does not, by itself, read the whole response. To do that, you need to listen for the data event on the res object. Installing the listener for the data event will cause the stream to start reading data from the incoming request. This data will not necessarily come all at once. It can come broken into multiple data events so you would really need to accumulate all the data and then in the end event, you know you have all the data. You can see a full example for how to implement that here in the http.get() doc.
Here's a simpler version that will just show you what's in each data event.
const https = require('https');
let url = 'https://v2.jokeapi.dev/joke/Any';
https.get(url, function(res) {
res.on('data', data => {
console.log(data.toString());
});
});
FYI, I no longer use plain https.get() because all the higher level libraries listed here are more convenient to use and all support promises which is generally an easier way to program with asynchronous requests.
My favorite library from that list is got(), but you can choose whichever one you like the API and/or features for. Here's an implementation using got():
const got = require('got');
const url = 'https://v2.jokeapi.dev/joke/Any';
got(url).json().then(result => {
console.log(result);
}).catch(err => {
console.log(err);
});
I have an http server listening on port 9090 - piping the request to stdout like so:
let server = http.createServer((req, res) => {req.pipe(process.stdout)})
server.listen(9090)
When I send it something with curl like so:
curl -XGET -T - 'http://localhost:9090' < /tmp/text-input
it works, and I see the output on the server's terminal
but when I try the following in node:
const http = require('http')
const nurl = new URL("http://localhost:9090")
let request = http.request(nurl)
request.on('response', (res) => {
process.stdin.pipe(request)
})
request.end() // If I emit this, nothing happens. If I keep this, I get the below error
and try to run it like so: node request.js < /tmp/text-input, I'm getting the following error:
node:events:368
throw er; // Unhandled 'error' event
^
Error [ERR_STREAM_WRITE_AFTER_END]: write after end
at new NodeError (node:internal/errors:371:5)
at write_ (node:_http_outgoing:748:11)
at ClientRequest.write (node:_http_outgoing:707:15)
at ClientRequest.<anonymous> (/home/tomk/workspace/js-playground/http.js:17:7)
at ClientRequest.emit (node:events:390:28)
at HTTPParser.parserOnIncomingClient (node:_http_client:623:27)
at HTTPParser.parserOnHeadersComplete (node:_http_common:128:17)
at Socket.socketOnData (node:_http_client:487:22)
at Socket.emit (node:events:390:28)
at addChunk (node:internal/streams/readable:324:12)
Emitted 'error' event on ClientRequest instance at:
at emitErrorNt (node:_http_outgoing:726:9)
at processTicksAndRejections (node:internal/process/task_queues:84:21) {
code: 'ERR_STREAM_WRITE_AFTER_END'
}
I want to pipe my stdin to an http server the same way I can with curl -T -. What is wrong with my request code?
Short answer
To send chunked encoding messages in node, use the POST method:
let request = http.request(url, { method: 'POST' })
process.stdin.pipe(request)
Edit: A more straigt forward approach
Or, to send any request method with chunked encoding:
let request = http.request(url)
request.setHeader("transfer-encoding", "chunked")
request.flushHeaders()
process.stdin.pipe(request)
Slightly longer (yet partial) answer
I opened a listening netcat (listen on plain tcp) like so nc -l 9090 to view how the request from curl differs from my code and found a few key differences in the headers.
In curl, the header Transfer-Encoding: chunked appeared, but was missing from the request my code sent out. Also, my code had a header Connection: closed
I logged the request object and found that useChunkedEncodingByDefault is set to false, which was confusing given the quote from the nodejs http docs:
Sending a 'Content-Length' header will disable the default chunked encoding.
Implying that it should be the default.
But then I found this in the source of node
if (method === 'GET' ||
method === 'HEAD' ||
method === 'DELETE' ||
method === 'OPTIONS' ||
method === 'TRACE' ||
method === 'CONNECT') {
this.useChunkedEncodingByDefault = false;
} else {
this.useChunkedEncodingByDefault = true;
}
Edit
To send chunked encoding anyway, I (eventually) found that I need to explicitly add the Transfer-Encoding: Chunked header explicitly:
request.setHeader("transfer-encoding", "chunked")
# and then
request.flushHeaders()
So, in conclusion, node doesn't allow sending send by default GET requests with chunked encoding, but curl does. Odd, and unfortunately not documented (as far as I could find), but the important thing I got it working
I run a NodeJS server with two new error types in the logs:
[2021-05-21T09:11:33.891Z] SyntaxError: Unexpected token h in JSON at position 0
at JSON.parse (<anonymous>)
at createStrictSyntaxError (~/server/node_modules/body-parser/lib/types/json.js:158:10)
at parse (~/server/node_modules/body-parser/lib/types/json.js:83:15)
at ~/server/node_modules/body-parser/lib/read.js:121:18
at invokeCallback (~/server/node_modules/raw-body/index.js:224:16)
at done (~/server/node_modules/raw-body/index.js:213:7)
at IncomingMessage.onEnd (~/server/node_modules/raw-body/index.js:273:7)
at IncomingMessage.emit (events.js:323:22)
at IncomingMessage.EventEmitter.emit (domain.js:482:12)
at endReadableNT (_stream_readable.js:1204:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
The stacktrace shows only node_modules paths, not where in my code this error may have started. The stdout logs do not show what could have originated this error around that time either.
The server code that handles JSON objects is:
// Use JSON parser (required to parse POST forms)
app.use((req, res, next) => {
bodyParser.json()(req, res, next);
});
app.use(bodyParser.urlencoded({extended: true}));
I added logging inside this function in case I have the same error in the future.
In general, how can I log information about the request that caused an error in the node modules?
update with client-side code
This error originated from a user and I am unable to replicate it. The client-side code sending JSON data is:
// `id` indicates the ID of the video
var body = {
percent: percent,
videoId: id,
eventLabel: eventLabel
}
async function view() {
return await fetch("/viewership", {
method: "POST",
credentials: "include",
headers: {
Accept: "application/json",
"Content-Type": "application/json"
},
body: JSON.stringify(body)
});
};
The Network tab of the Chrome debugger shows this request payload:
{percent: 0, videoId: ..., eventLabel: "play"}
Well, here's what we know.
The code is processing an IncomingMessage (and incoming http request)
The error comes from the body-parser module
The error comes from JSON.parse() on what is apparently supposed to be a JSON body
The error appears to come from this particular section of code in the body-parser module.
That code is this:
if (strict) {
var first = firstchar(body)
if (first !== '{' && first !== '[') {
debug('strict violation')
throw createStrictSyntaxError(body, first)
}
}
So, it is apparently failing to find a leading { or [ on the JSON and is instead finding an h.
We can deduce from that information that an incoming http request (probably a POST) is supposed to have a JSON body, but the data is not legal JSON.
Your first point of debugging is to see exactly the JSON body data is in the request. If this request is coming from a browser, you can look in the Chrome network tab of the debugger and see exactly what the browser is sending your server.
So, this is most likely a client-caused error. Either the content-type is set wrongly to JSON when the data is not JSON or the client is supposed to be sending JSON, but is not sending proper JSON.
If you can show us the client-side code for this, we may be able to spot the error in that code.
Do you know a way to log any request that throws an error, e.g. for other bad requests in the future unrelated to JSON?
When the body-parser gets bad JSON, it calls the Express error handler with the exception. If you go the the "Writing Error Handlers" on this Express doc page, it will show you how to catch these errors and handle them with some error page back to the client and as much logging as you want.
I am writing this question because I have a question while writing the current program.
If you go to the Chrome Developer Tools -> Network tab on a specific homepage, you are writing a program to receive this as a Node.js.
In the General tab of the Header
Request URL: wss://stream-live.bitsonic.co.kr:8443/socket.io/?streams=btckrwticker%2FbtckrwaggTrade%2Fbtckrw*depth_20&EIO=3&transport=websocket&sid=1p8mHxkKcBYxAvKVE1id
I am writing this program based on url as wss: // stream-live.bitsonic.co.kr:8443 /.
In the Query String Parameters tab,
streams: btckrwticker/btckrwaggTrade/btckrw*depth_20
EIO: 3
transport: websocket
sid: 1p8mHxkKcBYxAvKVE1id
Contains the value.
Below is the program I wrote.
const io = require('socket.io-client');
const socket_url = 'wss://stream-live.bitsonic.co.kr:8443/';
var socket = io.connect(socket_url);
socket.on('connect', function(err, data){
console.log('Connected.1');
socket.emit('DEPTH',{
query: {
'streams': 'btckrw*ticker/btckrw*aggTrade/btckrw*depth_20',
'EIO': '3',
'transports': ['websocket'], // ['websocket', 'flashsocket', 'polling']
// 'nonce': Date.now()*1000,
'sid': 'UzCyjO_8gXPpPrcEEoEg'
}
});
});
socket.on('error', function(data){
console.log('err',data);
});
socket.on('close', function(data){
console.log('cls',data);
});
socket.on('message',(body)=>{
console.log('msg',body);
});
In the web socket tap, the contents of message is stacked on the Frames tab, and io.on ('message') is supposed to contain the content.
Below is the error content.
err: { Error: xhr post error
at XHR.Transport.onError (G:\dev\js\bsx\node_modules\engine.io-client\lib\transport.js:64:13)
at Request.<anonymous> (G:\dev\js\bsx\node_modules\engine.io-client\lib\transports\polling-xhr.js:109:10)
at Request.Emitter.emit (G:\dev\js\bsx\node_modules\component-emitter\index.js:133:20)
at Request.onError (G:\dev\js\bsx\node_modules\engine.io-client\lib\transports\polling-xhr.js:309:8)
at Timeout._onTimeout (G:\dev\js\bsx\node_modules\engine.io-client\lib\transports\polling-xhr.js:256:18)
at ontimeout (timers.js:498:11)
at tryOnTimeout (timers.js:323:5)
at Timer.listOnTimeout (timers.js:290:5) type: 'TransportError', description: 403 }
I would really appreciate it if you let me know what went wrong.
And I apologize for my poor English ability.
you have to know that using socket.io is not the exact same thing as normal WebSocket
if I get what you want correctly then you should know that using
socket.on('message',(body)=>{
console.log('msg',body);
});
is not the same as
let socket = new WebSocket('url');
socket.onmessage(callback) //or using addEventListener
in socket.io the message event should be defined by you and it's not automatic as I know
in normal WebSocket on message will listen to everything and check every request and response
hope you can get somewhere from my answer
http.js:732
throw new Error('Can\'t render headers after they are sent to the client.'
^
Error: Can't render headers after they are sent to the client.
at ServerResponse.OutgoingMessage._renderHeaders (http.js:732:11)
at ServerResponse.writeHead (http.js:1153:20)
at ProxyServer.<anonymous> (/home/ec2-user/.meteor/packages/meteor-tool/.1.1.4.1tjewoi++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/tools/run-proxy.js:96:21)
at ProxyServer.emit (/home/ec2-user/.meteor/packages/meteor-tool/.1.1.4.1tjewoi++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/http-proxy/node_modules/eventemitter3/index.js:100:27)
at ClientRequest.proxyError (/home/ec2-user/.meteor/packages/meteor-tool/.1.1.4.1tjewoi++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/http-proxy/lib/http-proxy/passes/web-incoming.js:140:16)
at ClientRequest.emit (events.js:117:20)
at Socket.socketOnData (http.js:1593:9)
at TCP.onread (net.js:528:27)
Meteor server throws this error or Production instance only. I'm running the same repository in Staging and it runs smoothly. Meteor throws this error even when nobody is interacting with the client (Crons run on server during regular intervals). I'm not able to figure out the reason. People have faced this problem for different issues, but I didn't find it familiar with my case.
I suspect this code is throwing this error. Not sure though
updateFunction = function(event) {
var res = Meteor.http.call("GET", "http_url"+ event);
var contents = EJSON.parse(res.content).tracks["0"];
if(!contents) return;
var events = [];
contents.map(function(ele){
if(ele.type == "snap") {
ele._id = ele.id;
delete ele.id;
events.push(ele);
}
});
CollectionName.upsert(event,{"$set":{"data": events}});
}