I have a nodeJS server configured with Express and BodyParser
const express = require('express')
const expressWs = require('express-ws')
const bodyParser = require('body-parser')
app.ws('/ws', websocket)
When the websocket gets a message I pass it on
ws.onmessage = e => {
const {action, payload} = JSON.parse(e.data)
channel.send(action,payload)
}
However when it comes to the app via the channel it's got lots of extra characters in it
"{\"action\":\"guide_data_retreived\",\"payload\":[{\"id\":544,\"json\":\"{\\\"code\\\":\\\"lPvwP4rz\\\",\\\"coverDesign\\\":null,\\\"created\\\":1535018423000,\\\"description\\\":\\\"{\\\\\\\"blocks\\\\\\\":[{\\\\\\\"key\\\\\\\":\\\\\\\"dpcth\\\\\\\",\\\\\\\"text\\\\\\\":\\\\\\\"This is an example of a medical emergency. \\\\\\\",\\\\\\\"type\\\\\\\":\\\\\\\"unstyled\\\\\\\",\\\\\\\"depth\\\\\\\":0,\\\\\\\"inlineStyleRanges\\\\\\\":[],\\\\\\\"entityRanges\\\\\\\":[],\\\\\\\"data\\\\\\\":{}},
Which makes it unparseable.
Any idea where this is coming from and how to fix it?
The problem is that you have several levels of string-encoded JSON nested within your objects.
The \'s are escape characters. They are there to indicate that the quote following them is not a terminating quote, but rather is a character in the string. So for instance, let's say I want to have a javascript object which looks like this:
// myFile.javascript
{ "x" : "abc" }
const s = JSON.stringify(foo) // s is a string
Then s will include these escape characters, so that the quotes around "x" and "abc" will be interpreted as inside the string rather than as string terminators:
s == "{\"x\":\"abc\"}" // -> true
So since s is a string, you can also put it inside another object, like this:
const bar = { "nested" : s }
And if you stringify that, you will end up with another level of escapes to signify that s is a string and not a nested JSON object within the object:
JSON.stringify(bar) == "{\"nested\":\"{\\"x\\":\\"abc\\"}\"}
So it's clear that within your application, you're passing around strings instead of objects. For instance, inside your payload, json is a nested string-encoaded JSON, and inside json, description is string-encoaded-JSON.
If you have control over the message being sent on this websocket, then you should parse those strings before you put them in the payload.
So for instance, if you are building the payload like this:
func sendMessage(ws, action, id, json) {
ws.send(action, {id: id, json: json})
}
then you should change it to this:
func sendMessage(ws, action, id, json) {
ws.send(action, {id: id, json: JSON.parse(json)})
}
And so on, for each level of nested object.
Related
I am in the midst of writing a router that uses a JSON object as a manifest. The router uses the JSON manifest as a reference for where to route incoming requested url paths. The code below belongs to main.js that is ran using the node command to start the application. Near the top of the document, you can see the router object that takes the JSON Manifest as an argument. Below this bit of code, is another bit, that shows the router document where the router class is written into it, you can see there how I am attempting to route request using this JSON Manifest.
const app = require('./serv/katanaLib');
const Router = require('./serv/router');
var router = new Router.Router({
"^/(.*)$" : "/public/view/page/:[?]",
"^/user/(.*)$" : "/public/view/account/:[?]",
"^/style/(.*)$" : "/public/style/:[?]"
});
app.Server(8888, function(request, response)
{
const $_Request = app.parseHttpReq(request);
router.onReq($_Request);
});
Below is router.js, this is where I am having a problem, which I will explain at the bottem of this question.
module.exports.Router = class Router
{
constructor(JSON_manifest)
{
this.manifest = JSON_manifest;
this.keys = Object.keys(JSON_manifest);
this.values = Object.values(JSON_manifest);
};
onReq($_Request)
{
this.keys.forEach((key, i)=>
{
key = ('^' + key + '$');
var re = new RegExp(key, 'i');
let x = $_Request.path.search(re);
console.log('\n\n' + key);
console.log(re);
console.log(x + '\n\n');
});
};
};
As anyone can obviously see, I am writing Regular Expressions into the keys of the JSON manifest object. You can see that I search the requested path using a foreach loop, looping through each manifest key. The problem I am having is that my regex values are matching multiple times. When I run the code above, and make the follow request to my server.
http://localhost:8888/index
Request Recieved:
Parsed-Request:
{ path: '/index', method: 'GET', ext: '', type: 'text/html' }
^^/(.*)$$
/^^\/(.*)$$/i
0
^^/user/(.*)$$
/^^\/user\/(.*)$$/i
-1
^^/style/(.*)$$
/^^\/style\/(.*)$$/i
-1
that is a desireable result, but when I run:
http://localhost:8888/user/abc
i get...
Parsed-Request:
{ path: '/user/abc', method: 'GET', ext: '', type: 'text/html' }
^^/(.*)$$
/^^\/(.*)$$/i
0
^^/user/(.*)$$
/^^\/user\/(.*)$$/i
0
^^/style/(.*)$$
/^^\/style\/(.*)$$/i
-1
The Regular Expression is matching both, the first and second, which is undesirable. If I requested http://localhost/style/123 the requested path will match the first key and third key. If anyone has advice on how to write the regex so I can finish this that would be great, thank you...
To solve the issue with multiple regexes matching, it's best to test them in a specific order, from most specific to least specific. Then, the first match is to be used.
In your example, the regexes are provided as object keys.
Since object keys are unordered, you'd need to use an array instead to control the order of the regexes.
In this case, the order of regexes would be as follows (first and second can be swapped):
^/user/(.*)$
^/style/(.*)$
^/(.*)$
I am using Express with Body Parser. Given a below header key:
X-Master-Key
When I am using the below code snippet, it fails to output the value
req.headers['X-Master-Key'] // Fails
but when the above is changed to, it works
req.headers['x-master-key'] // Works
Further, when I tried to output req.headers, it turns out that Express outputs all the headers in a down-case format.
I started digging down further and tried using the below code, either of these snippets work
req.header('X-Master-Key'); // Works
// -- OR
req.header('x-master-key'); // Works
So what's the issue here? Why does Express changes all header keys to down-case? Moreover, how using req.header() is different from req.headers[]?
The problem arises because in the HTTP protocol, headers are case-insensitive. This means that content-type, Content-Type, and coNTEnt-tYPe all refer to the same header, and the Express framework needs to be able to handle any of them.
The different between req.headers (the object) and req.header (the function) is simply this:
If you want to get a property from a Javascript object, the property name is case-sensitive. So req.headers['content-type'] will work; req.headers['Content-Type'] will not. Why does the lower case version work? Because the Express framework, in an attempt to handle all the different possible cases (remember, HTTP will allow anything), converts everything to lower case.
But the developers of Express recognize that you (the developer) might be looking for Content-Type and you might not remember to convert to lower case, so they provided a function, req.header, which will take care of that for you.
So, in short:
This is recommended:
const myHeader = req.header('Content-Type');
Use whatever case you want - the function will convert it to lower case and look up the value in req.headers.
This is not recommended:
const myHeader = req.headers['Content-Type'];
If you don't use a lower-case header name, you won't get what you expect.
The problem comes down to case-sensitivity.
When you look at the documentation for req.get (which is aliased by req.header), it states:
Returns the specified HTTP request header field (case-insensitive match). The Referrer and Referer fields are interchangeable.
The w3 standard indicates that headers should be case-insensitive:
Each header field consists of a name followed by a colon (":") and the field value. Field names are case-insensitive.
So it would appear that node http module, which express uses, just treats them all as lower-case to "save you steps" according to this github issue
You can see that the express framework req object actually utilizes the node module http:
var accepts = require('accepts');
var deprecate = require('depd')('express');
var isIP = require('net').isIP;
var typeis = require('type-is');
var http = require('http');
var fresh = require('fresh');
var parseRange = require('range-parser');
var parse = require('parseurl');
Furthermore, in the code you can see that the req.header method converts whatever you give it to lower-case:
req.get =
req.header = function header(name) {
if (!name) {
throw new TypeError('name argument is required to req.get');
}
if (typeof name !== 'string') {
throw new TypeError('name must be a string to req.get');
}
var lc = name.toLowerCase();
switch (lc) {
case 'referer':
case 'referrer':
return this.headers.referrer
|| this.headers.referer;
default:
return this.headers[lc];
}
};
Finally, the http module parses headers using the matchKnownFields function which automatically lower-cases any and all headers that aren't "traditional headers", in which case it is case-insensitive.
Here is the responsible snippet, that implements the behavior you are seeing:
if (lowercased) {
return '\u0000' + field;
} else {
return matchKnownFields(field.toLowerCase(), true);
}
I use koa js with bodyparser, suppose client send body like this:
{ "first": "1" , "second": "2"}
what I want is the original body as string with no changes (JSON.stringify changes the order of fields and remove spaces then I can't use it). I try to use raw-body that gives me the string of body, so I have to parse it to JSON.
is there any middleware that give me body as both json and original string?
If you want both the raw string and the JSON, get the string, keep a copy, then parse it to JSON.
var getRawBody = require('raw-body')
app.use(function* (next) {
var string = yield getRawBody(this.req, {
length: this.length,
limit: '1mb',
encoding: this.charset
})
var json = JSON.parse(string)
// do something with "string"
// do something with "json"
})
Note: You have to run getRawBody() against this.req, since that's node's raw http request object. this.request is koa-specific and won't work.
I'm trying to use breeze.js with sails.js.
Therefore I'm using the breeze json uriBuilder.
I logged the req.query and got the following:
{ '{"where":{"name":"foo"},"orderBy":': { '"name"': '' } }
To query waterline objects I need to bring it into a format like this:
{ where: { name: 'foo' }, sort: 'name' }
First thing I tried:
var queryString = JSON.parse(Object.keys(req.query)[0]);
That works as long as I only put a where clause in. But with more parameters i get this strangely formatted json object.
How can I parse it to get the correct object?
Solution:
Don't parse he req.query. Parse the url and parse it. This way u will get a json uery that sails accepts. Now strip of unsupported parameters as select and you're done.
var parsedUrl = urlUtils.parse(req.url, true);
var jsonQueryString = Object.keys(parsedUrl.query)[0];
var jsonQuery = JSON.parse(jsonQueryString);
Good question! We haven't yet documented this adequately, but the basic idea is that there is a breeze-client npm package (https://www.npmjs.com/package/breeze-client) that you can use to parse the incoming json query into a breeze EntityQuery instance, from there you can interrogate the EntityQuery and turn it into whatever server side syntax you want, i.e. in your case Sails.)
// req = the HTTP request object.
var resourceName = req.params.slug; // the HTTP endpoint
var entityQuery = breeze.EntityQuery.fromUrl(req.url, resourceName);
// you would write the transform to sails below.
var sailsQuery = tranformToSails(entityQuery);
This is exactly what we do in the breeze-sequelize npm package where we take the incoming req.query and go thru the process above to create a 'Sequelize' query.
See http://www.getbreezenow.com/sequelize-mysqlpostgressql-lite
I have a nodejs express web server running on my box. I want to send a get request along with query parameters. Is there any way to find type of each query parameter like int,bool,string. The query parameters key value is not know to me. I interpret as a json object of key value pairs at server side.
You can't, as HTTP has no notion of types: everything is a string, including querystring parameters.
What you'll need to do is to use the req.query object and manually transform the strings into integers using parseInt():
req.query.someProperty = parseInt(req.query.someProperty);
You can also try
var someProperty = (+req.query.someProperty);
This worked for me!
As mentioned by Paul Mougel, http query and path variables are strings. However, these can be intercepted and modified before being handled. I do it like this:
var convertMembershipTypeToInt = function (req, res, next) {
req.params.membershipType = parseInt(req.params.membershipType);
next();
};
before:
router.get('/api/:membershipType(\\d+)/', api.membershipType);
after:
router.get('/api/:membershipType(\\d+)/', convertMembershipTypeToInt, api.membershipType);
In this case, req.params.membershipType is converted from a string to an integer. Note the regex to ensure that only integers are passed to the converter.
This have been answered a long ago but there's a workaround for parsing query params as strings to their proper types using 3rd party library express-query-parser
// without this parser
req.query = {a: 'null', b: 'true', c: {d: 'false', e: '3.14'}}
// with this parser
req.query = {a: null, b: true, c: {d: false, e: 3.14}}
let originalType = JSON.parse(req.query.someproperty);
In HTTP, querystring parameters are treated as string whether you have originally sent say [0,1] or 5 or "hello".
So we have to parse it using json parsing.
//You can use like that
let { page_number } : any = req.query;
page_number = +page_number;
Maybe this will be of any help to those who read this, but I like to use arrow functions to keep my code clean. Since all I do is change one variable it should only take one line of code:
module.exports = function(repo){
router.route('/:id',
(req, res, next) => { req.params.id = parseInt(req.params.id); next(); })
.get(repo.getById)
.delete(repo.deleteById)
.put(repo.updateById);
}