Node.js - Send and receive Array as GET/POST using querystring - node.js

I've got the following code, but it doesn't seem to work:
var post_req = {
array: [
[ {
param1: 'something',
param2: 123
} ],
[ ],
[ ],
[ {
param2: 'something',
param4: 1234,
param1: 'hello'
} ]
]
};
var data_send = querystring.stringify(post_req);
var request = client.request('POST', '/', headers);
request.end(data_send);
and
if( req.method == 'POST' ) {
req.addListener('data', function(chunk)
{
POST = querystring.parse(chunk);
console.log(POST);
}
}
I end up with 5 sub-arrays, corresponding to the 5 parameters in the objects but with extra '][' characters in their names:
{ array:
[ { '][param1': 'something' }
, { '][param2': '123' }
, { '][param2': 'something' }
, { '][param4': '1234' }
, { '][param1': 'hello' }
]
}

There is a new node package that fixes this: "npm install qs".
https://github.com/ljharb/qs
"query string parser for node supporting nesting, as it was removed from 0.3.x, so this library provides the previous and commonly desired behaviour (and twice as fast)"
If anyone can tell me why it was removed from 0.3.x, I will give you an upvote for your comment. (I want my confidence in Node.js restored.)

To confirm my comment above, node's querystring.stringify function won't handle nested arrays (at the time of writing).
You can see the source of stringify at https://github.com/ry/node/blob/master/lib/querystring.js
Note that it handles one level of arrays but it doesn't recurse. When it finds an array it uses stringifyPrimitive to encode the array's values. You can see that stringifyPrimitive doesn't handle arrays, only number, boolean and string.
As I suggested in my comment, given that you're using a POST request a better idea would be to use JSON encoding for your complex data structure.
Or use https://github.com/visionmedia/node-querystring as suggested by #FriendlyDev

Don't use querystring.parse for recreating a JSON object sent as string. Use instead JSON.parse. And use JSON.stringify instead of querystring.stringify
querystring is useful when you send parameter encoded in the url or when you post a form. But there is no point in using it if you send just one JSON object with all your data.
In your scenario I would dismiss the querystring library and use JSON library, which is already included. It would be cleaner and faster.
http://www.json.org/js.html

Related

Why is the JSON output different from the actual output as rendered in the browser? [duplicate]

I have a problem with logging out the contents of an array inside an object. The actual object looks like this
var stuff = { accepted: [ 'item1', 'item2' ],
rejected: [],
response: 'Foo',
envelope: { from: 'The sender', to: ['new item1', 'new item2'] },
messageId: 'xxxxxxxxxxxxx' } }
The console.log shows the items of the first array fine but the second array is being output as [Object].
{ accepted: [ 'item1', 'item2' ],
rejected: [],
response: 'Foo',
envelope: { from: 'The sender', to: [Object] },
messageId: 'xxxxxxxxxxxxx' } }
What is happening here and how can I get the items of the second array to show when I console.log. Thanks for any help!
UPDATE
Sorry, I forgot to add that I am working exclusively in Node.js so it's a server side log that needs to display the object exactly as it's received from a callback with a straight console.log, ie. no further stringify-ing.
I also just tried creating another random object with a similar structure like this.
var objText = {
fun: {
stuff: 'Some stuff',
list: ['this', 'it', 'takes']
}
};
The console.log for the above is:
{ fun: { stuff: 'Some stuff', list: [ 'this', 'it', 'takes' ] } }
This appears to be the same structure to me and yet the console.log works fine so it seems to be perfectly possible in Node to log arrays content even when it's embedded inside and an object inside an outer object.
It looks like this is an old topic, anyway
I've faced the same issue, embedded array printed as [Array].
It is because of console.log in the node.js uses util.inspect for print, the default depth is 2.
So, to print something which is deeper than 2 followings can be used:
const util = require('util')
console.log(util.inspect(errors, true, 10))
This is the default way for some browser and implementations of showing too complex or deep objects/arrays with console.log. An alternative is to use JSON.stringify with console.log:
var stuff = {
accepted: ['item1', 'item2'],
rejected: [],
response: 'Foo',
envelope: {
from: 'The sender',
to: ['new item1', 'new item2']
},
messageId: 'xxxxxxxxxxxxx'
}
console.log(JSON.stringify(stuff, null, 4));
EDIT:
Another alternative is to use console.dir in case you have a too complex or recursive object, see https://stackoverflow.com/a/27534731/6051261
Try it with: console.log(JSON.stringify(variable))
If you like Chrome devtools, that folds your json objects and let you observe a lot of things, you can use the --inspect flag:
node --inspect index.js
The console will then give you an URL and you just have to copy paste in Google Chrome to enjoy Google Chrome console.
More information on this link

How can I mask json using json-masker for fields with "-" in it?

My requirement is to mask certain fields of a JSON while logging them.I am working on node.js. I have used json-masker library of node.js. While passing the JSON path of attributes with "-" in the name in the "whitelist" parameter, I am getting lexical error.
JSON
{
"attribute1":"value1",
"attribute2":"value2",
"attribute-name":"value3"
}
Code
const masker = require('json-masker');
const mask= masker({
whitelist: ['$.attribute1','$.attribute-name']
});
Error
Error Lexical error on line 1. Unrecognized text.
$.attribute-name
Also, is there a way to specify only the attributes that needs to be masked rather that specifying the ones that need not be masked(as specified in whitelist).
Please suggest if there is a better approach to do this using any other function/library.
Please note that I am receiving this JSON , so I cannot change the key name
The correct syntax is '$["attribute-name"]' instead of '$.attribute-name'
The $ fields are processed by jsonpath, a dependency of json-masker. This issue is discussed in one of their github issues (#90) and the solution presented there.
Use maskdata npm module: https://www.npmjs.com/package/maskdata
You can mask json fields containing '-' without any extra effort. Also, you can mask the nested fields too.
Example:
const MaskData = require('./maskdata');
const maskJSONOptions = {
// Character to mask the data. Default value is '*'
maskWith : "*",
// It should be an array
// Field names to mask. Can give multiple fields.
fields : ['level1.level2.level3.field3', 'level1.level2.field2', 'level1.field1', 'value1']
};
const nestedObject = {
level1: {
field1: "field1",
level2: {
field2: "field2",
level3: {
field3: "field3",
}
}
},
value1: "value"
};
const maskedObj = MaskData.maskJSONFields(nestedObject, defaultJSONMaskOptions2);
//Output : {"level1":{"field1":"******","level2":{"field2":"******","level3":{"field3":"******"}}},"value1":"*****"}

Specification for complex url query strings

I've been experimenting with Node.js's query parser and I am able to get it to parse some really complex queries into pretty deeply nested objects and arrays. Some examples:
Query Strings:
1) '?$or=foo&$or=bar'
2) '?$or[foo]=bar'
3) '?$or[0][foo]=bar&or[1][bar]=baz'
4)'?$or[0][foo]=bar&or[1][bar][]=baz&or[1][bar][]=bing&or[1][bar][]=bang'
Node parses them into:
1) {
$or: ['foo', 'bar']
}
2) {
$or: {
foo: 'bar'
}
}
3) {
$or: [
{ foo: 'bar' },
{ bar: 'baz' }
]
}
4) {
$or: [
{ foo: 'bar' },
{ bar: ['baz', 'bing', 'bang'] }
]
}
This is cool, but I can't find any docs about why this works. I've seen a couple links to this document, and it has this to say:
A host identified by an Internet Protocol literal address, version 6 [RFC3513] or later, is distinguished by enclosing the IP literal within square brackets ("[" and "]"). This is the only place where square bracket characters are allowed in the URI syntax.
Sounds like I'm not safe to use this syntax outside of Node at the very least, but why did Node decide it should work this way, and did they document this anywhere?
After some more digging I discovered in the express.js 4.x docs that express's query parser is based on the qs parser. I'm running express 3.x, whose documentation doesn't seem to mention qs or any advanced parsing capabilities, yet all of the qs documentation still seems to apply.

Node dbus-native

The documentation for node module dbus-native is weak, and the none of the many examples apply to my use case, which seems like it ought to be simple.
My problem is trying to make a simple method call into the "connman" network manager. I can make a request (invoking GetProperties method) just fine, and get a complex structure back and generally figure out how to get data out of it. But when I try to send something back with SetProperty, I just can't figure out how to get my data into the form it wants.
There's not really any documentation about how to translate to and from DBus's type system and node's. I gathered from source that variant types are arrays with signature and value, e.g., [ 's', 'string ], but no matter how I try to massage the data I'm trying to send to SetProperty, I get an "Invalid struct data" error.
The SetProperty method I'm trying to call has an argument signature "sv"--name, value. The particular property I'm trying to set has a value with is itself (as far as I can determine--there's precious little documentation on that side as well) an "a{sv}". that is,property is a hash of property names and values. My initial attempt as calling this function was:
var sysbus = require('dbus-native').systemBus();
sysbus.invoke({
path: '/net/connman/service/ethernet_1cba8cfa0e57_cable',
destination: 'net.connman',
'interface': 'net.connman.Service',
member: 'SetPropertry',
signature: 'sv',
body: [
'IPv4.Configuration', [
'a{sv}',
{ 'Method': [ 's', 'dhcp' ] }
]
],
type: dbus.messageType.methodCall
}, function (err, res) {
// etc...
});
This gives me the "Invalid struct data" error. I've tried man other ways to wrap the data in arrays, hashes, etc., and just can't seem to find the right answer. I'm trying to emulate this working Python code:
import dbus
bus = dbus.SystemBus()
service = dbus.Interface(bus.get_object("net.connman",
"/net/connman/service/ethernet_1cba8cfa0e57_cable",
"net.connman.Service");
conf = { "Method": make_string_variant("dhcp") }
service.SetProperty("IPv4.Configuration", conf);
Any ideas?
I recently started using dbus-native for connman and also had problems with all the nested arrays. I'll be improving the marshaling/unmarshaling in the fork of the project at https://github.com/Digisoft-tv/node-dbus - you might want to have a look. Hopefully my changes will be accepted up-stream.
Anyway, to get it working now, the format of the parameters you need to pass in is as below:
mgr.CreateSession([ [ 'key', [ 's', 'value' ] ], [ 'key2', [ 'i', 2 ] ] ], '/object/path', function(error, response) {
if (error) {
return console.error('SetProperty error:', error);
}
console.info('SetProperty response', response);
});
The outermost array is the "object" (ARRAY of dict entries).
Each nested array holds a key-value pair (DICT ENTRY). key at index 0 and value at index 1.
The values is a variant, which is always encoded as an array, with "signature" at index 0 and the actual value at index 1.
Hope it helps.
The correct way to encode 'a{sv}' is [ ['string', ['signature', 'value']], ... ]
Arrays ( a ) are encoded as normal JS arrays as well as structs / hashes ( () / {} ). I'll probably add shortcut to allow JS object to be used in place of a{sv} but at the moment input has to be quite verbose.
var sysbus = require('dbus-native').systemBus();
sysbus.invoke({
path: '/net/connman/service/ethernet_1cba8cfa0e57_cable',
destination: 'net.connman',
'interface': 'net.connman.Service',
member: 'SetPropertry',
signature: 'sv',
body: [
'IPv4.Configuration', [
'a{sv}',
[ // a
[ // {
'Method', [ 's', 'dhcp' ] // sv
]
]
]
],
type: dbus.messageType.methodCall
}, function (err, res) {
// etc...
});

Express.JS url parameters aren't parsed

I have the following route configured
app.put('/v1/users/:uid', function(req, res){
res.send(req.route);
});
When sending a PUT request to http://localhost:3000/v1/users/blablabla
I get the following output back
{
"path": "/v1/users/:uid",
"method": "put",
"callbacks": [
null
],
"keys": [
{
"name": "uid",
"optional": false
}
],
"regexp": {},
"params": []
}
As you see the params array seems to be empty instead of having the value "blablabla". But the "uid" key appears in keys, which I don't really know what to make of.
Would appreciate any suggestions.
OK, the trick is that Express uses a sparse array to parse the params.
When you pass it to req.send, the array is converted with JSON.stringify. Here's what happens in a JS shell:
> var params = [];
> params['uid'] = 1;
> params;
[ uid: 1 ]
> JSON.stringify(params);
'[]'
What's happening is that adding a non-numeric to an array does not change its length:
> params.length
0
So the new value is ignored by JSON.stringify.
Well this is the weirdest thing I've seen.
When doing a console.log(req.params) or console.log(req.route.params) I get an empty array response ([]).
But when doing a console.log(req.params.uid) I get the value! Thats extremely weird but hey, it works :)
Cheers.

Resources