I need a way to parse my JSONP object on server side to save it, due to cross domain origin issue I have shifted my way of communication from JSON to JSONP but not finding any suitable way to parse JSONP on server side to save it to the database.
Following is the Model,
define(['backbone'],function(Backbone){
'use strict';
return Backbone.Model.extend({
url:"http://crossdomain:9847/page",
defaults: {
type:'text',
position:0,
align:'left',
text:{"en":""},
color:"#000",
weight:'normal',
size:"14px",
font:"Verdana",
pageid:'askdkasdkgaskdgks'
},
idAttribute:'_id',
sync: function(method, collections, options) {
options.dataType = "jsonp";
return Backbone.sync(method, collections, options);
}
});
});
Express Server,
var express = require('/root/node_modules/express');
var page = require('./routes/page.js');
var app = express();
app.configure(function () {
app.use(express.json());
app.use(express.urlencoded());
app.set("jsonp callback", true);
})
app.get('/page', page.updatePage);
app.listen(9847);
exports.updatePage = function(req, res) {
console.log(req.query);
// Here how I can parse the req is my problem
// so I can save object to database?
}
URL is generating like,
http://crossdomain:9847/page?callback=jQuery203010156283635587138_1384408493698&{%22text%22:{%22en%22:%22Text%22},%22type%22:%22text%22,%22position%22:0,%22align%22:%22left%22,%22color%22:%22#000%22,%22weight%22:%22normal%22,%22size%22:%2214px%22,%22font%22:%22Verdana%22,%22pageid%22:%22askdkasdkgaskdgks%22}&_=1384408493700
and I am able to receive,
{ callback: 'jQuery203010156283635587138_1384408493698',
'{"text":{"en":"Text"},"type":"text","position":0,"align":"left","color":"': '' }
Now how can I parse this ? I can get callback from callback parameter, but how to get actual data ?
You can't parse the result because it's not valid JSON. Your problem is probably in this line:
app.set("jsonp callback", true);
This is where you set the JSONP callback variable, for example changing it from the default of callback to instead be callbackVariable.
Just comment out that line, and the JSONP you get back will hopefully be parseable. Or, you might also have to fix how Backbone is constructing the JSONP URL. If you instead used a URL like http://crossdomain:9847/page?callback=jQuery203010156283635587138_1384408493698&type=text&position=0&align=left&color=%23000&weight=normal&size=14px&font=Verdana&pageid=askdkasdkgaskdgks I believe it would work. Backbone seems to be adding additional encoding into the values in the URL, which makes parsing harder.
Finally, if you need help easily picking specific values out of a (valid) JSON string that has been parsed into a Javascript object, take a look at the many useful function in lodash.
Related
Нello! I am looking to call a function which has been passed to an expressRouter.post(...) call.
This expressRouter.post(...) call is occurring in a file which I am unable to modify. The code has already been distributed to many clients and there is no procedure for me to modify their versions of the file. While I have no ability to update this file for remote clients, other developers are able to. I therefore face the issue of this POST endpoint's behaviour changing in the future.
I am also dealing with performance concerns. This POST endpoint expects req.body to be a parsed JSON object, and that JSON object can be excessively large.
My goal is to write a GET endpoint which internally activates this POST endpoint. The GET endpoint will need to call the POST endpoint with a very large JSON value, which has had URL query params inserted into it. The GET's functionality should always mirror the POST's functionality, including if the POST's functionality is updated in the future. For this reason I cannot copy/paste the POST's logic. Note also that the JSON format will never change.
I understand that the issue of calling an expressjs endpoint internally has conventionally been solved by either 1) extracting the router function into an accessible scope, or 2) generating an HTTP request to localhost.
Unfortunately in my case neither of these options are viable:
I can't move the function into an accessible scope as I can't modify the source, nor can I copy-paste the function as the original version may change
Avoiding the HTTP request is a high priority due to performance considerations. The HTTP request will require serializing+deserializing an excessively large JSON body, re-visiting a number of authentication middlewares (which require waiting for further HTTP requests + database queries to complete), etc
Here is my (contrived) POST endpoint:
expressRouter.post('/my/post/endpoint', (req, res) => {
if (!req.body.hasOwnProperty('val'))
return res.status(400).send('Missing "val"');
return res.status(200).send(`Your val: ${req.body.val}`);
});
If I make a POST request to localhost:<port>/my/post/endpoint I get the expected error or response based on whether I included "val" in the JSON body.
Now, I want to have exactly the same functionality available, but via GET, and with "val" supplied in the URL instead of in any JSON body. I have attempted the following:
expressRouter.get('/my/get/endpoint/:val', (req, res) => {
// Make it seem as if "val" occurred inside the JSON body
let fakeReq = {
body: {
val: req.params.val
}
};
// Now call the POST endpoint
// Pass the fake request, and the real response
// This should enable the POST endpoint to write data to the
// response, and it will seem like THIS endpoint wrote to the
// response.
manuallyCallExpressEndpoint(expressRouter, 'POST', '/my/post/endpoint', fakeReq, res);
});
Unfortunately I don't know how to implement manuallyCallExpressEndpoint.
Is there a solution to this problem which excludes both extracting the function into an accessible scope, and generating an HTTP request?
This seems possible, but it may make more sense to modify req and pass it, rather than create a whole new fakeReq object. The thing which enables this looks to be the router.handle(req, res, next) function. I'm not sure this is the smartest way to go about this, but it will certainly avoid the large overhead of a separate http request!
app.get('/my/get/endpoint/:val', (req, res) => {
// Modify `req`, don't create a whole new `fakeReq`
req.body = {
val: req.params.val
};
manuallyCallExpressEndpoint(app, 'POST', '/my/post/endpoint', req, res);
});
let manuallyCallExpressEndpoint = (router, method, url, req, res) => {
req.method = method;
req.url = url;
router.handle(req, res, () => {});
};
How about a simple middleware?
function checkVal(req, res, next) {
const val = req.params.val || req.body.val
if (!val) {
return res.status(400).send('Missing "val"');
}
return res.status(200).send(`Your val: ${val}`);
}
app.get('/my/get/endpoint/:val', checkVal)
app.post('/my/post/endpoint', checkVal)
This code isn't tested but gives you rough idea on how you can have the same code run in both places.
The checkVal function serves as a Express handler, with request, response and next. It checks for params first then the body.
Introduction
I have running Azure Function, which is written as Expressjs application.
I have simple routes there to test
// Test get
router.get('/test', (req, res) => {
return res.status(200).send({ result: req.query });
});
// Test post
router.post('/test', (req, res) => {
return res.status(200).send({ result: req.body });
});
The app configuration is the following
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
app.use(cors());
Issue
I'm trying to do POST /test request.
With Content-Type: application/javascript it works well
With Content-Type: application/json it calls function, but nothing executes, and timeout is returned.
There is no issue with GET /test route.
There is no issue running locally.
Need help to understand why Azure Function call doesn't work properly for POST requests with Content-Type: application/json. Thank you
use :
app.use(bodyParser.json({ type: 'application/*+json' }));
to allow custom json types
Going deeper how Azure Function and body parser work, I figured out the root of the problem.
Be careful with request object
In regular expressjs app we have request object, which is actually instance of Stream object, and all incoming data goes through this streaming.
So, we have raw data, and we need to parse it right way. bodyParser middleware has several methods to parse this data from stream based on it's type (Content-Type).
For example bodyParser.json() method tries to parse data in JSON format with application/json content type. Once this function parses data req.body is fulfilled with JSON object.
Azure Functions don't support streaming
This is where everything breaks :) Yes, it doesn't support streaming. And since this is true data is accepted different way. So request object is no longer Stream object. And req.body is already fulfilled with incoming data. For application/json it contains JSON object, for other content types it contains raw data which is needed to be parsed. As example I can share my simple middleware code for parsing data with type x-www-form-urlencoded.
'use strict';
// https://github.com/sindresorhus/query-string
const queryString = require('query-string');
/**
* Azure body parser
*/
function azureBodyParser() {
return function(req, res, next) {
// x-www-form-urlencoded
if (req.headers['content-type'] === 'application/x-www-form-urlencoded') {
req.body = queryString.parse(req.body, { arrayFormat: 'bracket' });
}
next();
};
}
module.exports = azureBodyParser;
Then use it like this
const app = require('express')();
const azureBodyParser = require('./middlewares/azureBodyParser');
// ...
app.use(azureBodyParser());
You can change it and add more handlers for other content types.
Also we need to provide some condition whether we run our app on server or as Azure Function, it's possible using ENV variables for example. It's your home task :)
The problem
Since Azure Functions don't support streaming, bodyParser.json() tries to get data from stream, and this is a point where app stuck. As a result function execution ends up with timeout.
Hope it will be helpful for all who struggling with Azure... Good luck :)
Might be late to this thread . Had a similar issue migrating an express app to Azure Functions and had to consume x-form-urlencoded values .
Used
const qs = require('qs');
While evaluating the request checked if the requst has rawBody and the relevant headers
if(req.rawBody)
and then simply
parsedBody = qs.parse(req.rawBody);}
Hope this helps someone.
It is a pain to host an Express API as is due to the discontinued support in Azure Functions for express
I am developping a RESTful server in node.js, using Express as framework, and Winston, for the moment, as logger module.
This server will handle a big amount of simultaneous request, and it would be very useful to me to be able to track the log entries for each specific request, using something like a 'request ID'. The straight solution is just to add this ID as another piece of logging information each time I want to make a log entry, but it will mean to pass the 'request ID' to each method used by the server.
I would like to know if there is any node.js/javascript module or technique that would allow me to do this in an easier way, without carrying around the request ID for each specific request.
If you auto-increment, your later log analytics won't be able to uniquely identify requests, because different instances will generate colliding IDs, and restarting the app will automatically cause ID collisions.
Here's another possible solution.
Install cuid:
npm install --save cuid
Then in your main app file:
var cuid = require('cuid');
var requestId = function requestId(req, res, next) {
req.requestId = cuid();
next();
};
// Then, at the top of your middleware:
app.use(requestId);
Now you'll get a friendly request ID that is unlikely to collide, and you'll be able to uniquely identify your requests for your log analytics and debugging, even across multiple instances, and server restarts.
You can use req object that does comes with every request in express.
So the first route you would do in your application would be:
var logIdIterator = 0;
app.all('*', function(req, res, next) {
req.log = {
id: ++logIdIterator
}
return next();
});
And then anywhere within express, you can access that id in req object: req.log.id;
You will still need to pass some data into functions that do want to create some logs. In fact you might have logging function within req.log object, so that way it will be guaranteed that logging will happen only when there is access to req.log object.
I was struggling search for a solution for this problem.
The thing I didn't like it about solutions suggested here was that they imply to share the req object among all the functions along the project.
I found out a solution mixing your approach (creating an uuid per request) and with a library (continuation-local-storage) that allows sharing namespaces among modules.
You can find the explanation in this other answer: https://stackoverflow.com/a/47261545/5710581
If you want more info, I wrote down all these ideas and all the code in a post, in order to explain everything in one place:
Express.js: Logging info with global unique request ID – Node.js
You shouldn't be using Global Variables.
What I like to do is to populate a META object before each request.
I use a UUID generator (https://github.com/kelektiv/node-uuid) to ID a request
Here's an example
app.all('*', function(req, res, next) {
req.meta = {
ip: req.headers['x-forwarded-for'] || req.connection.remoteAddress,
timestamp: uuid(),
user_agent: req.headers['user-agent'],
body: req.body,
}
return next();
})
As mentioned by #moka , Using the request ID in each request is the crux of solving the problem. Another way of abstracting all these is by making use of http-context and uuid
So set a UUID in the httpContext before all your middlewares (set as an application middleware and not as a router middlware). now you can get the uuid anywhere in your code and log it.
Here is a sample implementation I have used
You can get the complete reference here uuid in request
const uuid = require('node-uuid');
const httpContext = require('express-http-context');
....
this.expressApp.use(httpContext.middleware);
this.expressApp.use((req, res, next) => {
httpContext.set('reqId', uuid.v4());
next();
});
Now I have used the reqId set here in my custom pino logger'
public infoLogService (fileName): pino.Logger {
return pino({
level: 'info',
name: this.appService.getApp_name(),
messageKey: 'XXX-Logs',
base: {pid: process.pid, hostname: os.hostname,
timestamp: this.getTimeStamp(),
appName: this.appService.getApp_name(),
fileName: fileName,
request_id: **isNullOrUndefined(httpContext.get('reqId'))** ? 'Not an actual request ' : httpContext.get('reqId')
},
enabled: true,
useLevelLabels: true,
});
}
If the reqId is null it means that the loggers have been inserted in code that is used before starting the express App. Hope you can use this as an alternate solution
I have two situations to get data from DB
To show normal data
http://exampleapp.com/task/{{taskId}}
To edit data via posting
http://exampleapp.com/task/{{taskId}}/?state={{app.state}}
Both url have the same http://exampleapp.com/task/{{taskId}} just a little bit different with last phrase ?state={{app.state}}
I use Express routing as followed:
app.get('/task/:taskId/(?state=:status(pending|cancel|confirmed|deleted))?', routes.task.show);
But I dont know why it does not work ?
For example error: Cannot GET /task/51d2c53f329b8e0000000001 when going to h**p://exampleapp.com/task/51d2c53f329b8e0000000001
Query strings cannot be defined in routes. You access query string parameters from req.query.
app.get('/task/:taskId', function(req, res) {
if (req.query.state == 'pending') { ... }
});
However, if you're modifying a task, this is not the appropriate way to do it. GET requests SHOULD be idempotent: the request SHOULD NOT modify state. That's what POST requests are for.
app.get('/task/:taskId', function(req, res) {
// show task info based on `req.params.taskId`
});
app.post('/task/:taskId', function(req, res) {
// set task `req.params.taskId` to state `req.body.state`
});
You could either have a <form> that posts to the task, or make an ajax request:
$.post('/task/1', { state: 'pending' }, function() { ... });
According to the Express API, you cannot mix RegExp routes with string routes.
You should do something like this (I'm assuming taskId is an integer):
app.get(/^\/task/([0-9]+)/(?state=:status(pending|cancel|confirmed|deleted))?, routes.task.show);
However, I don't see why you cannot only check if req.query.state is defined in your route. It's probably less error prone and easier:
app.get("/task/:taskId", function( req, res, next ) {
if (req.query.state) {
// Do things
}
next();
});
Your problem is that query strings are not considered in routing. You will either have to redesign your urls (ie, include the state into the url itself, instead of the query string) or check the query string in your route handler function.
I'm using Backbone.js on the client and node.js on the backend, and I'm having a bit of trouble doing a 'limited' model save, as explained here : http://backbonejs.org/#Model-save
As in the example, if I do
book.save({author: "Teddy"});
how do I access the parameters of the save using express.js, i.e. the fact that I only want to save to the 'author' field? I've tried the following
req.body -> gives ALL parameters of the model being saved, I want only the 'author' field
req.query -> empty
Any help much appreciated!
As stated in Model.save documentation:
save model.save([attributes], [options])
[...] The attributes hash (as in set) should contain the attributes you'd like to change —
keys that aren't mentioned won't be altered — but, a complete
representation of the resource will be sent to the server.
You can however override the save method and provide a data attribute via the options which will be sent to the server instead of the full model representation. For example, this will only save the attributes passed to the method :
var M = Backbone.Model.extend({
save: function (attrs, options) {
options || (options = {});
options.contentType = 'application/json';
options.data = JSON.stringify(attrs);
Backbone.Model.prototype.save.call(this, attrs, options);
}
});
And a Fiddle http://jsfiddle.net/dLFgD/
As #mikebridge noted in the comments, this behavior can now be obtained by passing an attrs option. So either use
book.save(null, {
attrs: {author: "Teddy"}
});
or keep the override
var M = Backbone.Model.extend({
url: '/echo/json/',
save: function(attrs, options) {
options || (options = {});
options.attrs = attrs;
Backbone.Model.prototype.save.call(this, attrs, options);
}
});
http://jsfiddle.net/nikoshr/dLFgD/7/
You could also send a PATCH request if you're using a Backbone version that supports it (>=0.9.9) and your server understands that verb, as explained in #pkyeck's answer
with the current version of Backbone (1.1.0) you could also do a PATCH which only sends the changed attributes to the server.
If instead, you'd only like the changed attributes to be sent to the server, call model.save(attrs, {patch: true}). You'll get an HTTP PATCH request to the server with just the passed-in attributes.
taken from here: http://backbonejs.org/#Model-save
in your case this would be:
book.save("author", "Teddy", {patch: true});
The best way I have found to access POST parameters is to use the bodyParser middleware. This is referenced at this question: How to retrieve POST query parameters?
/* On startup, register the bodyParser middleware */
app.use(express.bodyParser());
/* Then, during route execution, each parameter can be accessed with req.param */
app.post('/author/:id', function(req, res) {
var author = req.param('author', null); // Second parameter is default.
...
});