I am trying to send an http response in node to print results in the browser. The simplified source code is down below. Basically, all the variables are defined somewhere in the program, so that shouldn't be problem. When I try to run the script, I keep getting the error:
http.js:783
throw new TypeError('first argument must be a string or Buffer');
TypeError: first argument must be a string or Buffer
So can someone familiar with node.js or javascript syntax let me know what the problem is?
upload = function(req, res) {
var fileInfos = [obj, obj]; //defined as an array of objects
var counter = 0;
counter -= 1;
if (!counter) {
res.end({files: fileInfos}); //files is defined.
}
};
async.forEach(urls, downloadFile, function (err) { //all params defined.
if(err){
console.error("err");
throw err;
}
else{
http.createServer(function(req, res){
upload(req1, res); //req1 defined as an array of objects.
}).listen(3000, "127.0.0.1");
console.log('Server running at http://127.0.0.1:3000/');
}
});
This error is often caused by an attempt to call response.write with the wrong type of parameter. Looking at the documentation it suggests:
response.end([data], [encoding])#
This method signals to the server that all of the response headers and body have been sent; that server should consider this message complete. The method, response.end(), MUST be called on each response.
If data is specified, it is equivalent to calling response.write(data, encoding) followed by response.end().
Now response.write( chunk, encoding ) expects the chunk to be a string, so it seems possible that when you are calling res.end({files: fileInfos}) it is unable to write the content of that object as a string.
You can use JSON.stringify() to convert the JavaScript object to string before sending it to the client.
res.end(JSON.stringify({files: fileInfos}));
Related
Ok so i am using a method to make a request and pull some tables from another URL
Meteor.methods({
gimmetitle: function () {
var url = 'http://wiki.warthunder.com/index.php?title=B-17G_Flying_Fortress';
request(url, function(err, response, body) {
$ = cheerio.load(body);
var text = $('.flight-parameters td').text();
console.log(text);
return text;
});
}
});
When called the td's in the table succesfully print to the server console: http://prntscr.com/721pjh
Buuut, when that text is returned from that method to this client code, undefined is printed to the console:
Template.title.events({
'click #thebutton': function () {
Meteor.call('gimmetitle', function(error, result){
Session.set('gogle', result);
});
var avar = Session.get('gogle');
console.log(avar);
}
});
Ideas?
You need to understand two different things here :
On the client side, making some calls to the server is always asynchronous, because we have to deal with network latency. That's why we use callbacks to fetch the result of Meteor methods : this code is executed some time in the future, not right away.
This is why Session.set('gogle', result); is actually executed AFTER var avar = Session.get('gogle'); even though it appears before in your event handler code flow.
Contrary to template helpers, event handlers are NOT reactive, so it means that when you set the Session variable to the result of the method, the event handler code is not automatically reexecuted with the new value of Session.get('gogle').
You'll need to either do something with the result right in the Meteor method callback, or use a reactive computation (template helpers or Tracker.autorun) depending on Session.get('gogle') to rerun whenever the reactive data source is modified, and use the new value fetched from the server and assigned to the Session variable.
Quick update..Was able to fix this with just 1 line of code lol.
instead of request(url, function(err, response, body) i used the froatsnook:request package and used var result = request.getSync(url, {encoding: null}); and then just replaced $ = cheerio.load(body); with $ = cheerio.load(result.body);.
I have got stream and I need to get stream content into string. I stream from internet using http.get. I also write stream into file, but I don't want to write file and after that open the same file and read from it...
So I need to convert stream into string
Thanks for all advices...
var http = require('http');
var string = '';
var request = http.get("http://www.google.cz", function(response) {
response.on('data', function(response){
string += response.toString();
});
response.on('end', function(string){
console.log(string);
});
});
This works for sure. I am using it.
Instead of using the second nested function, try the toString() method on the stream. This can then be piped wherever you want it to go, and you can also write a through() method that can assign it to a variable or use it directly in the one function.
var http = require('http');
var string = '';
var request = http.get('http://www.google.cz', function (err, data){
if (err) console.log(err);
string = data.toString();
//any other code you may want
});
//anything else
One last note--the http.get() method takes two parameters: the url, and a callback. This requires two parameters, and you may have been getting nothing because it was an empty error message.
I need to allow the user of my app to download a file with Meteor. Currently what I do is when the user requests to download a file I enter into a "fileRequests" collection in Mongo a document with the file location and a timestamp of the request and return the ID of the newly created request. When the client gets the new ID it imediately goes to mydomain.com/uploads/:id. I then use something like this to intercept the request before Meteor does:
var connect = Npm.require("connect");
var Fiber = Npm.require("fibers");
var path = Npm.require('path');
var fs = Npm.require("fs");
var mime = Npm.require("mime");
__meteor_bootstrap__.app
.use(connect.query())
.use(connect.bodyParser()) //I add this for file-uploading
.use(function (req, res, next) {
Fiber(function() {
if(req.method == "GET") {
// get the id here, and stream the file using fs.createReadStream();
}
next();
}).run();
});
I check to make sure the file request was made less than 5 seconds ago, and I immediately delete the request document after I've queried it.
This works, and is secure(enough) I think. No one can make a request without being logged in and 5 seconds is a pretty small window for someone to be able to highjack the created request URL but I just don't feel right with my solution. It feels dirty!
So I attempted to use Meteor-Router to accomplish the same thing. That way I can check if they're logged in correctly without doing the 5 second open to the world trickery.
So here's the code I wrote for that:
Meteor.Router.add('/uploads/:id', function(id) {
var path = Npm.require('path');
var fs = Npm.require("fs");
var mime = Npm.require("mime");
var res = this.response;
var file = FileSystem.findOne({ _id: id });
if(typeof file !== "undefined") {
var filename = path.basename(file.filePath);
var filePath = '/var/MeteorDMS/uploads/' + filename;
var stat = fs.statSync(filePath);
res.setHeader('Content-Disposition', 'attachment; filename=' + filename);
res.setHeader('Content-Type', mime.lookup(filePath));
res.setHeader('Content-Length', stat.size);
var filestream = fs.createReadStream(filePath);
filestream.pipe(res);
return;
}
});
This looks great, fits right in with the rest of the code and is easy to read, no hacking involved, BUT! It doesn't work! The browser spins and spins and never quite knows what to do. I have ZERO error messages coming up. I can keep using the app on other tabs. I don't know what it's doing, it never stops "loading". If I restart the server, I get a 0 byte file with all the correct headers, but I don't get the data.
Any help is greatly appreciated!!
EDIT:
After digging around a bit more, I noticed that trying to turn the response object into a JSON object results in a circular structure error.
Now the interesting thing about this is that when I listen to the filestream for the "data" event, and attempt to stringify the response object I don't get that error. But if I attempt to do the same thing in my first solution(listen to "data" and stringify the response) I get the error again.
So using the Meteor-Router solution something is happening to the response object. I also noticed that on the "data" event response.finished is flagged as true.
filestream.on('data', function(data) {
fs.writeFile('/var/MeteorDMS/afterData', JSON.stringify(res));
});
The Meteor router installs a middleware to do the routing. All Connect middleware either MUST call next() (exactly once) to indicate that the response is not yet settled or MUST settle the response by calling res.end() or by piping to the response. It is not allowed to do both.
I studied the source code of the middleware (see below). We see that we can return false to tell the middleware to call next(). This means we declare that this route did not settle the response and we would like to let other middleware do their work.
Or we can return a template name, a text, an array [status, text] or an array [status, headers, text], and the middleware will settle the response on our behalf by calling res.end() using the data we returned.
However, by piping to the response, we already settled the response. The Meteor router should not call next() nor res.end().
We solved the problem by forking the Meteor router and making a small change. We replaced the else in line 87 (after if (output === false)) by:
else if (typeof(output)!="undefined") {
See the commit with sha 8d8fc23d9c in my fork.
This way return; in the route method will tell the router to do nothing. Of course you already settled the response by piping to it.
Source code of the middleware as in the commit with sha f910a090ae:
// hook up the serving
__meteor_bootstrap__.app
.use(connect.query()) // <- XXX: we can probably assume accounts did this
.use(this._config.requestParser(this._config.bodyParser))
.use(function(req, res, next) {
// need to wrap in a fiber in case they do something async
// (e.g. in the database)
if(typeof(Fiber)=="undefined") Fiber = Npm.require('fibers');
Fiber(function() {
var output = Meteor.Router.match(req, res);
if (output === false) {
return next();
} else {
// parse out the various type of response we can have
// array can be
// [content], [status, content], [status, headers, content]
if (_.isArray(output)) {
// copy the array so we aren't actually modifying it!
output = output.slice(0);
if (output.length === 3) {
var headers = output.splice(1, 1)[0];
_.each(headers, function(value, key) {
res.setHeader(key, value);
});
}
if (output.length === 2) {
res.statusCode = output.shift();
}
output = output[0];
}
if (_.isNumber(output)) {
res.statusCode = output;
output = '';
}
return res.end(output);
}
}).run();
});
I am writing a node js application. I am using request and cheerio to load a set of URLs and get a bunch of information for the site, now let's assume all I am trying to get is the title:
var urls = {"url_1", "url_2", "url_3",...,"url_n"};
for(var i=0; i<urls.length; i++)
{
getDOMTitle(urls[i],function(error,title){
if(error)
console.log("Error while getting title for " + urls[i]);
else
console.log("The title for " + urls[i] + " is " + title);
});
}
This is how my getDOMTitle method looks:
function getDOMTitle(urlReq,callback)
{
var request = require('request');
var cheerio = require('cheerio');
request({url:urlReq},function(error, response, doc){
var $ = cheerio.load(doc);
if(error)
{
callback(true,null);
}
else
{
$('title', 'head').each(function (i, elem) {
var title = $(this).text();
callback(false,title);
});
}
}
}
In the case where the module throws an uncaught exception, how do I handle that situation?
I have tried adding the following:
process.on('uncaughtException', function (err) {
console.error(err);
console.log("Node NOT Exiting...");
callback(true,null);
});
When I do that, I get an error saying I cannot set the headers once they have been sent. If I remove the callback from the process error handling, I do not see that error but the client spins for a long time because I assume we are never calling the callback.
How can I solve this?
Also, I have read somewhere that you can catch uncaught exceptions at the application level so you don't have to replicate the code to catch it in every method, is that possible? and if it is and the method that threw the exception is expected to callback with some information, how can that be achieved?
Thank you,
To answer your stated question, using an uncaught exception handler as a general error-trapping mechanism is commonly regarded as poor design. It's a false economy to use it to handle anything other than non-recoverable situations where you just need to do some cleanup before exiting.
You've got some problems in your example code. In your for loop, all the callbacks are going to report that they were working with the very last URL in your array because they're all referring to the same copy of i, which will be at its highest value by the time any of them execute. You need to use a helper function or an immediate function invocation to give each callback a private copy of i.
In getDOMTitle the error callback should be callback(error) and the code in your loop should include the returned value in the error message. The success callback should use null as its first parameter, though this is just a matter of convention.
When creating a POST request with valid JSON, the bodyParser parses the body of the POST request correctly. However, if I submit an invalid JSON string as the body, I receive the error:
SyntaxError: Unexpected token ILLEGAL
at parse (native)
at IncomingMessage.<anonymous>(/home/.../middleware/bodyParser.js:69:15)
...
So, it appears that the body parser is failing during the parsing of the body. However, I would like to catch this failure and return an error. I'm unsure what I can do to catch it, so any help would be appreciated. Thanks.
This is in the connect.js bodyParser middleware. It DOES do a try/catch and then calls next(err). You should be able to catch this error and handle with additional custom code using the app.error() callback hook. http://expressjs.com/guide.html#error-handling
For some reason, when using express/connect, JSON.parse doesn't throw exceptions, which is why your error handler doesn't fire.
I've logged an issue with express to find out what's going on, but in the meantime you can use this workaround:
express.bodyParser.parse['application/json'] = function(data) {
var result = JSON.parse(data)
if (typeof result != 'object') {
throw new Error('Problems parsing JSON')
}
return result;
}
app.use(express.bodyParser());
update: this issue is not familiar to the author of express, so I am wondering if it's another library causing it. Will have to dismantle my code piece by piece to figure out where this behaviour is being introduced.
Try to put your
app.use(express.bodyParser());
after
app.use(express.errorHandler(...))
solved it for me.
You may also adapt the following code to manage the error
express.bodyParser.parse['application/json'] = function(req, options, fn){
var buf = '';
req.setEncoding('utf8');
req.on('data', function(chunk){ buf += chunk });
req.on('end', function(){
try {
req.body = buf.length
? JSON.parse(buf)
: {};
fn();
} catch (err){
fn(new Error('Problems parsing JSON'));
}
});
};
bodyParser must be above app.use(app.router), it doesn't matter relative location to error handler as Perki