I've tried to download this image a million ways and I always get an undefined value in NodeJS, can someone please help me!! The image does download, but I cannot do anything with it, so it essentailly becomes useless!
var rawImg = urlHtml('img').attr('src'); // got the image needed!
var download = function (rawImg, filename, callback) {
request.head(rawImg, function (err, res, body) {
console.log('content-type:', res.headers['content-type']);
console.log('content-length:', res.headers['content-length']);
request(rawImg).pipe(fs.createWriteStream(__dirname + '/public/img/test.jpg')).on('close', function(callback) {
console.log(callback); // always getting undefined
});
});
};
download(rawImg, function () {
console.log('done'); // doesn't even launch because gets undefined before
});
Below is some sample code that downloads and saves an image to your hard drive. Here are some of the reasons why it is different from your code:
I am not sure what function urlHtml is, but my guess is that the rawImg variable is actually just the src attribute on an img html element. If that is the case, then rawImg is actually just a string with the image's URL in it. This is why, in my example code, I have renamed that variable in my code to be url.
Your function download expects three parameters (rawImg, filename, and callback), but you only call the function with two parameters (rawImg and a callback function). Since no filename is provided, your download function thinks that the function you intended to be a callback function is actually the filename. And since there is no third argument, callback is undefined.
Using the request.head function will only retrieve the HTTP headers without the body. So you won't actually get the image bytes with an HTTP HEAD request. See [https://ochronus.com/http-head-request-good-uses/ this article] for more details on HTTP HEAD. Since we are interested in downloading the image itself, we should perform an HTTP GET insted of an HTTP Head. This is why my sample code calls request.get.
.on('close', function(callback) {...}) is incorrect because the on-close function will not be called with any parameters when the close event is triggered. So callback is undefined because it is not passed into the function when the close event fires. This is why, in my example, there is no callback parameter for the on-close function.
Note 1: My example code adds some require calls for completeness, since this question is tagged as a nodejs question. It looks like you already have the fs and request variables in scope, so you may not need to require them again.
Note 2: If you are getting the URL from your urlHtml('img').attr('src'); call, then replace the hardcoded url in my code with urlHtml('img').attr('src');.
Working, example code (tested and all!!):
var fs = require('fs')
var request = require('request')
var url = 'https://avatars0.githubusercontent.com/u/5980429?v=2&s=200'
var download = function (url, filename, callback) {
request.get(url).pipe(fs.createWriteStream(__dirname + '/' + filename)).on('close', function() {
callback()
})
};
download(url, 'image.png', function () {
console.log('Image downloaded successfully. It is now saved in a file called "image.png" in the current working directory!');
});
request(rawImg).pipe(
fs.createWriteStream(__dirname + '/public/img/test.jpg')).on('close',function(callback) {
console.log(callback); // always getting undefined
});
Replace the bolded line with callback();
The 'undefined' value you are seeing is the return value of console.log.
For anyone wondering, I figured out what my problem was. I was trying to download what was in a nested object. Ended up using underscore.js !!
Related
For a school project I'm creating a portal for KVM using NodeJS and Express.
I need to adjust an XML file and then use that XML File to create an VM.
So i created 2 functions
CreateXML:
function createXML(req, res, next) {
var parses = new xml2js.Parser();
fs.readFile('Debian7.xml', function(err, data){
parser.parseString(data, function (err, result){
result.domain.name = req.body.name;
result.domain.memory[0]['$'].unit = "GB";
result.domain.memory[0]['_'] = req.body.ram;
result.domain.currentMemory[0]['$'].unit = "GB";
result.domain.currentMemory[0]['_'] = req.body.ram;
result.domain.vcpu = req.body.cpus;
var builder = new xml2js.Builder({headless: true});
var xml = builder.buildObject(result);
fs.writeFile('./xmlfiles/' + req.body.name + '.xml', xml, function(err, data){
if(err) console.log(err);
});
});
});
};
CreateDomain:
function createDomain(req, res){
var domainXML = fs.readFileSync('./xmlfiles/' + req.body.name + '.xml', 'utf8');
hypervisor.connect(function(){
hypervisor.createDomainAsync(domainXML).then(function (domain){
console.log('Domain Created');
res.json({success: true, msg: 'succesfully created domain'})
});
});
}
then I call these functions as middleware in my post request
apiRoutes.post('/domainCreate', createXML, createDomain);
But then when I use Postman on the api route I get the following error:
Error: ENOENT: no such file or directory, open './xmlfiles/rickyderouter23.xml'
After the error it still creates the XML file and when I create the XML file before I use postman it works fine. It's like it needs to execute both functions before the creation of the XML file, how do I create the XML file after the first function and then use it in the second function.
The answer is "it's asynchronous" (just like many, many problems in node.js/javascript).
The fs.readFile function is asynchronous: when you call it, you give it a callback function which it will call when it finishes loading the file.
The parser.parseString is asynchronous - it will call your callback function when it finishes parsing the XML.
The fs.writeFile is the same - it will call your callback function when it finishes writing the file.
The hypervisor.connect function is the same - it will call your callback function when it finishes connecting.
The middleware functions are called in order, but they both contain code that may not have completed before they return. So when your code calls createDomain and tries to read the XML file created in createXML, the XML file probably doesn't exist yet. The fs.readFile might not be finished yet; even if it is, the parser.parseString function might not be finished yet; even if that one is finished, the fs.writeFile might not be finished yet.
One way to solve this would be to put the functionality of the createXML and createDomain functions together into one middleware function. That would allow you to rewrite it so that all the function calls that depend on previous asynchronous function calls could actually wait for those calls to complete before executing. A simple way to do it would be this:
function createXML(req, res, next) {
var parses = new xml2js.Parser();
fs.readFile('Debian7.xml', function(err, data){
parser.parseString(data, function (err, result){
result.domain.name = req.body.name;
result.domain.memory[0]['$'].unit = "GB";
result.domain.memory[0]['_'] = req.body.ram;
result.domain.currentMemory[0]['$'].unit = "GB";
result.domain.currentMemory[0]['_'] = req.body.ram;
result.domain.vcpu = req.body.cpus;
var builder = new xml2js.Builder({headless: true});
var xml = builder.buildObject(result);
fs.writeFile('./xmlfiles/' + req.body.name + '.xml', xml, function(err, data){
if(err) console.log(err);
// notice the call to createDomain here - this ensure
// that the connection to the hypervisor is not started
// until the file is written
createDomain(req, res);
});
});
});
};
And change your route to:
apiRoutes.post('/domainCreate', createXML);
Now, that's pretty ugly. I don't like the idea of lumping those two middleware functions into one and I'd prefer to rewrite it to use a promise-based approach, but that's the basic the idea.
I'm using pkgcloud to build a webservice (express based) that will manage my openstack storage. I have a problem with downloading a file.
the pkgcloud api for download is a bit wierd: it passes to the callback the error and file's metadata, and returns from the function a readable stream with the file's content:
client.download({
container: 'my-container',
remote: 'my-file'
}, function(err, result) {
// handle the download result
})).pipe(myFile);
The problem I have with this is- how can I get the error, the metadata and the stream together? I need the metadata in order to set the content type for my result and obviously I need the error to know if all went well (in case the file doesn't exist, the stream is still returned- just containing an html error message, so I can't count on that for error).
I tried calling it like this:
router.get("/download", function (res, req) {
var stream = client.download(options, function (err, file) {
if (err) {
return res.status(err.status).json({'message' : 'Error downloading file from storage: ' + err.details});
}
}
// I'd like to get the callback's err and file here somehow
stream.pipe(res);
});
but the problem is that the data is piped to res before callback is called, so the callback doesn't help- it is either giving me the error too late, or not giving me the metadata. I also tried moving the stream.pipe(res) into the callback, but it does not work, cause inside the callback I can't access the result stream.
I thought of using promise for that, but how can I tell the promise to 'wait' for both the callback and the returned value? I'm trying to wrap the download function with a function that will run the download with one promise in the callback and one on the returned stream, but how can I do that?
Ok so i am using a method to make a request and pull some tables from another URL
Meteor.methods({
gimmetitle: function () {
var url = 'http://wiki.warthunder.com/index.php?title=B-17G_Flying_Fortress';
request(url, function(err, response, body) {
$ = cheerio.load(body);
var text = $('.flight-parameters td').text();
console.log(text);
return text;
});
}
});
When called the td's in the table succesfully print to the server console: http://prntscr.com/721pjh
Buuut, when that text is returned from that method to this client code, undefined is printed to the console:
Template.title.events({
'click #thebutton': function () {
Meteor.call('gimmetitle', function(error, result){
Session.set('gogle', result);
});
var avar = Session.get('gogle');
console.log(avar);
}
});
Ideas?
You need to understand two different things here :
On the client side, making some calls to the server is always asynchronous, because we have to deal with network latency. That's why we use callbacks to fetch the result of Meteor methods : this code is executed some time in the future, not right away.
This is why Session.set('gogle', result); is actually executed AFTER var avar = Session.get('gogle'); even though it appears before in your event handler code flow.
Contrary to template helpers, event handlers are NOT reactive, so it means that when you set the Session variable to the result of the method, the event handler code is not automatically reexecuted with the new value of Session.get('gogle').
You'll need to either do something with the result right in the Meteor method callback, or use a reactive computation (template helpers or Tracker.autorun) depending on Session.get('gogle') to rerun whenever the reactive data source is modified, and use the new value fetched from the server and assigned to the Session variable.
Quick update..Was able to fix this with just 1 line of code lol.
instead of request(url, function(err, response, body) i used the froatsnook:request package and used var result = request.getSync(url, {encoding: null}); and then just replaced $ = cheerio.load(body); with $ = cheerio.load(result.body);.
var userLat = db.collection('users', function (err, document){
document.findOne({_id: loggedUserID}, function(err, docs) {
console.log(docs.currentUserLat);
})
});
This is my code, I'm trying to get the value that's console logged into the variable. I just can't find the correct syntax to do this. The console log does return the correct value just need to drop it into the variable. Grateful for some help.
What do you want to do with 'docs.currentUserLat'?
You can do what you need to do without saving docs.currentUserLat to a variable that has scope outside of your db.collection call. Some examples:
If you simply want to change the document in your database, take advantage of the many methods specified in the Collections API: http://mongodb.github.io/node-mongodb-native/2.0/api/Collection.html. For example, to update the document and simultaneously resave it in the database:
db.collection('users', function (err, document){
document.findOneAndUpdate({_id: loggedUserID},
{currentUserLat: [updated value]},
function(err, docs) {
if(err) console.log(err);
}
)
});
If you just wanted to use docs.currentUserLat inside some node function, you'll need to properly nest the document.findOne function inside a callback (or vice versa). For example, to write currentUserLat to a file using the fs module:
var fs = require('fs');
db.collection('users', function (err, document){
document.findOne({_id: loggedUserID}, function(err, docs) {
fs.writeFile("pathToYourFile", docs.currentUserLat, function(err) {
if(err) {return console.log(err);}
});
});
});
Or, if you want to send it in response to a simple http request:
var http = require('http');
http.createServer(function(request,response){
db.collection('users', function (err, document){
document.findOne({_id: loggedUserID}, function(err, docs) {
response.writeHead(200,{'Content-Type':'text/html'});
response.end(docs.currentUserLat);
});
});
});
The key thing to remember is what JohnnyHK said in their comment: docs.currentUserLat is only available inside the anonymous function passed to findOne. So, whatever it is that you need to do, do it inside this function.
(Reading the link JohnnyHK provided is a great way to get started with understanding asynchronous functions in Node. Another is https://github.com/rvagg/learnyounode)
First of all you have to understand how javascript callback works. After that you will see that nothing assigns docs.currentUserLat to your userLat variable. The reason behind this is that your docs.currentUserLat is available only inside the callback. Think about it in the following way:
You program started to execute and encountered the line: var userLat = .... This line tells: do a callback (which basically asks someone else to do the job), your while your job is being executed the program continues, by assigning userLat to undefined and executes further. Then at some period of time callback finishes and console.log your docs.currentUserLat.
One way to have the desired behavior is to make userLat global and instead of console.log(docs.currentUserLat); do userLat = docs.currentUserLat. The problem that if you will do this, your userLat eventually will have the desired value (if callback will not fail), but you can not predict when. So if you will do
var userLat = db.collection('users', function (err, document){ ... });
.. some other code
console.log(userLat);
you will not be sure that you will get the output. Another way to do put everything in another callback.
I'm using the node-soap client from milewise (create API by the way), but I have some difficulties to get the results of the callback to the right scope.
Here is the code I have for now:
function generateSoapRequest(req, res, next)
{
soap.createClient('http://127.0.0.1:' + cfg.service_port + cfg.service_url_path,
function(err, client) {
client.CXIf.CXIf.CXProcessXML(
{"XMLRequestData": {"CXLogon": {"UserID":"1901007", "Password":"2580", "LogonType":11 } } },
function(err, result, body) {
if (err) {
console.log(err);
return;
}
console.log(result);
var cxresponse = result.XMLResponse[0].CXResponse;
console.log('SessionID:' + cxresponse.SessionID + ' SessionInstanceID:' + cxresponse.SessionInstanceID);
});
});
}
function getVoiceMailInformation(req, res, next) {
var cxresponse = generateSoapRequest(req, res, next);
var something = doSomethingNext(cxresponse);
}
function doSomethingNext(cxresponse){....; return something;}
Basically, when I launch the getVoiceMailInformation(), it creates a soap client and request some information through the generateSoapRequest().
The next step would be to get the result of that function (not implemented in the code above, because I don't know how) and do something else.
My problem is soap.createClient is asynchronous, so the callback is fired well after the function is complete.
What would be the best approach ?
(Maybe it's something trivial, but the scope of an anonymous function in javascript is something that is killing me.)
Any help will be very appreciated.
Basically you can't do something like:
var cxresponse = generateSoapRequest(req, res, next);
because the function you're calling invokes asynchronous code, and therefore can't return a value that's determined by that code. The normal way around this is to give the function an extra callback parameter for a function that will be called with the result once the result becomes available. It doesn't have to be an anonymous function; it can be a named function. In your case, (assuming you've modified generateSoapRequest to take a callback as its fourth argument and call it when the results are ready, you could write
generateSoapRequest(req, res, next, doSomethingNext);
and then doSomethingNext will be called with cxresponse as an argument. Of course, since doSomethingNext also gets called asynchronously, it can't return a value either, so you'll have to apply the same technique to it.
The async module can make this sort of thing easier: in particular, its "waterfall" pattern is useful when you have a bunch of functions that have to run in sequence, each being called back from the previous one.