I used cloudinary in angularjs web SPA and there was nice callback named .progress:
.progress(function (info) {
//file.progress = Math.round((e.loaded * 100.0) / info.total);
//file.status = "Uploading... " + file.progress + "%"
console.log(info);
})
which was basically logging object with data about uploading status info.
Now I'm wondering if I'm able to somehow get same progress when uploading by nodeJS code on server side
cloudinary.v2.uploader.upload(path, { resource_type: "video" }, function(err, success) {
console.log('err: ' + JSON.stringify(err));
console.log('suc: ' + JSON.stringify(success.secure_url));
});
I'm using electron so I can communicate nodeJS side with front with ipcMain I can send custom event from node to front so I can just pass new event every 1% of uploading and display some progressbar on front.
But since cloduinary docs doesn't provide some additional info of how to get status of uploading with nodejs I don't know how to do that and if it is possible.
After much more googling I found information on github from cloudinary collaborator that progress isn't available for server-side uploading.
Related
I have created a Node JS server which does the following:
Uploads media files (videos and images) to the server using multer
If the media is an image, then resize it using sharp
It the media is a video , then resize and compress it using fluent-ffmpeg
Upload files to Firebase storage for backup
All this is working know fluently. The problem is that, when the size of an uploaded file is big, the request processing takes long time. So I want to show some progress on the client side as below:
State 1. The media is uploading -> n%
State 2. The media is compessing
State 3. The media is uploading to cloud -> n%
State 4. Result -> JSON = {status: "ok", uri: .., cloudURI: .., ..}
Firebase storage API has a functionality like this when we creating an upload task as shown below:
let uploadTask = imageRef.put(blob, { contentType: mime });
uploadTask.on('state_changed', (snapshot) => {
if (typeof snapshot.bytesTransferred == "number") {
let progress = (snapshot.bytesTransferred / snapshot.totalBytes) * 100;
console.log('Upload is ' + progress + '% done');
}
});
I have found that, it is possible to realize this using websockets, I am interested if there is other methods to do that.
The problem is described also here: http://www.tugberkugurlu.com/archive/long-running-asynchronous-operations-displaying-their-events-and-progress-on-clients
And there is one of the methods Accessing partial response using AJAX or WebSockets? but I am looking for a more flexible and professional solution.
I have solved this problem using GraphQL Subscriptions. The same approach can be realized using WebSockets. The steps to solve this problem are as below:
Post files to upload server
Generate operation unique ID and send it as response to the client
Ex: response = {op: "A78HNDGS89NSNBDV7826HDJ"}
Create a subscription by opID
Ex: subscription { uploadStatus(op: "A78HNDGS89NSNBDV7826HDJ") { status }}
Every time on status change send request to the GraphQL endpoint, which which publishes the data to the pubsub. To send GraphQL request from nodejs server you can use https://github.com/prisma-labs/graphql-request
Ex:
const { request } = require('graphql-request');
const GQL_URL = "YOUR_GQL_ENDPOINT";
const query = `query {
notify ("Status text goes here")
}`
request(GQL_URL, query).then(data =>
console.log(data)
)
notify resolver function publishes the data to the pubsub
context.pubsub.publish('uploadStatus', {
status: "Status text"
});
If you have more complicated architecture, you can use message brokers like RabbitMQ, Kafka etc.
If someone knows other solutions, please let us know )
I am stuck in retrieving multipart from cloudant using Node JS API. Hence, I used REST API to download the wav file from cloudant database. But its not downloading wav file from https URL. When I enter the https URL directly in browser, it prompts me to save file locally. So, the URL is correct.
Here is the code for REST API:
var request1 = require('request');
var filestream = fs.createWriteStream("input.wav");
var authenticationHeader = "Basic " + new Buffer(user + ":" + pass).toString("base64");
request1( { url : "example.com/data/1533979044129/female";, headers : { "Authorization" : authenticationHeader } },
function (error, httpResponse, body) {
const statusCode = httpResponse.statusCode;
httpResponse.pipe(filestream);
httpResponse.on('end', function () {
console.log("file complete");
filestream.close();
}); });
The file size of input.wav is 0. Its not downloading file. Please help.
Your callback has an error argument, which you are completely ignoring. Do something with this error, like print it out so your problem can tell you what you're doing wrong. I definitely see at least 1 problem in your source, and the error from request should tell you what it is.
Edit On second thought the above code shouldn't even execute. You should share code that you tested yourself. There's typos in there.
Apologize for my English.
I have a node js script that has to send AMQP messages to device using IoT hub. I've took thiss script from github of azure iot. Here is this sample.
Here is this sample
Here is my script, based on this one:
console.log("creating the client");
var Client = require('azure-iothub').Client;
console.log("client has been created");
var Message = require('azure-iot-common').Message;
console.log("message has been created");
var connectionString = "HostName=id**.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=***;
console.log(connectionString);
var targetDevice = 'devicesergey';
var client = Client.fromConnectionString(connectionString);
client.open(function (err) {
if (err) {
console.error('Could not connect: ' + err.message);
}
else {
console.log('Client connected');
var data = JSON.stringify({ text : 'foo' });
var message = new Message(data);
console.log("json message is created")
console.log('Sending message: ' + message.getData());
client.send(targetDevice, message, printResultFor('send'));
console.log("message has been sent");
}
});
function printResultFor(op) {
return function printResult(err, res) {
if (err) {
console.log(op + ' error: ' + err.toString());
} else {
console.log(op + ' status: ' + res.constructor.name);
}
};
}
That works fine locally and I see messages on my device emulator. But when I try to put it to Azure Mobile Services API and try to run it, I see this message on logs:
An unhandled exception occurred. Error: One of your scripts caused the service to become unresponsive and the service was restarted. This is commonly caused by a script executing an infinite loop or a long, blocking operation. The service was restarted after the script continuously executed for longer than 5000 milliseconds. at process.Server._registerUncaughtExceptionListenerAndCreateHttpServer._onUncaughtException (D:\home\site\wwwroot\node_modules\azure-mobile-services\runtime\server.js:218:17) at process.EventEmitter.emit (events.js:126:20)
And sometimes I see this IIS error
I know exactly that this line occurs this function: client.open(function....
I've evem tried to leave only client.open() and send a messages out of this function. But in this case I see "client is not connected".
I asked about this stuff on github. They advised me to asked here. Maybe someone know how to solve this issue (with script or Azure). I would be very very greatfull!
Thank you!
The Mobile Service Custom API is a script that expose the functionality of the express.js library, please see the section Overview of custom APIs of the offical document "Work with a JavaScript backend mobile service"
I reproduced the issue successfully. I guess your script was not wrapped in the code below as the body block, and not sent the response to the client like browser.
exports.get = function(request, response) {
// The body block
....
response.send(200, "<response-body>");
}
For more details of Mobile Service Custom API, please see https://msdn.microsoft.com/library/azure/dn280974.aspx.
Update:
I changed your code as below.
And In order to facilitate the test, I changed the permission for the api as below, then I can access the api link https://<mobile-service-name>.azure-mobile.net/api/test with browser.
I've just tried to execute my script on new Azure MS and it was unsuccesfully.
I will write my step-by-step actions, maybe you can see anything wrong, because I'm not so good in NodeJS.
Add a new Azure MS with new SQL Database
Add a new API "dev". Access - everyone for all points. Here is source code:
exports.get = function(request, response) {
console.log("creating the client");
var Client = require('azure-iothub').Client;
console.log("client has been created");
var Message = require('azure-iot-common').Message;
console.log("message has been created");
var connectionString = "HostName=i***.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey***";
console.log(connectionString);
var targetDevice = 'devicesergey';
var client = Client.fromConnectionString(connectionString);
client.open(function (err) {
if (err) {
console.error('Could not connect: ' + err.message);
}
else {
console.log('Client connected');
var data = JSON.stringify({ text : 'foo' });
var message = new Message(data);
console.log("json message is created")
console.log('Sending message: ' + message.getData());
client.send(targetDevice, message, printResultFor('send'));
console.log("message has been sent"); }
});
response(200, "Hello, world!");
};
function printResultFor(op) {
return function printResult(err, res) {
if (err) {
console.log(op + ' error: ' + err.toString());
} else {
console.log(op + ' status: ' + res.constructor.name);
}
};
}
If I try to execute this stuff it occurs "no azure-iothub" and "no azure-iot-common", so I need to use git to add these npm.
I clone this repository to my local dir using git access to Azure MS https://id.scm.azure-mobile.net/id.git
Enter the "API" folder and add the NPMs:
Then I perfom "Rescan", "Save changes", "Commit", "Push" on
After these actions I execute my script by path "http://id**.mobile-services.net/api/dev" and don't see anything o see the error "500.1013" and these messages on logs (id depends):
An unhandled exception occurred. Error: One of your scripts caused the
service to become unresponsive and the service was restarted. This is
commonly caused by a script executing an infinite loop or a long,
blocking operation. The service was restarted after the script
continuously executed for longer than 5000 milliseconds. at
process.Server._registerUncaughtExceptionListenerAndCreateHttpServer._onUncaughtException
(D:\home\site\wwwroot\node_modules\azure-mobile-services\runtime\server.js:218:17)
at process.EventEmitter.emit (events.js:126:20)
I can't realize what I'm doing wrong
UPDATE:
I've tried to use Kudu console for installing the npms and it returns many errors. If i figured out correctly, I need to update my node js and npm. But I don't know how to do this and I didn't manage to find a solution.
Here are logs:
I have lack of reputation, so I am not allowed to past log scripts.
I've tried to do these actions, but it doesn't help:
at the root of the repo, you'll find a .deployment file that has:
command = ..\ZumoDeploy.cmd Change it to
command = deploy.cmd And create a deploy.cmd next to it containing:
set
NPM_JS_PATH=%ProgramFiles(x86)%\npm\1.4.9\node_modules\npm\bin\npm-cli.js ..\ZumoDeploy.cmd Commit both files and push.
I'm confused. How is it possible? Azure Mobile services don't permit to install azure-iot-hub npm). What can I do with this issue?
UPDATE2:
Peter Pan - MSFT, you advised me to use Kudu DebucConsole to install necessary npm. But when I try to do it - I see errors.
I've messaged about this issue to "npm" command on github, they say that the version of npm which Azure is used to - is completely unsupported.
htt ps://github.com/npm/npm/issues/12210#event-615573997
UPDATE3 (04/12/2016)
I've solved this issue by different way. Created my own node JS script that is listening a port, read GET params(deviceId and message) and send D2C messages.
Unfortunately, I still can't get trow the Azure issue.
UPDATE4
Peter Pan gave me an advise how to use another version of nodejs and npm. Now I've succesfully installed necessary NPM modules. But now Azure Mobile Script APIs don't work, it shows me {"code":404,"error":"Error: Not Found"} on any script that I try to get in my browser.
Maybe I've deleted something when I tried to do these stuffs.
I am trying to build a front end to my ACS ( Appcelerator Cloud Service) database. As a part of the admin front end, users will upload images and I am using Photos object to save them. I am using following code to upload the photos to cloud db and it works very well on my local system/PC.
var data = {
session_id:req.session.session_id,
photo: req.files.photo_file
};
data['photo_sizes[medium_500]'] = '500x333';
data['photo_sync_sizes[]'] = 'medium_500';
ACS.Photos.create(data, function(e) {
if(e.success && e.success === true){
// Update custom object with this photo
ACS.Objects.update({
session_id:req.session.session_id,
classname:objname,
id:objid,
fields: {
photo_id:e.photos[0].id,
photo_url:e.photos[0].urls.medium_500
}
},function(data) {
if(data.success) {
// console.log('Updated successfully:' + JSON.stringify(data));
res.send(data);
}else {
console.log('Error:\n' +
((data.error && data.message) || JSON.stringify(data)));
}
}
);
//res.send(data);
}else{
logger.debug('Error: ' + JSON.stringify(e));
req.session.flash = {msg:e.message, r:0};
res.redirect('/');
}
});
What's happening here is, a mutipart HTML form is uploading the file. That file is read on server and passed to the ACS.Photos.create call. However, when I publish the app to the cloud, it gives following error and application crashes.
[ERROR] [1233] Error: EACCES, open '/tmp/292fb15dcab44f58a315515bd9e70a8a'
Looking at the error it's clear that, server is not able to access the /tmp directory.
Node.acs is built on top of Node.js, I saw several node.js examples using this approach. How this issue is handled when the application/website is published or goes live on a web server?
Thanks,
Niranjan
Looks like there was indeed some file permission issue. Take a look at this post on the node.acs group.
https://groups.google.com/forum/#!topic/node-acs/XrRxBTtwiO4
The problem is now SOLVED !
I have managed to get file uploading work in Node.js with Express, and in the code i'm checking whether it's an image or not that the user is trying to upload.
If the file was successfully uploaded I want to show a message to the user, directly to the HTML page with the uploading form. The same should be if the file the user tried to upload wasn't an image, or something else happened during the upload.
The code below works (res.send...) but it opens up a new page containing only the message.
My question is: How can I change my code so that the message is sent directly to the HTML page instead? If it could be of any use, i'm using Jade.
Thanks in advance!
app.post('/file-upload', function(req, res, next) {
var fileType = req.files.thumbnail.type;
var divided = fileType.split("/");
var theType = divided[0];
if (theType === "image"){
var tmp_path = req.files.thumbnail.path;
var target_path = './public/images/' + req.files.thumbnail.name;
fs.rename(tmp_path, target_path, function(err) {
if (err) throw err;
fs.unlink(tmp_path, function() {
if (err) {
throw err;
res.send('Something happened while trying to upload, try again!');
}
res.send('File uploaded to: ' + target_path + ' - ' + req.files.thumbnail.size + ' bytes');
});
});
}
else {
res.send('No image!');
}
});
from what I understand you are trying to send a message to an already open browser window?
a few things you can do,
Ajax it, send the post, and process the return info.
Submit it as you are doing now, but set a flash message (look at http://github.com/visionmedia/express-messages) and either res.render the form page, or res.redirect to the form function
now.js or a similar solution. This would let clientside use serverside functions and serverside code to run clientside functions. So what you would do would be on submit, pass the post values to a serverside function, which will process it and trigger a clientside function (display a message)
For my money option #2 is probably the safest bet, as clients without javascript enabled will be able to use it. As for usability #1 or #3 would give a more streamlined appearance to the end user.
You can use WebSockets. I recommend using Socket.IO, it's very easy to work with. On the client-side you would have an event-handler which would use JavaScript to append the new information to that page.
You could then have the server for example say:
socket.emit('error', "Something happened while trying to upload, try again!");
and the client would use:
socket.on('error', function(data){
//alert?
alert(data);
});
http://socket.io/#how-to-use