I know that it's possible to upload files to my cloud-files account in Node.js, using the following module: node-cloudfiles.
But is it also possible to upload a filestream directly?
In my case I am dowloading an image from a certain location in Node.js and want to upload this directly to my cloud-files account without saving the image temporary on my server.
Of course it is possible - you can just read the documentation on Rackspace Cloud Files API ( http://docs.rackspacecloud.com/files/api/cf-devguide-latest.pdf ) and implement the necessary parts yourself.
However, I'd suggest to wait until https://github.com/nodejitsu/node-cloudfiles/pull/11 gets integrated into the trunk - then node-cloudfiles library will support uploading files using streams so you won't have to create files before uploading.
Related
I am using multer npm library to read files from postman and I am getting the file and its details in my node js code(checked via logging req.file), but my concern is I don't want the file to get stored in my local machine, I just want to extract the data from the file and process further for my requirements.
Is this possible or anyone can suggest me some solutions to this.
Thanks in advance
As I read multer libraries, it streams the uploaded file in the disk, and then it processes the file.
If you only want to use the properties like mime/type or originalfilename of the file you can use multer and in the storage options after getting the uploaded file properties, use cb(null, false) to prevent storing the file, but if you want to process the file, you can remove it from the disk after your process is done.
So I have a node js web app, this web app has a folder to store images uploaded by users from a mobile app. How I upload the image to the folder is by using the image's base64 string, and using fs.writeFile to save the image to the folder, like this:
fs.writeFile(__dirname + '/../images/complaintImg/complaintcase_' + data.cID + '.jpg', Buffer.from(data.complaintImage, 'base64'), function (err) {
if (err) {
console.log(err);
} else {
console.log("success");
}
});
The problem is, whenever the application is redeployed to google cloud, the images gets deleted. This is because the image folder of the local version of the application is empty - when the user uploads an image, i don't get a local copy of that image.
How do i prevent the images from getting deleted with every deployment? because the app is constantly updated (changes to js or html files), i can't have the images getting deleted with every deployment. How do i update a deployment to only deploy certain files? the gcloud app deploy command seems to deploy the entire project. or should i upload the images directly to google cloud storage?
please help, currently the mobile app isn't released to the public yet, so having the images deleted with every deployment is still not a big problem now, but it will be once it's released to the public. because the images they upload are very important. thank you in advance!
It appears that your __dirname directory you chose may be under /tmp or, if you use the flexible environment, some other directory local to your instance. If so the images will disappear whenever new instances are started (which always happens at new deployment, but it can happen in between deployments as well). This is expected, the instances are always started "from scratch".
You need to store the files that your app creates and you want to survive instance (re)starts on a persistent storage product, like Cloud Storage, see Using Cloud Storage (or Using Cloud Storage for flexible env). Note that you can't use the regular filesystem calls with Cloud Storage, you need to use the documented client library.
As stated in Dan Cornilescu's answer, for user uploaded files, you should store them in Cloud Storage for GAE Standard or for GAE Flexible.
Just as a reference, there is an alternative for those who are using Python 2.7, Java 8 or PHP 5, which is the BlobStore API
I'm currently writing images to my filesystem using jimp.
For example: image.write('path')
This cannot be used on Google App Engine because its read only filesystem.
How can I write this to a Google storage bucket? I've tried writestreams but keep getting read only errors so I feel like jimp is still writing to the drive.
Thanks heaps
As I can understand you are modifying some images on your App Engine and you want to upload them to a bucket but you didn't mention if you are using standard or flex environment. In order to do this you want to make your publicly readable so it can serve files.
Following this Google Cloud Platform Node.js documentation you can see that to upload a file to a bucket you need to create object first using:
const blob = bucket.file(yourFileName);
Then using createWriteStream you can upload the file
I am developing a web app using MEAN Stack and Angular 6. I want to upload tiff files(maps) with large file size.(max size is around 1.4 GB).I used 'ng2fileUpload' and 'multer' for that but it does not allow to store large files. Is there any good method to use for uploading large files in nodejs. My preavious post is on this link. 'Uploading files using multer gives 'request entity too large' error'
You should check first if there is a limit on max upload on the server site instead of the client side. There is a limit with apache service.
I want to upload a file which is being generated on my server conditionally in a directory, and I am using multer-s3 package to upload files to S3 service.
Is it possible to upload that generated file from server directory to S3 using multer-s3?
No, it is not possible to upload the generated file to s3 using multer-s3, because multer-s3 library has been designed and written in a way to provide alternative storage engine to multer, not as a library to upload file to s3. You can use some other library to upload files to s3 or you can read it here https://github.com/badunk/multer-s3/blob/master/index.js#L172