Just want simple file upload functionality . I have used fs-path that serves my purpose of creating dynamic folder structure and file at upload location. I am not able to achieve streaming of request file, that will have to be uploaded. My code is as follows :
fsPath.writeFile(path, **req.body**, function (err) {
if (err) {
throw err;
} else {
console.log('File Upload successful...');
res.send({ code: 200, message: `File Upload successful` });
}
});
Need some insights about, how can I send request file as input in the above code snippet. How do I steam my request file, that will be written in respective upload location?
If you want to stream the request body then instead of using a body parser or multr you should use the req stream directly. Remember that the request object is a stream and you can use it as such:
req.on('data', data => {
console.log(data);
});
You can also pipe it to some other stream, like a writable stream created with fs.createWriteStream etc.
Related
I have an API in one nodejs project as below which receive multiple attachment from UI:
const upload = multer()
router.post('/upload', upload.array("attachments"),controller.getSomething);
getSomething is supposed to call another POST API using Axios only, which is in another NodeJs project which accept these attachments and process it. It as well accept multiple files via multer.
I am unsure how could i send multiple files as a request from one Nodejs project to another at once. could you please favour.
I had to set formdata as below:
const formData=new FormData();
for(let file of req.files){
formData.append("attachments",file.buffer,file.originalname);
}
And passed the formdata to other api via axios.
You can do the following steps:
When you upload the temporary files (coming from UI), save them in the temporary folder.
Pass all the files names to the POST API using Axios.
In the post API, read all the files from the temporary folder and stream them to the destination.
controller.getSomething = (req, res, next) => {
// get the file names
const fileNames = req.files.map(filename => filename);
// now post this filenames to the
axios.post('/pathname', {fileNames})
// or, you can use get request
}
Reading files in the post Request:
var promises= ['file1.css', 'file2.css'].map(function(_path){
return new Promise(function(_path, resolve, reject){
fs.readFile(_path, 'utf8', function(err, data){
if(err){
console.log(err);
resolve(""); //following the same code flow
}else{
resolve(data);
}
});
}.bind(this, _path));
});
Promise.all(promises).then(function(results){
//Put your callback logic here
response.writeHead(200, {"Content-Type": "text/css"});
results.forEach(function(content){response.write(content)});
response.end();
});
#copied from this link. You should check the different answers that can help you.
I've been stuck on this problem for awhile now and can't seem to figure out why this file isn't being uploaded to firebase correctly. I'm using the code below in firebase functions to generate a document, then I convert that document to a stream, finally I create a write stream declaring the path that I want the file to be written to and pipe my document stream to the firebase storage WriteStream.
Example 1: PDF uploaded from file system through firebase console. (Link works and displays pdf)
Example 2: PDF generated in firebase functions and written to storage using code below
Considerations:
I know the PDF is valid because I can return it from the function and see it in my web browser and everything is as I would expect.
When trying to open the bad file it doesn't display anything and redirects me back to the overview screen.
const pdfGen = require("html-pdf");
pdfGen.create(html, { format: "Letter" }).toStream(function (err, stream) {
if (err) return res.status(500).send(err);
var file = storage.bucket()
.file(job.jobNum + "/quote-doc.pdf")
.createWriteStream({
resumable : false,
validation : false,
contentType: "auto",
metadata : {
'Cache-Control': 'public, max-age=31536000'}
});
stream.pipe(file)
.on('error', function(err) {
return res.status(500).send(err);
})
.on('finish', function(data) {
stream.close();
file.end();
res.end();
console.log('finished upload');
});
});
I am implementing a web app using MEAN Stack and Angular 6. There I want to submit a form with file upload. '.png' files should be uploaded.
I want to save the file in a different file server and send the url to the image.Currently I upload files into a folder in my project and save the image in db (I used ng2fileupload and multer for that.). Then it saves like this.
"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAV4AAAFUCAYAAABssFR8AAAK..."
But I want to save the image url and the image should be retrived by the url. Does anyone can explain a proper method for that?
I faced the same problem a month ago and find out a solution to this problem. Though I haven't used multer in the app.
From my frontend, I will be sending an object to Node API endpoint /event which will look like:-
let img = {
content: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUg...",
filename: 'yourfile.png'
}
At the backend, I'm using Cloudinary to store my images (Its free plan allows 10GB storage) and returns secure https URLs. So install it using npm i cloudinary and require in your api.js file.
And add the below configuration
cloudinary.config({
cloud_name: 'yourapp',
api_key: 'YOUR_KEY',
api_secret: 'YOUR_SECRET_KEY'
});
Last Step:- (Not so optimized code)
Let say I have an event Schema which has images array, where I'll be storing the URLs returned by cloudinary.
app.post('/event', (req, res) => {
try {
if (req.body.images.length > 0) {
// Creating new Event instance
const event = new Event({
images: [],
});
// Looping over every image coming in the request object from frontend
req.body.images.forEach((img) => {
const base64Data = img.content.split(',')[1];
// Writing the images in upload folder for time being
fs.writeFileSync(`./uploads/${img.filename}`, base64Data, 'base64', (err) => {
if (err) {
throw err;
}
});
/* Now that image is saved in upload folder, Cloudnary picks
the image from upload folder and store it at their cloud space.*/
cloudinary.uploader.upload(`./uploads/${img.filename}`, async (result) => {
// Cloudnary returns id & URL of the image which is pushed into the event.images array.
event.images.push({
id: result.public_id,
url: result.secure_url
});
// Once image is pushed into the array, I'm removing it from my server's upload folder using unlinkSync function
fs.unlinkSync(`./uploads/${img.filename}`);
// When all the images are uploaded then I'm sending back the response
if (req.body.images.length === event.images.length) {
await event.save();
res.send({
event,
msg: 'Event created successfully'
});
}
});
});
}
} catch (e) {
res.status(400).send(e);
}
});
P.S. Go ahead and suggest some optimization solution for this code here
This question already has answers here:
Nodejs POST request multipart/form-data
(5 answers)
Closed 5 years ago.
File uploading using multer is not happening
My code:
File read from html and pass to external url
router.post('/fileUpload',function (req,res) {
request({
uri: http//example.com/upload, // url of other server
method: "POST",
form: {
"name":req.body.name,
"image":? //image from html - no idea how to pass the req.files here
}
}, function (error, response, body) {
------------
------------
}
});
other server url : /example.com/upload
This is the code to upload image using multer
app.post('/upload',function(req,res){
var storage = multer.diskStorage({
destination: function (req, file, callback) {
callback(null, 'uploads');
},
filename: function (req, file, callback) {
callback(null, file.fieldname + '-' + Date.now());
}
});
var upload = multer({ storage : storage }).array('productImage');
upload(req,res,function(err) {
if(err) {
return res.json({'success':0,'result':{},'errorMessage':'Unknown Error'});
}
return res.json({'success':1,'result':{},'errorMessage':''});
});
});
Create readStream from file uploaded and pipe it to the another server.check this link https://www.npmjs.com/package/request, you will easily get this done.
The simple answer would be to create a read stream from the uploaded file and pipe it to the second server, like so:
fs.createReadStream(req.files[0].path).pipe(request.post('http//example.com/upload'))
However, there are many ways to make this work. The most efficient of which is to use a binary stream from the initial upload all the way to the second server. You want to avoid using your first server as a storage for all of the uploads.
Here's how I would do it instead:
Use jQuery file upload on client side upload
(Or any other library/approach to upload the raw binary stream)
$('#fileupload').fileupload({
url: 'https://server1.com/upload'
})
Use Formidable (or other library like multer) to handle upload server-side
(This will allow you to read the binary stream in parts, and handle each part as it comes)
app.post('/upload',function(req,res){
var form = new formidable.IncomingForm();
form.parse(req);
form.onPart = function(part) {
fs.createReadStream(part).pipe( request.post('http//example.com/upload'))
}
}
When each part of the binary upload is received, you can to stream the binary directly to the second server via pipe() to avoid having to store it on the first server whatsoever.
The key to making this work is to look at the file upload and transfer as a stream of binary bits. When the user uploads a file (req.body) you want to create a read stream from the file, and pipe the binary over the request.post().
I am using Twitter module for nodejs to upload media to my twitter account. As example program from here https://github.com/desmondmorris/node-twitter/tree/master/examples#media.
Instead of using image from system i am passing a base64 encoded string of the image. When i run this program it output the base64 string(image) in the console in a loop and after a while it crashes and doesn't upload my image.
// Load your image
var data = require('fs').readFileSync(image);
// Make post request on media endpoint. Pass file data as media parameter
client.post('media/upload', {media: image}, function(error, media, response){
if (!error) {
// If successful, a media object will be returned.
console.log(media);
// Lets tweet it
var status = {
status: 'I am a tweet',
media_ids: media.media_id_string // Pass the media id string
}
client.post('statuses/update', status, function(error, tweet, response){
if (!error) {
console.log(tweet);
}
});
}
else{
console.log("error upload");
}
});
So i tried different method. I saved the base64 encode image in to out.png
fs.writeFile("out.png", data.image, 'base64', function (err) {
console.log(err);
});
var data = require('fs').readFileSync("out.png");
and this code output error saying
Error: ENOENT: no such file or directory, open 'D:\Local server\nodejs\out.png'
This file does exist i have confirmed
Update
Api returns [Error: Status Code: 400]
hi there are several nodejs api which allows to serve as a client library for Twitter, and with which you can upload your media to your twitter account, the most popular twitter and are node-twitter-api