busyboy not piping file to specified folder in node js (windows) - node.js

I am trying to upload mp4 files using typescript, node js and mongodb but it is not working.
I am using busboy library for the file upload
I am using postman to test my api
when I send the file I receive the file on busboy
I can display the file info like the filename
but the file does not move to the new location
video.controller.ts
const MIME_TYPES = ["video/mp4"];
function getPath({
videoId,
extension,
}: {
videoId: Video["videoId"];
extension: Video["extension"];
}) {
return `${process.cwd()}/videos/${videoId}.${extension}`;
}
export async function uploadVideoHandler(req: Request, res: Response) {
const bb = busboy({ headers: req.headers });
const user = res.locals.user;
const video = await createVideo({ owner: user._id });
console.log(video);
bb.on("file", async (_, file, info) => {
const extension = info.mimeType.split("/")[1];
const filePath = getPath({
videoId: video.videoId,
extension,
});
video.extension = extension;
const { filename } = info;
console.log("filename ", filename);
await video.save();
console.log(filePath);
const stream = fs.createWriteStream(filePath);
file.pipe(stream);
});
bb.on("close", () => {
res.writeHead(StatusCodes.CREATED, {
Connection: "close",
"Content-Type": "application/json",
});
res.write(JSON.stringify(video));
res.end();
});
return req.pipe(bb);
}
Api route
const router = express.Router();
router.post("/", requireUser, uploadVideoHandler);
export default router;

Related

Send blob pdf file from #react-pdf/renderer via rest API to Nest-js server that use fileInterceptor

Im unable to send blob pdf file that comes from #react-pdf/render.
first I'm tring to convert that blob into a file using new File()
<BlobProvider
document={<Document />}
>
{({ blob, url, loading, error }) => {
buildPdfFile(blob);
return <div />;
}}
</BlobProvider>
const fileRef = useRef<File | null>(null);
const buildPdfFile = (blob: any) => {
const file = new File(
[blob],
`${get(resumeData, "ownerName", "")}_${get(
resumeData,
"ownerId",
""
)}_ficha_de_inscripciĆ³n.pdf`,
{
type: "application/pdf",
}
);
fileRef.current = file;
console.log(fileRef.current);
};
const handleOnSubmit = () => {
dispatch(sendPdfToServer(fileRef.current!));
};
once I got that file I'm tried to send it using formdata in a POST request with application/pdf as content-type to my nestjs app
const sendPdfToServer = (inscriptionPdf) => {
const jwt = getJWT();
const options = {
headers: new Headers({
"content-type": "application/pdf",
Authorization: `Bearer ${jwt}`,
}),
};
const formData = new FormData();
formData.append("file", inscriptionPdf, inscriptionPdf.name);
const path = `${url}`;
try {
const response = await fetch(path, {
...options,
method: "POST",
body: formData,
});
}
catch (e) {
console.log(e);
}
}
but in the endpoint I'm using, the file is never intercepted, it shows as undefined
#Post('sendMail')
#UseInterceptors(
FileInterceptor('file', {
storage: diskStorage({
destination: './uploads/emailsTemporalFiles',
filename: (req, file, cb) => {
console.log('file ->', file);
const fileName: string = path
.parse(file.originalname)
.name.replace(/\s/g, '');
const extension: string = path.parse(file.originalname).ext;
cb(null, `${fileName}${extension}`);
},
}),
}),
)
async sendMail(#Res() response, #UploadedFile() file) {
this.logger.log(` | sendMail`);
console.log(file); // it prints undefined
}

Upload image to s3 bucket - react native and node js

Within my app a user can select a profile image and i would like that image to be uploaded to an s3 bucket when the user saves their profile data
I pass the image data (and json, which consists of name, email, telephone for example) from my app to an express server and upload there
At present I can pass the image data (the url it seems at present) to an s3 bucket and it saves
I don't think i'm actually saving the image itself though, as when downloading from s3 (manually) and trying to open on my mac it states it may be damaged and i cannot see the image
Feel daft for asking but how do i actually upload the image itself? Thanks
React Native Side
const handleFormSubmit = formData => {
const jsonData = JSON.stringify({
...formData,
});
// Handle profile image
if (imageProps && imageProps.uri) {
const data = new FormData();
data.append('formBody', jsonData);
data.append('image', {
uri:
Platform.OS === 'android'
? imageProps.uri
: imageProps.uri.replace('file://', ''),
type: imageProps.type,
name: imageProps.fileName,
});
sendRequest(data);
} else {
sendRequest(jsonData);
}
};
const sendRequest = data => {
let responseData;
fetch('http://localhost:8080/users/api/update_user_profile', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Accept: 'application/json',
},
body: data,
})
.then(response => {
responseData = response;
return response.json();
})
.then(jsonData => {
console.log(jsonData)
})
.catch(error => {
console.log(error)
});
};
Server Side
const s3 = new AWS.S3({
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
});
// Setting up S3 upload parameters
const params = {
Bucket: 'bucket-folder',
ACL: 'public-read',
Key: req.files.image.name,
Body: req.files.image.path
};
const stored = await s3.upload(params).promise();
You can use Multer for uploading files to s3.
const multer = require('multer');
const AWS = require('aws-sdk');
const uniqid = require('uniqid');
const storage = multer.memoryStorage();
const upload = multer({ storage });
// ? Posts new file to amazon and saves to db
router.post(
'/:id',
upload.single('attachment'),
async (req, res) => {
const unique = uniqid.time();
const { file } = req;
const { filePath } = req.body;
const { id } = req.params;
const s3FileURL = process.env.AWS_UPLOADED_FILE_URL;
const region = process.env.AWS_REGION;
const secretAccessKey = process.env.AWS_SECRET_ACCESS_KEY;
const accessKeyId = process.env.AWS_ACCESS_KEY_ID;
const Bucket = process.env.AWS_BUCKET_NAME + '/' + filePath;
const Key = `${id}/${unique}-${file.originalname}`;
const Body = file.buffer;
const ContentType = file.mimetype;
const ACL = 'public-read';
const s3bucket = new AWS.S3({
accessKeyId,
secretAccessKey,
region,
});
const params = {
Bucket,
Key,
Body,
ContentType,
ACL,
};
s3bucket.upload(params, async (err, data) => {
if (err) {
res.status(500).json({ error: true, Message: err });
} else {
console.log(params);
const newFileUploaded = {
description: req.body.description,
fileLink: `${s3FileURL}${filePath}/${id}/${unique}-${file.originalname}`,
s3_key: params.Key,
};
try {
const response = await postFile({
name: req.body.name,
attachment: newFileUploaded,
alt: req.body.alt,
user: req.body.user,
relatedID: req.body.relatedID,
});
res.status(200).json({
message: response.message,
success: response.success,
result: response.result,
});
} catch (e) {
res.status(500).json({
message:
'File upoladed but Db couldnt saved request (upload by ID)',
success: false,
result: [],
});
}
}
});
}
);

Uploding a profile picture in a React native Expo Application

So, I have an expo application that the user can set his profile picture, but I'm having trouble to store the picture in my backend.
I'm using Node.js in the backend and i have 2 libraries which are fs-extra and path.
So When a user set his profile picture, what i get in the backend is this file:///data/user/0/host.exp.exponent/cache/ExperienceData/%2540motaa%252FNatureApp/ImagePicker/783259e0-e3b5-41ee-8fd7-e03fd54425a0.jpg
And i have tried to use a form to transfer the file but i couldn't get it to work so i stuck with this.
But now i don't know how to extract the picture so i can store it on my backend or even in my database.
This is the code i'm using to store the picture in a directory in the backend:
const { image } = req.body;
console.log(image);
const newPath = path.resolve(__dirname, "../PostImages") + "/" + "image.jpg";
await fs.move(image, newPath);
Frontend Code:
const image = result;
const data = new FormData();
data.append("profilePicture", {
name: "hello.jpg",
type: image.type,
uri:
Platform.OS === "android"
? image.uri
: image.uri.replace("file://", ""),
});
uploadUserImage({ data });
Request code:
const uploadUserImage = (dispatch) => {
return async ({ data }) => {
console.log(data);
const token = await AsyncStorage.getItem("token");
try {
await axios({
method: "post",
url: "http://localhost:5000/userImg",
headers: {
Authorization: `Bearer$${token}`,
"Content-Type": "multipart/form-data",
},
body: data,
});
} catch (err) {
console.log(err);
}
};
};
You can use multer which is great to save images on backend. If you are passing the form data you can use it, this is how you can add it:
<form action="/profile" method="post" enctype="multipart/form-data">
<input type="file" name="avatar" />
</form>
var express = require('express')
var multer = require('multer')
var upload = multer({ dest: 'uploads/' })
var app = express()
app.post('/profile', upload.single('nameofImage'), function (req, res, next) {
// req.file is the `avatar` file
// req.body will hold the text fields, if there were any
})
For more details, you can check here.
For React Native/Expo projects, to upload/send a file to the backend you need to create a formData and use it in the Body of the request, like this
const formData = new FormData();
formData.append('FileData', {
name: 'FileData.jpg',
uri: 'file:///data/user/0/host.exp.exponent/cache/ExperienceData/%2540motaa%252FNatureApp/ImagePicker/783259e0-e3b5-41ee-8fd7-e03fd54425a0.jpg',
type: 'image/jpg', // mime type of file
});
Now perform a request to the backend with this formData in the body
Request Code would be
const uploadUserImage = (dispatch) => {
return async ({ data }) => {
console.log(data);
const token = await AsyncStorage.getItem("token");
try {
await axios({
method: "post",
url: "http://localhost:5000/profile", // make sure URL is correct
headers: {
Authorization: `Bearer$${token}`,
"Content-Type": "multipart/form-data",
},
body: data,
});
} catch (err) {
console.log(err);
}
};
};
And in the backend access the file as shown below
var express = require('express')
var multer = require('multer')
const upload = multer({ storage: multer.memoryStorage() });
var app = express()
app.post('/profile', upload.single('FileData'), function (req, res, next) {
// access it with --> req.file
})

Unable to upload image/video files from react-native to nodejs

I am using expo-image-picker in react-native to pick image/video files and multer in nodejs as middleware to download files in directory /public/upload. When i am uploading file along with other parameters from react-native, multer is unable to detect a file present in req.body and hence not downloading any file.
Here is my react-native code using axios
pickImage = async () => {
try {
let options = {
mediaTypes: ImagePicker.MediaTypeOptions.Images,
quality: 1,
// base64:true
}
let result = await ImagePicker.launchImageLibraryAsync(options)
if (!result.cancelled) {
this.setState({ content: result })
}
} catch (E) {
console.log("error in picking image:", E)
}
}
createFormData = (response) => {
const photo = {
uri: response.uri,
type: response.type,
name: "my-img.jpg",
};
const form = new FormData();
form.append('acivityFile',photo);
return form;
};
handleSubmit = async () => {
if (this.state.content) {
const formData = this.createFormData(this.state.content)
console.log("form data:", formData)
try {
const res = await axios.post('http://393ad751391b.ngrok.io/activities',
{
title: "This is the title",
description: "This is a description",
eventType: "LOST & FOUND",
file: formData
},
{
headers: {
'Accept': 'application/json',
'Content-Type': 'multipart/form-data'
},
},
)
console.log("res:", res.data);
} catch (err) {
console.log("err in post axios:",err)
}
}
}
Here is my route file handling http requests in server-side
const express = require('express');
const Upload = require('./../utils/multerSetup');
const activityController = require('../Controllers/activityController');
const router = express.Router();
router
.route('/')
.get(activityController.getAllActivities)
.post(
Upload.single('activityFile'),
activityController.addActivity
);
Here is my multerSetup.js file in server-side
const multer = require('multer');
const storage = multer.diskStorage({
destination: function (req, file, cb) {
cb(null, 'public/uploads/');
},
filename: function (req, file, cb) {
const ext = file.mimetype.split('/')[1];
cb(null, file.fieldname + '-' + Date.now() + '.' + ext);
},
});
const upload = multer({ storage });
module.exports = upload;
Here is my activityController.js file in server-side
const Activity = require('./../modals/activityModel');
const User = require('./../modals/user');
exports.getActivity = async (req, res, next) => {
console.log('here');
const activity = await Activity.findById(req.params.id);
res.status(200).json({
status: 'success',
data: {
activity,
},
});
};
exports.addActivity = async (req, res, next) => {
if (req.file) {
let file = {
Ftype: req.file.mimetype.split('/')[0],
name: req.file.filename,
};
req.body.file = file;
}
if (!req.body.location) {
req.body.location = {
coordinates: ['77.206612', '28.524578'],
};
}
if (req.body.votes) {
req.body.votes.diff = req.body.votes.up - req.body.votes.down;
}
req.body.creator = "12345" || "req.userId";
const activity = await Activity.create(req.body);
res.status(201).json({
status: 'success',
data: {
activity,
},
});
};
Also when Content-type:'multipart/form-data, then server console throws Error: Multipart:Boundary not found. When i use Content-type:'application/json', then multer does not work.
I just want to know what is the correct way of uploading files with additional parameters from react-native to nodejs multer. Any suggestions would be a great help!

how to upload file saved in memory by multer to another API

I have 2 separate NodeJS APIs that uses multer to save a file in memory.
My express middleware looks like this
import multer from 'multer';
const storage = multer.memoryStorage();
export default multer({ storage }).single('image');
I am able to receive the file which is saved in memory successfully so my req.file.image looks like this
{
fieldname: 'image',
originalname: 'image-2017-08-28-13-47-31-218.png',
encoding: '7bit', mimetype: 'image/png',
buffer: <Buffer 89 50 4e 47 0d 0a 1a 0 ... >,
size: 493181
}
After receiving the file, on the first API, I need to send it to the second API that also uses multer & express
function secondApiReceiveImage(req, res) {
console.log(req.file)
console.log(req.files)
console.log(req.body)
res.send('ok');
}
I tried sending using the following implementations
Via https://github.com/request/request#multipartform-data-multipart-form-uploads
import request from 'request';
function firstApiReceiveImage(req, res) {
const options = {
url:'http://SECOND_API/api/image',
formData: { image: req.file.buffer }
};
request.post(options, (err, httpResponse, body) => {
console.log('err', err);
console.log('body', body);
});
}
In this case, logs of req.file, req.files and req.body are all undefined on secondApiReceiveImage API handler function
My next try was with https://github.com/form-data/form-data
import multer from 'multer';
const storage = multer.memoryStorage();
export default multer({ storage }).single('image');
function firstApiReceiveImage(req, res) {
const CRLF = '\r\n';
const form = new FormData();
const opts = {
header: `${CRLF} + '--' + ${form.getBoundary()} + ${CRLF} + 'X-Custom-Header: 123' + ${CRLF} + ${CRLF}`,
knownLength: 1
};
form.append('image', req.file.image.buffer, opts);
form.submit('http://SECOND_API/api/image', (err, res) => {
console.log('err', err);
console.log('res', res);
});
}
I got the same result, undefined for req.file, req.files & req.body on the second API
These are my middlewares for both APIs aside from multer BTW
app.use(compression());
app.use(helmet());
app.use(cors());
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
I could have had an easier life if I can persist the file on the first API, but we are not allowed to save to disk in this case :(
Any advice for me?
I've stumbled upon the same issue and here is what I've managed to come up with.
First of all I used the form-data package to mimic the FormData data structure on the server side and here is the util file that does that.
const FormData = require('form-data')
const formData = new FormData()
const config = {
headers: {
'Content-Type': `multipart/form-data; boundary=${formData.getBoundary()}`
}
}
export default { data: formData, config }
and here is my node.js api file
import formSet from '../utils/formData'
const multer = require('multer')
const upload = multer()
const { Router } = require('express')
const router = Router()
......
router.post('/api/images', upload.single('image'), async (req, res) => {
formSet.data.append('image', req.file.buffer, { filename: req.file.originalname })
formSet.data.append('other_data', 'value')
const response = await axios.post('/images', formSet.data, formSet.config)
.......
})
The team was able to get through this by converting the buffer to Stream first and send form data differently.
import stream from 'stream';
import rq from 'request-promise';
const { Duplex } = stream;
function bufferToStream(buffer) {
const duplexStream = new Duplex();
duplexStream.push(buffer);
duplexStream.push(null);
return duplexStream;
}
async function store({
originalname, mimetype, size, buffer,
}) {
logger.debug(`Saving image name:${originalname} type:${mimetype} size:${size}`);
const formData = {
image: {
value: bufferToStream(buffer),
options: {
filename: originalname,
contentType: mimetype,
knownLength: size,
},
},
};
const options = Object.assign({}, IMAGE_SAVE_ENDPOINT, { formData });
const response = await rq(options);
return response.id;
}
Using Request, formdata should be set to the buffer:
formData: { image: req.file.buffer }
I've found the way to upload files with multer through buffer and not storing locally...
Here is my code....
var multer = require('multer');
var storage= multer.memoryStorage();
var upload = multer({storage: storage});
var cloudinary = require('cloudinary');
cloudinary.config({
cloud_name: 'your-cloud-name',
api_key: process.env.CLOUDINARY_API_KEY, //your api-key
api_secret: process.env.CLOUDINARY_API_SECRET //your api-secret
});
Configure your cloudinary and multer as above.....
Get the buffer from req as follows..
router.post("/",upload.single('image'),function(req,res){
var buf = req.file.buffer.toString('base64');
// Upload code Here
)};
Upload code:
cloudinary.uploader.upload("data:image/png;base64," + buf, function(result) {
//your code here
console.log(result);
},{
folder: 'folder-name' //if you want to store in a folder
});
The output of result would be as follows:
{
asset_id: 'ed58484fb2bdce823f1b27d*******8',
public_id: '**************',
version: 1594455479,
version_id: '68450****88842ce0aa319603e68d3',
signature: '3785dbfc3**********883720b',
width: 750,
height: 500,
format: 'png',
resource_type: 'image',
created_at: '2020-07-11T08:17:59Z',
tags: [],
pages: 1,
bytes: 70846,
type: 'upload',
etag: 'd04dd691de532e941faccda846ef3d76',
placeholder: false,
url: 'http://res.cloudinary.com/*****************',
secure_url: 'https://res.cloudinary.com/*******************'
}

Resources