error on azure table storage upload file using nodejs expreesjs - node.js

I m trying to upload images/videos file on to azure blob storage using
nodejs. But on uploading through postman Im getting error, even though
I can see the file object in console. Below is the code for it.
const addProductImage = async (req: any, res: Response) => {
try {
console.log(req.files.file);
if (!req.files) {
res.status(400).send({ status: Status.ERROR, error: "No file uploaded" });
}
let file = req.files.file;
const sharedKeyCred = new StorageSharedKeyCredential(accName, acckey)
const blobServClient1 = new BlobServiceClient(`https://${accName}.blob.core.windows.net`, sharedKeyCred)
const containerClient1 = blobServClient1.getContainerClient(containerName)
const blockBlobClient = containerClient1.getBlockBlobClient(file.name);
await blockBlobClient.upload(file, file.size);
res.status(200).send({ status: Status.SUCCESS, });
}
catch (error: any) {
res.status(500).send({ status: Status.ERROR, error });
}
}
the above codes console is -
{
name: 'hclTech.png',
data: <Buffer 81 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 00 00 02 7u 00 00 00 75 08 06 00 00 00 3f 4a 88 b4 00 00 00 01 73 52 47 42 00 ae ce 1c e9 00 00 00 07 ... 54120 more bytes>,
size: 54170,
encoding: '7bit',
tempFilePath: '',
truncated: false,
mimetype: 'image/png',
md5: 'd64ae80f1625e3c766b67bcf05a152a8',
mv: [Function: mv]
}
On uploading a file in postman under form-data with "file" as key and
hitting post, it gives error

Here If you use multer package to read the file in the form-data.
Then you can set Multer to read the form-data where the file is using the .single('<Name of the Feild>') . Here the name of the field must be the same one in which you are adding the file. here I am using the postman with field name as test
Now you can set up multer like this
var multer = require('multer');
var upload = multer();
app.use(upload.single('test'));
Complete Api code :
var express = require('express');
var multer = require('multer');
const { BlobServiceClient } = require("#azure/storage-blob");
var app = express();
var upload = multer();
app.use(express.json());
app.use(express.urlencoded({ extended: false }));
app.use(upload.single('test'));
const blobServiceClient = BlobServiceClient.fromConnectionString('<Connection String From Portal>');
const containerClient = blobServiceClient.getContainerClient('< Name of the Container>');
app.get('/', async (req,res)=>{
const blockBlobClient = containerClient.getBlockBlobClient(req.file.originalname.toString());
await blockBlobClient.upload(file.toString() , file.size);
res.send(req.file.size.toString());
});
app.listen(3000);
Postman output:
Portal:

Related

How to make request to amazon s3 using node js for downloading a file?

I made a request to amazon s3 using node js for downloading a file. In the response i receive the Buffer, here i have 2 problems:
If i send the buffer to frontend from node js like res.send(awsreq.Body.Buffer) and console log it once in node js and once in frontend, the buffer from frontend will look different than in node js.
Node js Buffer: <Buffer 50 4b 03 04 14 00 06 00 08 00 00 00 21 00 df a4 d2 6c 5a 01 00 00 20 05 00 00 13 00 08 02 5b 43 6f 6e 74 65 6e 74 5f 54 79 70 65 73 5d 2e 78 6d 6c 20 ... 11906 more bytes>
Vue js frontend buffer: PK\u0003\u0004\u0014\u0000\u0006\u0000\b\u0000\u0 Show more(36,8kb)
How can i directly download the file from a request made from frontend? First how to receive that buffer corecly and how to convert it in that way frontend will download the file automatically? If possible how to do all of that in node js and when frontend receive the response to start automatically the download?
To Download a file from amazon S3 follow these steps
Install aws sdk using this command npm install aws-sdk
Check below code
var AWS = require("aws-sdk")
const s3 = new AWS.S3({
endpoint: "ENDPOINT",
accessKeyId: "YOUR_ACCESS_KEY_ID",
secretAccessKey: "YOUR_SECRET_ACCESS_KEY",
region: "us-east-1",
signatureVersion: "v4"
})
const downloadParams = {
Bucket: "BUCKET_NAME",
Key: "KEY_NAME/FILE_NAME/FILE_PATH"
}
// Download the file
s3.getObject(downloadParams, function (error, fileData) {
if (!error) {
console.log(fileData, "file")
} else {
console.log(error, " ERROR ")
}
})
For more information you can check AWS official documentation using this link:
https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/getting-started-nodejs.html
At last you can use
Content-Disposition in the request header
Or you can use pre Signed url
Example:
const getpreSignedUrlForDownload = async () => {
try {
const parms = {
Bucket: "BUCKET_NAME",
Key: "FILE_KEY",
Expires: 60 * 1,
ResponseContentDisposition: 'attachment; filename"' + "FILE_NAME + '"'
}
return new Promise((resolve, reject) => {
s3.getSignedUrl("getObject", parms, (err, url) => {
err ? reject(err) : resolve(url)
})
})
} catch (error) {
throw new Error(error)
}
}
getpreSignedUrlForDownload()

Response body: Payload Too Large

I am writing a POST request to a mock api that will store a file and its binary. The error I'm receiving is:
<Buffer 49 44 33 03 00 00 00 00 01 10 54 49 54 32 00 00 00 0b 00 00 00 61 6c 61 72 6d 53 6f 75 6e 64 54 58 58 58 00 00 00 17 00 00 00 53 6f 66 74 77 61 72 65 ... 115969 more bytes>
Error: SyntaxError: Unexpected token P in JSON at position 0
at JSON.parse (<anonymous>)
at safeParseJSON (C:\Users\Student\Downloads\RequestMaker\fetchTest.js:36:19)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at async C:\Users\Student\Downloads\RequestMaker\fetchTest.js:72:5
Response body: Payload Too Large
const fs = require('fs');
const fetch = require('node-fetch');
async function makePostReq(time, date) {
let file = 'sound.mp3'
fs.readFile(file, 'utf8', async (err, data) => {
if (err) {
console.error(err)
return
}
let buf = Buffer.from(data, 'utf8');
console.log(buf)
let body = {
"File Name": "sound.mp3",
"Binary": buf
}
let url = 'https://6050ffd85346090017671094.mockapi.io/posts/mockData'
await fetch(url, {
method: 'POST',
body: JSON.stringify(body),
headers: { 'Content-Type': 'application/json' },
})
.then(safeParseJSON)
.then(json => console.log(json));
})
}
async function safeParseJSON(response) {
const body = await response.text();
try {
return JSON.parse(body);
} catch (err) {
console.error("Error:", err);
console.error("Response body:", body);
}
}
A lot of the solutions online talk about limiting the size using express. I'm only using Nodejs to do this because it's going to be part of an Alexa Skill once it works, so express won't work. Are there any ideas on how I can send the binary of a file as a POST request?
Thank you.

Imgur upload api error always returning 400 with multer and form-data nodejs

I'm using react antd to upload pictures. Here's my express handling the upload request:
router.post('/upload-image', upload.any(), async function(req, res, next) {
const file = req.files[0];
const form = new FormData();
form.append('image', file.buffer);
form.append('type', 'file');
const imgurInstance = new imgur({
refreshToken: user.imgurRefreshToken,
clientId: IMGUR_CLIENT_ID,
clientSecret: IMGUR_CLIENT_SECRET
});
await imgurInstance.getAccessToken();
const response = await imgurInstance.uploadImage(form);
const data = await response.json();
});
Here's the req.files[0] console logged:
{
fieldname: 'file',
originalname: 'headscratch.jpeg',
encoding: '7bit',
mimetype: 'image/jpeg',
buffer: <Buffer ff d8 ff e0 00 10 4a 46 49 46 00 01 01 01 00 01 00 01 00 00 ff e1 00 42 45 78 69 66 00 00 4d 4d 00 2a 00 00 00 08 00 01 87 69 00 04 00 00 00 01 00 00 ... 13803 more bytes>,
size: 13853
}
And here's my uploadImage function using node-fetch
uploadImage (form){
const path = `/3/upload`;
const headers = {
'Authorization': `Bearer ${this.accessToken}`,
...form.getHeaders()
};
const options = {
method: 'POST',
body: form,
headers
};
return fetch(`${this.BASE_API_URL}${path}`, options);
}
I always get a 400 from imgur and they don't provide any details why.
{
status: 400,
success: false,
data: { error: 'Bad Request', request: '/3/upload', method: 'POST' }
}
I have tried using base64, just uploading a url using form-data and I still get a 400 error. Does anybody have any guidance on how to successfully make this call work? Thanks.
I guess I needed to understand what form-data is actually doing. Apparently form-data won't send as a file unless you include the filename with {filename: ''} as an option
form.append('image', file.buffer, {filename: file.originalname});
imgur finally accepted my request and voila.

upload file is not storing the destination folder using multer in nodejs

router.post('/uploadDocuments', function (req, res) {
console.log("uploadDocuments");
console.log(req.files.file);
var storage = multer.diskStorage({
destination: (req, file, callback) => {
callback(null, './temp')
},
filename: (req, file, callback) => {
callback(null, file.fieldname + '-' + Date.now() + path.extname(file.originalname))
}
});
const upload = multer({ storage: storage });
});
In console i can see the upload file details
{
name: 'doc-file.jpg',
data: <Buffer ff d8 ff e0 00 10 4a 46 49 46 00 01 01 00 00 01 00 01 00 00 ff db 00 84 00 07 07 07 07 07 07 08 08 08 08 0b 0b 0a 0b 0b 10 0e 0d 0d 0e 10 18 11 12 11 ... 93509 more bytes>,
size: 93559,
encoding: '7bit',
tempFilePath: '',
truncated: false,
mimetype: 'image/jpeg',
md5: '19f413e98d9f275691ffd6b53062cf88',
mv: [Function: mv]
}
From postman to upload a file getting issues.I checked the folder permission also
Actually you need to specify the handler like the following.
const upload = multer({ storage: storage }).any();
Live Demo
FYI:
multer(options) is an object
multer(options).any() is a function
Other handlers
.single(fieldname) -> Accept a single file with the name fieldname
.array(fieldname[, maxCount]) -> Accept an array of files, all with the name fieldname
.fields(fields) -> Accept a mix of files, specified by fields
.none() -> Accept only text fields.
.any() -> Accepts all files that comes over the wire.
References
Mutter - NPM
Should I recommend using Formidable.
That would be:
const formidable = require('formidable');
const form = new formidable.IncomingForm();
form.uploadDir = 'Specify your folder';
(by default takes the folder at root directory);
This will automatically upload your image to your desired folder.

express-fileupload to Google Drive API

I can successfully save my uploaded image in my public uploads folder and integrate google drive api but it seems that it is uploading an empty file.
What should I put in the body parameter of the Google Drive API from my req.files data
{name: 'country.jpg',data: <Buffer ff d8 ff e0 00 10 4a 46 49 46 00 01 01 01 00 48 00 48 00 00 ff ed 26 0c 50 68 6f 74 6f 73 68 6f 70 20 33 2e 30 00 38 42 49 4d 04 04 00 00 00 00 00 22 ... 246290 more bytes>,size: 246340,encoding: '7bit',tempFilePath: '',truncated: false,mimetype: 'image/jpeg',md5: '8890c8336c58d854d490b41fa6ec0ad4',mv: [Function: mv]}
Here's my Google Drive API call after auth.
const drive = google.drive({ version: "v3", auth });
drive.files.create({
media: {
mimeType: "image/jpeg",
body: // WHAT TO PUT HERE
},
resource: {
name: "photo.jpg"
// if you want to store the file in the root, remove this parents
//parents: ['folder id in which he file needs to be stored.']
},
fields: "id"
})
.then(function(resp) {
//console.log("RESPONSE");
console.log(resp, "resp");
})
.catch(function(error) {
console.log("ERROR");
console.log(error);
});
Just like #DalmTo mentioned, you need to send your data to the Google Drive API somehow.
If you were sending a file in your filsystem you could use the code she provided:
var media = {
mimeType: 'image/jpeg',
body: fs.createReadStream(FILEPATH)
};
However, since you are not trying to save the file to your filesystem before uploading you have to adapt to your situation.
I'm assuming this files reaches your application thru a form upload since you are using Express. If you can user multer as well, you can get your files as a Buffer
You can then convert that Buffer to a Stream.
Your code for treating this would look like this:
Copyvar Readable = require('stream').Readable;
function bufferToStream(buffer) {
var stream = new Readable();
stream.push(buffer);
stream.push(null);
return stream;
}
app.post('/upload', upload.single('picture'), function (req, res, next) {
// req.file is the `picture` file
var media = {
mimeType: 'image/jpeg',
body: bufferToStream(req.file.buffer)
};
})
Thats because you are only uploading the file metadata you need to actually upload the file.
var media = {
mimeType: 'image/jpeg',
//PATH OF THE FILE FROM YOUR COMPUTER
body: fs.createReadStream('C:/Users/me/Downloads/photo.jpg')
};

Resources