image cannot able to read using jimp from s3 bucket & lambda - node.js

I'm using jimp plugin to read file & trying to optimize images. I can able to read file & optimize using public image url's.
The issue is while reading image from s3 bucket. I can able to fetch image from s3 bucket, while passing the image to jimp.read(buffer) as buffer it's not working and not able to get any error as well.
Here my code:
module.exports.handler = async (event, context, callback) => {
// Get the object from the event and show its content type
const bucket = event.Records[0].s3.bucket.name;
const key = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, " "));
const params = {
Bucket: bucket,
Key: key,
};
console.log("params ::::::::::", params);
try {
let objectId = uuidv4();
let objectKey = `resize-${width}x${height}-${objectId}.jpg`;
const origimage = await s3.getObject(params).promise();
console.log("origimage ====>", origimage.Body);
Jimp.read(origimage.Body)
.then(image => image.resize(width, height)
.getBufferAsync(imageType))
.then(resizedBuffer => uploadToS3(resizedBuffer, objectKey))
.then(function(response) {
console.log(`Image ${objectKey} was uploaed and resized`);
callback(null, {
statusCode: 200,
body: JSON.stringify(response)
});
})
.catch(error => console.log(error));
} catch (err) {
console.log(err);
const message = `Error getting object ${key} from bucket ${bucket}. Make sure they exist and your bucket is in the same region as this function.`;
console.log(message);
throw new Error(message);
}
here is the response which i got from s3 bucket:
2022-10-13T09:19:25.099Z 924b89b2-69ab-467e-8143-6f9e22e5e67a INFO CONTENT DATA: {
AcceptRanges: 'bytes',
LastModified: 2022-10-12T11:58:38.000Z,
ContentLength: 331338,
ETag: '"917eba20c634253a97ff6dfe3958db0a"',
ContentType: 'image/jpeg',
Metadata: {},
Body: <Buffer ff d8 ff e0 00 10 4a 46 49 46 00 01 01 00 00 01 00 01 00 00 ff db 00 43 00 03 02 02 02 02 02 03 02 02 02 03 03 03 03 04 06 04 04 04 04 04 08 06 06 05 ... 331288 more bytes>
}
Is there anything that i'm missing while sending buffer data to jimp.read function. I have tried passing imagename & base64 as well, still no luck.
Is there anyway i can access s3 bucket image using image url with in lambda fuction?
Thanks in advance

It doesn't look like you're waiting for the promise that Jimp.read(buffer) creates to resolve/reject.
Try await Jimp.read(...) and you should get something back in the way of the promise resolving/rejecting, triggering some of your logging.
Equally, you could also do return Jimp.read(...) and Lambda will wait for that promise to complete before ending execution.

Related

How to make request to amazon s3 using node js for downloading a file?

I made a request to amazon s3 using node js for downloading a file. In the response i receive the Buffer, here i have 2 problems:
If i send the buffer to frontend from node js like res.send(awsreq.Body.Buffer) and console log it once in node js and once in frontend, the buffer from frontend will look different than in node js.
Node js Buffer: <Buffer 50 4b 03 04 14 00 06 00 08 00 00 00 21 00 df a4 d2 6c 5a 01 00 00 20 05 00 00 13 00 08 02 5b 43 6f 6e 74 65 6e 74 5f 54 79 70 65 73 5d 2e 78 6d 6c 20 ... 11906 more bytes>
Vue js frontend buffer: PK\u0003\u0004\u0014\u0000\u0006\u0000\b\u0000\u0 Show more(36,8kb)
How can i directly download the file from a request made from frontend? First how to receive that buffer corecly and how to convert it in that way frontend will download the file automatically? If possible how to do all of that in node js and when frontend receive the response to start automatically the download?
To Download a file from amazon S3 follow these steps
Install aws sdk using this command npm install aws-sdk
Check below code
var AWS = require("aws-sdk")
const s3 = new AWS.S3({
endpoint: "ENDPOINT",
accessKeyId: "YOUR_ACCESS_KEY_ID",
secretAccessKey: "YOUR_SECRET_ACCESS_KEY",
region: "us-east-1",
signatureVersion: "v4"
})
const downloadParams = {
Bucket: "BUCKET_NAME",
Key: "KEY_NAME/FILE_NAME/FILE_PATH"
}
// Download the file
s3.getObject(downloadParams, function (error, fileData) {
if (!error) {
console.log(fileData, "file")
} else {
console.log(error, " ERROR ")
}
})
For more information you can check AWS official documentation using this link:
https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/getting-started-nodejs.html
At last you can use
Content-Disposition in the request header
Or you can use pre Signed url
Example:
const getpreSignedUrlForDownload = async () => {
try {
const parms = {
Bucket: "BUCKET_NAME",
Key: "FILE_KEY",
Expires: 60 * 1,
ResponseContentDisposition: 'attachment; filename"' + "FILE_NAME + '"'
}
return new Promise((resolve, reject) => {
s3.getSignedUrl("getObject", parms, (err, url) => {
err ? reject(err) : resolve(url)
})
})
} catch (error) {
throw new Error(error)
}
}
getpreSignedUrlForDownload()

Imgur upload api error always returning 400 with multer and form-data nodejs

I'm using react antd to upload pictures. Here's my express handling the upload request:
router.post('/upload-image', upload.any(), async function(req, res, next) {
const file = req.files[0];
const form = new FormData();
form.append('image', file.buffer);
form.append('type', 'file');
const imgurInstance = new imgur({
refreshToken: user.imgurRefreshToken,
clientId: IMGUR_CLIENT_ID,
clientSecret: IMGUR_CLIENT_SECRET
});
await imgurInstance.getAccessToken();
const response = await imgurInstance.uploadImage(form);
const data = await response.json();
});
Here's the req.files[0] console logged:
{
fieldname: 'file',
originalname: 'headscratch.jpeg',
encoding: '7bit',
mimetype: 'image/jpeg',
buffer: <Buffer ff d8 ff e0 00 10 4a 46 49 46 00 01 01 01 00 01 00 01 00 00 ff e1 00 42 45 78 69 66 00 00 4d 4d 00 2a 00 00 00 08 00 01 87 69 00 04 00 00 00 01 00 00 ... 13803 more bytes>,
size: 13853
}
And here's my uploadImage function using node-fetch
uploadImage (form){
const path = `/3/upload`;
const headers = {
'Authorization': `Bearer ${this.accessToken}`,
...form.getHeaders()
};
const options = {
method: 'POST',
body: form,
headers
};
return fetch(`${this.BASE_API_URL}${path}`, options);
}
I always get a 400 from imgur and they don't provide any details why.
{
status: 400,
success: false,
data: { error: 'Bad Request', request: '/3/upload', method: 'POST' }
}
I have tried using base64, just uploading a url using form-data and I still get a 400 error. Does anybody have any guidance on how to successfully make this call work? Thanks.
I guess I needed to understand what form-data is actually doing. Apparently form-data won't send as a file unless you include the filename with {filename: ''} as an option
form.append('image', file.buffer, {filename: file.originalname});
imgur finally accepted my request and voila.

Why does my request to the JIRA api to create an attachment return an empty array

I'm having trouble getting my request to jira cloud to add an attachment to an issue. It returns 200 but the response is an empty array.
const formData = new FormData();
for(var i = 0; i < files.length; i++){
let file = files[i];
formData.append("file", file.content, file.name);
}
console.log(formData); // see below
let params = {
method: 'post',
url: `https://submissive.atlassian.net/rest/api/3/issue/${createKey}/attachments`,
data: formData.getBuffer(),
headers: {
Accept: "application/json",
'X-Atlassian-Token': 'no-check',
'Authorization': getAuthHeader(user, apiToken),
... formData.getHeaders()
}
};
console.log('params: ', params);
let response = await axios(params);
console.log(response);
console.log(response.data); // [] empty array
console.log('attachments added');
formdata:
FormData {
_overheadLength: 143,
_valueLength: 2946927,
_valuesToMeasure: [],
writable: false,
readable: true,
dataSize: 0,
maxDataSize: 2097152,
pauseStreams: true,
_released: false,
_streams: [
'----------------------------470840621872458708605830\r\n' +
'Content-Disposition: form-data; name="file"\r\n' +
'Content-Type: application/octet-stream\r\n' +
'\r\n',
<Buffer ff d8 ff e0 00 10 4a 46 49 46 00 01 01 00 01 2c 01 2c 00 00 ff e1 28 3a 45 78 69 66 00 00 4d 4d 00 2a 00 00 00 08 00 06 01 06 00 03 00 00 00 01 00 02 ... 2946877 more bytes>,
[Function: bound ]
],
_currentStream: null,
_insideLoop: false,
_pendingNext: false,
_boundary: '--------------------------470840621872458708605830'
}
Is it a problem the length is 0? I've tried adding the forma data with and without the extension method with the same results. I've confirmed I can add an attachment with the account I built my API credentials from. The file name is correct: 'file'. My only thought is maybe im ussing buffers incorrectly but the form log above makes it look correct.
In case this helps anyone. I was having the same issue and the problem was with the call to formData.append(). The third argument needs to be an object.
// get the attachment file
// what I originally had
formData.append("file", attachment.Body) // in my case the attachment.Body is the buffer
// what fixed the issue
const fileInfo = {
filename: fileName, // e.g. MyDoc.pdf
contentType: attachment.ContentType, // e.g. application/pdf
knownLength: attachment.ContentLength, // e.g. 18059464
}
formData.append("file", attachment.Body, fileInfo)

upload file is not storing the destination folder using multer in nodejs

router.post('/uploadDocuments', function (req, res) {
console.log("uploadDocuments");
console.log(req.files.file);
var storage = multer.diskStorage({
destination: (req, file, callback) => {
callback(null, './temp')
},
filename: (req, file, callback) => {
callback(null, file.fieldname + '-' + Date.now() + path.extname(file.originalname))
}
});
const upload = multer({ storage: storage });
});
In console i can see the upload file details
{
name: 'doc-file.jpg',
data: <Buffer ff d8 ff e0 00 10 4a 46 49 46 00 01 01 00 00 01 00 01 00 00 ff db 00 84 00 07 07 07 07 07 07 08 08 08 08 0b 0b 0a 0b 0b 10 0e 0d 0d 0e 10 18 11 12 11 ... 93509 more bytes>,
size: 93559,
encoding: '7bit',
tempFilePath: '',
truncated: false,
mimetype: 'image/jpeg',
md5: '19f413e98d9f275691ffd6b53062cf88',
mv: [Function: mv]
}
From postman to upload a file getting issues.I checked the folder permission also
Actually you need to specify the handler like the following.
const upload = multer({ storage: storage }).any();
Live Demo
FYI:
multer(options) is an object
multer(options).any() is a function
Other handlers
.single(fieldname) -> Accept a single file with the name fieldname
.array(fieldname[, maxCount]) -> Accept an array of files, all with the name fieldname
.fields(fields) -> Accept a mix of files, specified by fields
.none() -> Accept only text fields.
.any() -> Accepts all files that comes over the wire.
References
Mutter - NPM
Should I recommend using Formidable.
That would be:
const formidable = require('formidable');
const form = new formidable.IncomingForm();
form.uploadDir = 'Specify your folder';
(by default takes the folder at root directory);
This will automatically upload your image to your desired folder.

express-fileupload to Google Drive API

I can successfully save my uploaded image in my public uploads folder and integrate google drive api but it seems that it is uploading an empty file.
What should I put in the body parameter of the Google Drive API from my req.files data
{name: 'country.jpg',data: <Buffer ff d8 ff e0 00 10 4a 46 49 46 00 01 01 01 00 48 00 48 00 00 ff ed 26 0c 50 68 6f 74 6f 73 68 6f 70 20 33 2e 30 00 38 42 49 4d 04 04 00 00 00 00 00 22 ... 246290 more bytes>,size: 246340,encoding: '7bit',tempFilePath: '',truncated: false,mimetype: 'image/jpeg',md5: '8890c8336c58d854d490b41fa6ec0ad4',mv: [Function: mv]}
Here's my Google Drive API call after auth.
const drive = google.drive({ version: "v3", auth });
drive.files.create({
media: {
mimeType: "image/jpeg",
body: // WHAT TO PUT HERE
},
resource: {
name: "photo.jpg"
// if you want to store the file in the root, remove this parents
//parents: ['folder id in which he file needs to be stored.']
},
fields: "id"
})
.then(function(resp) {
//console.log("RESPONSE");
console.log(resp, "resp");
})
.catch(function(error) {
console.log("ERROR");
console.log(error);
});
Just like #DalmTo mentioned, you need to send your data to the Google Drive API somehow.
If you were sending a file in your filsystem you could use the code she provided:
var media = {
mimeType: 'image/jpeg',
body: fs.createReadStream(FILEPATH)
};
However, since you are not trying to save the file to your filesystem before uploading you have to adapt to your situation.
I'm assuming this files reaches your application thru a form upload since you are using Express. If you can user multer as well, you can get your files as a Buffer
You can then convert that Buffer to a Stream.
Your code for treating this would look like this:
Copyvar Readable = require('stream').Readable;
function bufferToStream(buffer) {
var stream = new Readable();
stream.push(buffer);
stream.push(null);
return stream;
}
app.post('/upload', upload.single('picture'), function (req, res, next) {
// req.file is the `picture` file
var media = {
mimeType: 'image/jpeg',
body: bufferToStream(req.file.buffer)
};
})
Thats because you are only uploading the file metadata you need to actually upload the file.
var media = {
mimeType: 'image/jpeg',
//PATH OF THE FILE FROM YOUR COMPUTER
body: fs.createReadStream('C:/Users/me/Downloads/photo.jpg')
};

Resources