I am working on uploading PDF files to S3 using nodejs and react and I am running into an issue where some PDFs are being uploaded and some are not.
This endpoint gets a signed url from AWS
'/api/v1/upload/pdf',
requireAuth,
roleAuthorization(['admin']),
(req, res) => {
const date = new Date();
const year = date.getFullYear();
const key = `${date.toLocaleString('default', {
month: 'long',
})}-${short.uuid('0123456789').slice(0, 2)}-${year}.pdf`;
s3.getSignedUrl(
'putObject',
{
Bucket: 'bucket-name',
ContentType: 'application/pdf',
Key: key,
},
(err, url) => res.send({ key, url })
);
}
);
And this endpoint does the upload from react js and save the link to the file in the database
const createIssue = (dispatch) => async (issue, town) => {
try {
const token = await localStorage.getItem('token');
const uploadConfig = await axios.get('/api/v1/upload/pdf', {
headers: {
Authorization: `Bearer ${token}`,
},
});
const upload = await axios.put(uploadConfig.data.url, issue, {
headers: {
'Content-Type': issue.type,
},
});
const response = await api.post(`/api/v1/town/${town._id}/issue`, {
issue: uploadConfig.data.key,
});
dispatch({ type: 'create_issue', payload: response.data });
} catch (err) {
dispatch({
type: 'add_error',
payload: err.message,
});
}
};
This works but does not work for all PDF files, the file remains pending and does not upload.
Any help welcome.
thanks
Related
Im unable to send blob pdf file that comes from #react-pdf/render.
first I'm tring to convert that blob into a file using new File()
<BlobProvider
document={<Document />}
>
{({ blob, url, loading, error }) => {
buildPdfFile(blob);
return <div />;
}}
</BlobProvider>
const fileRef = useRef<File | null>(null);
const buildPdfFile = (blob: any) => {
const file = new File(
[blob],
`${get(resumeData, "ownerName", "")}_${get(
resumeData,
"ownerId",
""
)}_ficha_de_inscripción.pdf`,
{
type: "application/pdf",
}
);
fileRef.current = file;
console.log(fileRef.current);
};
const handleOnSubmit = () => {
dispatch(sendPdfToServer(fileRef.current!));
};
once I got that file I'm tried to send it using formdata in a POST request with application/pdf as content-type to my nestjs app
const sendPdfToServer = (inscriptionPdf) => {
const jwt = getJWT();
const options = {
headers: new Headers({
"content-type": "application/pdf",
Authorization: `Bearer ${jwt}`,
}),
};
const formData = new FormData();
formData.append("file", inscriptionPdf, inscriptionPdf.name);
const path = `${url}`;
try {
const response = await fetch(path, {
...options,
method: "POST",
body: formData,
});
}
catch (e) {
console.log(e);
}
}
but in the endpoint I'm using, the file is never intercepted, it shows as undefined
#Post('sendMail')
#UseInterceptors(
FileInterceptor('file', {
storage: diskStorage({
destination: './uploads/emailsTemporalFiles',
filename: (req, file, cb) => {
console.log('file ->', file);
const fileName: string = path
.parse(file.originalname)
.name.replace(/\s/g, '');
const extension: string = path.parse(file.originalname).ext;
cb(null, `${fileName}${extension}`);
},
}),
}),
)
async sendMail(#Res() response, #UploadedFile() file) {
this.logger.log(` | sendMail`);
console.log(file); // it prints undefined
}
Scenario
Front end is basically a React Native (Expo) application where users can issue a report - this includes taking multiple photos and filling in some details.
Backend is just node.js, with Express & Multer.
Problem
I use Axios to send the images and form data through FormData(), however on the server side, req.body and req.files consist of nothing.
One thing here is that sending the SAME data through POSTMAN works COMPLETELY fine, the images were stored into S3 and the form details were stored in the database. It is through the app/emulator that does not work.
I've tried removing the "multipart/form-data" header and this is the output from console.log(req.body) (req.files show undefined):
{
_parts: [
[ 'userId', '1f400069-07d0-4277-a875-cbb2807750c5' ],
[
'location',
'some location'
],
[ 'description', 'Aaaaa' ],
[ 'report-images', [Object] ]
]
}
When I put the "multipart/form-data" header back this output didn't even bother showing up.
What I've Done
I've been searching for solutions for the past hours and none of them worked. Those solutions are:
Adding boundary behind the "multipart/form-data" header
Putting the type to "image/jpeg"
Trimming the file uri to "file://"
Yet none of them works
Here is my code:
React Native Frontend (Expo)
const submitReport = async () => {
setShowLoading(true);
// Validate details (location & images)
if (location === "") {
setShowLoading(false);
showToast(7002);
return;
}
if (images.length === 0) {
setShowLoading(false);
showToast(7004);
return;
}
try {
const formData = new FormData();
formData.append("userId", user.userId);
formData.append("location", location);
formData.append("description", description);
images.forEach(img => {
const trimmedURI = (Platform.OS === "android") ? img.uri : img.uri.replace("file://", "");
const fileName = trimmedURI.split("/").pop();
const media = {
name: fileName,
height: img.height,
width: img.width,
type: mime.getType(trimmedURI),
uri: trimmedURI
};
formData.append("report-images", media);
});
const response = await axios.post(`http://<my-ip-address>:3003/api/report/submit`, formData, { headers: { 'Content-Type': "application/x-www-form-urlencoded" } });
console.log(response)
setShowLoading(false);
}
catch (error) {
console.log(error);
setShowLoading(false);
showToast(9999);
}
};
Backend
// Multer-S3 Configuration
const upload = multer({
storage: multerS3({
s3: s3,
bucket: process.env.AWS_S3_BUCKET_NAME,
contentType: (req, file, callback) => {
callback(null, file.mimetype);
},
metadata: (req, file, callback) => {
callback(null, { fieldName: file.fieldname });
},
key: (req, file, callback) => {
callback(null, `${process.env.AWS_S3_REPORT_IMAGES_OBJECT_PATH}${req.body.userId}/${new Date().getTime().toString()}-${file.originalname}`)
}
}),
fileFilter: (req, file, callback) => {
// Check if file formats are valid
if (file.mimetype === "image/png" || file.mimetype === "image/jpg" || file.mimetype === "image/jpeg") {
callback(null, true);
}
else {
callback(null, false);
return callback(new Error("Image File Type Unsupported"));
}
},
});
router.post("/submit", upload.array("report-images", 3), async (req, res) => {
try {
// req -> FormData consisting of userId, location & description
// multer-s3 library will handle the uploading to S3 - no need to code here
// Details of files uploaded on S3 (Bucket, Key, etc.) will be displayed in req.files
// Analyze from Rekognition
//Add to Database code blablabla
if (result.success === true) {
res.status(200).send({ message: result.data });
}
else if (result.success === false) {
res.status(404).send({ error: ERROR_MESSAGE });
}
}
catch (error) {
console.log(error);
res.status(404).send({ error: ERROR_MESSAGE });
}
});
I'm unsure if this an Axios problem or some problem on my side.
This project is for my Final Year Project.
So after diving through search results in Google, I've found this StackOverflow post: react native post form data with object and file in it using axios
I took the answer provided by user_2738046 in my code and it worked! Combining with Ali's suggestion here is the final code that worked.
const FormData = global.FormData;
const formData = new FormData();
formData.append("userId", user.userId);
formData.append("location", location);
formData.append("description", description);
images.forEach(img => {
const trimmedURI = (Platform.OS === "android") ? img.uri : img.uri.replace("file://", "");
const fileName = trimmedURI.split("/").pop();
const media = {
name: fileName,
height: img.height,
width: img.width,
type: mime.getType(trimmedURI),
uri: trimmedURI
};
formData.append("report-images", media);
});
const response = await axios({
method: "POST",
url: `http://${<my-ip-address>}:3003/api/report/submit`,
data: formData,
headers: {
'Content-Type': 'multipart/form-data'
},
transformRequest: (data, error) => {
return formData;
}
});
// If success, clear all text fields
if (response) {
showToast(7005);
setLocation("");
setImages([]);
setDescription("");
}
setShowLoading(false);
You need to change your image uploading code with this one, you also need to install mime npm package.
const formData = new FormData();
formData.append("userId", user.userId);
formData.append("location", location);
formData.append("description", description);
const formData = new FormData();
files = files || [];
if (files.length) {
for (let index = 0; index < files.length; index++) {
const filePayload = files[index];
const file = filePayload.value;
const localUri =
Platform.OS === "android" ?
file.uri :
file.uri.replace("file://", "");
const newImageUri = localUri;
const filename = newImageUri.split("/").pop();
const media = {
name: filename,
height: file?.height,
width: file?.width,
type: mime.getType(newImageUri),
uri: localUri,
};
formData.append(filePayload.name, media);
}
}
const response = await axios.post(`http://<my-ip-address>:3003/api/report/submit`, formData, {
headers: headers: {
"Content-Type": "application/x-www-form-urlencoded",
},
});
Within my app a user can select a profile image and i would like that image to be uploaded to an s3 bucket when the user saves their profile data
I pass the image data (and json, which consists of name, email, telephone for example) from my app to an express server and upload there
At present I can pass the image data (the url it seems at present) to an s3 bucket and it saves
I don't think i'm actually saving the image itself though, as when downloading from s3 (manually) and trying to open on my mac it states it may be damaged and i cannot see the image
Feel daft for asking but how do i actually upload the image itself? Thanks
React Native Side
const handleFormSubmit = formData => {
const jsonData = JSON.stringify({
...formData,
});
// Handle profile image
if (imageProps && imageProps.uri) {
const data = new FormData();
data.append('formBody', jsonData);
data.append('image', {
uri:
Platform.OS === 'android'
? imageProps.uri
: imageProps.uri.replace('file://', ''),
type: imageProps.type,
name: imageProps.fileName,
});
sendRequest(data);
} else {
sendRequest(jsonData);
}
};
const sendRequest = data => {
let responseData;
fetch('http://localhost:8080/users/api/update_user_profile', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Accept: 'application/json',
},
body: data,
})
.then(response => {
responseData = response;
return response.json();
})
.then(jsonData => {
console.log(jsonData)
})
.catch(error => {
console.log(error)
});
};
Server Side
const s3 = new AWS.S3({
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
});
// Setting up S3 upload parameters
const params = {
Bucket: 'bucket-folder',
ACL: 'public-read',
Key: req.files.image.name,
Body: req.files.image.path
};
const stored = await s3.upload(params).promise();
You can use Multer for uploading files to s3.
const multer = require('multer');
const AWS = require('aws-sdk');
const uniqid = require('uniqid');
const storage = multer.memoryStorage();
const upload = multer({ storage });
// ? Posts new file to amazon and saves to db
router.post(
'/:id',
upload.single('attachment'),
async (req, res) => {
const unique = uniqid.time();
const { file } = req;
const { filePath } = req.body;
const { id } = req.params;
const s3FileURL = process.env.AWS_UPLOADED_FILE_URL;
const region = process.env.AWS_REGION;
const secretAccessKey = process.env.AWS_SECRET_ACCESS_KEY;
const accessKeyId = process.env.AWS_ACCESS_KEY_ID;
const Bucket = process.env.AWS_BUCKET_NAME + '/' + filePath;
const Key = `${id}/${unique}-${file.originalname}`;
const Body = file.buffer;
const ContentType = file.mimetype;
const ACL = 'public-read';
const s3bucket = new AWS.S3({
accessKeyId,
secretAccessKey,
region,
});
const params = {
Bucket,
Key,
Body,
ContentType,
ACL,
};
s3bucket.upload(params, async (err, data) => {
if (err) {
res.status(500).json({ error: true, Message: err });
} else {
console.log(params);
const newFileUploaded = {
description: req.body.description,
fileLink: `${s3FileURL}${filePath}/${id}/${unique}-${file.originalname}`,
s3_key: params.Key,
};
try {
const response = await postFile({
name: req.body.name,
attachment: newFileUploaded,
alt: req.body.alt,
user: req.body.user,
relatedID: req.body.relatedID,
});
res.status(200).json({
message: response.message,
success: response.success,
result: response.result,
});
} catch (e) {
res.status(500).json({
message:
'File upoladed but Db couldnt saved request (upload by ID)',
success: false,
result: [],
});
}
}
});
}
);
This is the front-end logic , data is react-hook-form onSubmit data.
draftURL is type FileList. Request is successful but req.body is of type string.
const uploadData = new FormData();
uploadData.append("draft", data.draftURL[0], data.draftURL[0].name);
const response = await axios.post("/api/fileUpload", uploadData, {
headers: {
accept: "application/json",
"Content-Type": `multipart/form-data`,
},
});
I saw multiple solutions using next-connect and multer , but they did not work very well for me. And I am also confused about why is this not working.
async function handler(req: NextApiRequest, res: NextApiResponse) {
console.log(typeof req.body);
const params = {
Bucket: process.env.AWS_BUCKET,
Key: "cat.pdf", // File name you want to save as in S3
Body: req.body,
conditions: [{ acl: "public-read" }],
};
// Uploading files to the bucket
s3.upload(params, function (err, data) {
if (err) {
throw err;
}
console.log(`File uploaded successfully. ${data.Location}`);
res.json(data.Location);
});
}
Redux Code
export const uploadImage = data => async dispatch => {
const config = {
headers: {
"Content-Type": "multipart/form-data"
}
};
try {
const res = await axios.post("/api/profile/upload", data, config);
// const res = await axios.post("/api/profile/upload", data, {
// headers: {
// "Content-Type": "multipart/form-data"
// }
// });
console.log(res);
dispatch({
type: AVATAR_UPLOAD,
payload: res.data
});
} catch (err) {
const errors = err.response.data.errors;
if (errors) {
errors.forEach(error => dispatch(setAlert(error.msg, "danger")));
}
dispatch({
type: PROFILE_ERROR,
payload: { msg: err.response.statusText, status: err.response.status }
});
}
};
React Code
const data = new FormData();
data.append("file", avatar);
uploadImage(data);
I get bad request message, req.files is empty
I try to upload file with Postman same configuration,it seems api works but can't upload with react formdata.
Any idea would be appreciated. Thanks in advance
try doing this:
const formData = new FormData();
formData.append('file', file);
const res = await axios.post("/api/profile/upload", formData, config);