Using multipart/form-data in Nestjs - nestjs

I have a requirement of uploading a pdf file to the a third-party API to do that what we are using a controller and then sending its buffer to the endpoint in multipart/form-data but It's not working. What am i doing wrong??
I have my controller defined like
#Post('/upload')
#UseInterceptors(FileInterceptor('file'))
async uploadDocument(#UploadedFile() file: Express.Multer.File){
await this.httpRequestService.uploadDocument(file)
return [
Translation.translate('Create_Success_Message', {
Entity: 'File'
}),
{
file
}
]
}
This controller calls a service called uploadDocument which looks like:
async uploadDocument(file){
try{
const formData = new FormData()
formData.append('file', file)
const response = await this.httpService.post(`urlWhereItHits`,
formData,
{
headers:{
"X-AppSecretToken":"appsecrettoken",
"X-AgreementGrantToken":"agreementgranttoken",
'Content-Type' : 'multipart/form-data'
},
}).toPromise();
}
catch(error){
console.log(error)
}
}
Now i don't understand what's happening wrong here.
I get error saying
TypeError: source.on is not a function at Function.DelayedStream.create (C:\Users\Tushar\Squire-backend\node_modules\delayed-stream\lib\delayed_stream.js:33:10)

I had a similar task, receiving a .csv file and sending it to another server.
controller:
#UseInterceptors(FileInterceptor('file'))
#Post('upload')
async uploadFile(#UploadedFile() file: Express.Multer.File) {
await this.myService.uploadFile(file);
return {
message: 'File uploaded successfully.',
};
}
service:
import * as FormData from 'form-data';
import { Readable } from 'stream';
async uploadFile(file: Express.Multer.File) {
const form = new FormData();
form.append('files', Readable.from(file.buffer), {
filename: file.originalname,
});
try {
const result = await lastValueFrom(
this.httpService.post('The URL of your third-party service', form, {
headers: {
...form.getHeaders(),
},
}),
);
if (result.status === 200) return;
} catch {
throw new InternalServerErrorException(
'Error while uploading the .csv file.',
);
}
}
Some explanations here:
It requires importing Formdata from the package.
.append() requires a fieldName, and a readable Stream.
You can use the buffer of the file that's on memory (not recommended with large files) and use it to create a readable stream.
Also, you need to add the name of the file manually since Formdata only recognizes the filename if the file is saved on disk. (Which in our case is on memory as a Buffer).
You can create the stream in different ways, the example is not mandatory.
.getHeaders() will generate the default header 'Content-Type': 'multipart/form-data' and also it will generate the boundary of the file, like this: reference
{
'content-type': 'multipart/form-data; boundary=--------------------------286308325485491440714225'
}

Related

Cannot upload document via Telegram bot API

I'm using NestJS, there is axios module inside it. I read that Telegram accepts only multipart/form-data for files, so I use form-data package. I have only 400 errors, that don't represent anything.
I don't want to use packages like telegraf (it's awesome, I know) for this simple usecase.
const config: AxiosRequestConfig = {
headers: { 'Content-Type': 'multipart/form-data' },
};
const file = await readFile(path);
const body = new FormData();
body.append('chat_id', this.chatId.toString());
body.append('document', file, { filename: 'document.pdf' });
try {
const res = await firstValueFrom(this.http.post(`${this.tgURL}/sendDocument`, body, config));
} catch (e) {
console.log(e.message);
}
Telegram needs boundary in header.
So:
const config: AxiosRequestConfig = {
headers: { 'Content-Type': `multipart/form-data; boundary=${body.getBoundary()}` },
};

Convert buffer to readStream

I have endpoint which is receiving files and need to send these files to 3rd party. However i'm not converting these files properly cuz i receive Invalid request format but if i read the file from fs the file is successfully deployed. How to create readStream from the input file? I tried Readable.from(file.buffer) still error. If i pass createReadStream(join(process.cwd(), 'images...')) is working as expected, here is the script.
export const deployFile = async (
file: Express.Multer.File,
httpService: HttpService,
): Promise<string> => {
const formData = new FormData();
// Working
// formData.append('file', createReadStream(join(process.cwd(), '/images/1.jpeg')));
// Not working
formData.append('file', Readable.from(file.buffer));
try {
const observable = httpService
.post(process.env.PINATA_BASE_URL + '/pinning/pinFileToIPFS', formData, {
headers: {
'Content-Type': `multipart/form-data; boundary=${formData.getBoundary()}`,
pinata_api_key: process.env.PINATA_API_KEY,
pinata_secret_api_key: process.env.PINATA_API_SECRET_KEY,
},
})
.pipe(map((response) => response.data));
const response = await lastValueFrom(observable);
return response.IpfsHash;
} catch (error) {
logger.error(`Error deploying image reason: ${error.response.data.error}`);
}
};

why does a file written using writfFileSync method is corrupted for a video file in nodejs

I am uploading a video file to my s3 bucket using a presigned URL using the following code
function postsignedURL(req) {
return new Promise(function (resolve, reject) {
const params = {
Bucket: 'bucket1',
Expires: 60 * 60, // in seconds,
Fields: {
key: req,
'Content-Type': 'video/'
},
Conditions: [
['content-length-range', 300, 4000e+7],
{ 'Content-Type': 'video/' },
["starts-with", "$Content-Type", "video/"]
]
}
s3.createPresignedPost(params, (err, data) => {
if (err) {
reject(err)
} else {
resolve(data);
}
})
})
}
I am able to upload the file correctly and I can even download it using the following code in
const video = await readObjectSignedUrl(key)
console.log(video, " video ");
This one returns a presigned url. I can download the video using fetch method below
const objectResult = await fetchS3Object(video);
const bufferType = Buffer.from(objectResult);
const fileTypeResult = await fileType.fromBuffer(bufferTy[e);
console.log(fileTypeResult )
The result of fileTypeResult is the following which is correct
{ ext: 'mov', mime: 'video/quicktime' }
The code for the fetchS3Object is the following
async function fetchS3Object(key) {
console.log(key," KEY ")
const result = await fetch(key, {
method: 'GET',
mode: "cors",
headers: {
'Access-Control-Allow-Origin': '*',
}
})
const response = await result.text()
return response
}
When i print the objectResult variable it is some gibberish.
I am trying to write the objectResult to file using writeFileSync method.
fs.writeFileSync('/tmp/file', objectResult);
The file is written correctly in the specified location, but the video file is corrupted. When I try to play it, it is not working.
Is there is a reason why the written file is corrupted
I have Managed to solve the problem. Here is the issue. The problem arise from the fact that the original uploaded file which is a video had an encoding of
windows-1252,
However the fetch method return a binary which is encoded with
UTF-8.
As a result when you try to use writeFileSync the output is a corrupted file.
To solve this problem this what has to be done. The fetchS3Object function has to be changed to the following
async function fetchS3Object(key) {
console.log(key," KEY ")
const result = await fetch(key, {
method: 'GET',
mode: "cors",
headers: {
'Access-Control-Allow-Origin': '*',
}
})
const response = await result.arrayBuffer();
return response
}
then
const objectResult = await fetchS3Object(video);
const bufferType = Buffer.from(objectResult);
fs.writeFileSync('/tmp/file', bufferType);

How to post an image as a File with Axios after Getting it as an Arraybuffer from an attachment

As a POC I would like to make pictures of my receipts (gas, shop etc) and use a chatbot to send them to my accounting software. My problem has to do with the sending of the collected receipt (an image) to the accounting software using its API.
The first part (getting the attachment) results in an Arraybuffer with an image. I used one of the NodeJS samples for that (nr 15).
const attachment = turnContext.activity.attachments[0];
const url = attachment.contentUrl;
let image;
axios.get(url, { responseType: 'arraybuffer' })
.then((response) => {
if (response.headers['content-type'] === 'application/json') {
response.data = JSON.parse(response.data, (key, value) => {
return value && value.type === 'Buffer' ? Buffer.from(value.data) : value;
});
}
image = response.data;
}
).catch((error) => {
console.log(error);
});
I am struggling with the second part. Posting the image to the accounting software
const requestConfig = {
headers: {
'Authorization': 'Bearer ' + accessToken,
'Content-Type': 'application/x-www-form-urlencoded'
}
};
axios.post(postUrl, image, requestConfig)
.then((response) => { console.log(response); }
).catch((error) => {
console.log(error);
});
};
This results in 400. bad request. Probably the API needs a file and I cannot just send the buffer. I tested with Postman and the request is accepted by using application/x-www-form-urlencoded (by using a locally stored image file).
What is best practice to post an image retrieved in a bufferarray?
I think your comment is right on the money that you need to convert it to a file first. The channel isn't an issue because the file will be stored wherever the bot is hosted. The Attachments Sample actually has this code, which gets you close:
fs.writeFile(localFileName, response.data, (fsError) => {
if (fsError) {
throw fsError;
}
// Send the file
const url = '<yourApiUrl>';
const formData = new FormData();
formData.append('file',fs.createReadStream('<pathToFile>'), { knownLength: fs.statSync('<pathToFile>').size });
const config = {
headers: {
...formData.getHeaders(),
'Content-Length': formData.getLengthSync()
}
};
axios.post(url, forData, { headers });
});
I'm not super confident in the // Send the file section only because I can't test against your API. I got most of the code from here.

Angular 2 save file from Nodejs

I have the snipped in NodeJS to try to do a download from my server.
router.post("/download", function(req, res, next) {
console.log(req);
filepath = path.join(__dirname, "/files") + "/" + req.body.filename;
console.log(filepath);
res.sendFile(filepath);
});
This is my Angular 2 code:
Component.ts
download(index) {
var filename = this.attachmentList[index].uploadname;
this._uploadService
.downloadFile(filename)
.subscribe(data => saveAs(data, filename), error => console.log(error));
}
Service.ts
export class UploadService {
constructor(private http: HttpClient, private loginService: LoginService) {}
httpOptions = {
headers: new HttpHeaders({
"Content-Type": "application/json",
responseType: "blob"
})
};
downloadFile(file: string): Observable<Blob> {
var body = { filename: file };
console.log(body);
return this.http.post<Blob>(
"http://localhost:8080/download",
body,
this.httpOptions
);
}
In my HTML response I get the file correctly recording to my browser's network inspection tool. However Angular is trying to parse the response into a JSON object which obviously doesn't work since the response is a blob.
The error I get is this:
Unexpected token % in JSON at position 0 at JSON.parse ()
at XMLHttpRequest.onLoad angular 2
Does anyone have a solution for this?
Thanks in advance!
Try the following http options:
{
responseType: 'blob',
observe:'response'
}
You will then get a response of type HttpResponse<Blob>, so doing data.body in your service will give you the blob that you can pass to saveAs.
You can find more info on observe here:
https://angular.io/guide/http#reading-the-full-response
I solved that, in service code I deleted the after the post because it forced my return get a binary file, and not a JSON. Now the code it's like that:
downloadFile(file: string): Observable<Blob> {
var body = { filename: file };
console.log(body);
return this.http.post<Blob>(
"http://localhost:8080/download",
body,
this.httpOptions
);
}
Thank you all.

Resources