i got a problem here.
Lets start from we have API, this API returns data with next headers:
Content-Type image/png
Transfer-Encoding chunked
Connection keep-alive
And response body where only file is.
When im try to write this body data into file - it's always broken. I mean i've binary data, his mime, body.length is match original filesize, but this image could not be opened in any viewer after i save it.
What i'm do:
public userFile(req, res: Response) {
const data = {
fileId: parseInt(req.body.fileId),
};
let params = {
headers: {
'Authorization': keys.token,
},
};
axios.post('/api/getfile/', data, params,)
.then((response: AxiosResponse) => {
const fs = require('fs');
const dir = require('path').resolve(__dirname + '../../../files/storage');
const ext = {
'image/png': '.png'
};
fs.writeFile(dir + '/' + img + ext[response.headers['content-type']], response.data, (er) => {
res.send(response.data);
});
})
.catch((err) => {
logger.error("AXIOS ERROR: ", err)
})
}
BUT! When i get this file with postman... here it is!
So, i need your help - what i do wrong?
You must specifically declare which response type it is in params as
responseType: 'stream'
then save stream to a file using pipe
response.data.pipe(fs.createWriteStream("/dir/xyz.png"))
Related
having issues uploading file from NodeJs server side, found 100 posts and reasearches but nothing works, would appreciate any help.
Structure of the App
Front App - React Admin framework receving file and i encode in base64 the content of the image to send to API
Backend - NestJS App - receving base64 image in API
From my backend API need to send file to an external backend (Python API) to upload - here is the problem
Please see below my code, something wrong with the file from JS
i have tried several methods and all of them ends in same error
1 solution
converting base64 image in buffer and send to external backend to upload the file
have tried to pass as well cleanImageBuffer but no changes
import axios from 'axios';
import FormData from 'form-data';
export async function upload(
fileBase64: string,
filename: string
): Promise<any> {
const buffer = Buffer.from(fileBase64, 'base64')
const extension = fileBase64.substring(fileBase64.indexOf('/') + 1, fileBase64.indexOf(";base64"))
const cleanBase64 = fileBase64.replace(/^data:image\/png;base64,/, '')
const cleanImageBuffer = Buffer.from(cleanBase64, 'base64')
const formData = new FormData();
// have tried to pass as well cleanImageBuffer but no changes
formData.append('file', buffer);
formData.append('fileName', filename + '.' + extension);
formData.append('namespace', 'test');
return await axios
.post('external_api_url', JSON.stringify(formData), {
headers: {
Authorization: `Bearer token`,
ContentType: 'multipart/form-data'
}
})
.then((response) => {
console.log('response = ' + JSON.stringify(response))
})
result 1 solution
{
"status": "error",
"error": {
"code": "bad_request",
"message": "file Expected UploadFile, received: <class 'str'>"
}
}
2 solution
from base64 image received saving on my disk
after creating a stream and sending the image
export async function upload (
fileBase64: string,
filename: string
): Promise<any> {
const extension = fileBase64.substring(fileBase64.indexOf('/') + 1, fileBase64.indexOf(";base64"))
const cleanBase64 = fileBase64.replace(/^data:image\/png;base64,/, '')
const TMP_UPLOAD_PATH = '/tmp'
if (!fs.existsSync(TMP_UPLOAD_PATH)) {
fs.mkdirSync(TMP_UPLOAD_PATH);
}
fs.writeFile(TMP_UPLOAD_PATH + '/' + filename + '.' + extension, cleanBase64, 'base64', function(err) {
console.log(err);
})
const fileStream = fs.createReadStream(TMP_UPLOAD_PATH + '/' + filename + '.' + extension)
const formData = new FormData();
formData.append('file', fileStream, filename + '.' + extension);
formData.append('fileName', filename + '.' + extension);
formData.append('namespace', 'test');
return await axios
.post('external_api_url', formData, {
headers: {
Authorization: `Bearer token`,
ContentType: 'multipart/form-data'
}
})
.then((response) => {
console.log('response = ' + JSON.stringify(response))
})
}
result 2 solution
{
"status": "error",
"error": {
"code": "bad_request",
"message": "file Expected UploadFile, received: <class 'str'>"
}
}
other solution that ended in same result
tried to use fetch from node-fetch - same result
found out that some people had an outdated version of axios and having this issues, i have installed latest axios version 1.1.3 but same result
best scenario that i need
from base64 image received
convert in buffer and send file to external Python API so to avoid saving the file on local disk
would appreciate any help
below is a python example that works but not JS (JS nothing works)
import requests
url = "http://127.0.0.1:8000/external_api"
payload={'namespace': 'test'}
files=[
('file',('lbl-pic.png',open('/local/path/lbl-pic.png','rb'),'image/png'))
]
headers = {
'Authorization': 'Bearer token'
}
response = requests.request("POST", url, headers=headers, data=payload, files=files)
print(response.text)
Just a suggestion:
Here's a line which returns mentioned error https://github.com/tiangolo/fastapi/blob/41735d2de9afbb2c01541d0f3052c718cb9f4f30/fastapi/datastructures.py#L20, you might find it useful.
First see if you can make it work with regular HTML file input (don't complicate with Base64 yet), as described here https://stackoverflow.com/a/70824288/2347084
If (1) works, then try converting base64 into a File object as suggested here https://stackoverflow.com/a/47497249/2347084
Combine (2) and (1)
I want to post my solution that worked, because as i can see in internet everybody have issues with FormData on nodejs
i was using axios to send the buffer for uploading file
issue is with axios and specially with FormData, it does not add Content-Lenght in headers, any version of axios does not do this
python API had required Content-Lenght
So kindly was asking to make this header optionally in python API and the code started to work
The solution is if anybody goes in similar issues
axios does not add Content-Lenght when working with FormData (could not find any version of axios that works)
if you work with a buffer without having the file on local disk than will be issue because of Content-Lenght
if u have the file locally than using the module fs u are able to read the file and add all hedears and Content-Lenght
on axios GitHub issue is saying that this bug is fixed in latest axios, but it was still not working in my case
below is a code by using buffer and Content-Lenght is not required in 3rd API
function upload (image: {imageBase64: string, fileName: string}) {
const { imageBase64, fileName } = image;
const cleanBase64 = imageBase64.substr(imageBase64.indexOf(',') + 1);
// buffer should be clean base64
const buffer = Buffer.from(cleanBase64, 'base64');
const formData = new FormData();
// filename as option is required, otherwise will not work, will say that received file is string and UploadFile
formData.append('file', buffer, { filename: fileName });
return client
.post('url', formData, {
headers: {
...formData.getHeaders(),
},
})
.then((response) => response.data)
.catch((error) => {
return {
status: 'error',
error,
};
});
}
I have put together an AWS API Gateway (HTTP) that triggers a lambda function. The lambda function gets an object from an S3 bucket. I've followed the docs from AWS, and I'm getting the objects body contents successfully but the PDF is coming through corrupted.
I have also reviewed this post on AWS, but for some reason when I add isBase64Encoded: true the response returns as 500 and no detail gets logged to CloudWatch.
const getSupportingDocs = async (doc_id) => {
try {
const streamToString = (stream) =>
new Promise((resolve, reject) => {
const chunks = [];
stream.on("data", (chunk) => chunks.push(chunk));
stream.on("error", reject);
stream.on("end", () => resolve(Buffer.concat(chunks).toString("utf8")));
});
const command = new GetObjectCommand({
Bucket: "deq-waterlink-at",
Key: doc_id,
});
const { Body } = await s3Client.send(command);
//const { Body } = await s3Client.send(command).promise(); -> tried & nothing happened
const bodyContents = await streamToString(Body);
//console.log(bodyContents);
return bodyContents;
}
catch (err) {
console.log("ERROR DOWNLOADING: ", err);
}
};
exports.handler = async (event) => {
console.log("EVENT: ", event);
let response;
try {
if (event.queryStringParameters) {
let key = event.queryStringParameters['doc-id'];
let doc = await getSupportingDocs(`interactiveMap/${key}`);
response ={
"statusCode":200,
"body": JSON.stringify(doc),
/*"isBase64Encoded": true, --> Results in an Internal Server Error*/
"headers": {"content-type": "application/pdf"}
};
}
else {
response = {
statusCode: 200,
body: JSON.stringify('No Document ID included'),
};
}
return response;
}
catch (err) {
console.log("ERROR: ", err);
}
};
UPDATE:
After following the suggestion by Daniel Seichter below I was able to get the download triggered but I get an error message when trying to open the PDF that the file is damaged and cannot be repaired. If I removed the JSON.stringify around doc that file downloads but is blank.
I can directly download the file from S3 and it is not corrupted so something is happening in the download process.
I can log the response that the function generates and get the response below, but the browser returns a 500 response.
2022-08-03T21:56:12.163Z 9bd65c5c-6742-45a3-a6ef-6f314e4efa89 INFO RESPONSE: {
statusCode: 200,
headers: { 'content-type': 'application/pdf' },
isBase64Encoded: true,
body: '"%PDF-1.7\\n\\n4 0 obj\\n(Identity)\\nendobj\\n5 0 obj\\n(Adobe)\\nendobj\\n8 0 obj\\n<<\\n/Filter /FlateDecode\\n/Length 35942\\n/Length1 67496\\n/Type /Stream\\n>>\\nstream\\nx��\\u0007X\\u0014��?~�ݙ��.\\u001d�]\\u0016\\u0010\\\\p\\u0017V���bÂ�\\u00114(\\b(��J�+D�5�\\u0016M��ILL̛\\u00045�E\\r��Ę^Mb��f�2�;wv�\\u0012�����<�����a>s��{��{Ν\\u0001�\\u0000�\\u0019�\\u0003O^Ѱ!m�[;\\u0000\\u0016F\\u0001D?6$/\x7F0��\\u000b\\u00006���z\\f)\\u001c]t��u\\u0014�\x7F\\u0004(=2�h\\\\w�g\\u0003��l\\u001f]�Lm�d�\\t\\u0000�\\"�/�&odI�O��\\u0000BN\\u0001�\'*�*�\\u0003NNa\\u0001�^�2\'+�4Yg�x�\\u0006�\\u0017�g�S���~�=U�Ma}�ߦU4z!\\flH\\u000f�\\u0001ô���\\u0018#�G\\u0000}~��������X�\\b�O��\\u0004c��[��a<���i^����\\u0000�\\u0004��ٙ��\\u0015k�.l\x7F\\n��:UW1�+�Q���\\u000f`yk]uSE��\\\\\\u000bDY���Y\\u0015uպ��4\\u0001��\\u0007к���M�yp\\u001c`�)Z��P��\\u001b��\\u0001���y��K�xၭ������B�\\u0000�xxf�\\u001f�����s�w��> �Dz\\"0�?�9A-�\\u0005�eb�l�\\u0003rM\\u0017\\u001d�i��7C\\u001f`�\\u0004\\u0006\\f��\\tȵY���\\u0014�[�\\u001c\\u0002\\u001e\\u0004�vލU����68��\\u00020j�cy�c�m��y�:�r�>8��j\\u0005<�q�\\u001eR)�,��#V w��V��hO�\\u0015\\u0002$a\\u001b�%��\\u001dp��+���o�W���wa\\u0016�\\u000f\\u001e�t�\\f<��a�b;\\u001c����\\u0007�(\\u0016�4��I�P.���p8�܈y��\\u0011����wE\\u000e�=\\nu|\\u0013$*Z�A�&*!��\\r~\\u0015��\\u000f�\\u0004��7�N�q(��\\u0004�\\u0010J�\\np��=�\\u0013y���,�w\\nsa\'M���w�t�/|�I��އ�큻i��\\u001dH�\'#\\u0004�\\u0006=��\\u0018�\x7F|p/�f�Z��������\\u001e8��q��z\\\\=�\\u001eW������q��z\\\\=����vp\\b�����۾\\u0004�\\u000b�|$l���z�\\u0001����\\f���\\u0000���a;�;,��5c�tخX�y{��?\\u0001^^��r�L��\\txŢϻ��)�\\u0003r��`�\\u0013A�~3\\\\=�\\u001eW��\\u001d�\\u001e^��<�<��˺z�� #�\\u0001\\u001a�C��\\u0000�O\\u0002\\u0011T�*\\u0019ՠFԀ\\u0006Q\\u000bZ_\'�#��\\u0007=�AF#\\u0018\\u0011� �w\\u001eL`B4C0b��!\\u0010�\\u0018\\n��s\\u0010\\u0006a��\\u0010�\\u0018\\u0001���2FA��/��h�\\u001e�\\u0003�\\u0002VD��6����\\u0018�A��\\u001d1\\u0016�\\u0010� \\u001e1\\u001e�\\u000f�\\t=\\u0011\\u0013 \\u00011\\u0011\\u0012\\u0011{�\\u0003с�;$A\\u0012b2$#��ވNp!� ��\\u001b�Ș\\n��np#��>�i���\\u0015�è\\f�L�D̂,ľ��w\\u0016�A\x7F��0\\u0000q��\\u0003a b6d�~\\u0001\\u000f\\fB\\u001c$c\\u000e� �B.b\\u001e��~�|\\u0018�8\\u0018� \\u000e�q(\\fE\\u001c\\u0006�|?�p\\u0018�X\\u0000#\\u0010G�Hđ2��Q�\\u001fa4�F,�1�c`,�X�\\u001f�\\b�\\u0010��\\u0018q\\u001c�C�\\u0006�#��\\u0012�\\u0019\\\\a(�B)�\\u0004��8\\u0011�E�\\u0016�|��L�I0\\tq2LF,�r�\\n���\\u001e��X\\t��UP�X\\rՈSa��;�\\u00065�52�B-�t��8\\u0003f����P�X\'�,��X\\u000f��^�����Ѐ� c#4\\"6A\\u0013b34���90\\u0007q.�C�\'�|���\\u0000\\u0016�����\\u0010q\\u0011,F\\\\,�\\u0012X��\\u0002-�/�\\u0015Z\\u0011����Ka\\u0019�2\\u0019۠�w\\u0012���\\u0011��r�\\u0015�\\u0012q���`��\\u000bX\\r�\\u0011��\\u001a�\\u001b�F�\\u001b�&ě\\u0010?���f�[�\\u0016ĵ�\\u0016q\\u001d�G\\\\��\\u0019��\\u001b\\u00107�F�[�V�M�\\u0019q3��;\\u0001��x;lA�\\"�\\u001dp\\u0007�V���)�)�]�\\rq��w�݈����O�^؎�]���~��e�\\u0001;|\\u001f�\\u0003� �2\\u0010\\u001f��?�?|\\u001f���\\b�#�(�Ў�.�.���\\u0010v�n�=�\\u0018�c�8��\\u0017q/�\\u0007�\\u000f�!�\\u0003�\\u0007� �Ax\\u0002�\\t���\\u0010\\u001cB<\\f�\\u0011����OB\\u0007b\\u0007<�{\\u000f���ix\\u001a�\\u0019x\\u0016�Yx\\u000e�9�w�yx\\u001e�\\u0005x\\u0001�Ex\\u0011�%x\\u0019�ex��\\u000e�\\u0002�\\"�*�k�\\u001a�Qx\\u001d�ux��6�!�18��&��x\\u001c�#�\\u0005o���\\u001dx\\u0017�]\\u0019߃�\\u0010߇\\u000f|��\\u0003�\\u0010�C�\\b�#\\u0019?��\\u0011?�O|o§p\\u0002�爟��\\u0005|�;\\u0006\'�K�/�+į�kįe�\\u0006��\\u0001�·�������x\\nN!��Ӿ��\\f�A�\\u0001~D�\\u0011~B�\\t~F�\\u0019�(�\\u0002� ������o���#�\\u0006\x7F�\\u001f�\x7F\x9F�\x7F�9�sp��*��N�N�\\u0010%\\u0019}���wML�5�\\u0019X\\u001a\\"\\u0011x)��u\\u001aa\\u0018�~�vaM�¼\\u001cTk���b�\\u001d�\\u0010�3!�\\u0017j��NW\\n�ʹ�Lԉ�\\u0004�#/\x7F�Æ\\u0017�\\u0000\\u0018]8fl\\u0011��f|\\tL�xm\\u0019�7��w^\\u0012e�P�Ɏ\x7F[\\u000b��\\u0005H�\\u0015C\\u001a�\\u0018\\\\/�QS\\u000fA�;\\n��8Ԙ\\u0013P�yQ\\u000b-�y�\\u0013g��s��\\u0007 �N�K�b�\\u0002,=V.]�z�\\u00015UWi\\u001c\\u0002�\\u0017������I���p�Kځ=Fg\\u0018�&{ܓS:��h��\\u0011Æff���q�������\\u001c�\\u0012\\u0013z����clVK��Ȉ�А`�)�h��\\u001a�J\\u0014�\\n�c\\u0019\\u0002I�=,�dW��\\u0011i��J�\\u0003�K��l��g[;\\u0004]R(.�G_\\u0016��\\u001d\\u001f�\\u000e�����<Z�.\\u0018�U;�ډ�\\u001dh+�4\\u0012[\\n<�_5ݞ_�\\u001e�[U^�O��\\r���?:\\u0003��u�R�r��ժ�$إRcP�!,��E\\u0006\\u000f$r�\\u0019��w\\u0017\\u0003�69�=������kz�gM9\\u0006�yX\\u0013�.���u�pq\\u0016�c]!�?D�\\u0015��J�]km���\\u001d�Xw%u��a�\\u0001��;4U���k�s\\u0015H�.`��k�)\\u001f��U^cm�r\\u0019\\"1Ś_c]m��ȯ)G���SWL��ܒ\\u0015���� ��\\u001b\\u001d�C�Đ\\u0005\'#���a�V\\u001a]�z��}ۘ��sm\\u0014KKKÐ���v�\\u0010+˟��]\\ts&\'��\\u0014`#U�t���\\nJg�t��5�2�7�4�E�kp`*�]�ի���U\\u0015U9��s�=��\\r�\'��\\u001dD��\\u0006�\\u0002\\u00050��s��Jm~f\\u0017�-ɥ��+�\\"��ޝR\\u001eH����L+�`\\u0018V�n�����\\u0012;\\u0016ͤP�\\t�+3e᱕\\u0012|���S�|��n]�+��r��S��T\\u0004R\\u0014q�_�\\u0006\\u0007�\\u0007��^=�n\\u001d��|u�~_�\\u0014��`_���`�7�\\u001c[-,����\\u000e��l\\u001f|Ci�����E�S\\t\\u0018<�$;�f,�\\u0016vE\\u0001E\\n\\u0005K-w\\u0007��?�\\u00027�2\\u0014�جȨq%��ȧ\\u0012\\u001a.ư�N\\u0005\\t\\u00057\\u0013�8�6ʣ��n��\\u0006�6\\u001b��5�=0\\u0005#��cJ�q+L��\\r\\u001e�\\u0003ǣ��tt�\\u0004��9�]9ݏ�۱��d\\u0005\\u0016�.�w��\\r!�����$�_dW���M�%l$S�\\u000f1�,\\r�\\u001c8����:0��X�������h�K:\\"��Z\\rF�\\u0000t��\\u0005c&�X�WwK�?%�S*\\u0007(����Ձ�D���2�`+(�0�8M�:���?>�Kqa����ǻ�\\u0005A\\u0001)(F)G=���NV���!+�&�\\u001c�5Ժ��d7C����]��Wr�\\n��S\\u0019�J\\u0013i�J#���\\u0018\\u0011��\\u0007<\\u0000�r.\'\'����\\u0004�4�+�#�~Ɵf�7\\u0014/7��E�r?���t��0M��i�\\u000bhw=*�#xD���2��\\bMڍ)\\u0007q\\u0018E\\u0002{4DK\\"w�Sc����u����h�\\u0012\\u001e?�+�]hz܄�=\\u001a��dĆr��\'�\\u0006YYb��[��=�%�JkV��R\\u001d��2��P�Xb�\\u001bڭ\\u0011/E�6|Y�\\f��X<DVA/\\u0012\\u0005�H���|�A�$\\u000eRI,�cѫU��\\u000e6�\\u001e��\\u0003\\u000b�\\u0010H/\\r�W�\\n��,d<\\f%�`�\\u001a\x7F:��\\"+\x7F K0�\\u0018�80e\\u0011Y�+��,ĸ\\u0011�\\u000b\\u0002���<�D-d\\u001eƣ�>7p�C&�m6�\\u0006�ހ��x�\\u001dh�\\u001b��\\u0007�3\\u0003�3\\u0002��{m�^\\u0013�W��\\u0019-�Ie��)�bw�ų���1���}�Ɛ>HG�\\u0003tL\\u000e�\'��\\u000eR�GkH�\'1�~?\\u0019�}4a�G�6�,����z��\\u0010\\u0019��-��D\x7Fj\\u0002M�O�w��M���7�n��Bi�!\\"\\u0006x*�3��̈�!438Ј�_��߈\\u0002S9L�w\\u0013��y��|�u��믌\\u0016Tb��w�/��ɻ댖��z\\u0015�W�z\\u0019���z\\n�\\u0007��Z����-V�m[\\u0012,[�EZ~�\\u001clٱ9ܲis/˭��,\\u001b1��L6cq��dúp��u\\u000e��u6\\u000b�#��kש\\r��C�C�C��\\t\\u0002\\u0007\\f\\u0007\\u0018=ήlj���?\\u0018����=��-�\\u0012�Y�Y�z��\\f�<�}j�)����mf��\\u0004��=F�sO���vo��-�˓��/�r��\\r�y\\u001a;B\\u001b�=��7[z[���F���z��ҁדx�|�w��\\u001f&��dףF��Qbx��\\u0000�f�˲z�Ӳ��mY�\\u0016fY���a��ی�em}-mXM��m;�w�����M\\f�Z��\\\\���5.m\\t�\\\\�2�Ҋ�%��b�\\n[�[�-�Ao��\\u0004��(\\u00156KxX/\\u000b��,��^��d}/�.!Q\\u001f�S\\u0017\\u001b����6}\\u000f�.2*Z\\u001b\\u0016\\u001e�\\r\\u000e\\t�\\u0006��Z����hu\\u001aQ��(����x\\r�\\u0013\\u001a��U�x\\u0014�\\n�ö��\\u001e�a4��\\u001e�\\u0018�D�c�I��| D�\\u0013,�����\\u0012,�)X\\nݤ=�\\u0000P���P�\\n�r�ݎ\\u0002TQc�S\\u001d\\u0005�b�Ē]��T����J\\u001c��vn%�b��&L,�O�i���Q�!TJ��xcdw���\\u0011�^UPT��.mO��[�K��GcScc��\\u001f�]!����9����T��-ZS��\x7FG����H�ы�� V�\\u001d��\\\\t��YNo�[s�DǤ\x7F$��܅#l�\\u0003\\u0014fP\\u0001���҅��\\u0010�`�w�g2~�\\u0015��|����!\\u0004��/\\u00079�$�\x7F� 7�夕\\u0014�&�#ͨ?=�,K\\u0011�a�\\u001e\\u001e�\\u000b�\\u0007�\\u0012+\\t\':B��\\u0018Qs�#q$��P!�0N�g=+��*�Y�\\u0017~adn�\\u001a�����$z�\\u0012��\\u0011<��\\u0013�;l�\\u0007�I��P8��Ga��a\\u0017\\u001c�2/�3\\u001f�W�#\\u0011�\\u0004\\\\\\u001fV�\\r��\\u000ea&`�0�KV1#�s\\\\,(�\\\\&�Lc\\u000f��DA�q5;\\b��\\u0007l;��mp�Mf��<\\u0018\\u0001�I\\u001f�a�c{�\\u0016�\\u0018sߕ�!�A�,\\u0001;�c�����r��Jr ����|��\\u001b�;i\'��H˥��_:.ȁ�[�)\\u0014�I��R�S�\\u0007+`��e���C�I��z�\\"q\\u0004�$\\nǀ��8\\u0006��q�u�<ZF\\"�\\u0004��\\u0011y�D�!�#g�`y\x7F�W�>P�G8\\u0001_�3\\u001a��#�����\\u001b`�\\u0011�\\u0004\\u0005�6P�y���^��>~M�\\u0014��h3�!\\u0010,�W+\\u000f��\\u001dZ�v\\b\\u0003;�\\u00178��,?��+�,ð\\u0004^׳嬗Ń{\\u001d�\\u0006e��E�)��Gdʜe��xA��\\u0019����\\"�]>9M�����Ka��Ì�\\u0017�\\u0003xJ�b�`\\u000b��ߓ4�\\u00140\\u0005����d����3�\\n���0-�\\u0016�N\\t$R�S�Z��=\\u001eb8K�qCv6�QFl��\\u0010�a\\u000bV(�\\n&�\\u001c\\u0014\\u001a\\u0012\\u0012��Vt\\u001c8�A�Y�ݿ`����\\n����G$\\u0012Ϗ>�\\u001ev`�\\\\��\\u000fI�-���\\u0010J��H��.z�P�{Ly�B��TΔ��r���^�\\u000b\\u0002.\\u0012/�q�\\u000e�\\u001bj��\\u0012z\\f�Җ����\\u001e�և�\\u0019\\u001f�3�\\u0016\\u0012�n>�!�(��oం���ǐ�\\u001d\\u0007��җR�g�C:�.\\"!\\u000fm\'1s\\u0017\\u001d\\u0018\\u0016���6\\u001d�ʄp\'�\\u001a#��̞*�\\u001a\\u0019A\\u0010�aB�1QH4�fF��\\u0004�ڣ\\u001bf��L�K�1�1�\\u0012�4�:�\\\\]���7�bf1߬�/6�\\u001bu\\u000b\\f+\\r6�P\\u0000��\\u0018N\x7FBET�J\\u0003`4|�Uӽ\\u0011�Bۮ\\u0011���h(i�eg��\\f�C�\\fx�\\u0001���?�\\u0014W\\u0019vQ\\u0019g����qq\\u0019<�f�By�d�ԡ~[���#�ҝ3I��-�\\f��ng���#�J&�y���;�&\\u0013�����\\u0006�o\\u0016�`k�x��(����\\u0018����1N~4������4�\\u0013哭��s��X�D\\u0018��\\u001e8[�8�\\u001f�U0��\\u0012\\u0018r\\u001d&�*�[��\\fE��#����(V�)�(�(f+U��\\u0003�ދ�\\u0006�H8��_� \\n�\'\\f��.�D^\\u0005<��~ߧ� �!��!�^���ECxg��]�t8ʂB���Mo(�YY�\x7F\\u0010G�H~1,湲RR������!�Ёُe�#E\\u0006��Ge�$65��!5N�|o����\'\\u0007\\u001d���d�8\x7F�|\\u001d\\u0013��\\u001d��i�/�Y�\\u0019p��\\u0003��\\u0019�\\u0011��\x7F7��=�\\u0019\'j2�\\u0003\\u0011��h;\\u001b�%\\nN�\\u0019m��2��xUi�5���\\u0002Q�4e��M-&�d�X��ɮ��do2��\\u001c�\\u0016L���i�6:mr\\u001ak]���\\u0007gV��TȦ\\u0002-\\u0003���A\\u001c\\u000e>&�g<��\'(#=֝\\u001aB��ю�\\u001d\\u001fo��i�\\b6�4w*�}z��H����|��uuu���\\nb�9i�\'�.1)j\\\\zF�1�\\u0006�\\u001b6���[�\\r^�7�82!sj���)\\u0015\\u0015$��.b�V93�hr��Ma9Vk��_֡�7\\u001cJ�p����\\t��\\bO2\\u0004��4��T\\f#i�A4��^�A�\\"����9�|�\\"d����S�\\u0017�`\\u0001\\u0016�o�X<�B\\u000b\\u001bڢ|�\\u0007��2�YTV��٧�P�\\u0019������r�!A|\\u001f�kv��G�\\u0001G��I�ww��D�%\x84�wI�3��-\\\\TQ��u#�\\u0014��7�}%9}�c\\u00032\'IO���X����׆���DG\\u0013��~A*�0�\\u0013!�u)<�r�\\u0017��v\\u0005\\u001e�_˰��D�F��\\u0010�!p��h\\u0015b\\u000by\\\\c��3Ո���tȤ\\u0006Q�C�Muw��~9�\\n��9���s!��?&��|x����ز\\b9\\u001e3�օ\\u000eq9��\\\\I�\\n��e��i\\u0014�\\u0019\\u0000j�z��ax䎊����t��(�]��\\u000bM~�Y�,�^Z�9�]��҉�6l���g�\\u0017(��p�\\u0001���U��w�\\u0013��j�\\u001dJG�=�$�8j*_���/V.\\b�\\u001d��Y���\\u0019;%&K4d��\\"��fk�-ZN�5�\\u0013�����i\\u001f����jmK\\u0000�=�$b1�\\bV�)Xž8�oe�)�(�FJ1J�,�4���� �� [Q�\\u0001\\u001d��$\\u0018��3h��/���ۥ6i,�M��\\u0013ս����}���z\x7FN�04T $-Dz�yt\\\\ϑd+��\\u0013Ƀi��{�GFZ{�\\u001f����ߥ�\\u0018�Ēp��B\\\\���G?�\\u0013#r�\\u0015��]O\\u0004�ݪ\\u0016�g�\\u0006���VcѺ�\\u001e�W�����M5t�M\\u0005ggjvg�<�n�\\u001diC�d�8���Y���Y\\"�O\\u0011A��|v`�\\u0015I�\\u001d��b4�p2�\\u0011D\\rEcpW\\u0018\'\\u0002�;���/�I\\u0013F2�B\\u001dS-,d�tq�\\u000eI\\u0014�V\\u0006U�J��.ͨV�*�W\\u0011��<Z\\"ˌ\\u0001�\\t��\\u0016��\\u0016^\\u0016��,*2e�\\u0018Qh��\\\\a#\\r�wr\\u001d\\u0004��Ё*�j�rN\\u001d�N\\u0014��tu�8��\\u0013�s�*�13�l\\u0015.�ML�Z��*�f�D娉 Z��n�F��R\\u0000���Ũ�jƃ��Q��.�4~�\\u0004�6\\u0012�E2mee~�P\\u001e�\\bt �Tm;(����I��l6��ŝນ$%J�\\u0013�V�ʘ��Lq�\\u0014�\\u001f����9�k�������1#�֢�R\\u000e^��F��LV��1�W�����x�(V��?v��\\u000eŻX�\\u0006�=\\u0000*�\\u001b83\\rd\\u0014����Ҿ���<tf�#�\\u000e5\\u000b�F�W\\u001b�('... 702140 more characters
}
try to set in your response the correct mimetype: application/pdf
instead of using
response ={
statusCode:200,
body: doc
}
use this one:
response ={
statusCode:200,
headers: {"content-type": "application/pdf"},
body: doc
}
This should give your browser the information, of the correct content type.
Edit:
if this won't work, you probably need to follow this instructions: https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-payload-encodings.html
I have a code as shown below that uploads files from the browser and saves in the server, once it has been saved to the server, I want the server to connect to the Pinata API so the file can also be saved to the IPFS node.
let data = new FormData();
const fileBuffer = Buffer.from(`./public/files/${fileName}`, 'utf-8');
data.append('file', fileBuffer, `${fileName}`);
axios.post('https://api.pinata.cloud/pinning/pinJSONToIPFS',
data,
{
headers: {
'Content-Type': `multipart/form-data; boundary= ${data._boundary}`,
'pinata_api_key': pinataApiKey,
'pinata_secret_api_key': pinataSecretApiKey
}
}
).then(function (response) {
console.log("FILE UPLOADED TO IPFS NODE", fileName);
console.log(response);
}).catch(function (error) {
console.log("FILE WASNT UPLOADED TO IPFS NODE", fileName);
console.log(error);
});
The issue i'm having is that after creating a buffer of my file and wrapping it in a formdata, the pinata API returns an error :
data: {
error: 'This API endpoint requires valid JSON, and a JSON content-type'
}
If i convert the data to string like JSON.stringify(data) and change the content-type to application/json, the file buffer will be uploaded successfully as string.
I hope explained it well to get a solution. Thanks.
It looks like you're attempting to upload a file to the pinJSONToIPFS endpoint, which is intended to purely be used for JSON that is passed in via a request body.
In your situation I would recommend using Pinata's pinFileToIPFS endpoint
Here's some example code based on their documentation that may be of help:
//imports needed for this function
const axios = require('axios');
const fs = require('fs');
const FormData = require('form-data');
export const pinFileToIPFS = (pinataApiKey, pinataSecretApiKey) => {
const url = `https://api.pinata.cloud/pinning/pinFileToIPFS`;
//we gather a local file for this example, but any valid readStream source will work here.
let data = new FormData();
data.append('file', fs.createReadStream('./yourfile.png'));
return axios.post(url,
data,
{
maxContentLength: 'Infinity', //this is needed to prevent axios from erroring out with large files
headers: {
'Content-Type': `multipart/form-data; boundary=${data._boundary}`,
'pinata_api_key': pinataApiKey,
'pinata_secret_api_key': pinataSecretApiKey
}
}
).then(function (response) {
//handle response here
}).catch(function (error) {
//handle error here
});
};
The proper code to pin any file to IPFS is as below.
Apparently, even Pinata support staff didn't know this.
You need to set an object with the property name filepath as your last parameter. The name doesn't matter, it can be a duplicate, it can be the same as others, or it can be unique.
const url = "https://api.pinata.cloud/pinning/pinFileToIPFS";
const fileContents = Buffer.from(bytes);
const data = new FormData();
data.append("file", fileContents, {filepath: "anyname"});
const result = await axios
.post(url, data, {
maxContentLength: -1,
headers: {
"Content-Type": `multipart/form-data; boundary=${data._boundary}`,
"pinata_api_key": userApiKey,
"pinata_secret_api_key": userApiSecret,
"path": "somename"
}
});
Code to upload a file on IPFS using Pinata.
There are two methods available to upload files/images on Pinata. One is with Pinata SDK and the second is the pinFileToIPFS endpoint.
If you are uploading files from Next.js then you cannot convert your image into binary using fs.createReadStream or Buffer.from. These packages support the Node side. So if you want to upload the file with Next.js on Pinata then you can use this code.
// convert file into binary
const data = new FormData();
data.append("title", file.name);
data.append("file", file);
const url = "https://api.pinata.cloud/pinning/pinFileToIPFS";
// pass binary data into post request
const result = await axios.post(url, data, {
maxContentLength: -1,
headers: {
"Content-Type": `multipart/form-data; boundary=${data._boundary}`,
pinata_api_key: "your_pinata_key",
pinata_secret_api_key:
"your_pinata_secret",
path: "somename",
},
});
console.log("RESULT", result);
this will upload a file to ipfs under the path ipfs://{cid}/images/{fileId}
const PINATA_BASE_URL = "https://api.pinata.cloud";
const PINATA_PIN_URI = "/pinning/pinFileToIPFS";
const fileExt = file.type.split("/")[1];
let nftId = 1
// creates a 64byte string '0000...0001' to follow ERC-1155 standard
const paddedId = createPaddedHex(nftId);
const ipfsFileId = `${paddedId}.${fileExt}`;
const ipfsImageFilePath = `/images/${ipfsFileId}`;
const fileUploadData = new FormData();
// this uploads the file and renames the uploaded file to the path created above
fileUploadData.append("file", file, ipfsImageFilePath);
fileUploadData.append(
"pinataOptions",
'{"cidVersion": 1, "wrapWithDirectory": true}'
);
fileUploadData.append(
"pinataMetadata",
`{"name": "${ipfsImageFilePath}", "keyvalues": {"company": "Pinata"}}`
);
const pinataUploadRes = await axios.post(
PINATA_BASE_URL + PINATA_PIN_URI,
fileUploadData,
{
headers: {
Authorization: `Bearer ${PINATA_JWT}`,
},
}
);
const ipfsCID = pinataUploadRes.data.IpfsHash;
As a POC I would like to make pictures of my receipts (gas, shop etc) and use a chatbot to send them to my accounting software. My problem has to do with the sending of the collected receipt (an image) to the accounting software using its API.
The first part (getting the attachment) results in an Arraybuffer with an image. I used one of the NodeJS samples for that (nr 15).
const attachment = turnContext.activity.attachments[0];
const url = attachment.contentUrl;
let image;
axios.get(url, { responseType: 'arraybuffer' })
.then((response) => {
if (response.headers['content-type'] === 'application/json') {
response.data = JSON.parse(response.data, (key, value) => {
return value && value.type === 'Buffer' ? Buffer.from(value.data) : value;
});
}
image = response.data;
}
).catch((error) => {
console.log(error);
});
I am struggling with the second part. Posting the image to the accounting software
const requestConfig = {
headers: {
'Authorization': 'Bearer ' + accessToken,
'Content-Type': 'application/x-www-form-urlencoded'
}
};
axios.post(postUrl, image, requestConfig)
.then((response) => { console.log(response); }
).catch((error) => {
console.log(error);
});
};
This results in 400. bad request. Probably the API needs a file and I cannot just send the buffer. I tested with Postman and the request is accepted by using application/x-www-form-urlencoded (by using a locally stored image file).
What is best practice to post an image retrieved in a bufferarray?
I think your comment is right on the money that you need to convert it to a file first. The channel isn't an issue because the file will be stored wherever the bot is hosted. The Attachments Sample actually has this code, which gets you close:
fs.writeFile(localFileName, response.data, (fsError) => {
if (fsError) {
throw fsError;
}
// Send the file
const url = '<yourApiUrl>';
const formData = new FormData();
formData.append('file',fs.createReadStream('<pathToFile>'), { knownLength: fs.statSync('<pathToFile>').size });
const config = {
headers: {
...formData.getHeaders(),
'Content-Length': formData.getLengthSync()
}
};
axios.post(url, forData, { headers });
});
I'm not super confident in the // Send the file section only because I can't test against your API. I got most of the code from here.
I have the snipped in NodeJS to try to do a download from my server.
router.post("/download", function(req, res, next) {
console.log(req);
filepath = path.join(__dirname, "/files") + "/" + req.body.filename;
console.log(filepath);
res.sendFile(filepath);
});
This is my Angular 2 code:
Component.ts
download(index) {
var filename = this.attachmentList[index].uploadname;
this._uploadService
.downloadFile(filename)
.subscribe(data => saveAs(data, filename), error => console.log(error));
}
Service.ts
export class UploadService {
constructor(private http: HttpClient, private loginService: LoginService) {}
httpOptions = {
headers: new HttpHeaders({
"Content-Type": "application/json",
responseType: "blob"
})
};
downloadFile(file: string): Observable<Blob> {
var body = { filename: file };
console.log(body);
return this.http.post<Blob>(
"http://localhost:8080/download",
body,
this.httpOptions
);
}
In my HTML response I get the file correctly recording to my browser's network inspection tool. However Angular is trying to parse the response into a JSON object which obviously doesn't work since the response is a blob.
The error I get is this:
Unexpected token % in JSON at position 0 at JSON.parse ()
at XMLHttpRequest.onLoad angular 2
Does anyone have a solution for this?
Thanks in advance!
Try the following http options:
{
responseType: 'blob',
observe:'response'
}
You will then get a response of type HttpResponse<Blob>, so doing data.body in your service will give you the blob that you can pass to saveAs.
You can find more info on observe here:
https://angular.io/guide/http#reading-the-full-response
I solved that, in service code I deleted the after the post because it forced my return get a binary file, and not a JSON. Now the code it's like that:
downloadFile(file: string): Observable<Blob> {
var body = { filename: file };
console.log(body);
return this.http.post<Blob>(
"http://localhost:8080/download",
body,
this.httpOptions
);
}
Thank you all.