Remove line breaks form mult form data. Node - node.js

Data sent from Node via Mac. This fails:
POST / HTTP/1.1
Accept: application/json, text/plain, */*
Content-Type: multipart/form-data; boundary=--------------------------410170969577572462482590
Authorization: Basic U3dlY2q9DdlRvQ29uqdGFjdDpVM2RsWTI5RGRsUnZqRMjl1ZEdGamRB
User-Agent: axios/0.18.0
Content-Length: 437
Host: localhost:3000
Connection: close
{
"_overheadLength": 105,
"_valueLength": 5,
"_valuesToMeasure": {},
"writable": false,
"readable": true,
"dataSize": 0,
"maxDataSize": 2097152,
"pauseStreams": true,
"_released": false,
"_streams": {
"0": "----------------------------097921969493700670690484\r\nContent-Disposition: form-data; name=\"Domain\"\r\n\r\n",
"1": "Test"
},
"_currentStream": {},
"_insideLoop": false,
"_pendingNext": false,
"_boundary": "--------------------------097921969493700670690484"
}
Data sent from Postman from Windows. This works:
POST / HTTP/1.1
Content-Type: multipart/form-data; boundary=--------------------------214255515908701131866697
Authorization: Basic U3dlY29DwerdlRvQ29uwerdGFjdDpVM2RsWTwerI5RGRsUnZRMjl1ZEdGamRB
User-Agent: PostmanRuntime/7.15.0
Accept: */*
Cache-Control: no-cache
Postman-Token: 4af4ff14-1abd-4ab7-9e01-5ddd846acfa9
Host: localhost:3020
accept-encoding: gzip, deflate
content-length: 383
Connection: keep-alive
----------------------------214255515908701131866697
Content-Disposition: form-data; name="Domain"
test
--
It seem's node add row brakes: \r\n\r\n here and there.
This causes a failure from a windows server when I post data to it: "Line length 100 exceeded".
See this question: Sending post request multipart form data. Error from some microsoft service "Line length limit 100 exceeded"
I'm using this the form-data package to post data combined with axios.
Is it possible to add some filter/middleware etc that removes alla /n/r etc in my post request?
Update
My request from node:
const form_data = new FormData();
form_data.append('Domain', 'test');
const request_config = {
headers: {
"Authorization": "Basic dffdg",
"Content-Type": `multipart/form-data; boundary=${form_data._boundary}`
},
data: form_data
};
await axios.post(url, form_data, request_config).then(response => {

try removing data from request_config.
const form_data = new FormData();
form_data.append('Domain', 'test');
const request_config = {
headers: {
"Authorization": "Basic dffdg",
"Content-Type": `multipart/form-data; boundary=${form_data._boundary}`
},
//data: form_data
};
await axios.post(url, form_data, request_config).then( /* ... */ )
from axios/axios documentation
NOTE
When using the alias methods url, method, and data properties don't need to be specified in config.

Related

NodeJS / Axios: How to write a PDF to disk obtained in an HTTP multipart/form-data response?

I'm writing a REST API client that calls an API endpoint to obtain a PDF.
I'm using axios to consume the API:
const res = await axios({
method: 'GET',
url: url,
headers: {
'authorization': 'Bearer ' + at,
},
proxy: false,
httpsAgent: httpsAgent
});
console.log(res.data);
The response is of type multipart/form-data, in the following way:
content-type: multipart/form-data; boundary=95b77841-292d-4a0b-a689-787fcc0aa889
body:
--95b77841-292d-4a0b-a689-787fcc0aa889
Content-Disposition: form-data; name="metadata"
Content-Type: application/json
{"documentId":"123"}
--95b77841-292d-4a0b-a689-787fcc0aa889
Content-Disposition: form-data; name="document"; filename="document.pdf"
Content-Type: application/pdf
%PDF-1.4
....
%%EOF
--95b77841-292d-4a0b-a689-787fcc0aa889--
How can I save the 2nd part as a valid PDF file to disk?

How to delete multiple files on azure blob storage through rest api batch delete

I was trying to delete the multiple files from the container using the API provided by azure name as blob batch. So i followed the same instruction explained in this doc
I am getting the following body in response
'--batchresponse_ffab056e-336a-49e9-a878-371bd69b15f6
Content-Type: application/http
HTTP/1.1 400 One of the request inputs is not valid.
x-ms-error-code: InvalidInput
x-ms-request-id: <RequestId>
x-ms-version: 2020-04-08
Content-Length: 221
Content-Type: application/xml
Server: Windows-Azure-Blob/1.0
<?xml version="1.0" encoding="utf-8"?>
<Error><Code>InvalidInput</Code><Message>One of the request inputs is not valid.
RequestId: <RequestId>
Time:2021-11-23T13:01:31.5627264Z</Message></Error>
--batchresponse_ffab056e-336a-49e9-a878-371bd69b15f6--'
This was my request options and subrequest in batch
const boundary = "batch_357de4f7-6d0b-4e02-8cd2-6361411a9525";
const separation = "--"+boundary + "\n";
const ending = "--" + boundary + "--";
const fileList = ['sample1.json','sample2.json']
const subRequest = `--batch_357de4f7-6d0b-4e02-8cd2-6361411a9525\r
Content-Type: application/http\r
Content-Transfer-Encoding: binary\r
\r
DELETE /${azureDirectoryName}/${fileList[0].fileName} HTTP/1.1\r
Host: https://<accountName>.blob.core.windows.net\r
x-ms-date: ${strTime}\r
Authorization: Bearer ${authToken}\r
Content-Length: 0\r
--batch_357de4f7-6d0b-4e02-8cd2-6361411a9525--\r`
const requestObj = {
'Host': '<accountName>.blob.core.windows.net',
url: 'https://<accountName>.blob.core.windows.net/?comp=batch',
method: 'POST',
headers: {
'Content-Type': 'multipart/mixed; boundary=' + boundary,
"x-ms-date": new Date(),
"x-ms-version": "2020-04-08",
"Content-Length": subRequest.length,
Authorization: 'Bearer ' + authToken
},
body: subRequest
}
request(requestObj, (err, resp) => {
//internal logic
})
RequestObj JSON
'{"url":"https://<accountName>.blob.core.windows.net/?comp=batch","method":"POST","headers":{"Content-Type":"multipart/mixed; boundary=batch_357de4f7-6d0b-4e02-8cd2-6361411a9525","x-ms-date":"Wed, 24 Nov 2021 04:45:21 GMT","x-ms-version":"2018-11-09","Content-Length":2166,"Authorization":"Bearer <authtoken>"},"body":"--batch_357de4f7-6d0b-4e02-8cd2-6361411a9525\r\nContent-Type: application/http\r\nContent-Transfer-Encoding: binary\r\n\r\nDELETE /containerName/test.json HTTP/1.1\r\nHost: https://<accountName>.blob.core.windows.net\r\nx-ms-date: Wed, 24 Nov 2021 04:45:21 GMT\r\nAuthorization: Bearer <authToken>\r\nContent-Length: 0\r\n--batch_357de4f7-6d0b-4e02-8cd2-6361411a9525--\r"}

How to encode or resolve the response of request Module in nodejs

Hi I use request Module in Nodejs.
There is the code below:
const response = await request.post({
uri: "https://item.rms.rakuten.co.jp/rms/mall/rsf/item/vc",
headers: {
Host: "item.rms.rakuten.co.jp",
Origin: "https://item.rms.rakuten.co.jp",
Referer: "https://item.rms.rakuten.co.jp/rms/mall/rsf/item/vc",
"Content-Type": "application/x-www-form-urlencoded",
"Content-Length": buildForm(data).length,
Cookie: cookies~,
Accept:
"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "ko-KR,ko;q=0.9,en-US;q=0.8,en;q=0.7,id;q=0.6",
"Cache-Control": "max-age=0",
Connection: "keep-alive",
"sec-ch-ua": `"Google Chrome";v="87", " Not;A Brand";v="99", "Chromium";v="87"`,
"sec-ch-ua-mobile": "?0",
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "same-origin",
"Sec-Fetch-User": "?1",
"Upgrade-Insecure-Requests": 1,
"User-Agent":
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36",
},
formData: buildForm(data),
})
const buildForm = (data) => {
const keys = Object.keys(data);
const form = [];
for (let i = 0, l = keys.length; i < l; i++) {
if (typeof data[keys[i]] === "string") {
const data_value_buf = Buffer.from(data[keys[i]]);
const data_value_convert = encoding.convert(
data_value_buf,
"EUCJP",
"UTF8"
);
form.push(keys[i] + "=" + encoding.urlEncode(data_value_convert));
} else if (typeof data[keys[i]] === "undefined") {
form.push(keys[i] + "=" + "-");
} else {
form.push(keys[i] + "=" + data[keys[i]]);
}
}
return form.join("&");
}
the data is Json type.
When I call the request, I got the broken string response.
so, I added encoding: null, in request post option, like
const response = await request.post({
uri: "https://item.rms.rakuten.co.jp/rms/mall/rsf/item/vc",
encoding: null,
headers: {
Host: "item.rms.rakuten.co.jp",
~~~~
then I use Iconv Module.
const iconv = new Iconv("EUC-JP", "UTF-8//translit//ignore");
const res = iconv.convert(response);
How to show the response without broken string
Also the response headers is below
Cache-control: no-store, no-cache
Connection: close
Content-Encoding: gzip
Content-Type: text/html; Charset=EUC-JP
Date: Tue, 12 Jan 2021 05:28:50 GMT
Pragma: no-cache
Server: Apache
Transfer-Encoding: chunked
Vary: Accept-Encoding
X-Content-Type-Options: nosniff
X-frame-options: DENY
X-oracle-dms-ecid: 93mhk0e_y40000000
X-oracle-dms-rid: 0:1
X-XSS-Protection: 0
Please let me know how to get normal response like html tag response
You made a POST request. So you're SENDING data to the server. You don't get a HTML response. You get a response CODE and correspondant headers.
If you used a GET request instead, then the server could send you a HTML response. See that I said COULD. Not necessarily will. You can get JSON, or just a return code for example.
In this case, although it's not appearing completely, you can see a 200 result code in the top part of your image. This tell us that your request was successfull. And the headers are what you translated. But the relevant information is that the server returned a 200 code.
So or you GET something or you POST something. You can't get HTML while posting. In a post you send the data in the body of the request and then receive just the result code you received, or any relevant error code.
FYI, there are several other HTTP methods besides POST and GET, but for the moment stay with those 2 and when you really understood them you look the other ones.

axios equivalent of http requests?

I want to make the following http request
POST /v1/images HTTP/1.1
Host: api.medium.com
Authorization: Bearer 181d415f34379af07b2c11d144dfbe35d
Content-Type: multipart/form-data; boundary=FormBoundaryXYZ
Accept: application/json
Accept-Charset: utf-8
--FormBoundaryXYZ
Content-Disposition: form-data; name="image"; filename="filename.png"
Content-Type: image/png
IMAGE_DATA
--FormBoundaryXYZ--
It is Medium API
I have attempted following.
var axios = require("axios")
var data = (await axios("https://example.com/image.png")).data;
axios.post("https://api.medium.com/v1/images",{image: data},{
headers: {
"Content-Type" : "multipart/form-data",
"Authorization" : "Bearer " + process.env.key
}
}).then(x=>console.log(x.data))
And I get following error.
Error: Request failed with status code 400
It uses Medium API to upload image, I want to fetch a remote image, and convert it into multipart/form-data and upload it via API, the HTTP request seems confusing, I want the equivalent axios code, someone please help?
Try using FormData instead of plain object?
Example should be like this
const formData = new FormData();
formData.append('image', data);
axios.post('https://api.medium.com/v1/images', formData, {
headers: {
"Content-Type" : "multipart/form-data",
"Authorization" : "Bearer " + process.env.key
}
});
Reference: How to post a file from a form with Axios

Posting a multipart form with file in Axios, Nodejs

const form_data = new FormData();
form_data.append("File", fs.createReadStream(pathToFile));
form_data.append('Login', alias.toUpperCase());
const request_config = {
headers: {
"Authorization": "Basic 123",
"Content-Type": 'multipart/form-data'
},
data: form_data
};
await axios.post(url, params, request_config).then(response => {
Posting to an endpoint I can't debug. The response is a 500.
This is the error:
Is this the correct way do it?
Can I somehow see exactly what Axios is sending?
This is a postman request I received from the author of the API. This passes:
POST /api/upload HTTP/1.1
Host: api.test.contoso.se
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
Authorization: Basic 123
User-Agent: PostmanRuntime/7.13.0
Accept: */*
Cache-Control: no-cache
Postman-Token: 089af753-fa12-46c4-326f-dfc39c36faab,c5977145-ece3-4b53-93ff-057788eb0dcf
Host: api.test.contoso.se
accept-encoding: gzip, deflate
content-length: 18354
Connection: keep-alive
cache-control: no-cache
Content-Disposition: form-data; name="Lang"
SV
------WebKitFormBoundary7MA4YWxkTrZu0gW--
Content-Disposition: form-data; name="File"; filename="/C:/Users/file.docx
------WebKitFormBoundary7MA4YWxkTrZu0gW--
Content-Disposition: form-data; name="Login"
ABC
The error message in your screenshot is clear: "Missing content-type boundary".
To use axios for multipart/form-data request, you need to set boundary to upload the file. Example code is:
const form_data = new FormData();
...
const request_config = {
headers: {
"Authorization": "Basic 123",
"Content-Type": 'multipart/form-data; boundary=' + form._boundary
},
data: form_data
};
await axios.post(...)
"Can I somehow see exactly what Axios is sending?"
You can use HTTP proxy software such as Charles to intercept the request, and check what data is sent.

Resources