Problem
I use axios (that is not important, any alternative is ok) to make my HTTP calls to an external API.
When I do these HTTP calls via Postman to the API directly, I get this header at the API call response:
{ ..., Content-Type: application/json; charset=windows-1253, ... }
I do these HTTP calls via my NodeJS app like this:
router.get('/data',
async (req, res) => {
try {
let customers = await axios
.post(URL, {
...
})
res.send(customers.data)
} catch (error) {
res.send({ error: error })
}
}
)
and when I call it from Postman I get this header at the API call response:
{ ..., Content-Type: application/json; charset=utf-8, ... }
Pay attention that in the first case, the charset is (correctly) windows-1253, wherein the second case it is utf-8
Question
How can I set the charset to the response to be windows-1253 OR is there any way to decode utf-8 into windows-1253 in nodejs?
I found out about iconv, but as stated in this repository I have to install external tools to use it which is not my favorite think to do, so I would appreciate any alternatives.
The solution was to decode into ISO-8859-7 like this:
let customers = await axios.request({
method: 'POST',
url: url,
data: {
...
},
responseType: 'arraybuffer',
reponseEncoding: 'binary'
});
const decoder = new TextDecoder('ISO-8859-7');
let decodedCustomers = decoder.decode(customers.data)
Related
I'm developing a REST webservice in Node.js, with fastify framework, designed to respond to a specific client.
This is how the client calls my webservice:
POST /myws/scan HTTP/1.1
Host: myhost.io
User-Agent: AGENT
Content-Length: 45
Accept: */*
Content-Type: application/x-www-form-urlencoded
X-Forwarded-For: xxx.xxx.xxx.xxx
X-Forwarded-Proto: http
Accept-Encoding: gzip
{"FIELD1":"testvalue","FIELD2":True,"FIELD3":90}
As you can see the Content-Type is "application/x-www-form-urlencoded" but the request payload is in "application/json" format.
I can't change the client, so I have to adapt my webservice to manage this kind of calls.
I tried with #fastify/formbody package but I receive an error because I think it expects the request body in "application/x-www-form-urlencoded" format.
I tried also with this code:
app.addContentTypeParser('application/x-www-form-urlencoded', function (req, body, done) {
try {
var json = JSON.parse(body)
done(null, json)
} catch (err) {
err.statusCode = 400
done(err, undefined)
}
})
but I have a JSON deserializing error.
How I can manage these kind of calls and which is the best way to do that?
body is a stream.
You need to add the parseAs option:
fastify.addContentTypeParser(
'application/x-www-form-urlencoded',
{ parseAs: 'string' },
function (req, body, done) {
try {
const json = JSON.parse(body)
done(null, json)
} catch (err) {
err.statusCode = 400
done(err, undefined)
}
}
)
I would check the first char before running the JSON.parse and I would use https://www.npmjs.com/package/secure-json-parse (as Fastify does under the hood)
I'm struggling with AXIOS: it seems that my post request is not using my Cookie.
First of all, I'm creating an Axios Instance as following:
const api = axios.create({
baseURL: 'http://mylocalserver:myport/api/',
header: {
'Content-type' : 'application/json',
},
withCredentials: true,
responseType: 'json'
});
The API I'm trying to interact with is requiring a password, thus I'm defining a variable containing my password:
const password = 'mybeautifulpassword';
First, I need to post a request to create a session, and get the cookie:
const createSession = async() => {
const response = await api.post('session', { password: password});
return response.headers['set-cookie'];
}
Now, by using the returned cookie (stored in cookieAuth variable), I can interact with the API.
I know there is an endpoint allowing me to retrieve informations:
const readInfo = async(cookieAuth) => {
return await api.get('endpoint/a', {
headers: {
Cookie: cookieAuth,
}
})
}
This is working properly.
It's another story when I want to launch a post request.
const createInfo = async(cookieAuth, infoName) => {
try {
const data = JSON.stringify({
name: infoName
})
return await api.post('endpoint/a', {
headers: {
Cookie: cookieAuth,
},
data: data,
})
} catch (error) {
console.log(error);
}
};
When I launch the createInfo method, I got a 401 status (Unauthorized). It looks like Axios is not using my cookieAuth for the post request...
If I'm using Postman to make the same request, it works...
What am I doing wrong in this code? Thanks a lot for your help
I finally found my mistake.
As written in the Axios Doc ( https://axios-http.com/docs/instance )
The specified config will be merged with the instance config.
after creating the instance, I must follow the following structure to perform a post requests:
axios#post(url[, data[, config]])
My requests is working now :
await api.post('endpoint/a', {data: data}, {
headers: {
'Cookie': cookiesAuth
}
});
I'm trying to send an image and some data from an API to another. The image is stored in memory with multer. But when I want to send it, the body is just empty. I tried the same request with postman and it worked perfectly.
postman test
postman test image
server test
server test image
Here is some code. I removed some of it so you can read it better
export const saveImage = async ({ image, name, folder, options }: { image: any, name?: any, folder: string, options?: any }) => {
try {
const fd = new FormData();
fd.append("image", image.buffer, image.originalname);
if(options) {
fd.append("options[resize][height]", options?.resize?.height);
fd.append("options[resize][width]", options?.resize?.width);
}
if(name) fd.append("name", name);
fd.append("folder", folder);
fd.append("servideId", IMAGES_ID);
fd.append("serviceSecret", IMAGES_SECRET);
console.log(fd)
const formHeaders = fd.getHeaders();
const request = await axios.post(`${IMAGES_URL}/api/images`, {
headers: formHeaders,
body: fd
});
return request.data.id;
} catch (error) {
const { response } = error;
console.log(response.request.data)
if(error?.response?.data?.error) {
throw { statusCode: error.response.status, message: error.response.data.error }
}
console.error("Images API", error);
throw new InternalError("Something gone wrong");
}
}
When I log the FormData, I can see in _streams, the data that I'm sending, but the Images API receives an empty body.
FormData screenshot
If you need more information tell me, please! Thank you
The axios API for the post method is: axios.post(url[, data[, config]]). The second argument must always be the data you send along.
In your case axios thinks { headers: formHeaders, body: fd } is the body and the request ends up being application/json. To send a file with data using axios in Node.js, do the following:
const response = await axios.post(`${IMAGES_URL}/api/images`, fd, {
headers: {
...formHeaders,
'X-Custom-Header': 'lala', // optional
},
});
Your question inspired me to turn this answer into an article — Send a File With Axios in Node.js. It covers a few common pitfalls and you'll learn how to send files that are stored as a Buffer or coming from a Stream.
With Axios, you can directly use the form data without having to deal with headers.
axios.post("/api/images", fd)
If you wish to modify headers at some point in the future, you should pass the formData to the `data` field instead of `body`.
axios.post("/api/images", { headers: formHeaders, data: fd })
Correction in comments.
It can also be done using the Axios API syntax.
axios({method: 'post', url: 'url', data: fd, headers: {} })
In the backend, multer will add your files to req.file instead of req.body, if you have properly configured it to do so.
I'm trying to integrate our website with Converge API with Hosted Payments Page. Here is the link to their documentation https://developer.elavon.com/#/api/eb6e9106-0172-4305-bc5a-b3ebe832f823.rcosoomi/versions/5180a9f2-741b-439c-bced-5c84a822f39b.rcosoomi/documents?converge-integration-guide/book/integration_methods/../../book/integration_methods/hosted_payments.html
I'm having troubles getting past the first step which is requesting a transaction token from their API endpoint. I'm sending a POST request from my server using axios with the correct parameters and URL, but when I try and POST i get 400 Bad Request. When I make the same request in POSTMAN I get a 200 response with the transaction token. I talked to their developers and they said that everything I was doing was correct and that nothing seemed odd within my code, so even they were stumped as to why I couldn't make a POST request to their endpoint. Obviously there is something within my code that their API is not liking, or else I wouldn't be here trying to find answers for this.
Here is how I'm making the POST request:
app.get('/converge_token_req', (request, response) => {
let params = {
ssl_merchant_id: '*****',
ssl_user_id: '*****',
ssl_pin: '*****',
ssl_transaction_type: 'ccsale',
ssl_amount: '1.00'
}
axios.post('https://api.demo.convergepay.com/hosted-payments/transaction_token', params, {
headers: { 'Content_Type' : 'application/x-www-form-urlencoded' }
}).then((res) => {
response.send(res.data)
}).catch((error) => {
console.log('there was an error getting transaction token')
response.send(error.message)
})
})
Here are the Request Headers:
I'm honestly out of ideas to try. The developers say that everything looks just fine yet I'm unable to make a successful request to their API. If anyone has any thoughts on this that would be great. Thanks!
This code below worked for me:
app.get('/converge_token_req', (request, response) => {
let params = {
ssl_merchant_id: '*****',
ssl_user_id: '*****',
ssl_pin: '*****',
ssl_transaction_type: 'ccsale',
ssl_amount: '1.00'
}
axios({
method: 'post',
url: 'https://api.demo.convergepay.com/hosted-payments/transaction_token',
params: params
}).then((res) => { response.send(res.data)
}).catch((error) => {
console.log('there was an error getting transaction token: ',
error)
})
})
I've since found out the solution to my problem. The issue here is that converge expects a x-www-form-urlencoded string that needs to be Stringified before submitting the request. I found a library that works well for this called qs and I used it like so:
let params = qs.stringify({ // need this if content_type is application/x-www-form-urlencoded
ssl_merchant_id: env.CONVERGE.MERCHANT_ID,
ssl_user_id: env.CONVERGE.USER_ID,
ssl_pin: env.CONVERGE.PIN,
ssl_transaction_type: request.query.type,
ssl_amount: request.query.amount,
ssl_email: request.query.email,
ssl_company: request.query.company,
ssl_avs_address: request.query.address,
ssl_avs_zip: request.query.zip,
ssl_description: request.query.desc,
})
axios.post('https://api.convergepay.com/hosted-payments/transaction_token', params, {
headers: {
'Content_Type' : 'application/x-www-form-urlencoded; charset=UTF-8'
}
}).then((res) => {
response.send(res.data)
}).catch((error) => {
console.log('there was an error getting transaction token')
response.send(error.message)
})
I think you could also get away with just using JSON.stringify() but this way worked for me.
I'm switching one of my projects from request over to something a bit more light-weight (such as got, axios, or fetch). Everything is going smoothly, however, I'm having an issue when attempting to upload a file stream (PUT and POST). It works fine with the request package, but any of the other three return a 500 from the server.
I know that a 500 generally means an issue on the server's end, but it is consistent only with the HTTP packages that I'm testing out. When I revert my code to use request, it works fine.
Here is my current Request code:
Request.put(`http://endpoint.com`, {
headers: {
Authorization: `Bearer ${account.token.access_token}`
},
formData: {
content: fs.createReadStream(localPath)
}
}, (err, response, body) => {
if (err) {
return callback(err);
}
return callback(null, body);
});
And here is one of the attempts using another package (in this case, got):
got.put(`http://endpoint.com`, {
headers: {
'Content-Type': 'multipart/form-data',
Authorization: `Bearer ${account.token.access_token}`,
},
body: {
content: fs.createReadStream(localPath)
}
})
.then(response => {
return callback(null, response.body);
})
.catch(err => {
return callback(err);
});
Per the got documentation, I've also tried using the form-data package in conjunction with it according to its example and I still get the same issue.
The only difference between these 2 I can gather is with got I do have to manually specify the Content-Type header otherwise the endpoint does give me a proper error on that. Otherwise, I'm not sure how the 2 packages are constructing the body with the stream, but as I said, fetch and axios are also producing the exact same error as got.
If you want any of the snippets using fetch or axios I'd be happy to post them as well.
I know this question was asked a while ago, but I too am missing the simple pipe support from the request package
const request = require('request');
request
.get("https://res.cloudinary.com/demo/image/upload/sample.jpg")
.pipe(request.post("http://127.0.0.1:8000/api/upload/stream"))
// Or any readable stream
fs.createReadStream('/Users/file/path/localFile.jpeg')
.pipe(request.post("http://127.0.0.1:8000/api/upload/stream"))
and had to do some experimenting to find similar features from current libraries.
Unfortunately, I haven't worked with "got" but I hope the following 2 examples help someone else that are interested in working with the Native http/https libraries or the popular axios library
HTTP/HTTPS
Supports piping!
const http = require('http');
const https = require('https');
console.log("[i] Test pass-through: http/https");
// Note: http/https must match URL protocol
https.get(
"https://res.cloudinary.com/demo/image/upload/sample.jpg",
(imageStream) => {
console.log(" [i] Received stream");
imageStream.pipe(
http.request("http://localhost:8000/api/upload/stream/", {
method: "POST",
headers: {
"Content-Type": imageStream.headers["content-type"],
},
})
);
}
);
// Or any readable stream
fs.createReadStream('/Users/file/path/localFile.jpeg')
.pipe(
http.request("http://localhost:8000/api/upload/stream/", {
method: "POST",
headers: {
"Content-Type": imageStream.headers["content-type"],
},
})
)
Axios
Note the usage of imageStream.data and that it's being attached to data in the Axios config.
const axios = require('axios');
(async function selfInvokingFunction() {
console.log("[i] Test pass-through: axios");
const imageStream = await axios.get(
"https://res.cloudinary.com/demo/image/upload/sample.jpg",
{
responseType: "stream", // Important to ensure axios provides stream
}
);
console.log(" [i] Received stream");
const upload = await axios({
method: "post",
url: "http://127.0.0.1:8000/api/upload/stream/",
data: imageStream.data,
headers: {
"Content-Type": imageStream.headers["content-type"],
},
});
console.log("Upload response", upload.data);
})();
Looks like this was a headers issue. If I use the headers directly from FormData (i.e., headers: form.getHeaders()) and just add in my additional headers afterwards (Authorization), then this ends up working just fine.
For me just works when I added other parameters on FormData.
before
const form = new FormData();
form.append('file', fileStream);
after
const form = new FormData();
form.append('file', fileStream, 'my-whatever-file-name.mp4');
So that way I can send stream from my backend to another backend in node, waiting a file in multipart/form-data called 'file'