Submit POST form with latin1 charset in Node.js - node.js

Question

Short version of the problem : in my Node.js app using Sails.js framework, I'd like to send data (simple UTF-8 text charset) to a distant server POST form, which has ISO-8859-15 charset.
Details : using the request package, the form is submitted this way :
var postFields = {
'message' : message, // <== this var contains chars such as é,à,ç, etc.
'submit' : 'Send'
};
var opts = {
uri: url,
headers: {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:33.0) Gecko/20100101 Firefox/33.0',
'Cookie' : cookie,
'Content-type': "application/x-www-form-urlencoded; charset=ISO-8859-15"
},
form: postFields
};
request.post(opts, function(error, response, body) {
if (!error && response.statusCode == 200) {
return callback(null, true);
} else {
return callback(null, false);
}
});
This works fine, my form is sent successfully, except the "é" chars turned into ugly "é" to the distant server. In the 'Content-type' property of the opts object, I tried to set charset to ISO-8859-15 / UTF-8 / none, text/html or application/x-www-form-urlencoded, still doesn't work...
Don't know if this helps, but here is what I have to do :
1) get data from the distant server (latin1 encoded), then convert it to UTF-8 text :
var iconv = require('iconv-lite');
request(options, function(error, response, body) {
var decodedBody = iconv.decode(body, 'iso-8859-15');
return callback(null, decodedBody)
}
});
Great, now I have beautiful accents.
2) update this data, then send it back to the distant server using the first code block above.
Actually I used to do exactly the same operations with PHP, which worked fine : instead of sending the "raw" UTF-8 text message (postFields object), it was passed into the utf8_decode() func and the distant server displayed correctly the accents.
I gave a try to js implementation of this function => the accents are correctly sent, but this time all other normal chars are turned weird...
Hoping this is clear enough, if you need more details, code, or practical examples : no problem! Thanks for reading :)

Answer 1

Well, after 3 years and 10 months I faced a similar problem but I managed to find a solution!
To resolve this, you need to use urlencode and body instead of form, like this:
const urlencode = require('urlencode');
const request = require('request');
const fields = {
message: 'é à ç'
};
const opts = {
uri: 'https://localhost',
headers: {
'Content-type': 'application/x-www-form-urlencoded; charset=ISO-8859-1'
},
body: urlencode.stringify(fields, {
charset: 'latin1'
})
};
request.post(opts, (err, res, body) => {
if (err) {
throw err;
}
console.log(res.statusCode);
});
This is because request encodes form fields using UTF-8 so you never get Latin-1 characters even if you encode them before (you'll end up with re-encoded characters).
I've tested this and it works as expected, so I hope this helps someone else!