I'm using request.js and cheerio to capture some text of my site.
The original text is English, and I would like to capture the translated version.
Here's what I have for the request:
request.get({uri:
'http://immocostablancasofia.com/listing/villa-in-lliber-ref-p01638/?lang=nl',
'followAllRedirects': true}
It returns the English version instead of the Dutch one.
I also tried using formData, with no luck.
Add options for request:
var options = {
url: 'http://immocostablancasofia.com/listing/villa-in-lliber-ref-p01638/',
headers: {'Accept-Language': 'nl-NL'},
qs: {lang:'nl'}
};
And
request.get(options, callback);
I changed the code adding ',headers:{'Accept-Language': 'nl-NL'}' , and it works!
request.get({uri:
'http://immocostablancasofia.com/listing/villa-in-lliber-ref-p01638/?lang=nl',headers:{'Accept-Language': 'nl-NL'}
'followAllRedirects': true}
Related
This is in Google Script Editor. I've done a lot of searching but for some reason I can't find any information about a simple thing such as appending to an object.
I have an object like this:
var get_options = {
'method': 'get',
'muteHttpExceptions': true,
};
How would I go about appending to that object?
I want to add 'authorization': 'token', to that object so the object will look like this:
var get_options = {
'method': 'get',
'muteHttpExceptions': true,
'authorization': 'token',
};
Please help! Thank you in advance.
Google App Script (GAS) uses javascript platform. So whenever you have some basic coding syntax you want to look up, refer to javascript documentation
To answer your question in brief,to add key pair value to get_options you would do the following
var get_options = {
'method': 'get',
'muteHttpExceptions': true,
};
get_options.authorization = “token”
//or
get_options["authorization"] = “token”
More detailed explanation you can be found in this SO post
I am making a POST request to retrieve a pdf. The request works fine if I do it in postman, but I get an empty pdf if I do it through node.js using the request package. Here's my request using the request package:
let body = {
attr1: "attr1",
attr2: "attr2"
}
let opts = {
url: "some_url",
method: "post",
headers: {
"Content-Type": "application/x-www-form-urlencoded",
},
body
}
request(requestOpts).then(pdf => {
console.log(pdf) // prints out the binary version of the pdf file
fs.writeFileSync("testing.pdf", pdf);
});
I use the exact same request parameters when I use postman but it returns the pdf w/ the correct content.
Can someone help? Or is the way I am saving my pdf incorrect?
Thanks in advance!
Solution - i had to set encoding: false in the request options.
Try
fs.writeFileSync("testing.pdf", pdf, 'binary');
The third argument here tells fs to write binary rather than trying to UTF-8 encode it.
According to the docs the third paramter should be a string that represents the encoding.
For pdf files the encoding is 'application/pdf'
So this should work for you : fs.writeFileSync("testing.pdf", pdf, 'application/psf');
I am calling Google Place api web service with Request in Node js. The body of the request gives an error Invalid request. One of the input parameters contains a non-UTF-8 string. because I use Khmer characters in the url parameters (keyword parameter).
nearByUrl = 'https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=11.55082623,104.93225202&radius=400&keyword=ស្ថានីយប្រេង&key=' + MY_KEY;
request({
url: nearByUrl,
json: true
}, function (error, response, body) {
console.log(JSON.stringify(body, null, 2));
})
However, I can get a valid JSON with results when calling the exact same URL with Khmer characters from Chrome browser.
Is this problem related to Request?
How can I fix this?
So if you enter URL to which you want to send request in Chrome and open development tools, you will see that raw URL to which request is sent looks similar to this:
https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=11.55082623,104.93225202&radius=400&keyword=%E1%9E%9F%E1%9F%92%E1%9E%90%E1%9E%B6%E1%9E%93%E1%9E%B8%E1%9E%99%E1%9E%94%E1%9F%92%E1%9E%9A%E1%9F%81%E1%9E%84
Basically Chrome encoded all query parameters to ASCII, and when you enter directly parameters to URL, query parameters are not encoded. But if you send your parameters to request library through qs object, library will encoded them for you, and you will not have issue from the question.
var request = require("request")
var option = {
uri: 'https://maps.googleapis.com/maps/api/place/nearbysearch/json',
qs: {
location: '11.55082623,104.93225202',
radius: 1000,
keyword: 'ស្ថានីយប្រេង',
key: MY_KEY
}
};
request(
option, function (error, response, body) {
console.log(JSON.stringify(body, null, 2));
})
You could use method that is build into js library like this, but I personally think that first method is much better solution:
nearByUrl = 'https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=11.55082623,104.93225202&radius=400&keyword=' + encodeURIComponent(escape('ស្ថានីយប្រេង')) + '&key=' + MY_KEY;
request({
url: nearByUrl,
json: true
}, function (error, response, body) {
console.log(JSON.stringify(body, null, 2));
})
Why I think that first solution with qs param is better solution, because library is doing it for you and all parameters are encoded.
Better explanation of second method can be found here.
Hopefully this is solution to your issue :)
I'm extracting text from a Windows-1255-encoded webpage using Node.js. I'm trying to decode the text using the windows-1255.
After installing it using NPM and requiring it in the relevant file, I tried using it like this:
var title = windows1255.decode('#title').text());
This doesn't seem to have any effect. Any idea why?
Thanks!
Morgan
don't know if you still waiting for an answer about this issue, but the following worked for me...
When fetching the data (a file), I set the get options of encoding to be binary:
var options = {
method: 'GET',
url: 'myURL',
encoding: 'binary'
};
request(options, function (error, response, body) {
//deal with hebrew encoding
csvString = encoding.convert(body, 'UTF8', "CP1255").toString();
Then for I switch encoding from CP1255 (=windows1255) to UTF8.
Hope it helps :)
Is there any option for qs.stringify that will not encode the urls?
$ node
> var qs = require("querystring");
undefined
> qs.stringify({"url": "http://domain.com"});
'url=http%3A%2F%2Fdomain.com'
I want the folowing output:
'url=http://domain.com'
It's a little late, but for the next person, you can do:
querystring.unescape(myString)
Late answer again, but...
qs.stringify() has option encode:false which actually disable the URI encoding.
Qs.stringify documentation
You can also use it in nodejs request/request module as:
request({
url: 'http://url.domain'
qs: {
qs1: 'thisIsNotEncodedInTheRequest%20://асд'
},
qsStringifyOptions: {
// encoding: false /** (OLD VERSION - I think is deprecated yet) */
encode: false
}
});
Not directly, no. Although, if you are not escaping the value in the query string then there is hardly any benefit to using querystring at all. Mind as well just do: var q = 'url=http://domain.com'
EDIT: From looking at the source, the only way would be to change the behavior of (i.e. overwrite) the querystring escape() function - which is possible but not a good idea.
Here is the answer:
qs.stringify({url: "http://domain.com"}, { encodeURIComponent: uri => uri });
The option "encodeURIComponent: uri => uri" is to disable the encoding.
This worked for me
qs.stringify({url: "http://example.com"}, null, null, { encodeURIComponent: qs.unescape });