I would like to use the https library in node.js to send a request to this api:
https://rapidapi.com/dimas/api/NasaAPI?endpoint=apiendpoint_b4e69440-f966-11e7-809f-87f99bda0814getPictureOfTheDay
The given example on the RapidAPI website uses Unirest, and I would like to only use the https library. I've tried to write it like this:
const https = require('https');
var link = "https://NasaAPIdimasV1.p.rapidapi.com/getPictureOfTheDay";
var options = {host: "https://NasaAPIdimasV1.p.rapidapi.com/getPictureOfTheDay",
path: "/", headers: {"X-RapidAPI-Key": "---MY KEY(Yes, I've replaced it)---", "Content-Type": "application/x-www-form-urlencoded"}}
https.get(link, options, (resp) => {
let data = '';
resp.on('data', (chunk) => {
data += chunk;
});
resp.on('end', () => {
console.log(data);
});
}).on("error", (err) => {
console.log("https error 4: " + err.message);
});
But that returns the following response:
{"message":"Endpoint\/ does not exist"}
Thanks for any help
There are several mistakes.
First, you essentially pass URL in https twice - first as link param, second as combination of host and path properties for options param.
Second, your host is actually the full path - but it shouldn't be. In the end, looks like the library got confused and sent request to https://NasaAPIdimasV1.p.rapidapi.com/ instead.
Finally, this particular API requires using 'POST', not 'GET' method. That's actually mentioned in the documentation. That's why you have 'endpoint does not exist' error even on correctly formed request.
One possible approach is dropping link altogether, sending URL as part of options:
var options = {
host: 'NasaAPIdimasV1.p.rapidapi.com',
method: 'POST',
path: '/getPictureOfTheDay',
headers: {/* the same */}
};
https.request(options, (resp) => { /* the same */ }).end();
Related
Stack Overflow community, greetings. I'm trying to pass the response of a new request on the request object using the Node HTTP Module for a basic autocomplete search app for my website (i.e using Node as a proxy that will transform and redirect the requests within the server).
The flow basically is:
Client Browser - Node - ElasticSearch - Node - Client Browser
I've started with:
Listen to requests with http.createServer (function (req,res)
Get the body from the req object and use it in a new request with http.request(options, function (newReqResponse)
Get the body from that newReqResponse object and send it back to the client on the res object
The problem is that the content of newReqResponse is always outdated (trails behind the last typed character). i.e.:
If I type "te", the content of newReqResponse corresponds to that if I had typed only "t".
If I type "test", it corresponds to that if I had typed "tes".
And so on.
I've tried to solve it using Node.js streams and using the file system module to write and read files sync and async, but the result is the same. Here's a sample of the whole -code- picture:
var http = require('http');
var fs = require('fs');
http.createServer(function (req, res) {
var reqBody = '';
var newReqResponseBody = "";
req.on('data', function (chunk) {
reqBody += chunk;
fs.writeFile('reqbody.json', reqBody, function(err) {
if (err) {
throw err;
}
});
var options = {
hostname: '127.0.0.1',
port: 9500,
method: 'GET',
path: '/_search',
headers: { host: 'es',
'content-length': Buffer.byteLength(reqBody),
'content-type': 'application/json',
accept: 'application/json' },
};
var newReq = http.request(options, function (newReqResponse) {
newReqResponse.setEncoding("UTF-8");
newReqResponse.on('data', function (ch) {
newReqResponseBody += ch;
});
newReqResponse.on("end", function() {
fs.writeFile("newReqResponseBody.json", newReqResponseBody, function(err) {
if (err) {
throw err;
}
});
});
});
newReq.on("error", function(err) {
console.log(`problem with request: ${err.message}`);
});
newReq.write(reqBody);
newReq.end();
});
req.on('end', function() {
var responseBody = fs.readFileSync('newReqResponseBody.json', 'utf8');
console.log(responseBody);
res.end(responseBody);
});
}).listen(3000, '127.0.0.1');
Is there a workaround to work with requests and responses within the http server? If there isn't, I'll be very grateful if you give me any directions on how to solve this.
Since the planned use for Node is rather basic, I prefer to stick with core modules rather than having to get new ones from npm, unless that it's necessary.
Thanks in advance.
EDIT:
All I had to do was to call res.end(responseBody) within the newReqResponse.on("end") callback, which is totally counterintuitive for me, but... it works.
Glad you solved your own problem. However, I see room for improvement (not sure if you're new), especially if you're transferring data. Which you can do with streams.
You can see that I didn't calculate the content length, you're asked not to and should get ignore (for this specific case) according to HTTP specification as streams pass data in chunks with 'Transfer-Encoding': 'chunked' header.
const fs = require('fs');
const http = require('http');
const options = {
hostname: '127.0.0.1',
port: 9500,
method: 'GET',
path: '/_search',
headers: {
'Content-Type': 'application/json'
}
};
http.createServer((req, res) => {
req.pipe(fs.createWriteStream('reqBody.json'));
let request = http.request(options, (newRes) => {
newRes.pipe(res);
});
fs.createReadStream('reqBody.json').pipe(request);
res.setHeader('Content-Type', 'application/json');
}).listen(3000, '127.0.0.1');
You can shorten this snippet more if you don't want your data saved in the future and only want to pipe the req stream to request.
I am trying to get an authentication token from an API.
the request is supposed to look like
POST /oauth2/token HTTP/1.1
Host: mysageone.ca.sageone.com
client_id=4b64axxxxxxxxxx00710&
client_secret=iNumzTxxxxxxxxxxhVHstrqWesH8tm9&
code=12a0f9c12cxxxxxxxxxxxxxxx92a48cc1f237ead&
grant_type=authorization_code&
redirect_uri=https://myapp.com/auth/callback
My current code keeps giving me status 400. I have tried to modify the headers but it doesn't work. i have also tried to make the required parameters part of the path using ?.
const http = require('http');
var options = {
hostname: 'app.sageone.com',
path: '/oauth2/token',
method: 'POST',
headers: {
"client_id":"xxxxx",
"client_secret":"xxxxx",
"code":"xxxxxx",
"grant_type":"authorization_code",
"redirect_uri":"https://some link"
}
};
console.log('in users file point 2');
var req1 = http.request(options, (res1) => {
console.log('statusCode:', res1.statusCode);
console.log('headers:', res1.headers);
console.log('message',res1.statusMessage);
res1.on('data', (d) => {
res.json(d);
});
});
req1.on('error', (e) => {
console.error('error starts here',e);
});
req1.end();
});
Looks to me like your problem is not with Node.js, but with your use of Sage One's api. This is the relevant documentation that might solve your problem.
From a quick glance it looks like you want to send a GET not a POST, and you should send those parameters in the URL. Here is the example URL they give:
https://www.sageone.com/oauth2/auth?response_type=code&client_id=4b64axxxxxxxxxx00710&redirect_uri=https://myapp.com/auth/callback
&scope=full_access
I've never used Sage One before, but that would match my experience with other OAuth APIs.
I am trying to create http get request from node, to get information from youtube URL. When I click it in browser I get json response but if I try it from node, I get ssl and other types of error. What I have done is,
this.getApiUrl(params.videoInfo, function (generatedUrl) {
// Here is generated URL - // https://www.googleapis.com/youtube/v3/videos?key=AIzaSyAm_1TROkfNgY-bBuHmSaletJhVQmkycJc&id=_H_r9qVrf24&part=id%2Csnippet%2CcontentDetails%2Cplayer%2Cstatistics%2Cstatus
console.log(generatedUrl);
var req = http.get(generatedUrl, function (response) {
var str = '';
console.log('Response is ' + response.statusCode);
response.on('data', function (chunk) {
str += chunk;
});
response.on('end', function () {
console.log(str);
});
});
req.end();
req.on('error', function (e) {
console.log(e);
});
});
I get this error
{
"error": {
"message": "Protocol \"https:\" not supported. Expected \"http:\".",
"error": {}
}
}
When I make it without https I get this error,
Response is 403
{"error":{"errors":[{"domain":"global","reason":"sslRequired","message":"SSL is required to perform this operation."}],"code":403,"message":"SSL is required to perform this operation."}}
You need to use the https module as opposed to the http module from node, also I would suggest one of many http libraries that provide a higher level api such as wreck or restler which allow you to control the protocol via options as opposed to a different required module.
Your problem is obviously accessing content served securely with http request hence, the error. As I have commented in your question, you can make use of https rather than http and that should work but, you can also use any of the following approaches.
Using request module as follow:
var url = "https://www.googleapis.com/youtube/v3/videos?key=AIzaSyAm_1TROkfNgY-bBuHmSaletJhVQmkycJc&id=_H_r9qVrf24&part=id%2Csnippet%2CcontentDetails%2Cplayer%2Cstatistics%2Cstatus";
request(url, function (error, response, body) {
if (!error && response.statusCode == 200) {
console.log(body);
}
});
Using https module you can do like below:
var https = require('https');
var options = {
hostname: 'www.googleapis.com', //your hostname youtu
port: 443,
path: '//youtube/v3/videos?key=AIzaSyAm_1TROkfNgY-bBuHmSaletJhVQmkycJc&id=_H_r9qVrf24&part=id%2Csnippet%2CcontentDetails%2Cplayer%2Cstatistics%2Cstatus',
method: 'GET'
};
//or https.get() can also be used if not specified in options object
var req = https.request(options, function(res) {
console.log("statusCode: ", res.statusCode);
console.log("headers: ", res.headers);
res.on('data', function(d) {
process.stdout.write(d);
});
});
req.end();
req.on('error', function(e) {
console.error(e);
});
You can also use requestify module and
var url = "https://www.googleapis.com/youtube/v3/videos?key=AIzaSyAm_1TROkfNgY-bBuHmSaletJhVQmkycJc&id=_H_r9qVrf24&part=id%2Csnippet%2CcontentDetails%2Cplayer%2Cstatistics%2Cstatus";
requestify.get(url).then(function(response) {
// Get the response body
console.log(response.body);
});
superagent module is another option
var url = "https://www.googleapis.com/youtube/v3/videos?key=AIzaSyAm_1TROkfNgY-bBuHmSaletJhVQmkycJc&id=_H_r9qVrf24&part=id%2Csnippet%2CcontentDetails%2Cplayer%2Cstatistics%2Cstatus";
superagent('GET', url).end(function(response){
console.log('Response text:', response.body);
});
Last but not least is the unirest module allow you to make http/https request as simple as follow:
var url = "https://www.googleapis.com/youtube/v3/videos?key=AIzaSyAm_1TROkfNgY-bBuHmSaletJhVQmkycJc&id=_H_r9qVrf24&part=id%2Csnippet%2CcontentDetails%2Cplayer%2Cstatistics%2Cstatus";
unirest.get(url).end(function(res) {
console.log(res.raw_body);
});
There might be more options out there. Obviously you need to load the modules using require before using it
var request = require('request');
var https = require('https');
var requestify = require('requestify');
var superagent = require('superagent');
var unirest = require('unirest');
I provided extra details, not only to answer the question but, also to help others who browse for similiar question on how to make http/https request in nodejs.
I'm trying to create the express.js proxy to an external website to obtaining audio data from there. I know about modules like http-proxy, but think they are excessive for the case when only one url-bound request goes through proxy. I'm using the code below:
express.get('/proxy', function (req, res) {
var options ={
host: "website.com",
port: 80,
path: "/audio/test.mp3",
method: 'GET'
};
http.get(options, function (audioRes) {
var data = [], dataLen = 0;
audioRes.on('data', function(chunk) {
data.push(chunk);
dataLen += chunk.length;
})
.on('end', function() {
var buf = new Buffer(dataLen);
res.set(audioRes.headers);
res.send(buf);
});
})
.on('error', function (error) {
console.log(error.message);
});
});
I get response, but it cannot be decoded as a valid audio. While debugging with the Fiddler, I found out that the number of bites sent by a server mismatches the number specified in the Content-Length header (which indicates fewer bytes being retrieved).
I cannot figure out how to properly return the exact response that's been retrieved from the remote server. Would be grateful for any help.
To send request via proxy, you can set the proxy url in Host header. Also you have to specify the full URL of the external resource you are trying to access via proxy.
var http = require("http");
var options = {
host: "proxy",
port: 8080,
path: "http://www.google.com", //full URL
headers: {
Host: "10.1.2.3" //your proxy location
}
};
http.get(options, function(res) {
console.log(res);
});
I am not sure why it is not returning full response. Can you post your options.
Update
Try this inside /proxy after putting the options
http.get(options, function (audioRes) {
audioRes.pipe(res);
})
.on('error', function (error) {
console.log(error.message);
});
How would I go about downloading the contents of a URL in Node when using the Express framework? Basically, I need to complete the Facebook authentication flow, but I can't do this without GETing their OAuth Token URL.
Normally, in PHP, I'd use Curl, but what is the Node equivalent?
var options = {
host: 'www.google.com',
port: 80,
path: '/index.html'
};
http.get(options, function(res) {
console.log("Got response: " + res.statusCode);
}).on('error', function(e) {
console.log("Got error: " + e.message);
});
http://nodejs.org/docs/v0.4.11/api/http.html#http.get
The problem that you will front is: some webpage loads its contents using JavaScript. Thus, you needs a package, like After-Load which simulates browser's behavior, then gives you the HTML content of that URL .
var afterLoad = require('after-load');
afterLoad('https://google.com', function(html){
console.log(html);
});
Using http way requires way more lines of code for just a simple html page .
Here's an efficient way : Use request
var request = require("request");
request({uri: "http://www.sitepoint.com"},
function(error, response, body) {
console.log(body);
});
});
Here is the doc for request : https://github.com/request/request
2nd Method using fetch with promises :
fetch('https://sitepoint.com')
.then(resp=> resp.text()).then(body => console.log(body)) ;
Using http module:
const http = require('http');
http.get('http://localhost/', (res) => {
let rawHtml = '';
res.on('data', (chunk) => { rawHtml += chunk; });
res.on('end', () => {
try {
console.log(rawHtml);
} catch (e) {
console.error(e.message);
}
});
});
rawHtml - complete html of the page.
I just simplified example from official docs.
using Axios is much simpler
const axios = require("axios").default
const response = axios.get("https://google.com")
console.log(response.data)
or
const axios = require("axios").default
const response = axios.get("https://google.com").then((response)=>{
console.log(response.data)
})
for full docs, you can head over Axios Github