Calling the Riot-Api Im receiving incomplete JSON on a https GET-request.
After debugging, I realized that depending how much I wait (breakpoint) pre-executing the
https on'data' callback Im actually receiving the complete JSON object.
(Average API response time for me is 200-300ms)
let getOptions = function(url) {
return {
host: 'na.api.pvp.net',
port: 443,
path: `${url}?api_key=${apiKey}`,
method: 'GET'
};
}
exports.Call = function(url, callback) {
let response = {};
let req = https.request(getOptions(url), function(res) {
response.statusCode = res.statusCode;
res.on('data', function(data) {
response.json = JSON.parse(data);
callback(response);
});
});
req.on('error', function(err) {
response.err = err;
callback(response);
});
req.end();
};
Running the code without breakpoints or only breaking a short time I run either into error:
JSON.parse(data): Unexpected Token in JSON at position ...
or
JSON.parse(data): Unexptected end of JSON Input.
As I expect the 'data' callback to be executed only after the request is complete im confused about how to fix it (without artificially delaying it ofc.).
http.request returns a stream – its not a simple callback that contains the whole response.
You will have to buffer and concatenate everything if you want to parse the whole response.
I would strongly recomment to use a helper library like got or request
Related
I'm making a cryptocurrency dashboard for a project, and I'm completely new to Node and Express. This is what I have currently
app.get('/search', function(req,res){
res.writeHead(200, {'Content-Type': 'text/html'});
res.write(req.url);
data = []
var options = {
"method": "GET",
"hostname": "rest.coinapi.io",
"path": "/v1/assets",
"headers": {'X-CoinAPI-Key': 'MY_API_KEY_HERE'}
};
const request = https.request(options, (data, response) => {
response.on('data', d => {
data.push(d);
})
});
console.log(data);
request.end();
res.end();
})
The idea is on my front end, I have a button that when clicked, will make a request to the CoinAPI api, getting all reported assets and current values. I'm not quite sure how I'm supposed to send that response data back to my frontend as a response. So, I tried to pull the response data out of the JSON that gets returned by the https.request line. I have a data array data = [] as you can see at the top of my code.
I originally had my request set up like:
const request = https.request(options, response => {
but when I would try to push d onto data, I console logged and the data array was empty. This makes sense, the data array is out of scope of the request function, so data doesn't get updated. But when I tried to pass data into the function, I errored out.
Basically I want to be able to send the JSON data back to my front end after making the request to CoinAPI. If I do process.stdout.write(d) in my https.request callback, I do actually see the coinapi data coming back. I just don't know how to send it to the front end as part of my response.
Issues:
The use of (data, response) is incorrect. The first and only argument is response so it should be (response).
The on('data') event receives buffered data. Concatenating it to a final string is the standard usage, not appending an array.
You're missing an on('end') which you should use to output the final data or do any processing.
Using res.write you're sending a text/html content type and some content which you don't want if the goal is to output JSON which the frontend can parse and use.
Missing an error handler for the API call.
Complete updated code:
app.get('/search', function(req,res){
let str = '';
const options = {
"method": "GET",
"hostname": "rest.coinapi.io",
"path": "/v1/assets",
"headers": {'X-CoinAPI-Key': 'MY_API_KEY_HERE'}
};
const request = https.request(options, (response) => {
response.on('data', d => {
str += d;
});
response.on('end', () => {
try {
let obj = JSON.parse(str);
// do any manipulation here
res.json(obj);
} catch(e){
console.log(e);
res.status(500).json({ message: 'Something went wrong - parse error' });
}
});
});
request.end();
request.on('error', (e) => {
console.log(e);
res.status(500).json({ message: 'Something went wrong - req error' });
});
});
I added a JSON.parse() to show how you'd handle that if you wanted to do some manipulation of the data before sending it to the frontend. If you simply want to return the exact response of the coin API then use an end event like:
response.on('end', () => {
res.json(str);
});
To send JSON data back to the client as response all you need to do is :
return res.json(data);
Its as simple as this. :)
I have a NodeJS API. The logic in the API needs to make an http get request to google.com, capture the response from google.com, and then return the html response to the original API call. My problem is capturing the http response from google asynchronously and returning it to the original API call.
// Entry point to /api/specialday
module.exports = function(apiReq, apiRes, apiNext) {
var options = {
host: 'www.google.com'
};
callback = function(googleRes) {
var str = '';
// another chunk of data has been recieved, so append it to `str`
googleRes.on('data', function (chunk) {
str += chunk;
});
// capture the google response and relay it to the original api call.
googleRes.on('end', function () {
apiRes.send(str);
});
}
http.request(options, callback).end();
}
The error I get here is Uncaught TypeError: Cannot read property 'send' of undefined. I understand why I'm getting the error (because apiRes is out of scope), I just can't figure out how to do it right. Any help much appreciated!
The reason you are seeing the above error is because the original response object apiRes is gone by the time you have received the response from the google API.
As far as I can tell you will have to bind() the apiRes twice (untested):
callback = function(googleRes) {
var str = '';
// another chunk of data has been recieved, so append it to `str`
googleRes.on('data', function (chunk) {
str += chunk;
});
// capture the google response and relay it to the original api call.
googleRes.on('end', function () {
apiRes.send(str);
}.bind(apiRes));
}.bind(apiRes)
A more modern solution would be to use promises for this task https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Promise
Promises, that's it! Thanks Michal. Below is a simplified version of my implementation.
// Entry point to /api/specialday
module.exports = function(apiReq, apiRes, apiNext) {
var p1 = new Promise(
// The resolver function is called with the ability to resolve or
// reject the promise
function(resolve, reject) {
var options = {
host: 'www.google.com'
};
callback = function(googleRes) {
var str = '';
// another chunk of data has been recieved, so append it to `str`
googleRes.on('data', function (chunk) {
str += chunk;
});
// capture the google response and relay it to the original api call.
googleRes.on('end', function () {
resolve(str);
});
}
http.request(options, callback).end();
)};
p1.then(function(googleHtml) {
apiRes.status(200).send(googleHtml);
}
}
Then I can run my app and call the api using Postman at http://localhost:8080/api/gains:
Directly pipe output with apiRes, sample using request :
var request = require("request");
// Entry point to /api/specialday
module.exports = function(apiReq, apiRes, apiNext) {
request.get('http://www.google.fr').pipe(apiRes);
});
I try to put many callbacks in a callback, but the program will shut down after the return of several success requests and post "socket hang up" error. I try to collect data from response and output them at once, can someone tell me which part goes wrong...
By the way, I hide the details of request method, I promise the request method works on http call.
http.request(options1,function(data){
var condition=data.length;
var result=[];
data.foreach(item,function(data){
http.request(options2, function(data){
if(data) result.push(data);
condition--;
if(condition<=0) console.log(result);
}
});
});
for my http.request method
var request=function(options,callback){
http.request(options,function(res){
var body;
res.on('data',function(chunk){
body+=chunk;
});
res.on('end',function(){
callback(JSON.parse(body));
});
request.end();
};
That's not the correct usage of http.request().
The http.request() callback is passed an IncomingMessage object, not buffered response data.
EDIT: Your custom request() should look something like this, although you should handle errors too. Note the proper placement of request.end() and initializing var body = '':
function request(options, callback) {
http.request(options,function(res) {
var body = '';
res.setEncoding('utf8');
res.on('data',function(chunk) {
body += chunk;
}).on('end',function() {
callback(JSON.parse(body));
});
}).end();
}
You're missing .end() for your requests so that node.js knows you are ready to actually send the HTTP request: http.request(..., ....).end();. This is the cause of your particular error... the server hangs up the connection because it got tired of waiting for your request after the TCP connection was opened.
in my project I have to do a request to upcDatabase.com, I amworking with nodeJS, I get the answer from the server but I do not how to extractthe data this are the important part of my code:
module.exports = function (http,upc){
var upc_ApiKey = "XXX",
url = "http://upcdatabase.org/api/json/"+upc_ApiKey+'/'+upc;
http.get(url,function(resp){
// my code to read the response
I do not get any error, but the resp is a big Json and I do not know where to find the data
I would recommend you using the superagent module. It provides much more functionality than the built-in http request and it will automatically parse the response for you.
request
.get(url)
.end(function(err, res) {
if (res.ok) {
// Her ethe res object will be already parsed. For example if
// the server returns Content-Type: application/json
// res will be a javascript object that you can query for the properties
console.log(res);
} else {
// oops, some error occurred with the request
// you can check the err parameter or the res.text
}
});
You could achieve the same with the built-in http module but with much more code:
var opts = url.parse(url);
opts.method = "GET";
var req = http.request(opts, function (res) {
var result = "";
res.setEncoding("utf8");
res.on("data", function (data) {
result += data;
});
if (res.statusCode === 200) {
res.on("end", function () {
// Here you could use the result object
// If it is a JSON object you might need to JSON.parse the string
// in order to get an easy to use js object
});
} else {
// The server didn't return 200 status code
}
});
req.on("error", function (err) {
// Some serious error occurred during the request
});
// This will send the actual request
req.end();
I probably have some issues with the asyncness of Node.js.
rest.js
var Shred = require("shred");
var shred = new Shred();
module.exports = {
Request: function (ressource,datacont) {
var req = shred.get({
url: 'ip'+ressource,
headers: {
Accept: 'application/json',
},
on: {
// You can use response codes as events
200: function(response) {
// Shred will automatically JSON-decode response bodies that have a
// JSON Content-Type
if (datacont === undefined){
return response.content.data;
//console.log(response.content.data);
}
else return response.content.data[datacont];
},
// Any other response means something's wrong
response: function(response) {
return "Oh no!";
}
}
});
}
}
other.js
var rest = require('./rest.js');
console.log(rest.Request('/system'));
The problem ist if I call the request from the other.js I always get 'undefined'. If I uncomment the console.log in rest.js then the right response of the http request is written to the console. I think the problem is that the value is returned before the actual response of the request is there. Does anyone know how to fix that?
Best,
dom
First off, it is useful to strip down the code you have.
Request: function (ressource, datacont) {
var req = shred.get({
// ...
on: {
// ...
}
});
}
Your Request function never returns anything at all, so when you call it and console.log the result, it will always print undefined. Your request handlers for the various status codes call return, but those returns are inside of the individual handler functions, not inside Request.
You are correct about the asynchronous nature of Node though. It is impossible for you to return the result of the request, because the request will still be in progress when your function returns. Basically when you run Request, you are starting the request, but it can finish at any time in the future. The way this is handled in JavaScript is with callback functions.
Request: function (ressource, datacont, callback) {
var req = shred.get({
// ...
on: {
200: function(response){
callback(null, response);
},
response: function(response){
callback(response, null);
}
}
});
}
// Called like this:
var rest = require('./rest.js');
rest.Request('/system', undefined, function(err, data){
console.log(err, data);
})
You pass a third argument to Request which is a function to call when the request has finished. The standard Node format for callbacks that can fail is function(err, data){ so in this case on success you pass null because there is no error, and you pass the response as the data. If there is any status code, then you can consider it an error or whatever you want.