I am using nodejs streams to fetch data from my level database (using leveldb). Here is a snippet of the code I use:
app.get('/my/route', function (req, res, next) {
leveldb.createValueStream(options)
.pipe(map.obj(function (hash) {
level.get(somekeys, function (err, item) {
return item
})
}))
.pipe(res)
})
So far, I have been using the list-stream module to get the data on the client side.
Now I'd like to retrieve the information on the client side as a stream. I've read this post (http://www.kdelemme.com/2014/04/24/use-socket-io-to-stream-tweets-between-nodejs-and-angularjs/) on how to do it using socket.io but I don't want to use socket.io
Is there any simple way to do it?
This can be done with Shoe. You have to compile the client code with browserify and you can have a stream in the browser that receives the data from the server.
The createValueStream basically is a read stream, then you can listen to the event eg., data, end: https://github.com/substack/node-levelup#createValueStream
Just need to listen to end event to finish the stream.
app.get('/my/route', function (req, res, next) {
leveldb.createValueStream(options)
.pipe(map.obj(function (hash) {
level.get(somekeys, function (err, item) {
return item
})
}))
.pipe(res)
.on('end', res.end)
})
Related
I am trying to develop a MERN stack application, and I have done numerous attempts at this. So, what I am trying to achieve is have some data I am pulling from an api and dump it to a database, then query from the database to create a JSON file every 5 minutes(using jenkins and python, the best approach I can think of). Below is a method I am implementing and it does not work. If I remove the setInterval() function and un-comment the callback function, the code works but does not update the data.
const express = require('express');
const fs = require('fs');
const app = express();
// Read JSON File
function readJSON(callback) {
fs.readFile('./name.json', "utf8", function(err, result) {
if (err) callback (err);
callback(JSON.parse(result));
});
}
// Process JSON File during callback
// readJSON(function(res) {
// app.get('/api/customers', (request, response) => {
// response.json(res);
// });
// });
// Attempt to run every 5 minutes
setInterval(readJSON(function(res) {
app.get('/api/customers', (request, response) => {
response.json(res);
})}, 60000 * 5); // 5 Minutes
const port = 5000
app.listen(port, () => `Server running on port ${port}`);
I thought of using sockets, but I don't want it to be real-time, only live data on an interval. Restful API's I don't believe are a good fit here either, I don't want a 2-way communication to modify/update the data. If my approach is bad, please let me know why you'd pick another approach. I am just trying to establish a foundation in full-stack web dev. Thanks!
A logical code would be:
On server side:
function readJSON(callback) {
fs.readFile('./name.json', "utf8", function(err, result) {
if (err) callback(err);
callback(null, JSON.parse(result));
});
}
app.get('/api/customers', (request, response) => {
readJSON((err, nameContent) => {
if(err) {
response.status(500).send(err);
}
response.send(nameContent);
})
});
And in the client side ask for the data every 5 minutes:
someAjaxMethod('/api/customers', (err, nameContent) => console.log(err, nameContent));
I am currently working on an admin panel for this website I am creating, so I am able to accept payments via Braintree but I need to implement the ability to retrieve a customers transactions but once a header is sent it sends just one of them and not the whole thing. Is it possible to combine the json to an array so it will send in the one header?
CODE:
router.get('/:cid/test', function(req, res) {
var stream = gateway.transaction.search(function (search) {
search.customerId().is(req.params.cid);
}, function (err, response) {
response.each(function (err, transaction) {
return res.render('admin/test', {transaction: transaction});
});
});
});
This is solely following the Braintree documentation and I know exactly why the error occurs. Any help is really appreciated and I am terrible at explaining so if you need to know more information please give me a holler!
UPDATE: So, I figured I would explore another method and I noticed the 'response' gives back an array of ids. So I will just use EJS to loop through all those and then have a seperate page for each transaction.
Disclaimer: I work for Braintree :)
As Robert noted, you can only call res.render (or any of the response methods that end the request) once per request (hence the error from express).
Unfortunately, you cannot treat response as an array, so you will need to use
one of the two documented ways of interacting with search responses. I personally prefer the streams approach because it is clearer:
app.get('/stream', function (req, res) {
var transactions = []
var transactionStream = gateway.transaction.search(function (search) {
search.customerId().is(req.params.cid);
})
transactionStream.on('data', function (transaction) {
transactions.push(transaction)
})
transactionStream.on('error', function () { /* handle errors */ })
transactionStream.on('end', function () {
res.json({transactions: transactions});
})
})
Alternately, you can use the ids property of response to compare the transactions array that you build from each to know when to end the request:
app.get('/calback', function (req, res) {
var transactionStream = gateway.transaction.search(function (search) {
search.customerId().is(req.params.cid);
}, function (err, response) {
var transactions = []
response.each(function (err, transaction) {
transactions.push(transaction)
if (transactions.length === response.ids.length) {
res.json({transactions: transactions});
}
})
})
})
You can only render one response per route. So you can only call this once and not in a loop:
res.render('admin/test', {transaction: transaction}); });
You can use the each method to iterate through the response and build up a result:
var transactions =[];
response.each(function (err, transaction) { transactions.push(transaction) });
return res.render('admin/test', {transaction: transactions});
That would work if the each method is synchronous. If it's not (and Nick would know), use the solution below.
I'm starting to use NodeJs recently and I'm trying to create a API that will get some information from web compile it and show to the user.
My question is the follow
router.get('/', function (req, res, next) {
https.get(pageUrl, function (res) {
res.on('data', function (responseBuffer) {
//Important info;
info = responseBuffer;
}
}
res.render('page', { important: info});
}
How can I wait until I have the "info" var and then send the res.render. Because right now if I try to wait it usually the program ends and don't wait the result.
Thanks.
Assuming your https.get call gives you a stream with an 'end' event [1], you can do the following:
router.get('/', function (req, res, next) {
https.get(pageUrl, function (res) {
var info;
res.on('data', function (responseBuffer) {
//Important info;
info = responseBuffer;
}
res.on('end', function() {
res.render('page', { important: info});
})
}
}
Note that the above code will not work because you shadowed the base res parameter with the res parameter from the https.get callback.
Also, note that the 'data' event may be emitted several times (again, assuming a standard stream implementation[1]), so you should accumulate the results inside your info variable.
[1] Could you please post more information about your code, such as where the https library comes from (is it the standard HTTPS lib?).
Personal thought: I highly suggest using the request module, disponible on NPM via npm install request, for HTTP(S) requests to external services. It's got a neat interface, is simple to use and handles a lot of situations for you (redirects are one example, JSON and Content-Type another).
I am writing an app in node.js, I have the following code.
API for retrieving topic from DB
allTopics = function (req, res) {
db.Topic.all({limit: 10}).success(function (topics) {
res.send(topics)
});
};
Route for topics index
app.get('/topics', function (req, res){
res.render('topics/index.ejs',{ topics : allTopics })
});
Is the above code correct for route?
Also I have index.ejs file where I want to list all the topics (i.e. retrieve data from json response). How do I achieve this?
Your code as-is won't work but you could rewrite it as follows:
// notice how I am passing a callback rather than req/res
allTopics = function (callback) {
db.Topic.all({limit: 10}).success(function (topics) {
callback(topics);
});
};
// call allTopics and render inside the callback when allTopics()
// has finished. I renamed "allTopics" to "theData" in the callback
// just to make it clear one is the data one is the function.
app.get('/topics', function (req, res){
allTopics(function(theData) {
res.render('topics/index.ejs',{ topics : theData });
});
});
I have a code as follow:
app.get('/list', function(req, res) {
serialport.list(function (err, ports) {
ports.forEach(function(port) {
var temp = port.manufacturer;
console.log(temp);
res.send(temp);
});
});
});
As it can be seen, port.manufacturer value is saved onto a variable temp and the resulting content of the temp is displayed onto console.
When I run the above code, i get something like this on my console:
Listening on port 3000...
Tinkerforge GmbH
GET /list 200 219ms - 16
Arduino (www.arduino.cc)
GET /favicon.ico 404 4ms
But when I call the api http://localhost:3000/list,
Only Tinkerforge GnbH is displayed and not Arduino.
Is there something i am missing over here??
Do I have to save the list in a array or something??
Any help would be really appreciated.
Thanks a lot in advance. And BTW i am kinda begineer in both node.js and javascript.
The problem here is that .send, unlike .write, can be called only once. When called, Express will analyze the data and detect the correct headers (res.writeHead) to send before writing (res.write) to the socket and finally close the connection (res.close).
The solution is to either use .write or send all of your data in one go.
Using .write
app.get('/list', function(req, res) {
serialport.list(function (err, ports) {
ports.forEach(function(port) {
var temp = port.manufacturer;
res.write(temp);
});
res.send();
});
});
Using .send
app.get('/list', function(req, res) {
serialport.list(function (err, ports) {
var manufacturers = [];
ports.forEach(function (port) {
manufacturers.push(port.manufacturer);
});
res.send(manufacturers.join(" "));
});
});
I'd probably go with something like JSON.stringify(manufacturers) instead of using .join, since JSON is so easy to work with.