I'm still finding my feet with node.js / express.js. I would like to pass the result of one rest web service to another... or create a new routed service from two web service functions...
function A connects to mysql and builds a json object
e.g.
url:port/getMySQLRecord/:recordid
function B add a new document (JSON object) to a mongoDB collection
it accepts an AJAX POST
e.g.
url:port/insertMongoDoc
function A and B both currently work as REST web services... (how can I best pipe the result of A to B?)
it seems inefficient for the HTTP client to call A and pass the results to B.
I mean using 2 x bandwidth when the server already has the object data doesn't seem the best option.
if this were nix I'd be using | ...
//pseudocode loadRecord middleware just queries mysql for req.params.recordid
// and stores the result as req.body, then calls next()
//storeReqDotBodyInMongo just takes req.body and does a mongo insert, then calls next()
//sendResponse is just res.send(req.body)
app.get('/getMySQLRecord/:recordid', loadRecord, sendResponse);
app.post('/insertMongoDoc', express.bodyParser(), storeReqDotBodyInMongo, sendResponse);
app.get('/getMySQLAndInsertMongo/:record', loadRecord, storeReqDotBodyInMongo, sendResponse);
Note the similarity of connect middleware to unix pipes. Just instead of stdio they use req/res/next.
Related
I have a firebase cloud function that is an endpoint for an external API, and it handles a POST request.
This external API POSTS data to my cloud function endpoint at random intervals (this cloud function gets pinged with a POST request based on when a result is returned from this external API, and there can be multiple at once and its unpredictable)
exports.handleResults = functions.https.onRequest((req, res) => {
if (req.method === 'POST') {
// run code here that handles the POST payload
}
})
What happens when there is more than one POST request that come in at the same time?
Is there a queue? Does it finish the first request before moving on to the next?
Or if another request comes in while the function is running, does it block/ignore the request until the function is done?
Cloud Functions will automatically scale up the server instances running your functions when it determines that more capacity is needed. Those instances will run your function concurrently. The instances will be scaled down when they are no longer needed. The exact behavior is not documented - it should be considered an implementation detail that may change over time.
To learn more about this, watch my video about Cloud Functions scaling and isolation.
Beginner here, I'm using Firebase real time database and I need my API to constantly return that value when something has been added see my code below.
apiCalls.get('/api/getallusers',function(req,res){
userFunc.getAllUsers(function(err,result){
if (err) return res.status(500).send('internal server error!');
res.status(200).write(JSON.stringify(result));
res.end();
return res;
})
})
this will return the error
Error [ERR_STREAM_WRITE_AFTER_END]: write after end
but if i remove res.end it will show 1 record and constantly load until the page times out..
is what I'm doing possible or are there different ways to do it.
also I'm using firebase cloud functions for this api.
UPDATE:
Uploaded the API but it does not return anything...
here is the link https://us-central1-testproject-e6819.cloudfunctions.net/api1/api/getUser
tried axios and Event Source
Firebase functions logs the values but it does not return it..
If you're viewing the API response like a web page, your browser is buffering the data it's received until there's enough of it to form a more full page. Your browser is expecting content that ends, not some endless stream of data.
You should remove .end() if you expect to be able to continue to write to the output stream.
Also, I recommend using the Server-Sent Events (SSE) protocol for this. https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events It provides a nice standards-based abstraction that makes it very easy to handle event streams client-side.
const eventSource = new EventSource('https://api.example.com/someApi');
eventSource.addEventListener('userupdate', (e) => {
console.log(e.data);
});
Server-side, there are a couple Express-based middlewares to make this even easier than it already is.
Operations in Cloud Functions must be relatively short-lived and end deterministically. There is no way to keep a connection open from Cloud Functions to the client.
Typically consider what triggers the need to send new data. For example, if it is triggered by the fact that a new user is registered, you can use trigger your Cloud Functions from Firebase Authentication. Then the function could for example write to the Realtime Database (or Cloud Firestore), and your client/app listens to the database for realtime updates. That way you're using all the pieces of Firebase in the way they're designed: Cloud Functions for short-lived updates triggered from events in the system, and the Realtime Database or Cloud Firestore for sending realtime updates.
If that doesn't work for your use-case, you'll need a runtime environment that allows you to keep processes alive. Something like App Engine flex, Kubernetes, or many other options come to mind for that.
I need a legacy java application to pull information from a meteor's collection.
Ideally, I would need a simple service where my app would be able to download the latest list of items prices. A scenario like going on (through an http GET):
www.mystore.com/listOfPrices
would return a json with an array
[{"item":"beer", price:"2.50"}, {"item":"water":, price:"1"}]
The problem is that I cannot make a meteor page printing the result "as is" because meteor assumes the client supports javascript. Note that I do plan to implement the java DDP client in a latter stage but here I would like to start with a very simple service.
Idea: I thought of doing my own Node.js request aside of the running meteor service in order to retrieve a snapshot of the collection. Then this request would be using a server based javascript DDP client in order to subscribe and filter to then return the collection once loaded as a jSON document (array).
Any idea on how to achieve this ?
Looks like you want to provide a REST interface. See the MeteorPedia page on REST for how to expose collection data. It might be as simple as
prices = new Mongo.Collection('prices');
// Add access points for `GET`, `POST`, `PUT`, `DELETE`
HTTP.publish({collection: prices}, function (data) {
// here you have access to this.userId, this.query, this.params
return prices.find({});
});
I'm trying to decide between two methods for inserting a new document to a collection from the client using Meteor.js. Call a Server Method or using the db API directly.
So, I can either access the db api directly on the client:
MyCollection.insert(doc)
Or, I can create a new Server Method (under the /server dir):
Meteor.methods({
createNew: function(doc) {
check(doc, etc)
var id = MyCollection.insert(doc);
return project_id;
}
});
And then call it from the client like this:
Meteor.call('createNew', doc, function(error, result){
// Carry on
});
Both work but as far as I can see from testing, I only benefit from latency compensation (the local cache updating and showing on the screen before the server responds) if I hit the db api directly, not if I use a Server Method, so my preference is for doing things this way. But I also get the impression the most secure approach is to use a Method on the server (mainly because Emily Stark gave it as an example in her video here) but then the db api is available on the client no matter what so why would a Server Method be better?
I've seen both approaches taken when reading source code elsewhere so I'm stumped.
Note. In both cases I have suitable Allow/Deny rules in place:
MyCollection.allow({
insert: function(userId, project){
return isAllowedTo.createDoc(userId, doc);
},
update: function(userId, doc){
return isAllowedTo.editDoc(userId, doc);
},
remove: function(userId, doc){
return isAllowedTo.removeDoc(userId, doc);
}
});
In short: Which is recommended and why?
The problem was that I had the method declarations under the /server folder, so they were not available to the client and this broke latency compensation (where the client creates stubs of these methods to simulate the action but in my case could not because it couldn't see them). After moving them out of this folder I am able to use Server Methods in a clean, safe and latency-compensated manner (even with all my Allow/Deny rules set to false - they do nothing and only apply to direct db api access from the client, not server).
In short: don't use the db api on the client or allow/deny rules on the server, forget they ever existed and just write Server Methods, make sure they're accessible to both client and server, and use these for crud instead.
Im working on a socket.io based server/client connection instead of ajax.
Client uses Backbone and I overwritten the Backbone.sync function with one
half assed of my own:
Backbone.sync = function (method, collection, options) {
// use the window.io variable that was attached on init
var socket = window.io.connect('http://localhost:3000');
// emit the collection/model data with standard ajax method names and options
socket.emit(method,{collection:collection.name,url:collection.url});
// create a model in the collection for each frame coming in through that connection
socket.on(collection.url,function(socket_frame){
collection.create(socket_frame['model']);
})
};
Instead of ajax calls I simply emit through socket attached to window.io
global var. Server listens to those emits and based on the model url, I don't want to change that behaviour and I use the default crud method names (read,patch...) inside each emited frame. The logic behind it (its a bit far thought, but who knows) that in case the client doesn't support Websockets I can easily fallback to default jQuery ajax. I attached the orginal Backbone.sync to a var so I can pass the same arguments to it when no websocket is available.
All it that greatness behalves properly and the server answers to the client events. The server emits then each model data as a seperate websocket frames in one connection.
I see the frames in the Network/Websocket filter as one (concurrent/established) connection
and things seems to be working
Currently the function assumes I pass a collection and not a model.
Questions:
Is that approach ok with you?
How can I use the socket.io callbacks on 'success' and 'failure' etc in Backbone the right way so I don't have to call the collection.create function 'by-hand'?
Is it better to establish different concurrent connections for models/collections or use the one already established instead?