I have an api exposed using node.js. I have to restrict the API call only once at a time. I tried the following approach.
app.js
globals.consumed = false;
routes.js
app.get("/route1", checkForAlreadyConsumed,controller.method);
controller.method
globals.consumed = true;
//Business logic goes here
//After all the stuff is done
globals.consumed = false;
checkForAlreadyConsumed
checkForAlreadyConsumed() {
if(consumed == false) {
next();
} else {
retrun res.send(403);
}
}
So by above logic i am try to make nobody should consume that API if its already doing its process. But by the above approach if i do 2 concurrent calls , the second call is waiting for the first call to finish so the 403 code is sent as response to frotend. Plz share your ideas.
Related
So I'm working with websockets to process data from website's API. For every new event I also send some http requests back to the website in order to obtain more data. Up untill now everything has worked fine, but now that I started using async requests to speed it up a bit things got a bit different. My code used to process one event and then move on to the next one (these events come in extremely quick - around 10 per second) but now it just seems to ignore the async (non blocking) part and move on to the next event and that way it just skips over half of the code. Note that the code works fine outside the Pusher. I'm using the 'pusher-client' module. My code looks like this:
var Request = require("request");
var requestSync = require('sync-request');
var Pusher = require('pusher-client');
var events_channel = pusher.subscribe('inventory_changes');
events_channel1.bind('listed', function(data)
{
var var2;
//Async request (to speed up the code)
function myFunction(callback){
request("url", function(error, response, body) {
if (!error && response.statusCode == 200)
{
result = JSON.stringify(JSON.parse(body));
return callback(null, result);
}
else
{
return callback(error, null);
}
});
}
myFunction(function(err, data){
if(!err)
{
var2 = data
return(data);
}
else
{
return(err);
}
});
//The part of the code below waits for the callback and the executes some code
var var1 = var2;
check();
function check()
{
if(var2 === var1)
{
setTimeout(check, 10);
return;
}
var1 = var2;
//A CHUNK OF CODE EXECUTES HERE (connected to the data from the callback)
}
});
In conclusion the code works, but not inside the pusher due to the pusher skipping the asynchronous request. How would I make the pusher wait for my async request to finish, before processing the next event (I have no idea)? If you happen to know, please let me know :)
You need to implement a queue to handle events one after another. I'm curious how it worked before, even without Pusher you'd have to implement some queue mechanism for it.
const eventsQueue = []
events_channel1.bind('listed', function(data) {
eventsQueue.push(data)
handleNewEvent()
})
let processingEvent = false
function handleNewEvent() {
if (processingEvent) return // do nothing if already processing an event
processingEvent = true
const eventData = eventsQueue.shift() // pick the first element from array
if (!eventData) return // all events are handled at the moment
... // handle event data here
processingEvent = false
handleNewEvent() // handle next event
}
Also, you should call clearTimeout method to clear your timeout when you don;t need it anymore.
And it's better to use promises or async/await instead of callbacks. Your code will be much easier to read and maintain.
I'm trying to implement and existing solution in node.js, specifically, using express.js framework. Now, the existing solution works as follows:
server exposes a GET service that clients can connect to
when a client calls the GET service, the client number increments (a global variable) and then the number of clients is checked;
if there are not at least 3 clients connected, the service is in endless loop, waiting for other clients to connect
if (or rather, when) the rest of the two clients connect, the service sends respond to everyone that enough clients are connected (a 'true' value).
So what basically happens is, the client connects and the connection is active (in a loop) until enough clients connect, then and only then there is a response (to all clients at the same time).
Now I'm not expert in these architectures, but from what I think, this is not a correct or good solution. My initial thought was: this must be solved with sockets. However, since the existing solution works like that (it's not written in node.js), I tried to emulate such behaviour:
var number = (function(){
var count = 0;
return {
increase: function() {
count++;
},
get: function(){
return count;
}
};
})();
app.get('/test', function(req, res){
number.increase();
while (number.get() < 3) {
//hold it here, until enough clients connect
}
res.json(number.get());
});
Now while I think that this is not a correct solution, I have a couple of questions:
Is there any alternative to solving this issue, besides using sockets?
Why does this "logic" work in C#, but not in express.js? The code above hangs, no other request is processed.
I know node.js is single-threaded, but what if we have a more conventional service that responds immediately, and there are 20 requests all at the same time?
I would probably use an event emitter for this:
var EventEmitter = require('events').EventEmitter;
var emitter = new EventEmitter();
app.get('/', function(req, res) {
// Increase the number
number.increase();
// Get the current value
var current = number.get();
// If it's less than 3, wait for the event emitter to trigger.
if (current < 3) {
return emitter.once('got3', function() {
return res.json(number.get());
});
}
// If it's exactly 3, emit the event so we wake up other listeners.
if (current === 3) {
emitter.emit('got3');
}
// Fall through.
return res.json(current);
});
I would like to stress that #Plato is correct in stating that browsers may timeout when a response takes too much time to complete.
EDIT: as an aside, some explanation on the return emitter.once(...).
The code above can be rewritten like so:
if (current < 3) {
emitter.once('got3', function() {
res.json(number.get());
});
} else if (current === 3) {
emitter.emit('got3');
res.json(number.get());
} else {
res.json(number.get());
}
But instead of using those if/else statements, I return from the request handler after creating the event listener. Since request handlers are asynchronous, their return value is discarded, so you can return anything (or nothing). As an alternative, I could also have used this:
if (current < 3) {
emitter.once(...);
return;
}
if (current === 3) {
...etc...
Also, even though you return from the request handler function, the event listener is still referencing the res variable, so the request handler scope is maintained by Node until res.json() in the event listener callback is called.
Your http approach should work
You are blocking the event loop so node refuses to do any other work while it is in the while loop
You're really close, you just need to check every now and then instead of constantly. I do this below with process.nextTick() but setTimeout() would also work:
var number = (function(){
var count = 0;
return {
increase: function() {
count++;
},
get: function(){
return count;
}
};
})();
function waitFor3(callback){
var n = number.get();
if(n < 3){
setImmediate(function(){
waitFor3(callback)
})
} else {
callback(n)
}
}
function bump(){
number.increase();
console.log('waiting');
waitFor3(function(){
console.log('done');
})
}
setInterval(bump, 2000);
/*
app.get('/test', function(req, res){
number.increase();
waitFor3(function(){
res.json(number.get());
})
});
*/
Rather than copy and pasting my code onto here, I have uploaded it to github. The RequireJS module does have a dependency on jquery.signalr and in tern has a dependency on jquery but also have a dependency on the javascript held in /signalr/hubs. There is a bit of config to do with Require.Config.
Basically what is happening is on the first time you load the page the connection is made to the hubs within signalr and the "server side" code is executed and does the desired thing. When you refresh the page it does not. All client side code is called, so for example:
var myViewModel = new MyViewMode();
myViewModel.init();
and within your init method you have
var connection = $.connection.myHub;
this.init = function() {
connection.server.myMethod();
}
this would then go off to
public MyHub : Hub
{
public void MyMethod()
{
Client.Request.populateSomeInformation() // I think it's request but I'm doing this from memory!
}
}
and then call
connection.client.populateSomeInformation = function () { .. )
but doesn't call this :(
It looks like a connection has been made (using the good old console.log() to see what it outputs) and indeed debugging the project it executes the code within the hub but there is no response made back to the javascript.
So wonderful people of the internet, where am I going wrong? Do I need to check the state of $.connection.hub.start(); before attempting to start it again?
Time for beer :)
I believe it should be
connection.client.populateSomeInformation = function () { .. )
(not connection.server)
http://www.asp.net/signalr/overview/hubs-api/hubs-api-guide-javascript-client#callclient
(observations on the code you have on github right now)
var isLoaded = false;
// ... some code that doesn't change isLoaded ...
if (isLoaded == false) {
scrollIntervalId = window.setInterval(function () {
signalRLoaded();
}, 30);
}
I think isLoaded will always be false at this point. Not sure what you intended this to accomplish.
var connection = $.connection.hub.start();
I don't think you're supposed to open the connection before defining any client functions. I don't see any client functions being defined here, so maybe you're doing that somewhere else? I don't know if it really matters other than if the server attempts to call a client function that hasn't yet been defined...
function SignalRReady(callback) {
if (isLoaded) {
callback(connection);
} else {
readyCalls = callback;
}
return SignalRReady;
}
SignalRReady.version = "1.0.0";
SignalRReady.load = function(name, request, onLoad, config) {
if (config.isBuild) {
onLoad();
} else {
SignalRReady(onLoad);
}
};
return SignalRReady;
I'm confused by this bit of code, probably because I don't see how it's being used. Is this an attempt at a kind of singleton? I see that SignalRReady is the "class" being returned for the module. You're not really returning an object, you're returning a constructor which implies that you're instantiating it in other places, something like
define(['SignalRReady'], function(sigR)
{
var srr = new sigR();
});
But then you have that load function defined that calls the constructor and makes this look all weird. How are you using this?
Anyways, I'm thinking you might be hitting some kind of race condition where the client function may not always be available at the time the server is trying to call it.
(additional comments/code 2013-09-06)
Your connection object is actually a jQuery promise ( http://api.jquery.com/category/deferred-object/ ).
If you're unfamiliar with promises, think of them generically as a queue of callbacks to be executed later. In this case, when connected, all the callbacks will be executed (in the order they were added). If a callback is added after being connected, it will get executed immediately. This is how your code is working now. You add the callback to the .done queue after the connection is made and is executed immediately.
If you insist on creating the connection object yourself, then you do not need to use the stateChanged event. You just add the callback to the .done queue:
define(function()
{
function signalRReady(callback)
{
if (window.connection == undefined) {
window.connection = $.connection.hub.start();
}
window.connection.done(callback);
}
signalRReady.version = "1.0.0";
return signalRReady;
});
However, I believe it's not a good idea to initiate the connection yourself. Because your module isn't a complete wrapper around SignalR such that people would only use your module to do SignalR stuff, you are not guaranteed (and cannot expect) other code will not initiate the connection. Especially if someone is adding your module to an existing codebase.
Your module is simply adding a new event, so keep it simple. Take the callback and execute it yourself when appropriate:
define(function()
{
function signalRReady(callback)
{
$.connection.hub.stateChanged(function (state)
{
if(state.newState === $.signalR.connectionState.connected)
{
callback();
}
});
}
signalRReady.version = "1.0.0";
return signalRReady;
});
Nowadays, promises are pretty popular. You might want to implement a promise-based module like:
define(function()
{
var deferred = $.Deferred();
$.connection.hub.stateChanged(function (state)
{
if(state.newState === $.signalR.connectionState.connected)
{
// executes all callbacks attached by the "ready" function below
deferred.resolve();
}
});
return {
ready: function(callback)
{
deferred.done(callback);
},
version: "1.0.0"
};
});
If callbacks are attached after the connection has been made, they are executed immediately.
Also, notice this example module's init function returns an object instead of a function. Since RequireJS will pass the same instance around to any module that requires it, state is maintained - we can use local variables instead of global.
mongoosejs async code .
userSchema.static('alreadyExists',function(name){
var isPresent;
this.count({alias : name },function(err,count){
isPresent = !!count
});
console.log('Value of flag '+isPresent);
return isPresent;
});
I know isPresent is returned before the this.count async function calls the callback , so its value is undefined . But how do i wait for callback to change value of isPresent and then safely return ?
what effect does
(function(){ asynccalls() asynccall() })(); has in the async flow .
What happens if var foo = asynccall() or (function(){})()
Will the above two make return wait ?
can process.nextTick() help?
I know there are lot of questions like these , but nothing explained about problem of returning before async completion
There is no way to do that. You need to change the signature of your function to take a callback rather than returning a value.
Making IO async is one of the main motivation of Node.js, and waiting for an async call to be completed defeats the purpose.
If you give me more context on what you are trying to achieve, I can give you pointers on how to implement it with callbacks.
Edit: You need something like the following:
userSchema.static('alreadyExists',function (name, callback) {
this.count({alias : name}, function (err, count) {
callback(err, err ? null : !!count);
console.log('Value of flag ' + !!count);
});
});
Then, you can use it like:
User.alreadyExists('username', function (err, exists) {
if (err) {
// Handle error
return;
}
if (exists) {
// Pick another username.
} else {
// Continue with this username.
}
}
Had the same problem. I wanted my mocha tests to run very fast (as they originally did), but at the same time to have a anti-DOS layer present and operational in my app. Running those tests just as they originally worked was crazy fast and ddos module I'm using started to response with Too Many Requests error, making the tests fail. I didn't want to disable it just for test purposes (I actually wanted to have automated tests to verify Too Many Requests cases to be there as well).
I had one place used by all the tests that prepared client for HTTPS requests (filled with proper headers, authenticated, with cookies, etc.). It looked like this more or less:
var agent = thiz.getAgent();
thiz.log('preReq for user ' + thiz.username);
thiz.log('preReq for ' + req.url + ' for agent ' + agent.mochaname);
if(thiz.headers) {
Object.keys(thiz.headers).map(function(header) {
thiz.log('preReq header ' + header);
req.set(header, thiz.headers[header]);
});
}
agent.attachCookies(req);
So I wanted to inject there a sleep that would kick in every 5 times this client was requested by a test to perform a request - so the entire suite would run quickly and every 5-th request would wait to make ddos module consider my request unpunishable by Too Many Requests error.
I searched most of the entries here about Async and other libs or practices. All of them required going for callback - which meant I would have to re-write a couple of hundreds of test cases.
Finally I gave up with any elegant solution and fell to the one that worked for me. Which was adding a for loop trying to check status of non-existing file. It caused a operation to be performed long enough I could calibrate it to last for around 6500 ms.
for(var i = 0; i < 200000; ++i) {
try {
fs.statSync('/path' + i);
} catch(err) {
}
};
I am trying to write a node.js application, and we need to deploy it in production.
We need to make sure that node.js does not hang when there are any long running processes/operations, like querying, or the database server access.
So, i am trying to make a call to mongo or to filesystem which takes very long time to finish, so that i can verify that other node.js server is free to serve any other requests while that takes place.
Sadly, i am not able to insert a record for which mongo takes really long time to finish or to make a synch call to the file system.
Can someone tell me how to do it?
Thanks
Tuco
The trick is do a console log of the data after the block that do a call and a console.log in the callback if in the console apears first the message is actually asynchronous
Im using mongojs as driver for mongo:
collection.find({}, function(err, res) {
console.log("done")
});
console.log("sendign signal")
If its asynchronous, in the console:
sendign signal
done!
Now for the chained behavior you can make something like that
dbChain = (function() {
var chain = [], cursor = 0, busy = false;
chainin = {
push : function(aFn) {
if(!busy) {
chainin.reset();
aFn();
busy = true;
} else {
chain.push(aFn)
}
},
next : function() {
cursor++;
if(chain[cursor]) {
chain[cursor]();
} else {
chainin.reset();
}
},
reset : function() {
chain = [];
cursor = 0;
busy = false;
}
}
return chainin;
})()
and, in all the db calls you have to do:
dbChain.push(...(a function ) ...)
in all your callbacks
dbChain.next()