I would like to have a Node.js app running in multiple instances in Cloud Foundry.
The app has - among others - a scheduler ("node-schedule": "^0.2.7") which writes a status to a mongo-db once every minute. Now with multiple instances of the app, I get multiple entries - for each instance one - in the database.
I'm looking for something to synchronize instances to ensure just one instance "gets the lock" to write to the database once every minute. I already found suggestions to use singleton frameworks or middleware but they are request-centric, not "from the inside" of the app - the scheduler.
Is there a solution for this?
Each application instance will have an environment variable set, CF-INSTANCE-INDEX, to expose the instance index. You could make only the instance with index '0' responsible for scheduling logs entries.
http://docs.run.pivotal.io/devguide/deploy-apps/environment-variable.html#CF-INSTANCE-INDEX
Related
I am trying to solve a problem where I am listening on a Mongo Changestream but in a production environment, we have multiple instances of the service running so in each instance I get the 'on change' event which I need to process.
But I want to process the change ONLY ONCE but here it is being processed multiple times(Number of instances of the service)
I need a way to ensure that each change Event is processed only once.
Thought of using Redis but then there is a chance of collision when two instances try to add the same on change event at the same time.
Please refer the diagram below for a rough idea of my architecture.
I'm fairly new at google cloud and node. Based on Google's recommendation (the requirement to watch Firebase at all times) I deployed managed VM node app, instead of just app engine. There are now 22-23 instances every time I deploy. Is this expected? I thought it would only scale when necessary.
This node app has a method which watches Firebase variables and change, in turn, the script fires off a notification.
What happens now is that multiple notifications are being fired and I only expect one. I suspect it's because there are multiple instances of this app.
What is the right way to do this so that only one is watching?
Thanks.
You can use the method suggested by google for flexible server environments and firebase. https://cloud.google.com/solutions/mobile/mobile-firebase-app-engine-flexible and https://cloudplatform.googleblog.com/2016/06/learn-to-build-a-mobile-backend-service-with-Firebase-and-App-Engine.html .
Have the instance "claim users" by transactioning it's instance ID at a location where the user can reach it and send updates to that instance by setting the instanceID in the path.
I have setup 2 instance under aws load balancer. I have deployed node.js web services + mongodb in both instance. load balancer works fine with web services.
But, Problem is I have one timer service (node.js service only). the behavior of this timer is updating my mongodb based on some calculation.
My problem is, I must need to run this timer service (timer.js) at only one aws instance (out of 2) at same time. and expected that if one aws instance goes down then timer service at other instance will come up.
i know elb not providing this kind of facility.Can any one please help me to make it done ?
Condition : At a time only one timer service must be run with amazon load balancer.
Thanks.
You would have to implement this yourself using a locking algorithm using a shared data store that supports atomic operations
Alternatively, consider starting a "timer" server in an Auto Scale Group of Min:1, Max: 1 so Amazon keeps it running. This instance can be a t2.micro which is very cheap. It can either run the job itself, or just make an http request to your load balancer to run the job at the desired internal. If you so that, only one of your servers will run each job
Wouldn't it make more sense to handle this like any other "service" that needs to keep running?
upstart service
running node.js server using upstart causes 'terminated with status 127' on 'ubuntu 10.04'
This guy had a bad path in his file but his upstart script looks okay
monit
Node.js (sudo) and monit
Given: One Worker role + several Quartz.net jobs. Quartz jobs are host-agnostic and are executed in Worker role.
A worker role can be scaled to multiple instances.
The Goal is: have the ability to define what job to run in what instance at runtime (or define it with configuration only, no code changes). For example:
MyRole Instance 1: Job1, Job2, Job3
MyRole Instance 2: Job4,
MyRole Instance 3: Job4,
MyRole Instance 4: Job4,
MyRole Instance 5: Job5, Job6
In my example Job4 receives a lot of load. So I'd like it to run on more instances. And I also want it to be scalable at runtime (or at least via configuration, w/o code changes).
AFAIK it is not possible to have azure configuration per instance (only per role itself). Search online on similar issues haven't given any results.
Question: Did anybody have similar situation? What would be the best approach? Any other design advice is very appreciated. Thanks.
This is more of an architectural problem then one specific to Azure. The most common solution is to set up a "broker" that each instance/process reaches out to and asks for its individual workload. The challenge, regardless of the platform you are deploying the solution is how to a) identify the broker and b) ensure the "state" information being managed by the broker is persisted in case the broker process dies.
The most common approach to address these concerns in Azure is the use of a blob with a lease on it that allows the broker to be 'self elected' (the first process to get the lease is the broker), and stores both the address of the broker and the broker's state (metadata stored within the blob). You then put the logic for assigning jobs into this broker in a way that best suits your task distribution needs.
I have two instances of a worker role.
I want to run a sub-task (on a Thread Pool thread) only on one of the Worker Role instances.
My initial idea was to do something like this:
ThreadPool.QueueUserWorkItem((o) =>
{
if (RoleEnvironment.CurrentRoleInstance.Id == RoleEnvironment.Roles[RoleEnvironment.CurrentRoleInstance.Role.Name].Instances.First().Id)
{
emailWorker.Start();
}
});
However, the above code relies on Role.Instances collection always returning the instances in the same order. Is this the case? or can the items be returned in any order?
Is there another approved way of running a task on one role instance only?
Joe, the solution you are looking for typically rely on:
either acquiring on lease (similar to a lock, but with an expiration) on a specific blob using the Blob Storage as a synchronization point between your role instances.
or queuing / dequeuing a message from the Queue Storage, which is usually the suggested pattern to delay long running operations such as sending an email.
Either ways, you need to go through the Azure Storage to make it work. I suggest to have a look at Lokad.Cloud, as we have designed this open-source framework precisely to handle this sort of situations.
If they need to be doing different things, then it sounds to me like you don't have 2 instances of a single worker role. In reality you have 2 different worker roles.
Especially when looking at the scalability of your application, processes need to be able to run on more than one instance. What happens when that task that you only want to run on one role gets large enough that it needs to scale to 2 or more role instances?
One of the benefits of developing for Azure is that you get scalability automatically if you design your app properly. If makes you work extra to get something that's not scalable, which is what you're trying to do.
What triggers this task to be started? If you use a message on Queue Storage (as suggested by Joannes) then only one worker role will pick up the message and process it and it doesn't matter which instance of your worker role does that.
So for now if you've got one worker role that's doing the sub task and another worker role that's doing everything else, just add 2 worker roles to your Azure solution. However, even if you do that, the worker role that processes the sub task should be written in such a way that if you ever scale it to run more than a single instance that it will run properly. In that case, you might as well stick with the single worker role and code for processing messages off the Queue to start your sub task.