I have created a web application in which I'm using sails.js framework and mongodb for back-end purpose. I want to implement a cron job running at midnight of each day so that I can calculate previous day's power consumption of a property. This data has to be used for reporting purpose.
The cron job has to run w.r.t properties location, i.e, if 2 properties are located at different time zones, I have to run the job when midnight occurs for each time zone.
Related
I am using spring cloud and have various microservices for an online shopping vendor. Everything is working as expected.
But, I got a requirement where I need to run a cron job over customer's records, get the customer's who's statement date matches the current date and calculate the rate of interest to be paid. This needs to be run every day.
I am confused about how to accommodate this cron job with MS architecture. Do I need to have another server having just this cron job?
Depending on the platform (eg: cf, k8s..) that you're orchestrating the batch-jobs in SCDF, you could write a simple Quartz based Boot Application that can interact with SCDF's REST endpoints to schedule the Task definitions defined in SCDF.
There are several online literatures on Quartz + Boot solution.
We are also working on a native scheduler integration for Cloud Foundry (via PCF Scheduler). Once it ready, you'd be able to schedule (i.e., cron-expressions) for Tasks from SCDF's Dashboard natively.
As I understand you should have one centralized supervisor of jobs, because multiplied instances can potentially run the same job at the same time.
This supervisor can be a microservice, which delegates job execution to other services via rest call or message queue, and wait for result.
It means that job supervisor becomes part of infrastructure, like message queue or database.
Currently from the Azure portal (old portal) when setting up a scheduled job start time, you can only specify times as half hour intervals like 12:00 or 3:30 but you can't tell it to start from say 12:55. Is there another way to set up a scheduled job that allows for any start time?
You can either use the new portal, which enables you more advanced options along side with the ability to set the scheduling to a minute granularity. Or use the REST api to do a request with a specific time/interval.
Hope this helps!
Mert
I am developing an application using Azure Cloud Service and web api. I would like to allow users that create a consultation session the ability to change the price of that session, however I would like to allow all users 30 days to leave the session before the new price affects the price for all members currently signed up for the session. My first thought is to use queue storage and set the visibility timeout for the 30 day time limit, but this seems like this could grow the queue really fast over time, especially if the message should not run for 30 days; not to mention the ordering issues. I am looking at the task scheduler as well but the session pricing changes are not a recurring concept but more random. Is the queue idea a good approach or is there a better and more efficient way to accomplish this?
The stuff you are trying to do should be done with a relational database. You can use timestamps to record when prices for session changed. I wouldn't use a queue at all for this. A queue is more for passing messages in a distributed system. Your problem is just about tracking what prices changed on what sessions and when. That data should be modeled in a database.
I think this scenario is more suitable to use Azure Scheduler. Programatically create a Job with one time recurrence with set date as 30 days later to run once. Once this job gets triggered automatically by scheduler, assign an action to callback to one of your API/Service to do the price & other required updates and also remove this Job from the scheduler as part of this action to have a clean jobs list. Anyways premium plan of Azure Scheduler Job Collection will give you unlimited number of jobs to run.
Hope this is exactly what you were looking for...
I would consider using Azure WebJobs. A WebJob basically gives you the ability to run a .NET console application within the context of an Azure Web App. It can be run on demand, continuously, or in response to a reoccurring schedule. If your processing requirements are low and allow for it they can also run in the same process that your Web App is running in to save you $$$ as they are free that way.
You could schedule the WebJob to run once or twice per day and examine the situation and react as is appropriate. Since it's really just a .NET worker role you have ultimate flexibility.
Now my website is on windows azure. I make a store procedure in database, then I execute it to update database every night on 0:00:00. I use azure automation to achieve this goal. So I write a Runbook which execute this store procedure. And add a scheduler to this task. Now this all work well, but in the scheduler, I set the time "0:00:00" which can't set the time zone.
(I am in China, but this task always run before 0:00:00 compared to my local time)
How can I specify the time zone when setting a scheduler on azure automation?
As you may already know that everything in Azure runs in UTC. If you want to run your task at midnight China time, what you could do is calculate the time difference (which I believe is 8:00 hours) and run your task at that time which would be at 4:00 PM UTC ( assuming my math is right).
I have a symfony based web application that runs on mongodb and mysql backend. The principal of the application is that for each user there is a python script that runs 4-12 times a day on cronjobs and populates the mysql and mongodb databases. The script takes between 1.5 minutes to 2 minutes to execute. At the moment the cronjob runs on sequential basis. That means that the script executes a job and waits for the job to end before executing the next one. The moment my web application has a new user the cronjobs are auto created for a duration of time. With 24 hours in a day I can run a limited number of cronjobs thereby, limited number of users (around 250-300)
What would I need to do if I wanted to host 1000 to a million users on my web application? Can I run my script on multithread basis? That means instead of waiting for a job to finish, launch hundreds of job at the same time. This way I can grow my user base exponentially.
But, what concurrency will mongodb and mysql be able to sustain? how many jobs can I execute parallelly? What system factors do I need to consider to grow my user base? Do I need to add more machines to my application?