I'm completely new to Python and Bottle (and apscheduler) so would appreciate some simple instructions on implementing a scheduled job.
I have written some Python code that can check the status and data displayed on one of our websites. I have also written some code that emails the status to us.
Now, I would like to add functionality to call this code regularly (e.g. every 15 minutes) and, for this, apscheduler seems to be a suitable, lightweight, self-contained solution.
What I would like to know is:
Can I start a BackgroundScheduler when the Bottle website starts? (Where is the "start up" code situated?)
And can I make my instance of the BackgroundScheduler global? The reason is that I want to access it from elsewhere on the site and add/remove/pause jobs
I saw that someone else had a similar idea here is it possible to run a task scheduler in bottle web framework but the details and answer have too little detail for a noob like me.
I understand that I could probably wrangle something similar using sleep() but I first want to explore the apscheduler option first.
I did also look at Celery but I think that is overkill for this small, single-purpose web application that I am building.
I should add that I am using a Bottle project available in Visual Studio 2013, which comes with certain configuration and files already included.
You could use threading.Timer :
from threading import Timer
def my_task():
# do stuff ...
Timer(15*60, my_task).start() # schedule for every 15 minutes
# start your scheduler
my_task()
#Alexander: I did this too. It works with the Botte WSGIRefServer(), here you can have threads living as long as you like. On an Apache with mod_wsgi your threads supposed to be running only for so long - then they are killed. So a scheduler inside Bottle did not work for me.
Personaly I'm writing a little schedule server in zmq (pyzmq) which receives scheduled task and uses schedule-package to run them. Works well for me.
Related
I have a web interface built by flask which needs to flush one of its dictionaries every 5 minutes. How can I code this feature in? I have looked up threading, multiprocessing and Asyncio almost all examples are about complicated stuff and none of them is relevant to my very simple basic need.
Advanced Python Scheduler can do this for you and it is easy to use. It runs within the application and runs jobs that you schedule at intervals.
job = scheduler.add_job(myfunc, 'interval', minutes=5)
The documentations are clear and easy to setup. Also you can find Flask-APScheduler which is almost the same with few flask features.
I have an issue currently where I've got a cron job set to run at midnight each day to reset daily API requests for a service that I run. The job failed recently which caused me a whole bunch of headaches and I've been trying to find a solution to monitor all of my cron jobs so I don't have a situation like this happen again.
I haven't been able to find a sufficient solution however, and in response I am considering creating a platform that allows you to monitor cron jobs, see logs (and past logs), last run date, failure/success of the last run, etc... in real-time and would notify you if your job hasn't completed within a specified window of time or the job failed.
I believe this might be a pain point and a good solution for others as well.
What are you thoughts? Do you think that this would be useful, have any suggestions, or just think this would be a waste of time?
Did you hear about Rundeck? (https://www.rundeck.com/open-source)
It looks like it's exactly what you're looking for.
You install it on a server, and it's like a Web UI for a crontab.
You define jobs you want to run using the Web UI, how often you want them to run and you can see some history of the past executions, their status and their output. You can also see when the next execution will happen.
I think there are also some alerting features to notify you if a job is on failure. I'm not sure if it can notify you based on the job execution time though.
This might be a good fit for what you're looking for.
2 years later, I am asking myself exactly the same questions ) Definitely you should have created such service already, haven't you? Every backend coder needs this time from time, in theory. I'm surprised this question hasn't received enough activity/voting. I got an answer leading to this though: https://uptimerobot.com/cron-job-monitoring/ that might be a good solution. Need to test it out. It does not seem to be promoted enough, as it's not easy to find. Also there is https://cronitor.io/docs/cron-job-monitoring that has ability to transmit (somewhat limited) telemetry data, +a lot of SDKs to be used from within programming languages.
So, I see that heroku provides the option to run a command at a specific time. Information on the scheduler here.
LINK: https://elements.heroku.com/addons/scheduler
However, if you go through the steps when setting it up, they do not provide a lot of flexibility on when you can run your code daily. For example, you can only run code at 4:00pm or 4:30pm, not 4:10pm.
How can I make it so that a node.js file would run on heroku at a specific time (like 4:10pm or 2:15pm, some time outside the options given on heroku) on a daily basis?
There appears to be no support for node.js explaining this either on their website.
Might be just a work around, but you could start the process at the nearest time slot, that is before your desired time, let it run and wait passively until your desired time and let it do the actual task just then.
However notice as Heroku mentions in the documentation Heroku Scheduler isn't guaranteed to run the task, even though it's very reliable. If you need something critical or have something that has to been run everyday for sure, you should probably make a separate process, which handles the scheduling.
We've added a simple Heroku add-on called Cron To Go that does exactly that - you can use Cron expressions for accuracy and schedule one-off Dynos, just like with Heroku Scheduler.
There's also a simple Node example here.
Our node app gets quite big and one job takes quite some time to execute. We run this job with a cronjob, but by calling the URL. Now Heroku has problems with this, because the job takes more than 30 seconds to finish. So we receive a time-out and after that it tries to execute it immediately again, and again, till our Memory quota is about 300% and the app crashes.
Now I want to fix this. Locally we don't have any problems running this script at all. It takes about a minute (for now, but in the future if we have more users it may take more time) to finish and memory stays stable.
Now running this script on the background should fix the problem according https://devcenter.heroku.com/articles/request-timeout#debugging-request-timeouts
Overe here https://devcenter.heroku.com/articles/asynchronous-web-worker-model-using-rabbitmq-in-node#getting-started I read about JackRabbit. But it seems like it's used for systems like RabbitMQ https://github.com/hunterloftis/jackrabbit
So my question: anyone who has experience with background tasks in node? Can and should I use JackRabbit for my background tasks, or are there better solutions? My background task just contains a very complex ExpressJS task, which takes some time to execute so....
I'm the Node.js platform owner at Heroku (and I actually wrote the web worker article you referenced).
Your use case sounds like it may fit the scheduler very well:
https://devcenter.heroku.com/articles/scheduler
It's a great replacement for cron-type jobs.
I'm in the process of designing an application that will run on a headless Windows CE 6.0 device. The idea is to make an application that will be started at startup and run until powered off. (Basically it will look like a service, but an application is easier to debug without the complete hassle to stop/deploy/start/attach to process procedure)
My concern is what will happen during development. If I debug/deploy the application I see no way of closing it in a friendly and easy way. (Feel free to suggest if this can be done in a better/user friendly way) I will just stop the debugger and the result will be WSACleanup is not called.
Now, the question. What is the consequence of not calling WSACleanup? Will I be able to start and run the winsock application again using the debugger? Or will there be a resource leak preventing me to do so?
Thanks in advance,
Jef
I think that Harry Johnston comment is correct.
Even if your application has no UI you can find a way to close it gracefully. I suppose that you have one or more threads in loops, you can add a named manual reset event that is checked (or can be used for waits instead of Sleep()) inside the loop condition and build a small application that opens the event using the same name, sets it and quits. This would force also your service app to close.
It may not be needed for debugging, but it may be useful also if you'll need to update your software and this requires that your main service is not running.