Basic Tutorial
Contents
Basic Tutorial#
This is a basic level tutorial.
Topics of the tutorial:
Time scheduling
Execution options
Changing log destination
Time Scheduling#
There are a lot of scheduling options in Red Engine: the tasks can run at specific time, after some other tasks have run or when other conditions are met. In this tutorial we focus on the time specific scheduling as that is most used. In later tutorials we discuss other options.
@app.task('every 10 seconds')
def do_constantly():
...
@app.task('every 1 minute')
def do_minutely():
...
@app.task('every 1 hour')
def do_hourly():
...
@app.task('every 1 day')
def do_daily():
...
@app.task('every 2 days 2 hours 20 seconds')
def do_custom():
...
You may also schedule tasks to run on fixed time periods (ie. daily, weekly, monthly):
@app.task('daily')
def do_daily():
...
@app.task('weekly')
def do_weekly():
...
@app.task('monthly')
def do_monthly():
...
But what if you wanted to schedule to run on specific time on those periods? That’s also easy:
@app.task('daily after 10:00')
def do_daily_after():
...
@app.task('daily before 22:00')
def do_daily_after():
...
@app.task('daily between 10:00 and 22:00')
def do_daily_between():
...
@app.task('weekly on Monday')
def do_on_monday():
...
@app.task('weekly between Saturday and Sunday')
def do_on_weekend():
...
@app.task('monthy after 5th')
def do_monthly_after_fifth():
...
@app.task('monthy before 5th')
def do_monthly_before_fifth():
...
Our previous examples were scheduled to run once in the
time periods we specified. There are also time of ...
scheduling options for situations in which you wish
to run the task constantly in the given period or
if you wish to add them to other scheduling options
(we get back to this later):
@app.task('time of day between 10:00 and 18:00')
def do_constantly_during_day():
...
@app.task('time of week between Saturday and Sunday')
def do_constantly_during_weekend():
...
Execution Options#
There are three options for how tasks are executed:
process
: Run the task in a separate processthread
: Run the task in a separate threadmain
: Run the task in the main process and thread (default)
Here is a quick example of each:
@app.task("daily", execution="main")
def do_main():
...
@app.task("daily", execution="thread")
def do_thread():
...
@app.task("daily", execution="process")
def do_process():
...
You may also put the default execution method if there is one that you prefer in your project:
app = RedEngine(config={'task_execution': 'main'})
@app.task("daily")
def do_main():
...
There are pros and cons in each option. In short:
Execution |
Parallerized? |
Can be terminated? |
Can modify the session? |
---|---|---|---|
|
Yes |
Yes |
No |
|
Yes |
Yes if task supports |
Yes |
|
No |
No |
Yes |
Changing Logging Destination#
Logging the states of the tasks is vital for Red Engine’s system. This includes the logs about when each task started, succeeded and failed. This information is used in many of the scheduling statements and to prevent setting the same task running multiple times.
Red Engine extends logging library’s loggers extending them with Red Bird to enable reading the logs.
By default, the logs are stored in-memory and they do not
persist over if the scheduler is restarted. You may change
the destination simply by creating a Red Bird repo and
pass that as the logger_repo
to the application.
Storing the logs in-memory:
from redengine import RedEngine
from redbird.repos import MemoryRepo
app = RedEngine(
logger_repo=MemoryRepo()
)
Storing the logs to CSV file:
from redengine import RedEngine
from redbird.repos import CSVFileRepo
app = RedEngine(
logger_repo=CSVFileRepo(
filename="logs.csv"
)
)
See more repos from Red Bird documentation. We will get back on customizing the loggers in later tutorials.