Basic Tutorial#

This is a basic level tutorial.

Topics of the tutorial:

Time scheduling
Execution options
Changing log destination

Time Scheduling#

There are a lot of scheduling options in Red Engine: the tasks can run at specific time, after some other tasks have run or when other conditions are met. In this tutorial we focus on the time specific scheduling as that is most used. In later tutorials we discuss other options.

@app.task('every 10 seconds')
def do_constantly():
    ...

@app.task('every 1 minute')
def do_minutely():
    ...

@app.task('every 1 hour')
def do_hourly():
    ...

@app.task('every 1 day')
def do_daily():
    ...

@app.task('every 2 days 2 hours 20 seconds')
def do_custom():
    ...

You may also schedule tasks to run on fixed time periods (ie. daily, weekly, monthly):

@app.task('daily')
def do_daily():
    ...

@app.task('weekly')
def do_weekly():
    ...

@app.task('monthly')
def do_monthly():
    ...

But what if you wanted to schedule to run on specific time on those periods? That’s also easy:

@app.task('daily after 10:00')
def do_daily_after():
    ...

@app.task('daily before 22:00')
def do_daily_after():
    ...

@app.task('daily between 10:00 and 22:00')
def do_daily_between():
    ...


@app.task('weekly on Monday')
def do_on_monday():
    ...

@app.task('weekly between Saturday and Sunday')
def do_on_weekend():
    ...


@app.task('monthy after 5th')
def do_monthly_after_fifth():
    ...

@app.task('monthy before 5th')
def do_monthly_before_fifth():
    ...

Our previous examples were scheduled to run once in the time periods we specified. There are also time of ... scheduling options for situations in which you wish to run the task constantly in the given period or if you wish to add them to other scheduling options (we get back to this later):

@app.task('time of day between 10:00 and 18:00')
def do_constantly_during_day():
    ...

@app.task('time of week between Saturday and Sunday')
def do_constantly_during_weekend():
    ...

Execution Options#

There are three options for how tasks are executed:

process: Run the task in a separate process
thread: Run the task in a separate thread
main: Run the task in the main process and thread (default)

Here is a quick example of each:

@app.task("daily", execution="main")
def do_main():
    ...

@app.task("daily", execution="thread")
def do_thread():
    ...

@app.task("daily", execution="process")
def do_process():
    ...

You may also put the default execution method if there is one that you prefer in your project:

app = RedEngine(config={'task_execution': 'main'})

@app.task("daily")
def do_main():
    ...

There are pros and cons in each option. In short:

Execution	Parallerized?	Can be terminated?	Can modify the session?
`process`	Yes	Yes	No
`thread`	Yes	Yes if task supports	Yes
`main`	No	No	Yes

Changing Logging Destination#

Logging the states of the tasks is vital for Red Engine’s system. This includes the logs about when each task started, succeeded and failed. This information is used in many of the scheduling statements and to prevent setting the same task running multiple times.

Red Engine extends logging library’s loggers extending them with Red Bird to enable reading the logs.

By default, the logs are stored in-memory and they do not persist over if the scheduler is restarted. You may change the destination simply by creating a Red Bird repo and pass that as the logger_repo to the application.

Storing the logs in-memory:

from redengine import RedEngine
from redbird.repos import MemoryRepo

app = RedEngine(
    logger_repo=MemoryRepo()
)

Storing the logs to CSV file:

from redengine import RedEngine
from redbird.repos import CSVFileRepo

app = RedEngine(
    logger_repo=CSVFileRepo(
        filename="logs.csv"
    )
)

See more repos from Red Bird documentation. We will get back on customizing the loggers in later tutorials.

Red Engine: Advanced Scheduler

Basic Tutorial

Contents

Basic Tutorial#

Time Scheduling#

Execution Options#

Changing Logging Destination#