Advanced Guide
Contents
Advanced Guide#
This is an advanced level tutorial.
App Configuration#
There are several configurations options for the application.
The configurations can be set by:
app = RedEngine(config={
'task_execution': 'process',
'task_pre_exist': 'raise',
'force_status_from_logs': True,
'silence_task_prerun': False,
'silence_cond_check': False,
'max_process_count': 5,
'restarting': 'replace'
})
These are the default options.
task_execution: How tasks are run by default. Options:
process: on separate process
thread: on separate thread
main: no parallelization
task_pre_exist: What happens if a task with given name already exists. Options:
raise
: An error is thrownrename
: the task is renamed (numbers added after the name)ignore
: The task is simply not inserted to the session
force_status_from_logs: Use logs always to determine the task statuses. If:
True
: Logs are always read when checking the statuses. Robust but less performant.False
: If cached status found, it is used instead.
silence_task_prerun: Whether to silence errors occurred before running a task. If:
True
: The scheduler does not crash on errors occurred on the startup of a taskFalse
: The scheduler crashes. Useful for debug but not for production
silence_cond_check: Whether to silence errors occurred when checking conditions’ values. If:
True
: The scheduler does not crash on errors occurred on checking conditionsFalse
: The scheduler crashes. Useful for debug but not for production
max_process_count: Maximum number of processes allowed to be started
By default the number of CPUs
restarting: How the scheduler is restarted (if restart is called)
replace
: Restart by replacing the current processrelaunch
: Restart by starting a new processfresh
: Restart by starting a new process (on new window on Windows)recall
: Restart by calling the start method again. Useful for testing the restart
Task Types#
So far, we have only used FuncTask
with passing a callable.
There are other task types as well to cover most common use cases:
FuncTask
: Executes a Python functionCommandTask
: Executes a shell commandCodeTask
: Executes raw code as string. Potentially dangerous.
Here are the ways to initialize tasks:
@app.task('daily')
def do_things():
...
def run_things():
...
app.task('daily', func=run_things)
app.task('daily', func_name="main", path="path/to/example.py")
app.task('daily', command='echo "Hello world"')
app.task('daily', code='print("Hello world")')
Modifying the System on Runtime#
The scheduler system can be modified in runtime. You could during the runtime:
shut down the scheduler
restart the scheduler
force a task to be run
disable a task
create, update or delete tasks
To do there, you can create a task that runs either as a separate thread or on the main loop. Tasks parallelized as separate processes cannot alter the scheduling environment due to limitations with sharing memory.
To alter the session:
from redengine.args import Session
@app.task('every 20 hours', execution="thread")
def do_shutdown(session=Session()):
session.shutdown()
@app.task('every 20 hours', execution="thread")
def do_restart(session=Session()):
session.restart()
@app.task('every 10 minutes', execution="thread")
def do_modify_tasks(session=Session()):
task = session['do_restart']
task.force_run = True
for task in session.tasks:
task.disable = True