Meta Controller: Meta-Task & Meta-Dataset & Meta-Model¶
Introduction¶
Meta Controller
provides guidance to Forecast Model
, which aims to learn regular patterns among a series of forecasting tasks and use learned patterns to guide forthcoming forecasting tasks. Users can implement their own meta-model instance based on Meta Controller
module.
Meta Task¶
A Meta Task instance is the basic element in the meta-learning framework. It saves the data that can be used for the Meta Model. Multiple Meta Task instances may share the same Data Handler, controlled by Meta Dataset. Users should use prepare_task_data() to obtain the data that can be directly fed into the Meta Model.
-
class
qlib.model.meta.task.
MetaTask
(task: dict, meta_info: object, mode: str = 'full')¶ A single meta-task, a meta-dataset contains a list of them. It serves as a component as in MetaDatasetDS
The data processing is different - the processed input may be different between training and testing
- When training, the X, y, X_test, y_test in training tasks are necessary (# PROC_MODE_FULL #)
- but not necessary in test tasks. (# PROC_MODE_TEST #)
- When the meta model can be transferred into other dataset, only meta_info is necessary (# PROC_MODE_TRANSFER #)
-
__init__
(task: dict, meta_info: object, mode: str = 'full')¶ The __init__ func is responsible for - store the task - store the origin input data for - process the input data for meta data
Parameters: - task (dict) – the task to be enhanced by meta model
- meta_info (object) – the input for meta model
-
get_meta_input
() → object¶ Return the processed meta_info
Meta Dataset¶
Meta Dataset controls the meta-information generating process. It is on the duty of providing data for training the Meta Model. Users should use prepare_tasks to retrieve a list of Meta Task instances.
-
class
qlib.model.meta.dataset.
MetaTaskDataset
(segments: Union[Dict[str, Tuple], float], *args, **kwargs)¶ A dataset fetching the data in a meta-level.
A Meta Dataset is responsible for - input tasks(e.g. Qlib tasks) and prepare meta tasks
- meta task contains more information than normal tasks (e.g. input data for meta model)
The learnt pattern could transfer to other meta dataset. The following cases should be supported - A meta-model trained on meta-dataset A and then applied to meta-dataset B
- Some pattern are shared between meta-dataset A and B, so meta-input on meta-dataset A are used when meta model are applied on meta-dataset-B
-
__init__
(segments: Union[Dict[str, Tuple], float], *args, **kwargs)¶ The meta-dataset maintains a list of meta-tasks when it is initialized.
The segments indicates the way to divide the data
The duty of the __init__ function of MetaTaskDataset - initialize the tasks
-
prepare_tasks
(segments: Union[List[str], str], *args, **kwargs) → List[qlib.model.meta.task.MetaTask]¶ Prepare the data in each meta-task and ready for training.
The following code example shows how to retrieve a list of meta-tasks from the meta_dataset:
# get the train segment and the test segment, both of them are lists train_meta_tasks, test_meta_tasks = meta_dataset.prepare_tasks(["train", "test"])
Parameters: segments (Union[List[Text], Tuple[Text], Text]) – the info to select data Returns: A list of the prepared data of each meta-task for training the meta-model. For multiple segments [seg1, seg2, … , segN], the returned list will be [[tasks in seg1], [tasks in seg2], … , [tasks in segN]]. Each task is a meta task Return type: list
Meta Model¶
General Meta Model¶
Meta Model instance is the part that controls the workflow. The usage of the Meta Model includes: 1. Users train their Meta Model with the fit function. 2. The Meta Model instance guides the workflow by giving useful information via the inference function.
Meta Task Model¶
This type of meta-model may interact with task definitions directly. Then, the Meta Task Model is the class for them to inherit from. They guide the base tasks by modifying the base task definitions. The function prepare_tasks can be used to obtain the modified base task definitions.
Meta Guide Model¶
This type of meta-model participates in the training process of the base forecasting model. The meta-model may guide the base forecasting models during their training to improve their performances.
Example¶
Qlib
provides an implementation of Meta Model
module, DDG-DA
,
which adapts to the market dynamics.
DDG-DA
includes four steps:
- Calculate meta-information and encapsulate it into
Meta Task
instances. All the meta-tasks form aMeta Dataset
instance. - Train
DDG-DA
based on the training data of the meta-dataset. - Do the inference of the
DDG-DA
to get guide information. - Apply guide information to the forecasting models to improve their performances.
The above example can be found in examples/benchmarks_dynamic/DDG-DA/workflow.py
.