Meta Controller: Meta-Task & Meta-Dataset & Meta-Model¶
Meta Controller provides guidance to
Forecast Model, which aims to learn regular patterns among a series of forecasting tasks and use learned patterns to guide forthcoming forecasting tasks. Users can implement their own meta-model instance based on
Meta Controller module.
A Meta Task instance is the basic element in the meta-learning framework. It saves the data that can be used for the Meta Model. Multiple Meta Task instances may share the same Data Handler, controlled by Meta Dataset. Users should use prepare_task_data() to obtain the data that can be directly fed into the Meta Model.
MetaTask(task: dict, meta_info: object, mode: str = 'full')¶
A single meta-task, a meta-dataset contains a list of them. It serves as a component as in MetaDatasetDS
The data processing is different - the processed input may be different between training and testing
- When training, the X, y, X_test, y_test in training tasks are necessary (# PROC_MODE_FULL #)
- but not necessary in test tasks. (# PROC_MODE_TEST #)
- When the meta model can be transferred into other dataset, only meta_info is necessary (# PROC_MODE_TRANSFER #)
__init__(task: dict, meta_info: object, mode: str = 'full')¶
The __init__ func is responsible for - store the task - store the origin input data for - process the input data for meta data
- task (dict) – the task to be enhanced by meta model
- meta_info (object) – the input for meta model
get_meta_input() → object¶
Return the processed meta_info
Meta Dataset controls the meta-information generating process. It is on the duty of providing data for training the Meta Model. Users should use prepare_tasks to retrieve a list of Meta Task instances.
MetaTaskDataset(segments: Union[Dict[str, Tuple], float], *args, **kwargs)¶
A dataset fetching the data in a meta-level.
A Meta Dataset is responsible for - input tasks(e.g. Qlib tasks) and prepare meta tasks
- meta task contains more information than normal tasks (e.g. input data for meta model)
The learnt pattern could transfer to other meta dataset. The following cases should be supported - A meta-model trained on meta-dataset A and then applied to meta-dataset B
- Some pattern are shared between meta-dataset A and B, so meta-input on meta-dataset A are used when meta model are applied on meta-dataset-B
__init__(segments: Union[Dict[str, Tuple], float], *args, **kwargs)¶
The meta-dataset maintains a list of meta-tasks when it is initialized.
The segments indicates the way to divide the data
The duty of the __init__ function of MetaTaskDataset - initialize the tasks
prepare_tasks(segments: Union[List[str], str], *args, **kwargs) → List[qlib.model.meta.task.MetaTask]¶
Prepare the data in each meta-task and ready for training.
The following code example shows how to retrieve a list of meta-tasks from the meta_dataset:
# get the train segment and the test segment, both of them are lists train_meta_tasks, test_meta_tasks = meta_dataset.prepare_tasks(["train", "test"])
Parameters: segments (Union[List[Text], Tuple[Text], Text]) – the info to select data Returns: A list of the prepared data of each meta-task for training the meta-model. For multiple segments [seg1, seg2, … , segN], the returned list will be [[tasks in seg1], [tasks in seg2], … , [tasks in segN]]. Each task is a meta task Return type: list
General Meta Model¶
Meta Model instance is the part that controls the workflow. The usage of the Meta Model includes: 1. Users train their Meta Model with the fit function. 2. The Meta Model instance guides the workflow by giving useful information via the inference function.
Meta Task Model¶
This type of meta-model may interact with task definitions directly. Then, the Meta Task Model is the class for them to inherit from. They guide the base tasks by modifying the base task definitions. The function prepare_tasks can be used to obtain the modified base task definitions.
Meta Guide Model¶
This type of meta-model participates in the training process of the base forecasting model. The meta-model may guide the base forecasting models during their training to improve their performances.
Qlib provides an implementation of
Meta Model module,
which adapts to the market dynamics.
DDG-DA includes four steps:
- Calculate meta-information and encapsulate it into
Meta Taskinstances. All the meta-tasks form a
DDG-DAbased on the training data of the meta-dataset.
- Do the inference of the
DDG-DAto get guide information.
- Apply guide information to the forecasting models to improve their performances.
The above example can be found in