The Figure below illustrates how the workﬂows designed by the AI planner are ranked based on meta-models built by three workﬂow assessment components. Meta-models are generated in oﬄine mode (right side of the ﬁgure) using meta-data from the DM experiment repository (DMER), e.g., input data characteristics computed by the DCTool, descriptions of workﬂows and performance results. For the meta-miner, these meta-data are parsed and structured into a Data Mining Experiment Database (DMEX-DB) using concepts from the Data Mining Optimization Ontology (DMOP).
The three meta-level components use these meta-data in diverse ways to build workﬂow assessment models. First, an ontology-based meta-miner builds a predicitve model by correlating dataset and workﬂow/algorithm characteristics with observed performance in past experiments. A second component uses expert rules to build a qualitative workﬂow assessment model, whereas a Time and Memory Analyser builds a model that estimates time and memory consumption of candidate workﬂows.
In production mode, the AI Planner generates a typically large number of workﬂows that need to be ranked. First, the meta-mined model is applied to score the candidate workﬂows, and only the k top-ranked workﬂows are further assessed by the non-functional (qualitative and time/memory-based) models. Finally, the probabilistic ranker aggregates the scores produced by the three meta-models using predeﬁned weights and delivers a ﬁnal ranking of the preselected workﬂows.