]> CURRENT VERSION: DMOP-v5.4 (under development). v5.3 (uploaded 2012-01-10) Editors/Major Contributors: Melanie Hilario, Maria Keet, Agnieszka Lawrynowicz. Other Contributors: Huyen Do, Simon Fischer,Dragan Gamberger, Lina Al-Jadir, Simon Jupp, Alexandros Kalousis, Petra Kralj.Novak, Babak Mougouie, Phong Nguyen, Raul Palma, Robert Stevens, Anze Vavpetic, Jun Wang, Derry Wijaya, Adam Woznica e-lico Data Mining Ontology (DMO) for Data Mining Optimization (DMOP) Melanie.Hilario (v0.1 created Mar 2009) achieves: an Operation O achieves a Task T indicates that the operation achieves the task addressed by the operator or the workflow executed by the operation. addresses: an algorithm A addresses a task T == A specifies a way of performing T. assumes: an object property that connects an algorithm to one or several of its assumptions. If these assumptions are not met, the algorithm will not produce the expected results or may not even run at all. executes: an Operation (or process) P executes an Operator (or ground workflow) O == P is the process of running O to achieve the task for which O was designed. handlesFeatureType: a DM algorithm A expectsFeatureType T (a DataType) == A can handle only dataset features or variables of type T. Typically these features can be of type categorical, real (continuous) or, more rarely, ordinal. hasCombinationFunction: a model evaluation function may comprise two components (typically a performance measure and a complexity measure) whose trade-off is regulated by a combination function (often, their sum or product) and their relative weights. hasComplexityComponent: A model evaluation function E hasComplexityComponent F (a model complexity function). A learning algorithm's cost function is often composed of two components: a loss component (that quantifies the estimated error of the algorithm) and a complexity component (that quantifies the complexity of the model produced by the algorithm. These two components are not always present. The property "hasComplexityComponent" takes a value other than "None" when the cost function comprises some estimation of model complexity. hasComponent is a subproperty of hasPart that represents the component-integral object relationship identified by Winston et al. (1987) and Odell (1994). It defines a configuration of parts within a whole, i.e., the parts bear a particular functional or structural relationshop to one another as well as to the whole. M. E. Winston, R. Chaffin, D. Hermann. A taxonomy of part-whole relations. Cognitive Science 11, 417-444 (1987). J. J. Odell, Six different kinds of composition. Journal Of Object-Oriented Programming 5, 10-15 (1994). hasDataType: specifies the data type of a given data item. The range of this property is the DataType class, which is a hierarchy that overlaps with but is different from OWL's built-in set of datatypes. hasDecisionRule: see DecisionRule. hasDecisionStrategy: see DecisionStrategy. hasDistinctValue relates a CategoricalFeature to each of the different values (class: CategoricalFeatureValue) that it can take. Each of these values can then be characterized by a set of statistical properties. hasDistributionParameter: The distribution parameters of a Normal density estimate are an N x D Gauss-MeanVector and a D x D Gauss-CovarianceMatrix. hasFeature: links a DataSet or DataTable to any of its component features. hasFeatureEvaluator designates the algorithm used by a feature selection algorithm to measure the quality of the candidate features, either individually or as feature subsets. hasFeatureTestEvaluator: For decision trees and rules, the criterion used to select the feature to use in the next tree node or rule condition. Aka split criterion. hasHypothesisComplexityControlStrategy: A classification modeling algorithm A hasComplexityControlStrategy S == A follows strategy S to restrict the complexity of the learned hypothesis (model or pattern set) and thus avoid overfitting. hasLeafPredictor: In a decision tree, the predictive model (classifier or regressor) that predicts the value of the target variable for all the instances grouped in a specific leaf. This can be the default classifier (majority rule) or regressor (mean value rule) as in CART or a learned model, e.g. a NaiveBayes classifier in NBTree. hasMember is a subproperty of hasPart in which the parts are members of a collection. Based on Winston et al. (1987) and Odell (1994). hasModalValue: refers to the value of a categorical feature that appears most frequently in a given dataset. hasOptimizationProblem: designates the optimization problem solved by a data mining algorithm in order to achieve its task (in particular induction and feature extraction). See OptimizationProblem. hasOptimizationStrategy refers to the optimization strategy adopted by a data mining algorithm to solve its optimization problem. Related concepts: OptimizationStrategy, OptimizationProblem. hasPreprocessor: Replace with hasComponentStep and its subObjectProperties, step1, step2, etc.? hasRecoveryOfPursuit: a property that relates a heuristic search algorithm to a value set named after Pearl's (1984, p. 65) "recovery of pursuit, which can be described as either tentative or Irrevocable.See the concept RecoveryOfPursuit. hasScopeOfSelection:a dimension along which Pearl (1984) describes search algorithms. At one extreme, certain search algorithms select from all available candidates (in a graph-based search, all open nodes); algorithms at the other extreme evaluate and select only from the most recent candidates (e.g. the successors of the current node). hasSearchDirection: refers to the direction in which a search procedure is conducted: forward, backward, bidirectional or random. hasSimilarityFunction: designates the similarity function (Kernel) used by a data mining algorithm. hasTargetFeature: specifies which, among the values of hasFeature, is the outcome or response variable/feature. This is applicable only to labeled datasets, where labels can be continuous or categorical. hasTargetLearner: designates the induction algorithm for which a given feature processing algorithm is deemed most appropriate, if any. isImplementedBy: A DM-Algorithm isImplementedBy a DM-Operator. In other words, a DM-Operator is the implementation of a DM-Algorithm in a specific programming language. An operator is executable, an algorithm is not. Inverse of the property (DM-Operator) implements (DM-Algorithm. realizes (Domain: OperatorExecution, Range: Algorithm) the execution of an operator realizes the specifications contained in the algorithm implemented by the operator. Note an algorithm is basically an abstract specification of a process which comes into existence through an operator execution. The composition of d-uses and valued-by relations: a description d-uses a parameter that is valued by a region. A.K.A. 'selected-by'.The referencing relation between concepts defined by descriptions and constituents of situations. It can be understood as a reification of a 'satisfiability' relation holding between elements of theories and elements of models.It has a time index, but this should not be intended as a partial compresence, since the time only refers to a part of the classified particular life or extension. A.K.A. 'selects'.The referencing relation between concepts defined by descriptions, and constituents of situations. It can be understood as a reification of a 'satisfiability' relation holding between elements of theories and elements of models.It has a time index, but this should not be intended as a partial compresence, since the time only refers to a part of the classified particular life or extension. A relation holding between two endurants participating in a same perdurant. This typically subsumes many common sense, verbally encoded, relations, such as "making", "moving", "transforming", etc. 'Component' is a proper part with a role (or function) in a system or a context. Roles can be different for the same entity, and the evaluation of them changes according to the kind of entity. For instance, components of endurants can 'play functional roles' in a whole, while components of perdurants are the essential 'episodes' in their whole.As a functional part relation, component is not transitive, because functions depend on intentions and/or designs, and something intentionally essential for a direct whole, can be non-essential for another, indirect whole. The role shared by all members of a collection has a covering relation towards the collection. Descriptions can d-use (descriptively use) concepts or figures, provided that used ones are defined by some description. Descriptions define either concepts or (social) figures. Once defined, they can be d-used by other descriptions. Anti-transitive predecessor. Anti-transitive succession. A partial order relation that holds between descriptions. It represents the proper part relation between a description and another description featuring the same properties of the former, with at least an additional one.Descriptions can be expanded either by adding other descriptions as parts, or by refining concepts or figures that are d-used by them.Specializing the concepts or figures that are d-used by them is on the contrary a case of description specialization.For descriptions, an intention to expand must be present (unless purely formal theories are considered, but even in this case a criterion of relevance is usually active). The composition of d-uses and sequences relations: a description d-uses a course that sequences a perdurant. A method can exploit an involved endurant when it plays a device-like role. A relation between information objects that are used as representations (signs) and the content (meaning, conceptualization) they represent. In this ontology content is reified as a 'description'.Information objects are 'systemic' objects created by the system of rules of a semiotic code. For the representation between the physical implementation of information objects (physical representations) and information objects, the 'realized-by' relation is used. An activity expected by a method. The maximal specialization achievable according to some criterion. Only applicable to social objects that logically reify set-related entities (relations, classes, sets). The composition of d-uses and played-by relations: a description d-uses a role that is played by an endurant. A particular case of an endurant participating in a perdurant that meets (is connected to the beginning of) the life of another endurant. Being a (generic, temporary) constituent in a countable collection, for example: member of a society, bacterium in a colony, etc. A task (as any other concept) can be optional within some plan (or any description). In this case, it can be ignored in plan execution without affecting the satisfaction of the plan.Within plans, an task said to be optional should be placed in a way that preserves the topology (the connectedness) of the maximal task, except for sequential tasks, where it can be skipped without affecting the control structure. In fact, an optional task must either be component of a bag or sequential task, or have the concurrent task or the any-order task as a direct predecessor. a.k.a. 'expressed according to'. The relation between information objects and the languages, codes, grammars, etc. that they are ordered by. E.g. Dante's Comedy is ordered by Middle Age Italian language (in this case, a complex of encoding systems).In principle, any description can be used as an encoding system, but in practice, only some combinatorial systems are used for encoding (see module on 'information objects'). The mediated relation between an entity and a parameter through the region at which the entity is localized and that is the value for the parameter. This is the immediate relation between roles and endurants. A role classifies the position (function, use, relevance, ...) of an endurant within a context (description). Roles can be ordered, interdependent, at different layers. etc. This is the immediate relation between roles and endurants. A role classifies the position (function, use, relevance, ...) of an endurant within a context (description). Roles can be ordered, interdependent, at different layers. etc. To be understood as 'entity x has predecessor y'.This is the transitive version, but it results to be a complex property in OWL-DL, and transitivity should be overruled. A relation holding between non-physical objects and entities whatsoever (thus including non-physical objects themselves). An intuition for the references relation could be that a non-physical object adds 'information' to an entity. In fact, non-physical objects depend on a communication setting. In most cases, this is the characteristic relation that provides a unity criterion to objects, events, etc. For example, cars are objects and not mere aggregates because there is a project, a design, a social value, a functional structure, a personal emotional structure, etc. attached to them. This attachment can be represented by means of 'non-physical objects' that 'reference' cars. The most obvious application is for situations, which do not exist without a description, although they still are extensional entities: a situation without a part is no more the same situation, but a situation is not a mere aggregate, since it has references to a description as its unity criterion. Adding information to an entity can also be thought as an intentional solution to a holistic stance. Defenders of this view -within different frameworks- are Kant, Brentano, Husserl, Gestalt psychologists, Merleau-Ponty ... References is distinguished according to the kinds of non-physical objects and referenced ground entities: referencing between descriptions and situations is called 'SATISFIED-BY', while referencing between description components and situation components is called 'CLASSIFIES'. 'SETTING-FOR' is a referencing relation between a situation and the entities in its setting (it was formerly a constitution relation, but since situation appear to be social objects from the DOLCE viewpoint, the constitution solution is no more applicable). 'EXPRESSES' is bound to information objects and the meaning (description of a representation or conceptualization) in which they are involved. 'REALIZED-BY' is bound to information objects and physical representations that are used to communicate them, etc. 'ABOUT' is bound to information objects and entities whatsoever (aboutness of intentionality). A relation holding between non-physical objects and entities whatsoever (thus including non-physical objects themselves). An intuition for the references relation could be that a non-physical object adds 'information' to an entity. In fact, non-physical objects depend on a communication setting. In most cases, this is the characteristic relation that provides a unity criterion to objects, events, etc. For example, cars are objects and not mere aggregates because there is a project, a design, a social value, a functional structure, a personal emotional structure, etc. attached to them. This attachment can be represented by means of 'non-physical objects' that 'reference' cars. The most obvious application is for situations, which do not exist without a description, although they still are extensional entities: a situation without a part is no more the same situation, but a situation is not a mere aggregate, since it has references to a description as its unity criterion. Adding information to an entity can also be thought as an intentional solution to a holistic stance. Defenders of this view -within different frameworks- are Kant, Brentano, Husserl, Gestalt psychologists, Merleau-Ponty ... References is distinguished according to the kinds of non-physical objects and referenced ground entities: referencing between descriptions and situations is called 'SATISFIED-BY', while referencing between description components and situation components is called 'CLASSIFIES'. 'SETTING-FOR' is a referencing relation between situation and the entities in its setting (it was formerly a constitution relation, but since situation appear to be social objects from the DOLCE viewpoint, the constitution solution is no more applicable). 'EXPRESSES' is bound to information objects and the meaning (description of a representation or conceptualization) in which they are involved. 'REALIZED-BY' is bound to information objects and physical representations that are used to communicate them, etc. 'ABOUT' is bound to information objects and entities whatsoever (aboutness of intentionality). Concepts and figures can be refined by adding components, e.g. an elementary task can become complex, a complex task can increase its complexity, maximal tasks can be composed, etc.A description gets expanded if one of the concepts or figures it uses are refined.Refinement applies also to collections, situations, and information objects.Descriptions are refined by adding component descriptions. This is the immediate relation between courses and perdurants. A course can be either atomic, being a simple 'perdurant role', or it can be complex, thus creating an abstract ordering over a temporal or causal sequence of processes or actions. The ontology of plans develops in detail intentional complex courses. Specialization as reification of a partial-order relation between type- or set-reified social objects, i.e. descriptions, concepts, and collections.For example, concepts that are apparently classified by other concepts; e.g. a manager that plays the role of buyer, where the role manager actually specializes the role buyer. Descriptions can be specialized by other descriptions that specialize their concepts. For descriptions, an intention to specialize must be present (unless purely formal theories are considered, but even in this case a criterion of relevance is usually active). Specialization does not imply expansion (proper part) for descriptions. If there exists a concept that is defined by the specialized description, which is not d-used in the specializing one, the second only specializes a part of the first. If there exists a concept that is defined by the specializing description, which is not d-used by the specialized one, the first both specializes and expands the second. Specialization as reification of a partial-order relation between social objects. For example, concepts that are apparently classified by other concepts; e.g. a manager that plays the role of buyer, where the role manager actually specializes the role buyer. Descriptions can be specialized by other descriptions that specialize their concepts or figures. For descriptions, an intention to specialize must be present (unless purely formal theories are considered, but even in this case a criterion of relevance is usually active). Specialization does not imply expansion (proper part) for descriptions. If there exists a concept that is defined by the specialized description, which is not d-used in the specializing one, the second only specializes a part of the first. If there exists a concept that is defined by the specializing description, which is not d-used by the specialized one, the first both specializes and expands the second. To be understood as 'entity x has successor y'. Succession does not exclude connection, but it excludes overlapping. It can be direct or indirect, and assumes a choice (temporal, spatial, abstract, etc.) Cf. the cognitive 'path' schema. This is the transitive version. The use relations between endurants: an endurant e1 uses e2 within a perdurant in which both are participating. A rule then states that if e1 uses e2, e2 is used *in* a perdurant. The "selected by" relations holding between regions and parameters. At least one region is supposed to be a value for a parameter. hasExplicitFeatureSpace: For kernel functions, indicates if the mapping function $\phi$ is explicitly computable or not. isAdmissible: A Search Strategy or Algorithm is admissible if it terminates with an optimal solution when one exists. isComplete: A Search Strategy or Algorithm is complete if it terminates with a solution when one exists. isMultiAlgorithmOperator: An Operator mapsToMultipleAlgorithms if it implements several algorithms simultaneously, allowing the user to choose one algorithm by setting one or several operator hyperparameters. 1 1 METAL characteristic: Average absolute correlation between continuous features. METAL characteristic: Average mutual information between pairs of categorical features. METAL characteristic: Average feature Entropy METAL characteristic: Average mutual information METAL characteristic: A matrix containing the difference between the matrix of total and the matrix of within-groups sums of squares and cross products. METAL characteristic: Canonical correlation of the best linear combination of features to distinguish between classes. METAL characteristic: Absolute class frequencies. Stored in a vector indexed by each class value. METAL characteristic: Class covariance matrices. Stored in a vector indexed by class and each containing a matrix of (features x features) METAL characteristic: Class entropy. METAL characteristic: Relative class frequencies. Stored in a vector indexed by each class value. METAL characteristic: A vector of eigen values of linear discriminant functions. From Mitra Basu and Tin Kam Ho. Data Complexity in Pattern Recognition. Springer, 2006. From Mitra Basu and Tin Kam Ho. Data Complexity in Pattern Recognition. Springer, 2006. METAL characteristic: For each continuous feature, its correlation with other continuous features. Stored in a vector indexed by each continuous features. METAL characteristic: For each categorical feature, the feature entropy. A categorical value should be marked as frequent, iff it is 50% more frequent than we would expect under a uniform distribution. (e.g. 75% for a binary value) METAL characteristic: For each continuous feature, the ratio between the standard deviation and the standard deviation of alpha trimmed mean. If the standard deviation is 0, then the ratio is set 1. METAL characteristic: For each categorical feature, the mutual information between the feature and the class. It is stored in a vector indexed by each categorical feature. A categorical value should be marked as rare, iff it is 50% less frequent than we would expect under a uniform distribution. (e.g. 25% for a binary value) METAL characteristic: For each k value of the feature, the value frequency. It is stored in a vector indexed by each feature value. METAL characteristic: For each k value of each j categorical feature and each i class, the proportion of cases that have the k value in the j feature and belong to the i class. It is stored in a vector indexed by each categorical feature and containing a flat contingency tables that combine the values of the categorical feature with the class values. A categorical value should be marked as very frequent, iff it is 90% more frequent than we would expect under a uniform distribution. (e.g. 95% for a binary value) A categorical value should be marked as very rare, iff it is 90% less frequent than we would expect under a uniform distribution. (e.g. 5% for a binary value) From Mitra Basu and Tin Kam Ho. Data Complexity in Pattern Recognition. Springer, 2006. From Mitra Basu and Tin Kam Ho. Data Complexity in Pattern Recognition. Springer, 2006. From Mitra Basu and Tin Kam Ho. Data Complexity in Pattern Recognition. Springer, 2006. METAL characteristic: Noise signal ratio From Mitra Basu and Tin Kam Ho. Data Complexity in Pattern Recognition. Springer, 2006. From Mitra Basu and Tin Kam Ho. Data Complexity in Pattern Recognition. Springer, 2006. METAL characteristic: Number of continuous features with outliers. From Mitra Basu and Tin Kam Ho. Data Complexity in Pattern Recognition. Springer, 2006. From Mitra Basu and Tin Kam Ho. Data Complexity in Pattern Recognition. Springer, 2006. METAL characteristic: Proportion of continuous features with outliers. From Mitra Basu and Tin Kam Ho. Data Complexity in Pattern Recognition. Springer, 2006. From Mitra Basu and Tin Kam Ho. Data Complexity in Pattern Recognition. Springer, 2006. METAL characteristic: Matrix of total sums of squares and cross products of features. From Mitra Basu and Tin Kam Ho. Data Complexity in Pattern Recognition. Springer, 2006. METAL characteristic: matrix of within-groups sums of squares and cross products of features. AlgorithmAssumption: is a hypothesis based on which an algorithm has been developed, and which should be true if the algorithm is to achieve the task it was designed to address. 1 AssociationDiscoveryAlgorithm is designed to solve AssociationDiscoveryTask, and hence it mines data for associations, where association is a relation between objects, or measured quantities that results from interaction or dependence. This relationship is not necessarily causal. In statistics, an association is any such relationship that renders two measured quantities statistically dependent. AssociationDiscoveryTask consists in mining data for associations, where association is a relation between objects, or measured quantities that results from interaction or dependence. This relationship is not necessarily causal. In statistics, an association is any such relationship that renders two measured quantities statistically dependent. AttributeValueTableFormat: the format of an attribute-value table, i.e. a table where each table cell designates a value of a particular attribute (or feature) of a particular object (or example or instance). Bayesian networks specialist needed for this sub-ontology. Volunteers welcome. 1 Blind or uninformed search is search that uses no additional information about states beyond that provided in the problem definition (definition of goal state(s) and operators). All they can do is generate successors and distinguish a goal from a nongoal state [1]. As a result, the order in which the search progresses (e.g. the order in which nodes are expanded in a search tree/graph) does not depend on the nature of the solution sought [2]. [1] S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice-Hall, 2003. [2] J. Pearl. Heuristics: intelligent search strategies for computer problem-solving. Addison-Wesley, 1984. true true BranchAndBound is informed exhaustive search. It uses knowledge of states to cut off unpromising paths, i.e., which are known not to contain the goal state. It is non-heuristic in the sense that it maintains a provable upper and lower bound on the (globally) optimal objective value. [S. Boyd and J. Mattingley, Branch-and-Bound Methods, notes for EE364b. Stanford. March 11, 2007) CSVC-Algorithm: An SVC algorithm that uses the hyperparameter C (as opposed to Nu-SVC-Algorithm which uses the hyperparameter Nu). The hyperparameter C controls the trade-off between the loss function (hinge loss) and the regularizer (squared $L_2$ norm of the weight vector $\mathbf{w}$) in the SVM cost function. SVMs usually address binary classification problems (with labels y = {+1, -1}), also they can be extended to multi-class (> 2 classes) classification problems. The C-SVCAlgorithm handles noise through the use of slack variables.The degree of noise tolerance is adjusted through the parameter C. 1 1 1 1 1 1 1 1 1 1 CategoricalFeature: a feature (attribute or variable) that takes values from a finite set of discrete, non-numeric, unordered labels. Sometimes called a nominal feature. 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ClassCondMeanMatrix is the KxD matrix of class conditional means -- the mu parameter of Normal L/Q Discriminant Models. ClassCondProbMatrix: A k x d matrix where k = number of classes and d = number of words/features, each cell M_{ci}, c = 1 ... k, i = 1 ... d, contains the probability of feature i in class c, and each row sums to 1. Used in all NaiveBayes models built from discrete or mixed discrete/continuous datasets. ClassCondVarianceMatrix is actually a concatenation of D univariate vectors representing the class-conditional variance of each of the D features. It has dimensions KxD where each cell $M_{kd}$ is the variance of feature d within class k. Based on feature independence assumption of Naive Bayes. 1 1 ClassificationModelingAlgorithm: an algorithm that builds (learns) a classifier. For the moment, ClassificationModelingAlgorithm implicitly stands for propositional supervised classification algorithms. This is the reason its input range is specificed as LabeledDataSet and PropositionalDataSet. In the future we will add relational classification modeling algorithms as well as semi-supervised classification modeling algorithms. ClassificationProblemType refers to the number of classes that can be distinguished by a classifier (and by the classification algorithm that produced it). A binary classification problem consists in distinguishing two classes, a multiclass classification problem more than two classes. More recently one-class classifications consists in distinguishing instances of a selected class from non-instances, regardless of the class membership of these non-instances. 1 1 1 TreeInductionAlgorithm: Hand et al 01, p. 335: Decision trees can be considered discriminant functions if they predict crisp classes at the leaves and discriminative if they predict probabilities. TreeInductionAlgorithm: This used to RecursivePartitioningAlgorithm but has been renamed TreeInductionAlgorithm because the RuleInductionAlgorithm overlaps with RecursivePartitioningAlgorithm, which cannot therefore be a primitive classe. ComplementClassCondProbMatrix: A complement class-conditional probability matrix is a k x d matrix where k = number of classes and d = number of words/features, each cell $M_{ci}, c = 1 ... k, i = 1 ... d$, contains the probability of feature i NOT in class c, and each row sums to 1. Used in ComplementNaiveBayes [Rennie, 2003]. 1 ComplementNaiveBayesModel's ModelParameter: is a k-list of vectors $\theta_c$, c=1 to k (number of classes), where $\theta_{ci}$ is the probability of word/feature i NOT in class c, with i = 1 to p (number of words/boolean features). We can therefore represent this model parameter as a k x p matrix whose rows are classes and columns are words/boolean features. We call this a ComplementCCProbMatrix. Constraint: In mathematics, a constraint is a condition that a solution to an optimization problem must satisfy. 1 1 1 1 1 1 ContinuousFeature:a feature/attribute/variable that takes its values from a range of real numbers. 1 CoreDMTask: a data mining task that can be achieved by a number of alternative methods organized in a more or less complex subtree under the Algorithm class. Thus a CoreDMTask cannot be accomplished without preliminary search in a space of potential methods/algorithms. This restrictive definition of a core DM task excludes utility tasks such as dataset reading/writing, model application, etc. Contrary to other DM ontologies, the e-lico DMOP does not classify CoreDMTasks according to the standard DM phases: Pre-processing, Modeling and Post-Processing. The reason is that many of these tasks can be performed in any phase: for example, one can nest a ModelingTask in the Pre-processing phase; for instance, the MissingValueImputationTask can be cast as a PredictiveModelingTask whereby a classification or regression learner is used to predict the missing values of a selected feature. In the reverse direction, Feature(Set)ProcessingTasks can be integrated into the modeling phase, as in the case of embedded feature selection techniques. 1 1 DM-Algorithm: An algorithm in general is a well defined sequence of steps that specifies how to solve a problem or perform a task. It typically accepts an input and produces an output. A DM algorithm is an algorithm that has been designed to perform any of the DM tasks, such as feature selection, missing value imputation, or modeling (or induction). The higher-level classes of the DM-Algorithm hierarchy correspond to DM-Task types. Immediately below are broad algorithm families or what data miners more commonly call paradigms or approaches. The Algorithm hierarchy bottoms out in individual algorithms such as CART, Lasso or ReliefF. A particular case of a DM algorithm is a Modeling (or Learning) algorithm, which is a well-defined procedure that takes data as input and produces output in the form of models or patterns. DM-Data: In SUMO, Data is defined as 'an item of factual information derived from measurement or research' [http://sigma.ontologyportal.org:4010/sigma/WordNet.jsp?word=data&POS=1] In IAO, Data is an alternative term for 'data item' =def 'an information content entity that is intended to be a truthful statement about something (modulo, e.g., measurement precision or other systematic errors) and is constructed/acquired by a method which reliably tends to produce (approximately) truthful statements.' [http://purl.obolibrary.org/obo/IAO_0000027] In the context of DMOP, DM-Data is the generic term that englobes different levels of granularity: data can be a whole dataset (one main table and possibly other tables), or only a table, or only a feature (column of a table), or only an instance (row of a table), or even a single feature-value pair. 1 Model: a simplified description of a complex entity or process [http://virtual.cvut.cz/ksmsaWeb/browser/title]. DM-Operation: a process in which a DM-Operator is executed. Synonym: DM-OperatorExecution. DM-Operator: a programmed, executable implementation of a DM-Algorithm. DM-PatternSet: A pattern set, as opposed to a model which by definition has global coverage, is a set of local hypotheses, i.e. each applies to a limited region of the sample space. DM-Software: Restriction to the Data Mining area of the general concept of Software: (computer science) written programs or procedures or rules and associated documentation pertaining to the operation of a computer system and that are stored in read/write memory; "the market for software is expected to expand". [http://sigma.ontologyportal.org:4010/sigma/Browse.jsp?kb=SUMO&lang=en] DM-Task: A task in general is any piece of work that is undertaken or attempted [SUMO]. A DM-Task is any task that needs to be addressed in the data mining process. DMOP's DM-Task hierarchy models all the major task classes. DataFormat: the organization of information according to preset specifications (usually for computer processing) [http://sigma.ontologyportal.org:4010/sigma/Browse.jsp?kb=SUMO&lang=en]. DataProcessingAlgorithm: an algorithm that specifies a procedure for solving a DataProcessingTask (see this concept). DataProcessingTask: takes in any subclass of Data (a DataSet, DataTable or Feature) and outputs a some transformation of its input. 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 DataSet: in data mining, the term data set is defined as a set of examples or instances represented according to a common schema. 1 DataTable: a set of data arranged in rows and columns. [http://virtual.cvut.cz/ksmsaWeb/browser/title] DecisionRule: a subclass of Decision strategy that selects a course of action by applying a yes/no rule. The various instances of decision rules are based on the specific test or criterion used. In data mining, a decision rule is most often applied to a sequence of scalar values that represent measures of quality (e.g. estimated feature quality), performance (e.g. misclassification rate), probability, etc. The test takes the form of a triple <Focus, RelOp, Threshold>, where Focus represents the specific criterion on which the decision is based: the observed values themselves, their ranks or their probabilities. RelOp is one of the relational operators {LessThan, Leq, Eq, GreaterThan, Geq}, and Threshold is the cutoff point on the magnitude, rank, percentage or probability. Note that Threshold is not necessarily a constant, it can be a function (e.g., max, min) of the observed values. For example, the Max Rule is a particular case of the Top K Rule where K=1, and the Maximum a posteriori rule is a special case of the Max rule, where the focus is a probability distribution the RelOp is Eq, and the threshold is the maximum of the observed probabilities. DecisionRule hasFixedThreshold: takes on a value only if the threshold on the decision criterion is a constant in the algorithm; if this property is empty, the threshold is a user-defined parameter. DecisionRule DecisionStrategy: a strategy followed to make a decision at some point or a data mining or optimization process, e.g. to select a feature subset after scoring or ranking features, to predict an outcome after building a probabilistic model or an SVM. A decision strategy can take the form of a decision rule or a statistical test. DecisionTree:a tree-structured predictive model where each path from the root to a leaf can be read as a rule, or a conjunction of conditions (tests on the nodes along the path). The different paths represent an exclusive disjunction (XOR) of conjunctions, i.e., the rules are non-overlapping and each example is covered by exactly one rule. 1 GenerativeModel is pairwise disjoint with DiscriminativeModel and DFModel. But DiscriminativeModel and DiscriminantFunctionModel may or may not be disjoint because some DF models output probabilities. Those who argue for disjointness say that even if DF models output probabilities, what distinguished them from discriminative models is their ModelStructure. If this is not a posterior probability distribution as in logistic regession, then the model is a DFmodel. A potential counter-argument is that maybe it is the concept of ModelStructure that needs to be revised. The issue remains open. LearningPolicy: An InductionAlgorithm's learning policy is its basic approach to the problem of learning, which can be divided into two broad categories: 1) Eager: The learning policy consists in condensing a collection of data into a compact hypothesis, i.e. a model or a pattern set, that can later be applied to new data to achieve the purpose for which the hypothesis was built. 2) Lazy: the policy is simply to store the data, deferring the data analysis or inductive process to the moment when the hypothesis is needed. In short, lazy learning is data memorization as opposed to eager learning which consists in hypothesis construction. 1 1 Feature: a property of an instance (case, example, record). Synonyms: Attribute, Variable. FeatureBasedModelParameter: Model parameter conceptualization to be reviewed. The categorization of model parameters based on whether they concern the class, features or instances was a good idea. Only caveat: Three of the 4 feature-based probabilistic parameters concern BOTH the class and the features; they will therefore have two primitive parents, which is contrary to ontological best practices. [mh 20091215]. FeatureClassMutualInformation: For each categorical feature, the mutual information between the feature and the class. It is stored in a vector indexed by each categorical feature. [From METAL project]. FeatureDiscretizationAlgorithm: an algorithm designed to achieve a FeatureDiscretizationTask, defined as the task of transforming a real or continuous feature into a discrete or categorical feature. 1 1 1 1 1 FeatureExtractionAlgorithm: an algorithm designed to achieve the FeatureExtractionTask, defined as the task of of converting a set of observed features (syn. variables, attributes) into a (potentially smaller) set of features deemed useful for the modeling or pattern discovery task at hand. Ref: I. Guyon, S. Gunn, M. Nikravesh and L. A. Zadeh. Feature Extraction: Foundations and Applications, Springer, 2006. FeatureNormalizationAlgorithn: rescales all values of a Feature by dividing each of them by the norm of the Feature vector. FeatureRankingAlgorithm: an algorithm designed to achieve a FeatureRankingTask, defined as a task that takes in a set of features and outputs an ranked list of these features based on the demands of a given data mining task. 1 1 FeatureSelectionAlgorithm: an algorithm designed to achieve a FeatureSelectionTask, defined as a task that takes in a set of p features and outputs a subset p' of the same features, where p' < p. FeatureSelectionAlgorithm is abbreviated as FSAlgorithm or FSA when used as a suffix. FeatureSetEvaluationFunction: TODO: Fill up the latexFormula annotation property based on Liu and Setiono (see primaryReference annotation for ConsistencyBasedFeatureSelectionAlgorithm) FeatureSquashToIntervalAlgorithm: rescales all values of a Feature in order to squash them within a user-defined interval, e.g., [0,1] 0r [-1,1] for classification. FeatureStandardizationAlgorithm: rescales all values of a continuous Feature through diverse operations on its mean, standard deviation, etc., or combinations of such operations. FeatureTransformationAlgorithm: an algorithm designed to achieve a FeatureTransformationTask, defined the task of transforming a feature by re-expressing its values in terms that facilitate the induction process. This is typically done on an individual feature without affecting or taking account of the other features in a data set. For instance, an individual feature can be rescaled to a fixed interval, or it can be standardized or normalized, or subjected to some functional transform (e.g., sqrt, log, logit). 1 1 FeatureWeightingAlgorithm: an algorithm designed to achieve a FeatureWeightingTask, defined as the task of assigning to a feature (set) a weight or score that quantifies the quality or usefulness of a feature (set) relative to a given modeling or pattern discovery task. FeatureWeightingAlgorithm is abbreviated as FWAlgorithm or FWA when used as a suffix. 1 FormalExpression: the union of MathematicalExpression and LogicalExpression. 1 1 GenerateAndSelectStrategy: a strategy for controling model complexity that consists in generating models/patterns of varying complexity/generality and selecting the least complex or the most general. 1 1 1 Graph: a set of nodes connected by arcs. false 1 1 1 1 false HeuristicBestFirstSearch: The term "heuristic search" is used in two different senses in the literature on Ai and optimization. In the very broad sense, heuristic search has two defining characteristics. First, heuristic search uses information other than that given in the problem definition to guide search. Second, heuristic search methods are not guaranteed to find the optimal solution; this characteristic distinguishes heuristic methods from branch and bound methods, which are informed but not heuristic methods. In the early AI literature on search, however, the term "heuristic search" was used in the framework of search viewed as the problem of going from an initial to a goal state, and typically used the graph model, where states are represented by nodes. In the absence of any information outside that given in the problem definition, the only solution was systematic search; the choice of the next node to expand (and ultimately of the solution path) was based on criteria such as node recency or an evaluation function f(n)=g(n)= the cost of going from the initial node to the current node (note that this function relies only on the problem definition and record of past problem solving steps). In contrast to this uninformed approach,other methods relied on a so-called heuristic evaluation function f(n)=g(n)+h(n), where h(n) is an estimate of the path from the current state to the goal state; the choice of this function requires domain-specific information. In this context, heuristic search in the narrow technical sense refers to methods based on some form of h(n) (with or without g(n)) were called heuristic search. The general approach used in heuristic search (in this specific sense) is best-first search, where the "best" node was that with the lowest evaluation function f(n). For this reason, we use the term HeuristicBestFirstSearch to designate heuristic search in this narrow technical sense and distinguish it from heuristic search in the first, broader sense defined above. BiasVarianceProfile: The bias-variance profile of an algorithm is a qualitative indication of the so-called capacity (or complexity) of the models it generates. High-bias algorithms are algorithms that can produce only low capacity (high bias) models; examples are linear discriminants and standard Naive Bayes. High-variance algorithms, on the contrary, can generate extremely complex models, so that a complexity control parameter is typically used to control the bias-variance trade-off. Depending on the value of the complexity parameter, an individual model can have (very) high bias or (ver) high variance. But the algorithm itself is classified as high-bias or high-variance based on the maximum variance or capacity that its generated models can attain. HillClimbing: an irrevocable search strategy based on local optimizations. Repeatedly expands a node moving in the direction of increasing value of some objective function. Terminates when it reaches a "peak" where no neighbor has a higher value. Aka greedly local search. [Pearl, 1984, Russell and Norvig, 2003]. HingeLoss: Primal SVM Loss function HypothesisApplicationAlgorithm: an algorithm designed to achieve a HypothesisApplicationTask, defined as the task of applying an induced model of pattern set to new data. A HypothesisApplicationAlgorithm is typically (and ambiguously) called an interpreter. HypothesisApplicationTask: the task of applying an induced model or pattern set to new data. HypothesisEvaluationTask: the task of quantifying the quality of an induced model or pattern set with respect to a specific criterion (e.g., predictive performance, interestingness). HypothesisProcessingAlgorithm: an algorithm designed to achieve a HypothesisProcessingTask, defined as the task of transforming an induced model or pattern set in view of a specific goal such as increasing its readability or improving its performance. HypothesisProcessingTask: the task of transforming an induced model or pattern set in view of a specific goal such as increasing its readability or improving its performance. 1 HypothesisTestBasedFSAlgorithm: This type of univariate feature selection algorithm is very common in biological data analysis but doesn't seem to be implemented in RapidMiner.Can be implemented easily in R. IncrementalReducedErrorPruning: Contrary to ReducedErrorPruning, which is a model pruning algorithm (prunes the decision tree or ruleset after model construction), IncrementalReducedErrorPruning prunes each individual rule immediately after it has been grown. 1 1 1 1 1 InductionAlgorithm: an algorithm designed to achieve an InductionTask, defined as the task of analysing a data set in view of extracting a hypothesis (either a model or a pattern set) that can later be used for predictive or descriptive purposes. Based on whether it outputs a model or a pattern set, an InductionAlgorithm is either a ModelingAlgorithm or a PatternDiscoveryAlgorithm. InductionTask: the task of analysing a data set in view of extracting a hypothesis (either a model or a pattern set) that can later be used for predictive or descriptive purposes. InformedSearch: search that uses (often problem-specific) knowledge beyond that which is built into the state and operator definitions. This additional knowledge guides search in the sense that it allows us to determine whether one nongoal state is more promising than another. [Pearl, 1984; Russell, 2003]. Instance: as used in DMOP, should be taken to denote DM-Instance. In the general sense, an object is an instance of a set or class if it is included in that set or class [http://virtual.cvut.cz/ksmsaWeb/browser/title]. In data mining, a (DM-)Instance is an instance of a dataset and is therefore synonymous with case or example or observation as used in statistics. InstanceWeightVector: a vector of length N, with N the number of cases in the training set, whose elements are the alpha coefficients of the instances. Instances with alpha value > 0 are the support vectors. ItemSequenceTableFormat: the format of a data table (see DataTable) whose instances are tuples of sequences. The elements of the sequences themselves can be of any (primitive or structured) type (see DataType). ItemSetTableFormat: The format of a DataTable whose instances are sets of items belonging to a predefined category. IterativeImprovementSearch: a subclass of search strategies that differs from path-based search in that what is important is not the path to the goal, but the goal state itself. This goal state need not always be a precisely defined configuration, but a "best" state that optimizes some objective function. Local search starts from a single or several arbitrary states and typically moves only to neighbors of that state. The term "iterative improvement search" was introduced by Andrew Moore for what Norvig & Russell [2003] call "local search". NumNearestNeighbors: KNN parameter indicating the number of nearest neighbors to take into account in order to predict the target value of the query or test instance. KNearestNeighborsAlgorithm: Classified under the 'regression approach' (=Bishop's discriminative models) by Hand et al 01, p. 348 -- 'they directly estimate the posterior probabilities of class membership'. See also Hand, 1997: p. 79: suggests very clearly that KDE-based classification: generative <> KNN: discriminative. KernelWeightVector: a vector of the kernel coefficients in a support vector machine. 1 LabeledDataSet: a data set whose instances are pairs (X, Y), where X is a set of predictive/explanatory/independent attribute/feature/variable values and Y is a target/response/independent value (in multitask learning, a set of such values). 1 1 1 LinearDiscriminantModel: the value of the NumberOfProbabilities complexity metric = $C\times D_{cont}+D(D_{cont}+1)/2$ if we take the joint probabilitydistribution. The class-conditional densities $P(\mathbf{x}_{i}|y_{j};\Theta_{j})\sim N(\mathbf{\mu}_{j},\mathbf{\Sigma}), i.e., \Theta_{j}=(\mathbf{\mu}_{j},\mathbf{\Sigma})$ can be simplifed into a series of linear functions $f_{i}(\mathbf{x})=ln(P(\mathbf{x}_{i}|y_{i})P(y_{i}))=\mathbf{w}_{i}^{T}\mathbf{x}+w_{i0}$, where $\mathbf{w}_{i}=\mathbf{\Sigma}^{-1}\mathbf{\mu}_{i}$ and $w_{i0}=-\frac{1}{2}\mathbf{\mu}_{i}^{T}\mathbf{\Sigma}^{-1}\mathbf{\mu}_{i}+ln\, P(y_{i})$ List: an ordered, variable-length array of items (as opposed to a tuple which is fixed-length). Sequences are special cases of lists which contain elements of primitive data types. Function:(mathematics) a mathematical relation such that each element of a given set (the domain of the function) is associated with an element of another set (the range of the function) [http://sigma.ontologyportal.org:4010/sigma/Browse.jsp?kb=SUMO&lang=en] 1.0 Measure: This can be considered the equivalent of the concept of 'measurement datum' in IAO, though I haven't checked thoroughly. 1.0 ModelComplexityMeasure: this concept revolves around the (free) parameters of a model, but the crucial difference is whether one measures the number or the magnitude of the model parameters. For the moment, the concept of ModelComplexityMeasure is used as the range of the object property hasComplexityComponent of CostFunction, but may be needed elsewhere since it's a crucial concept in DM. ModelParamter: a parameter of a learned model, contrary to hyperparameters which are runtime parameters of a learning algorithm. Ultimately hyperparameters impact the learned model's parameters, e.g., by restricting the search space or by limiting the range of possible values a model parameter may take. 1 1 ModelParameterMagnitude: Added the disjoint subclasses of ModelComplexityMeasure: MagnitudeOfModelParameters and NumberOfModelParameters [mh 20091229]. ModelProcessingTask: a task that takes a Model or set of Models as input and outputs some transformation or combination of the input models. ModelStructure subclasses are inherited by the model descriptor generated after execution of a specific operator. The model descriptor should then contain an instantiation of the model structure with its specific model parameters. 1 ModelingAlgorithm: an algorithm designed to build a model, defined as an induced hypothesis that ensures global coverage of the instance space represented by the training data. For instance, a predictive model is assumed to output a prediction for any new example drawn from the same population as the training data. ModelingTask: the task of building a model, defined as an induced hypothesis that ensures global coverage of the instance space represented by the training data. For instance, a predictive model is assumed to output a prediction for any new example drawn from the same population as the training data. 1 NN-Parameter: parameter of an algorithm that builds a neural network. NaiveBayesAlgorithm might subsequently be subsumed by BayesNetAlgorithm. 1 NaiveBayesDiscreteModel: The quantifier restriction for hasModelParameter is "exactly 1 ClassPriorVector" and "exactly 1 ClassCondProbMatrix" (and not some) because these distribution parameters for discrete variables will surely be used after discretization. 1 NiaveBayesKernelModel: The model parameters ClassPriorVector and ClassCondProbMatrix are for the joint PD of the discrete features if any. Hence we use only and not some (because there will be zero such model parameters if there are no discrete variables). 1 1 NaiveBayesMultinomialModel: The ModelParameter of NaiveBayesMultinomialModel is a k-list of vectors $\theta_c$, c=1 to k (number of classes), where $\theta_{ci}$ is the probability of word/feature i in class c, with i = 1 to p (number of words/boolean features). We can therefore represent this model parameter as a $k x p$ matrix whose rows are classes and columns are words/boolean features. We call this a MatrixOfCCProbs. 1 1 1 NaiveBayesNormalModel: The model parameters ClassPriorVector and ClassCondProbMatrix are for the JPD of the discrete features if any. Hence we use only and not some (because there will be zero such model parameters if there are no discrete variables). NaiveBayesNormalModel hasDecisionBoundary {ArbitraryLinearBoundary}: Holds only if the class variable is binary and distribution of X is Gaussian. [Mitchell 2010] 1 ANonlinearPolynomialKernel: a polynomial kernel with degree > 1. NuSVC-Algorithm: A subclass of SVC-Algorithm, where SVC-Algorithm is formulated such that the C hyperparameter is replaced by the Nu hyperparameter NumberOfLeaves: Replaced SizeOfTree with NumberOfLeaves for consistency with the division of ModelComplexityMeasures into MagnitudeOfModelParameters and NumberOfModelParameters [mh 20091229]. NumberOfWeights: model complexity measure that corresponds to the L0 norm of the model weights. 1 1 OperatorParameter: a user-tunable runtime parameter that impacts either the learning process (e.g., by limiting the search space) or the learned model (e.g. , by controlling model complexity). A hyperparameter often comes with a default value. OptimizationProblem: In mathematics and computer science, an optimization problem is the problem of finding the best solution from all feasible solutions. OptimizationProblem: is the minimization or maximization of a function subject to constraints on its variables. Let $x$ be the vector of variables (aka unknowns or parameters), $f$ the objective function, a function of $x$ that we want to min/maximize, and $c$ the vector of constraints that the unknowns must satisfy. The otimization problem can then be written as $min_{x\in R^{n}}f(x)\, subject\, to\begin{cases} c_{i}(x)=0, & amp;i\in\mathcal{E}\\ c_{i}(x)\geq0 & amp;i\in I\end{cases}.$ Here $f$ and each $c_{i}$ are scalar-valued functions of the variables $x,$ and $\mathcal{I},$ $\mathcal{E}$ are sets of indices. [J. Nocedal and S. J. Wright, Numerical Optimization, Springer 1999]. 1 1 The concept of Parameter is used in a very broad sense in the DMO. It englobes the following senses: Parameter (computer science): Parameter (statistics): a quantity (such as the mean or variance) that characterizes a statistical population and that can be estimated by calculations from sample data [http://sigma.ontologyportal.org:4010/sigma/Browse.jsp?kb=SUMO&lang=en]. TO BE COMPLETED 1 1 PathBasedSearch: the problem statement consists of an initial and a goal state, and the solution is a path from the initial to the goal state. PatternDiscoveryAlgorithm: an algorithm designed to detect patterns in data, where a pattern is any regularity, relation or structure inherent in the data, be it exact, approximate or statistical. As opposed to a model which by definition have global coverage, a patter is a local hypothesis, i.e. it applies to a limited region of the sample space. PatternDiscoveryTask: deals with the automatic detection of pattterns in data, where a pattern is any regularity, relation or structure inherent in the data, be it exact, approximate or statistical. In a more restricted sense, a pattern is a learned predictive hypothesis that has local coverage as opposed to a model (again in the restricted sense) which has global coverage. A hypothesis is said to have global coverage if it is capable of returning a prediction for any instance of the sample space, whereas a local hypothesis typically returns a prediction for a limited region of the sample space. PatternEvaluationFunction is a function that evaluates a single pattern. It may be used to evaluate the interestingness of the discovered pattern from the user's viewpoint, and in such a way govern the selection of patterns which are of the interest for the user. Hence, it also may play an important role in reducing the search space during mining for patterns. Formally, this function maps from a pattern to a (usually) numeric value. The evaluation might consider each pattern by taking its statistical properties, and the context of the set of patterns into account. It may also account for the pattern usefulness, novelty and validity in the context of the particular application. PatternSetBasedClassificationModel: a model which is based on a pattern set. Together with a decision strategy it forms a predictive model. 1 PiecewiseLinearBoundary: (Almost) synonymous with PiecewiseMultivariateLinearBoundary or PiecewiseObliqueLinearBoundary, but includes PiecewiseAxisParallelBoundary as well. 1 1 PropositionalDataSet: a dataset that consists of a single data table (see DataTable). PropositionalLogicStructure: The prevalent logical structure in propositional learning algorithms is a disjunction of conjunctions. When the disjunction is exclusive (XOR), the different "rules" are non-overlapping, so order among disjuncts has no importance), the result is a decision tree. When disjunction is inclusive (OR), the different rules constitue a RuleSet which can be either ordered (a decision list) or unordered. RuleSet: a disjunction of conjunction of logical tests, i.e. in which the disjuncts (rules) are not mutually exclusive so that a test example can be matched by several rules. Conflicts due to overlapping rules are avoided by imposing an order on the rules (an ordered ruleset is called a decision list). Alternatively, rules can be weighted and, in case simultaneously applicable rules have conflicting consequents, a decision is taken by aggregating the rule weights. 1 1 1 RecoveryOfPursuit: term used by Pearl (1985, pp. 65-66) to designate a heuristic search algorithm's behavior wrt to choices made in the past. A tentative strategy can backtrack over a decision taken in the past and shift attention back to previously suspended alternatives which had appeared to offer less promise than the current path, whereas an irrevocable strategy can only pursue its search from the current node/state. RegularizedClassificationModelEvalFunction: a predictive model evaluation function that contains both a loss component and a model complexity component linked by some combination function. A regularization parameter governs the trade-off between the two components. 2 RelationalDataSet: a dataset that has at least 2 data tables. RipperModel: hasHypothesisStructure {RuleSet} UnweightedRuleSet: Ripper's ruleset is not a proper decision list (Frank, 2000). Rule: is composed of a head (conclusion) and a body (a conjunction of logical tests). 1 1 SVC-Algorithm (SupportVectorClassifierModelingAlgorithm): a learning algorithm that builds a classifier using the SVM (SupportVectorMachines) approach. In DMOP's task-based hierarchy, we distinguish SVC from SVR (SVMs for regression problems). SVC algorithms are typically designed fro binary classification problems (with labels y = {+1, -1}) but have also be extended for multi-class problems, or one-class problems. An SVC-Algorithm optimizes the margin between classes in a feature space induced by some kernel function. The margin can be hard or soft. A hard margin does not allow any violation while a soft margin tolerate violations, i.e it allows some instances to be inside the margin, between the margin and the separating hyperplane. The level of error tolerance can be controlled by the hyperparameter C. If C is very large, no error is allowed and the soft margin becomes hard. Therefore both the hard-margin and the soft-margin SVC-Algorithm can be formulated in terms of a single optimization problem, which optimizes the trade off between the margin and the hinge loss error of the margin violation. This formulation of SVC-Algorithm is called C-SVC-Algorithm (since hyperparameter C is used). The SVC-Algorithm can be also reformulated in another way, with the C hyperparameter replaced by Nu hyperparameter, yielding the so-called Nu-SVC-Algorithm. Hand et al. 01, p. 335, cite SVMs as examples of what they call the discriminative approach, which is in fact what Bishop 06 calls the discriminant function approach. Hand calls regression approach what Bishop calls discriminative. Both use the term generative for the same thing. SVC-Parameter: parameter of an SVC-Algorithm, including the hyperparameter C as well as parameters of the kernel function such as kernel matrix K and label Y. 1 1 SVC-Model: a classification model produced by SVC-Algorithm: $y=wx_{sv} + b$ or $y = alpha_{sv} * K(x, x_{sv}) + b$. ScopeOfSelection: a property of heuristic search algorithms which designates the extent of the set of alternatives considered by the algorithm at each choice point. Some algorithms have a global scope of selection: they consider all available alternatives, i.e. all the nodes/states that have been generated and remained unexpanded at a given step. Others have a local scope in the sense that they limit their choice to the most recent alternatves -- concretely, to the immediate successors of the current node/state. SearchDirection: Forward, Backward, and Bidirectional Search. For the moment they are just a value partition of the object property hasSearchDirection (Domain: GlobalSearch, both Blind/Uninformed and Heuristic). But based on these property values we can if needed create equivalent classes defined by (GlobalSearch and hasSearchDirection value ...). In this case GlobalSearch will be subcategorized along two dimensions: degree of information in search and direction of search. 1 1 SelfConfiguring NNAlgorithm: can be formally defined as: SelfConfiguringNNAlgorithm == NeuralNetworkAlgorithm and hasSelfTunedHyperParameter some NNParameter. However, there is no need for this equivalent class definition because membership in this class is fully determined by the definition of the object property hasSelfTunedHyperParameter with domain SelfConfiguringNNAlgorithm and range NNParameter. SemiSupervisedInductionAlgorithm: an InductionAlgorithm that builds a hypothesis from input data that comprise both a LabeledDataSet and an UnlabeledDataSet. Set: (mathematics) an abstract collection of numbers or symbols [http://virtual.cvut.cz/ksmsaWeb/browser/title] Set: A finite, unordered collection of objects. A bag or multi-set is a special case of sets which can contain the same object several times. SharedCovarianceMatrix: The CovarianceMatrix is a $DxD$ covariance matrix -- the $\Sigma$ parameter of Normal L/Q Discriminant Models. An NLDModel has only one shared Sigma whereas an NQDModel has K Sigma matrices (one per class). This is because the NormalLinearDiscriminantAnalysisAlgorithm assumes that the covariance of the data is class-independent. SharedCovarianceMatrix: $P(\mathbf{x}|y_{i};\mathbf\Theta_{i})\sim N(\mathbf\mu_{i},\mathbf\Sigma), i.e. \mathbf\Theta_{i}=(\mathbf\mu_{i},\mathbf\Sigma)$ StatisticBasedFWAlgorithm: A statistic-based feature weighting algorithm uses a sample statistic (e.g., mean, correlation coefficient) or a test statistic (e.g. Chi square statistic) to compute feature weights. StochasticSearch: Informally, stochastic search is any search that incorporates randomness. Strategy: In general, the term strategy refers to a plan of action designed to achieve a particular goal. In a computing context, the plan of action specified in a strategy is expressed as an algorithm. In DMOP, we reserve the term Algorithm to specifications that are sufficiently self-contained as to be implemented as distinct operators, and use the term Strategy to refer to a clearly identifiable plan of action that is embedded so tightly into an algorithm that it cannot be implemented as an independent operator. For instance, a learning algorithm typically embeds an optimization strategies and/or a strategy for controlling the complexity of the learned hypothesis; these are integral components that cannot be factored out without changing the way the learning algorithm operates, and are therefore classified as strategies rather than as algorithms. StructuredPredictionAlgorithm: This node is a placeholder for the family of structured prediction algorithms, and awaits contributions from experts on the subject. SumOfSquaredWeights: $\min_{\mathbf{w},b}\langle\mathbf{w}.\mathbf{w}\rangle$ $\|\mathbf{w}\|_{2}^{2}=<\mathbf{w}\cdot\mathbf{w}>=\sum_{i=1}^{p}w_{i}$ SumOfWeights: $\|\mathbf{w}\|_{2}^{2}=<\mathbf{w}\cdot\mathbf{w}>=\sum_{i=1}^{p}w_{i}$ L1Weights: the L1 norm of model weights, i.e. the sum of model weights. SupervisedInductionAlgorithm: An InductionAlgorithm that builds a hypothesis from a LabeledDataSet. TableFormat: the format (see DataFormat) of a data table (see DataTable) as determined by the type of data that can be assigned to its different cells. TargetFeature: in data mining, a feature/attribute/variable whose value depends on other (explanatory, independent or predictive) features. Synonyms: Dependent/response feature/attribute/variable. TimeSeriesTableFormat: the format of a time series table, i.e, a DataTable each instance of which is a sequence of observations/measurements ordered in time. 1 1 1 1 1 1 1 TopKRule is a DecisionRule which makes a specific decision for the top K values of the set of observed or computed values of a given cost function, and the opposite decision for all the others. The DecisionThreshold K can be any value from 1 to N-1, where N is the total number of available values. When K=1, TopKRule is known as the MaxRule. 1 A Tree is a directed graph that has no graph loops [http://virtual.cvut.cz/ksmsaWeb/browser/title]. 1 Tuple: a fixed-length, ordered array of objects which can be of different, possibly structured data types. Vectors are special cases of tuples which contain only primitive (numerical or symbolic) data types. Tuple: In mathematics and computer science a tuple represents the notion of an ordered list of elements. UnlabeledDataSet: a data set whose instances are described exclusively by independent/explanatory variables/features/attributes, i.e., they contain no dependent or target features. UnsupervisedFeatureExtractionAlgorithm: the division into projective and manifold methods is due to: C.J.C. Burges (2004). Geometric methods for feature extraction and dimensional reduction: A guided tour. In L. Rokach and O. Maimon (Eds.), Data Mining and Knowledge Discovery Handbook: A Complete Guide for Practitioners and Researchers. Springer. UnsupervisedInductionAlgorithm: an InductionAlgorithm that builds a hypothesis from an UnllabeledDataSet. UtilityAlgorithm: in DMOP, any algorithm that is used in data mining algorithms without itself being specifically aimed at data analysis, e.g. sort algorithm. UtilityOperator: In DMOP, an operator which performs a certain routine task that has no impact on the quality of the induced hypothesis and is therefore of no interest in meta-learning. Executions of utility operators in DM processes are ignored by the meta-miner. See comment on GenericDMTask for the semantics of UtilityTask. 1 ValueThresholdRule is a binary DecisionRule where a decision is taken in one direction if the observed or computed value is greater than or equal to the specified threshold. In probabilistic binary classification, for instance, one can decide to predict the positive class if the posterior class probability is greater than a given threshold, eg 0.25, 0.5, etc. 1 1 WrapperFeatureSelectionAlgorithm: an algorithm in which the feature selection process is wrapped around a learning algorithm: feature subsets generated by a feature subset generator are used to train the learning algorithm and the performance of the learned models is estimated on a separate test set. The feature subset that yields the highest model performance is selected. Wrapper methods differ based on the following components: (1) the feature generation mechanism used (random feature sampling, genetic algorithms), (2) the learning algorithm used (several dozen candidates, based on the task);(3) the model evaluation strategy (cross-validation, hold-out). 1 SubgroupDiscoveryAlgorithm: takes input data labeled by a class label (some of them require binary class labels). It returns a set of subgroup descriptions, which are most comonly in the form of conjunctive classification rules. Some algorithms are heuristic while other are exhaustive. A few algorithms have been developed that work on more complex data types. 1 A region at which only abstract qualities can be directly located. It assumes some metrics for abstract (neither physical nor temporal) properties. Most of the subclasses under abstract-region represent ways of carving out discrete value regions for the different DM data, algorithm and hypothesis characteristics. However, ceertain subclasses need to be sorted out and refined further. Eventive occurrences (events) are called achievements if they are atomic, otherwise they are accomplishments.Further developments: being 'achievement', 'accomplishment', 'state', 'event', etc. can be also considered 'aspects' of processes or of parts of them. For example, the same process 'rock erosion in the Sinni valley' can be seen as an accomplishment (what has brought the current state that e.g. we are trying to explain), as an achievement (the erosion process as the result of a previous accomplishment), as a state (collapsing the time interval of the erosion into a time point), as an event (what has changed our focus from a state to another).In the erosion case, we could have good motivations to shift from one aspect to another: a) causation focus, b) effectual focus, c) condensation d) transition (causality). Eventive occurrences (events) are called achievements if they are atomic, otherwise they are accomplishments.Further developments: being 'achievement', 'accomplishment', 'state', 'event', etc. can be also considered 'aspects' of processes or of parts of them. For example, the same process 'rock erosion in the Sinni valley' can be seen as an accomplishment (what has brought the current state that e.g. we are trying to explain), as an achievement (the erosion process as the result of a previous accomplishment), as a state (collapsing the time interval of the erosion into a time point), as an event (what has changed our focus from a state to another).In the erosion case, we could have good motivations to shift from one aspect to another: a) causation focus, b) effectual focus, c) condensation d) transition (causality). An occurrence-type is stative or eventive according to whether it holds of the mereological sum of two of its instances, i.e. if it is cumulative or not. A sitting occurrence is stative since the sum of two sittings is still a sitting occurrence.In general, events differ from situations because they are not assumed to have a description from which they depend. They can be sequenced by some course, but they do not require a description as a unifying criterion.On the other hand, at any time, one can conceive a description that asserts the constraints by which an event of a certian type is such, and in this case, it becomes a situation.Since the decision of designing an explicit description that unifies a perdurant depends on context, task, interest, application, etc., when aligning an ontology do DLP, there can be indecision on where to align an event-oriented class. For example, in the WordNet alignment, we have decided to put only some physical events under 'event', e.g. 'discharge', in order to stress the social orientedness of DLP. But whereas we need to talk explicitly of the criteria by which we conceive discharge events, these will be put under 'situation'.Similar considerations are made for the other types of perdurants in DOLCE.A different notion of event (dealing with change) is currently investigated for further developments: being 'achievement', 'accomplishment', 'state', 'event', etc. can be also considered 'aspects' of processes or of parts of them. For example, the same process 'rock erosion in the Sinni valley' can be conceptualized as an accomplishment (what has brought the current state that e.g. we are trying to explain), as an achievement (the erosion process as the result of a previous accomplishment), as a state (if we collapse the time interval of the erosion into a time point), or as an event (what has changed our focus from a state to another).In the erosion case, we could have good motivations to shift from one aspect to another: a) causation focus, b) effectual focus, c) condensation d) transition (causality).If we want to consider all the aspects of a process together, we need to postulate a unifying descriptive set of criteria (i.e. a 'description'), according to which that process is circumstantiated in a 'situation'. The different aspects will arise as a parts of a same situation. Formerly known as description. A unitary endurant with no mass (non-physical), generically constantly depending on some agent, on some communication act, and indirectly on some agent participating in that act. Both descriptions (in the now current sense) and concepts are non-physical objects. AKA 'entity'.Any individual in the DOLCE domain of discourse. The extensional coverage of DOLCE is as large as possible, since it ranges on 'possibilia', i.e all possible individuals that can be postulated by means of DOLCE axioms. Possibilia include physical objects, substances, processes, qualities, conceptual regions, non-physical objects, collections and even arbitrary sums of objects.The class 'particular' features a covering partition that includes: endurant, perdurant, quality, and abstract. There are also some subclasses defined as unions of subclasses of 'particular' for special purposes: spatio-temporal-particular (any particular except abstracts)- physical-realization (any realization of an information object, defined in the ExtendedDnS ontology). A quality space is a topologically maximal region. The constraint of maximality cannot be given completely in OWL, but a constraint is given that creates a partition out of all quality spaces (e.g. no two quality spaces can overlap mereologically). We distinguish between a quality (e.g., the color of a specific rose), and its value (e.g., a particular shade of red). The latter is called quale, and describes the position of an individual quality within a certain conceptual space (called here quality space) Gardenfors (2000). So when we say that two roses have (exactly) the same color, we mean that their color qualities, which are distinct, have the same position in the color space, that is they have the same color quale. Dummy class for optimizing some property universes. It includes all entities that are not reifications of universals ('abstracts'), i.e. those entities that are in space-time. Within stative occurrences, we distinguish between states and processes according to homeomericity: sitting is classified as a state but running is classified as a process, since there are (very short) temporal parts of a running that are not themselves runnings.In general, states differ from situations because they are not assumed to have a description from which they depend. They can be sequenced by some course, but they do not require a description as a unifying criterion.On the other hand, at any time, one can conceive a description that asserts the constraints by which a state of a certian type is such, and in this case, it becomes a situation.Since the decision of designing an explicit description that unifies a perdurant depends on context, task, interest, application, etc., when aligning an ontology do DLP, there can be indecision on where to align a state-oriented class. For example, in the WordNet alignment, we have decided to put only some physical states under 'state', e.g. 'turgor', in order to stress the social orientedness of DLP. But whereas we need to talk explicitly of the criteria by which we conceive turgor states, these will be put under 'situation'.Similar considerations are made for the other types of perdurants in DOLCE.A different notion of event (dealing with change) is currently investigated for further developments: being 'achievement', 'accomplishment', 'state', 'event', etc. can be also considered 'aspects' of processes or of parts of them. For example, the same process 'rock erosion in the Sinni valley' can be conceptualized as an accomplishment (what has brought the current state that e.g. we are trying to explain), as an achievement (the erosion process as the result of a previous accomplishment), as a state (if we collapse the time interval of the erosion into a time point), or as an event (what has changed our focus from a state to another).In the erosion case, we could have good motivations to shift from one aspect to another: a) causation focus, b) effectual focus, c) condensation d) transition (causality).If we want to consider all the aspects of a process together, we need to postulate a unifying descriptive set of criteria (i.e. a 'description'), according to which that process is circumstantiated in a 'situation'. The different aspects will arise as a parts of a same situation. An occurrence-type is stative or eventive according to whether it holds of the mereological sum of two of its instances, i.e. if it is cumulative or not. A sitting occurrence is stative since the sum of two sittings is still a sitting occurrence. A Perdurant that exemplifies the intentionality of an agent. Could it be aborted, incomplete, mislead, while remaining a (potential) accomplishment ... The point here is that having a result depends on a method, then an action remains an action under incomplete results. As a matter of fact, if we neutralize intentionality, a purely topological, post-hoc view is at odds with the notion of incomplete accomplishments. In dependency terms, an activity is an action that is generically constantly dependent on a (at least partly) shared plan adopted by participants. This condition implies that an action must be sequenced by a task.Intuitively, activities are complex actions that are at least partly conventionally planned. A state of the (embodied) mind Collections are social objects which, although not defined by a description, depend both on member entities and on some concepts or figures, hence indirectly on descriptions. While we could talk in general of collections of any kind of entities (events, objects, abstracts, etc.), we restrict here our attention to collections of endurants, and to the concepts that classify them (i.e. roles). Here communication is taken in a rather wide sense, being possible as an (intentional) activity as well as a phenomenon. AKA C-Description. A non-physical object that is defined by a description s, and whose function is classifying entities from a ground ontology in order to build situations that can satisfy s. A concept that classifies (in particular, it 'sequences') perdurants (processes, events, or states), as a component of some description. Courses are the descriptive counterpart of perdurants, and, since perdurants have endurants as participants, they are usually the function of some role. A description is a social object which represents a conceptualization (e.g. a mental object or state), hence it is generically dependent on some agent and communicable. Descriptions define or use concepts or figures, are expressed by an information object and can be satisfied by situations. The typology of descriptions is still preliminary. A role played by descriptions only. Usable for metalinguistic notions, like those that deal with granular partitions of knowledge, strata of reality, argumentation, etc. A parameter valued by regions that are used asindicators for some behaviour or event to be checked. An information encoding system is a description that involves information objects. They can be divided into 1) axiomatic systems, which provide roles and operations to define formal descriptions (e.g. theories), 2) combinatorial systems, which provide roles and operations to create valid information objects (e.g. grammars), 3) classification systems, which are contexts of (ev. ordered) lists of information objects, and 4) informal encoding systems, which provide roles and operations to define informal descriptions (e.g. narratives). Information objects are social objects. They are realized by some entity. They are ordered (expressed according to) by some system for information encoding. Consequently, they are dependent from an encoding as well as from a concrete realization.They can express a description (the ontological equivalent of a meaning/conceptualization), can be about any entity, and can be interpreted by an agent.From a communication perspective, an information object can play the role of "message". From a semiotic perspective, it playes the role of "expression". A role used to express logical levels within some layering description or granular partition. A typical example is the Linnean taxonomic ordering, where Phylum or Species are hierarchical roles. A description that contains a specification to do, realize, behave, etc. Subclasses are plan, technique, practice, project, etc. A social object that is not assumed to internally represent a description. Since social objects are dependent on physical ones, it is not trivial to interpret the local sense in which a social object 'internally represents' a plan. See 'agentive-social-object' for some discussion. A concept that classifies (in particular, it is 'valued by') regions, as defined by some description. Parameters are the descriptive counterpart of regions, and, as regions represent the qualities of perdurants or endurants, they can be requisites for some role or course.A parameter has at least one region that is a value for it. A course used to sequence phenomena (non-intentional processes). Any physical particular that realizes a non-physical endurant. Such physical particulars can be either physical endurants, physical qualities, physical regions, perdurants with at least one physical participant, or a situation with one physical entity in its setting.Ultimately, a physical realization depends on at least one physical endurant (each of the others physical entity types depend on a physical endurant to be considered as such). A plan is a method for executing or performing a procedure or a stage of a procedure.A plan must use both at least one role played by an agent, and at least one task.Finally, a plan has a goal as proper part, and can also have regulations and other descriptions as proper parts. A project is a proactively satisfied method. Differently from a plan, a project includes at least one 'product' role to be played by some endurant (e.g. a house), or one 'result' role played by a perdurant with a definite participant (e.g. a restored state of a house). Also known as 'functional role'.A concept that classifies (in particular, it is 'played by') endurants, as used in some description. Roles are the descriptive counterpart of endurants, and, as endurants participate in perdurants, they usually have courses as modal targets (see).The typology of roles is still preliminary. A catch-all class for entities from the social world. It includes agentive and non-agentive socially-constructed objects: descriptions, concepts, figures, collections, information objects. It could be equivalent to 'non-physical object', but we leave the possibility open of 'private' non-physical objects. A course used to sequence activities or other controllable perdurants (some states, processes), usually within methods. They must be defined by a method, but can be *used* by other kinds of descriptions. They are desire targets of some role played by an agent. Tasks can be complex, and ordered according to an abstract succession relation. Tasks can relate to ground activities or decision making; the last kind deals with typical flowchart content. A task is different both from a flowchart node, and from an action or action type.Tasks can be considered shortcuts for plans, since at least one role played by an agent has a desire attitude towards them (possibly different from the one that puts the task into action). In principle, tasks could be transformed into explicit plans. A technique is a practical method to obtain some modification in the environment (or evaluation of an environment) that fulfils some task. ArbitraryLinearBoundary: a decision boundary that can be axis parallel or oblique. (Almost) synonymous with oblique linear or multivariate linear boundary, but includes axis parallel linear boundary as well. AxisParallelLinearBoundary. Synonym: UnivariateLinearBoundary. BasisFunctionRadius: minimal radius or width of the radial basis function used for the RBF Network's hidden units (e.g. sigma for Gaussian basis functions). false false BestFirstSearch: an instance of HeuristicBestFirstSearch that refers specifically to the (non greedy) best-first algorithm described by Pearl (1984, p. 48). BestFirstSearch maintains a list of all nodes generated so far, and at each step selects the best node/state (in the sense of some criterion, e.g., distance to goal state) for expansion. As illustrated in BestFirstAlgorithm, Best-First search is on the global endpoint of the global-local continuum and on the tentative extreme of the tentative-irrevocable continuum. It is global because it each step it reexamines the set of all unexpanded nodes (and not just the successors of the current node); it is tentative because selecting the best node which is not ncessarily the successor of the current node is tantamount to revoking the choice of the path from the current node. false true BreadthFirstSearch: an uninformed search strategy that expands the shallowest unexpanded node first. The fringe (set of generated but unexpanded nodes is a first-in first-out queue). true BFS is complete under the assumption that the branching factor is finite. CategoricalFeature: a feature (attribute or variable) that takes values from a finite set of discrete, non-numeric, unordered labels. Sometimes called a nominal feature. ClassSpecificCovarianceMatrix: $P(\mathbf{x}|y_{i};\mathbf\Theta_{i})\sim N(\mathbf\mu_{i},\mathbf\Sigma_{i}), i.e. \mathbf\Theta_{i}=(\mathbf\mu_{i},\mathbf\Sigma_{i})$ ComplementNaiveBayesModel's ModelParameter: is a k-list of vectors $\theta_c$, c=1 to k (number of classes), where $\theta_{ci}$ is the probability of word/feature i NOT in class c, with i = 1 to p (number of words/boolean features). We can therefore represent this model parameter as a k x p matrix whose rows are classes and columns are words/boolean features. We call this a ComplementCCProbMatrix. ConditionalFeatureIndepenceAssumption is the assumption of class-conditional feature independence, that is, that the features are independent given the class: $x_1 \bot x_2 \bot \dots \bot x_n \equiv P(x_1, \dots, x_p|c)=\Pi_i P(x_i|c)$ ContinuousFeature:a feature/attribute/variable that takes its values from a range of real numbers. D-TrainFinalModel is a dummy operator that represents the sequence of operators that is needed to train the final model in a modeling task. The goal is to train the final model based on the operators and the set of settings that achieved the highest performance during the model selection process. Training the final model to be deployed is an important step in all data mining processes, but since it simply reexecutes the training process all over again on the full dataset, it has no impact on meta-learning and is therefore classified as a UtilityOperator (see definition under this concept). In DMOP-based meta-analysis of meta-learning, what is important is this operator's output, since it is this final model that should be described in the experiment database rather than any of the models generated using resampling (hold-out, cross-validation, bootstrapping) procedures. DM-Data: In SUMO, Data is defined as 'an item of factual information derived from measurement or research' [http://sigma.ontologyportal.org:4010/sigma/WordNet.jsp?word=data&POS=1] In IAO, Data is an alternative term for 'data item' =def 'an information content entity that is intended to be a truthful statement about something (modulo, e.g., measurement precision or other systematic errors) and is constructed/acquired by a method which reliably tends to produce (approximately) truthful statements.' [http://purl.obolibrary.org/obo/IAO_0000027] In the context of DMOP, DM-Data is the generic term that englobes different levels of granularity: data can be a whole dataset (one main table and possibly other tables), or only a table, or only a feature (column of a table), or only an instance (row of a table), or even a single feature-value pair. Model: a simplified description of a complex entity or process [http://virtual.cvut.cz/ksmsaWeb/browser/title]. DM-PatternSet: A pattern set, as opposed to a model which by definition has global coverage, is a set of local hypotheses, i.e. each applies to a limited region of the sample space. DataFrame: a tuple of arbitrary primitive types. DataSet: in data mining, the term data set is defined as a set of examples or instances represented according to a common schema. DataTable: a set of data arranged in rows and columns. [http://virtual.cvut.cz/ksmsaWeb/browser/title] DecisionList: an ordered ruleset with a default rule in the lowest-priority position. false false DepthFirstSearch: a blind or uninformed search technique that expands the deepest unexpanded node (recursively expands the first child node of the current node). The fringe (set of generated by unexpanded nodes) is implement as a LIFO queue or stack. GenerativeModel is pairwise disjoint with DiscriminativeModel and DFModel. But DiscriminativeModel and DiscriminantFunctionModel may or may not be disjoint because some DF models output probabilities. Those who argue for disjointness say that even if DF models output probabilities, what distinguished them from discriminative models is their ModelStructure. If this is not a posterior probability distribution as in logistic regession, then the model is a DFmodel. A potential counter-argument is that maybe it is the concept of ModelStructure that needs to be revised. The issue remains open. DualFormHardMarginAndL1SVC: the dual form cost function that corresponds to the dual form of the HardMarginAndL1SVC optimization problem. The dual form typically leads to a more efficient solution. However, contrary to the primal SVM cost function , the dual form cannot be decomposed into a LossComponent and a ComplexityComponent. DualFormHardMarginAndL1SVC: $\max_\mathbf{\alpha}W(\mathbf{\alpha})=\sum_{i=1}^{l}\alpha_{i}-\frac{1}{2}\sum_{i,j=1}^{n}y_{i}y_{j}\alpha_{i}\alpha_{j}K(\mathbf{x}_{i},\mathbf{x}_{j})$ DualFormHardMarginConstraint: $\sum_{i=1}^{n}y_{i}\alpha_{i}=0,\alpha_{i}\geq0,i=1,...,n$ $\sum_{i=1}^{n}y_{i}\alpha_{i}=0,C\geq\alpha_{i}\geq0,i=1,...,n$ DualFormL2SVCConstraint: $\sum_{i=1}^{n}y_{i}\alpha_{i}=0,\alpha_{i}\geq0,i=1,...,n$ DualFormL2SVCCostFunction: $\max_{\mathbf{\alpha}}W(\mathbf{\alpha})=\sum_{i=1}^{l}\alpha_{i}-\frac{1}{2}\sum_{i,j=1}^{n}y_{i}y_{j}\alpha_{i}\alpha_{j}(K(\mathbf{x}_{i},\mathbf{x}_{j})+\frac{1}{C}\delta_{ij})$ ErrorEpsilon: The lowest value of error that justifies pursuit of training. Below this value, the optimization process is stopped. FOLDecisionTree: First-order logic decision tree (e.g. TILDE). FOLRuleSet: A rule set expressed in first-order logic (e.g. FOIL). Feature: a property of an instance (case, example, record). Synonyms: Attribute, Variable. false HardMarginConstraint: $y_{i}(\langle\mathbf{w}.\Phi(\mathbf{x}_{i})\rangle+b)\geq1,i=1,...,l$ InformationGain: $IG(S,A)=H(S)-\sum_{v\in Values(A)}\frac{|S_{v}|}{|S|}H(S_{v})$ InformationGain: Aka Mutual Information InformationGainRatio: $SplitInformation(S,A)=\sum_{i=1}^{c}\frac{|S_{i}|}{|S|}log_{2}\frac{|S_{i}|}{|S|}$ $GainRatio(S,A)=\frac{IG(S,A)}{SplitInformation(S,A)}$ For the formula of $IG(S,A)$, see Information Gain. InformationGainRatio: InformationGain normalized by the entropy of the predictive feature. Instance: as used in DMOP, should be taken to denote DM-Instance. In the general sense, an object is an instance of a set or class if it is included in that set or class [http://virtual.cvut.cz/ksmsaWeb/browser/title]. In data mining, a (DM-)Instance is an instance of a dataset and is therefore synonymous with case or example or observation as used in statistics. Irrevocable: An irrevocable strategy is one which never reconsiders past decisions to explore other alternatives available in a given search state/node. In short, an irrevocable strategy never backtracks over past choices. L1HingeLoss: $\sum_{i=1}^{n}\xi_{i}$ L1HingeLoss is the L1 norm of Hinge Loss, i.e., the sum of hinge loss measured over all training cases. L1HingeLossPlusL2WeightsCostFunction is the regularized sum of the L1 norm of the hinge loss and the L2 norm of the model weights. L1HingeLossPlusL2WeightsCostFunction: $\min_{\mathbf{\xi},\mathbf{w},b}\langle\mathbf{w}.\mathbf{w}\rangle+C\sum_{i=1}^{n}\xi_{i}$ L2HingeLoss: the (squared) L2 norm of the hinge loss, i.e. the sum of squared hinge loss measured over all training cases. L2HingeLoss: $\sum_{i=1}^{n}\xi_{i}$ L2HingeLossPlusL2WeightsCostFunction is the regularized sum of the L2 norm of the hinge loss and the L2 norm of the model weights. L2HingeLossPlusL2WeightsCostFunction; $\min_{\xi,\mathbf{w},b}\langle\mathbf{w}.\mathbf{w}\rangle+C\sum_{i=1}^{n}\xi_{i}^{2}$ L2Weights: $\|\mathbf{w}\|_{2}^{2}=<\mathbf{w}\cdot\mathbf{w}>=\sum_{i=1}^{p}w_{i}$ L2Weights designates the L2-norm of the model weights, i.e., the sum of squared weights. LabeledDataSet: a data set whose instances are pairs (X, Y), where X is a set of predictive/explanatory/independent attribute/feature/variable values and Y is a target/response/independent value (in multitask learning, a set of such values). LearningRate: a value between 0 and 1 that determines the magnitude of weight adjustments at each step. A higher value leads to faster learning but also to the risk of instability. LearningRateDecay: a boolean that indicates if the learning rate should be decreased during the learning process. LinearCombinationOfFunctions: a model structure representing a linear combination of functions other than linear (identity) or kernel functions. LinearDiscriminantModel: the value of the NumberOfProbabilities complexity metric = $C\times D_{cont}+D(D_{cont}+1)/2$ if we take the joint probabilitydistribution. The class-conditional densities $P(\mathbf{x}_{i}|y_{j};\Theta_{j})\sim N(\mathbf{\mu}_{j},\mathbf{\Sigma}), i.e., \Theta_{j}=(\mathbf{\mu}_{j},\mathbf{\Sigma})$ can be simplifed into a series of linear functions $f_{i}(\mathbf{x})=ln(P(\mathbf{x}_{i}|y_{i})P(y_{i}))=\mathbf{w}_{i}^{T}\mathbf{x}+w_{i0}$, where $\mathbf{w}_{i}=\mathbf{\Sigma}^{-1}\mathbf{\mu}_{i}$ and $w_{i0}=-\frac{1}{2}\mathbf{\mu}_{i}^{T}\mathbf{\Sigma}^{-1}\mathbf{\mu}_{i}+ln\, P(y_{i})$ true LogLikelihoodOfClassPosteriors: $l(\boldsymbol{\Theta})=\sum_{i=1}^{N}[\sum_{j=1}^{k-1}(y_{j}(x_{i})|\mathbf{x}_{i};\mathbf{\Theta})\ln P(y_{j}|\mathbf{x}_{i};\mathbf{\Theta})+(1-\sum_{j=1}^{k-1}y_{j}(\mathbf{x}_{i}))\ln(1-\sum_{j=1}^{k-1}P(y_{j}|\mathbf{x}_{i};\mathbf{\Theta}))]$ LogLikelihoodOfMultivarNormalClassCondProbsClassSpecificCovariance: $\ell(\Theta)=\sum_{i=1}^{n}\sum_{j=1}^{k}y_{j}(\mathbf{x}_{i})ln[P(y_{j})P(\mathbf{x}_{i}|y_{j};\Theta_{j})]$ where the class-conditional densities are multivariate normal: $P(\mathbf{x}_{i}|y_{j};\Theta_{j})\sim N(\mathbf{\mu}_{j},\mathbf{\Sigma}),i.e.,\Theta_{j}=(\mathbf{\mu}_{j},\mathbf{\Sigma})$. The maximum likelihood model parameters are $\mathbf{\Theta}_{ML}=argmax_{\mathbf{\Theta}}\,\ell(\mathbf{\Theta})$ LogLikelihoodOfMultivarNormalClassCondProbsSharedCovariance: $\ell(\Theta)=\sum_{i=1}^{n}\sum_{j=1}^{k}y_{j}(\mathbf{x}_{i})ln[P(y_{j})P(\mathbf{x}_{i}|y_{j};\Theta_{j})]$ where the class-conditional densities are multivariate normal: $P(\mathbf{x}_{i}|y_{j};\Theta_{j})\sim N(\mathbf{\mu}_{j},\mathbf{\Sigma}),i.e.,\Theta_{j}=(\mathbf{\mu}_{j},\mathbf{\Sigma})$. The maximum likelihood model parameters are $\mathbf{\Theta}_{ML}=argmax_{\mathbf{\Theta}}\,\ell(\mathbf{\Theta})$ LogLikelihoodOfNaiveClassCondProbs = (negative of) the loss function based on class conditional probabilities where the assumption of class-conditional feature independence has been integrated, i.e., $P(\mathbf{x}|y_{i})=\prod_{i=1}^{d}P(x_{i}|y_{i};\mathbf\theta)$. $l(\mathbf\theta)&=&\sum_{i=1}^{N}[\sum_{j=1}^{k-1}(y_{j}(x_{i})|\mathbf{x}_{i};\mathbf\theta)\ln\frac{P(\mathbf{x}|y_{i};\mathbf\theta)P(y_{i})}{P(\mathbf{x;\mathbf\theta})}\\&&+(1-\sum_{j=1}^{k-1}y_{j}(\mathbf{x}_{i}))\ln(1-\sum_{j=1}^{k-1}\frac{P(\mathbf{x}|y_{i};\mathbf\theta)P(y_{i})}{P(\mathbf{x;\mathbf\theta})}]$ Matrix: a tuple of vectors. MaxAPosteriori is a specific instance of MaxRule where the maximum value is selected from a set of posterior class probabilities. Measure: This can be considered the equivalent of the concept of 'measurement datum' in IAO, though I haven't checked thoroughly. MinConditionalEntropyOptimizationProblem: An alternative (more generic) name is ConditionalClassEntropyMinimizationProblem 1 ModelBasedEarlyStopping uses a halt criterion based on some feature of the model being built (e.g. minimal leaf size or minimal size for split in decision trees, where early stopping is called pre-pruning). ModelComplexityMeasure: this concept revolves around the (free) parameters of a model, but the crucial difference is whether one measures the number or the magnitude of the model parameters. For the moment, the concept of ModelComplexityMeasure is used as the range of the object property hasComplexityComponent of CostFunction, but may be needed elsewhere since it's a crucial concept in DM. ModelParameterMagnitude: Added the disjoint subclasses of ModelComplexityMeasure: MagnitudeOfModelParameters and NumberOfModelParameters [mh 20091229]. Momentum: a value between 0.0 and 1.0 that is indicates the fraction of the previous weight update to add to the current one. This adds inertia to the motion through weight space and smooths out oscillations. MultinomialClassPriorAssumption: $\hat{\pi}_{c}^{MLE}=\frac{N_{c}}{N}$ MultinomialClassPriorAssumption: the assumption that the class prior probability follows a multinomial distribution. Under this assumption, the maximum likelihood estimate of class c is its relative frequency in the training set (see latexFormula). MultivariateDecisionTree: a disjunction of conjunction of multivariate tests resulting in oblique or non axis-parallel splits at each node MultivariateSeries: a list of vectors. NaiveBayesDiscreteModel: The quantifier restriction for hasModelParameter is "exactly 1 ClassPriorVector" and "exactly 1 ClassCondProbMatrix" (and not some) because these distribution parameters for discrete variables will surely be used after discretization. NiaveBayesKernelModel: The model parameters ClassPriorVector and ClassCondProbMatrix are for the joint PD of the discrete features if any. Hence we use only and not some (because there will be zero such model parameters if there are no discrete variables). NaiveBayesMultinomialModel: The ModelParameter of NaiveBayesMultinomialModel is a k-list of vectors $\theta_c$, c=1 to k (number of classes), where $\theta_{ci}$ is the probability of word/feature i in class c, with i = 1 to p (number of words/boolean features). We can therefore represent this model parameter as a $k x p$ matrix whose rows are classes and columns are words/boolean features. We call this a MatrixOfCCProbs. NaiveBayesNormalModel: The model parameters ClassPriorVector and ClassCondProbMatrix are for the JPD of the discrete features if any. Hence we use only and not some (because there will be zero such model parameters if there are no discrete variables). Negentropy: (from negative entropy) is a measure of distance to gaussianity (or normality). It is based on the definition of the differential entropy H of a random vector $\mathbf{y}=(y_{1},\dots,y_{n})^{T}$ with density $f(.)$: $H(\mathbf{y})=-\int\, f(\mathbf{y})\log\, f(\mathbf{y})\mathrm{d\mathbf{y}}$. Negentropy is defined by normalizing differential entropy, i.e., subtracting $H(\mathbf{y}$) from a Gaussian random vector of the same covariance matrix as $\mathbf{y}$. Negentropy is always nonnegative, and it is zero iff $\mathbf{y}$ has a gaussian distribution. Negentropy is used in Independent Component Analysis to measure the nongaussianity (and therefor the mutual independence) of the tranformed variables (components). Ref. Hyvarinen et al. (2001). Independent Component Analysis, Wiley. NormalClassCondProbabilityAssumption: the assumption that the probability distribution of the data given the class follows a multivariate Gaussian distribution. NormalClassConditionProbabilityAssumption: $p(x|y)\sim\mathcal{N}(\mathbf{\mu},\Sigma)$ NumHiddenLayers: The number of hidden layers in a multilayer perceptron. NumHiddenUnits: a vector of integers giving the number of hidden units in each of the hidden layers of a multilayer perceptron. The length of the vector is the value of the parameter NumHiddenLayers. NumOutputUnits: The number of output units in a neural network. This depends on the task (classification or regression) and, in the case of classification, on the number of classes. NumTrainingCycles: the maximal number of training iterations (aka epochs) allowed. Training stops when this limit is attained, regardless of the magnitude of the observed error. NumberOfLeaves: Replaced SizeOfTree with NumberOfLeaves for consistency with the division of ModelComplexityMeasures into MagnitudeOfModelParameters and NumberOfModelParameters [mh 20091229]. NumberOfWeights: model complexity measure that corresponds to the L0 norm of the model weights. PatternSetBasedClassificationModel: a model which is based on a pattern set. Together with a decision strategy it forms a predictive model. ModelBasedEarlyStopping: uses a halt criterion based on some observation (eg elbow in LossFunction curve) that reveals performance stagnation or degradation. PiecewiseAxisParallelLinearBoundary or PiecewiseUnivariateLinearBoundary: better known as a hyperrectangle. PiecewiseObliqueLinearBoundary: aka PiecewiseMultivariateLinearBoundary. 2 3 true ProcessBasedEarlyStopping: uses a halt criterion relative to the learning process, such as a fixed limit on the number of iterations (eg number of epochs in neural networks) PropositionalDataSet: a dataset that consists of a single data table (see DataTable). RelationalDataSet: a dataset that has at least 2 data tables. SVC-Model: a classification model produced by SVC-Algorithm: $y=wx_{sv} + b$ or $y = alpha_{sv} * K(x, x_{sv}) + b$. Sequence: a list of nominals. Series: a list of numbers. 0 -1 SoftMarginConstraint: $y_{i}(\langle\mathbf{w},\Phi(\mathbf{x}_{i})\rangle+b)\geq1-\xi_{i},\xi_{i}\geq0,i=1,...,n$ SumOfSquaredWeights: $\min_{\mathbf{w},b}\langle\mathbf{w}.\mathbf{w}\rangle$ $\|\mathbf{w}\|_{2}^{2}=<\mathbf{w}\cdot\mathbf{w}>=\sum_{i=1}^{p}w_{i}$ TargetFeature: in data mining, a feature/attribute/variable whose value depends on other (explanatory, independent or predictive) features. Synonyms: Dependent/response feature/attribute/variable. TrainingDataModel or NullModel is an ersatz model assigned to lazy learners and classified under PosteriorProbabilityDistribution in order to group them with Discriminative Models. true true UniformCostSearch (UCS): an uninformed search strategy which consists in expanding a candidate (generated but unexpanded) node with the lowest cost of the path from the initial to the current state). If step costs are all equal, UCS produces the same solution as BreadthFirstSearch. A UnivariateDecisionTreeOrDecisionList is an ordered disjunction of conjunction of univariate tests. This holds only for orthogonal decision trees. UnivariateTest: a boolean function consisting of <Feature> <Rel> <Value>, where <Rel> is any of <, <=, = >=, >. UnlabeledDataSet: a data set whose instances are described exclusively by independent/explanatory variables/features/attributes, i.e., they contain no dependent or target features. Unweighted RuleSet: a RuleSet whose member rules are all assumed to carry unit weight. Vector: a tuple of numbers. WeightNeighborVotes: a KNN boolean parameter indicating whether to weight the votes of the K nearest neighbors based on their similarity to the query or test instance. WeightedRuleSet: an unordered set of rules, each of which carried an associated weight or score. AKA Agentive-role.A role that can only be played by agents. The course of events typical of the life of an object (kind). A collection with only agents as members.