Supervised learning

• AODE
  AODE
  Averaged one-dependence estimators is a probabilistic classification learning technique. It was developed to address the attribute-independence problem of the popular naive Bayes classifier...
  
  
• Artificial neural network
  Artificial neural network
  An artificial neural network , usually called neural network , is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks. A neural network consists of an interconnected group of artificial neurons, and it processes...
  
  
  • Backpropagation
    Backpropagation
    Backpropagation is a common method of teaching artificial neural networks how to perform a given task. Arthur E. Bryson and Yu-Chi Ho described it as a multi-stage dynamic system optimization method in 1969 . It wasn't until 1974 and later, when applied in the context of neural networks and...
    
    
• Bayesian statistics
  Bayesian statistics
  Bayesian statistics is that subset of the entire field of statistics in which the evidence about the true state of the world is expressed in terms of degrees of belief or, more specifically, Bayesian probabilities...
  
  
  • Naive Bayes classifier
    Naive Bayes classifier
    A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions...
    
    
  • Bayesian network
    Bayesian network
    A Bayesian network, Bayes network, belief network or directed acyclic graphical model is a probabilistic graphical model that represents a set of random variables and their conditional dependencies via a directed acyclic graph . For example, a Bayesian network could represent the probabilistic...
    
    
  • Bayesian knowledge base
• Case-based reasoning
  Case-based reasoning
  Case-based reasoning , broadly construed, is the process of solving new problems based on the solutions of similar past problems. An auto mechanic who fixes an engine by recalling another car that exhibited similar symptoms is using case-based reasoning...
  
  
• Decision trees
  Decision tree learning
  Decision tree learning, used in statistics, data mining and machine learning, uses a decision tree as a predictive model which maps observations about an item to conclusions about the item's target value. More descriptive names for such tree models are classification trees or regression trees...
  
  
• Inductive logic programming
  Inductive logic programming
  Inductive logic programming is a subfield of machine learning which uses logic programming as a uniform representation for examples, background knowledge and hypotheses...
  
  
• Gaussian process regression
• Group method of data handling
  Group method of data handling
  Group method of data handling is a family of inductive algorithms for computer-based mathematical modeling of multi-parametric datasets that features fully automatic structural and parametric optimization of models....
  
   (GMDH)
• Learning Automata
  Learning Automata
  A branch of the theory of adaptive control is devoted to learning automata surveyed by Narendra and Thathachar which were originally described explicitly as finite state automata...
  
  
• Learning Vector Quantization
• Minimum message length
  Minimum message length
  Minimum message length is a formal information theory restatement of Occam's Razor: even when models are not equal in goodness of fit accuracy to the observed data, the one generating the shortest overall message is more likely to be correct...
  
   (decision trees, decision graphs, etc.)
• Lazy learning
  Lazy learning
  In artificial intelligence, lazy learning is a learning method in which generalization beyond the training data is delayed until a query is made to the system, as opposed to in eager learning, where the system tries to generalize the training data before receiving queries.The main advantage gained...
  
  
• Instance-based learning
  Instance-based learning
  In machine learning, instance-based learning or memory-based learning is a family of learning algorithms that, instead of performing explicit generalization, compare new problem instances with instances seen in training, which have been stored in memory...
  
  
  • Nearest Neighbor Algorithm
  • Analogical modeling
    Analogical modeling
    Analogical modeling is a formal theory of exemplar-based analogical reasoning, proposed by Royal Skousen, professor of Linguistics and English language at Brigham Young University in Provo, Utah. It is applicable to language modeling and other categorization tasks...
    
    
• Probably approximately correct learning
  Probably approximately correct learning
  In computational learning theory, probably approximately correct learning is a framework for mathematical analysis of machine learning. It was proposed in 1984 by Leslie Valiant....
  
   (PAC) learning
• Ripple down rules
  Ripple down rules
  Ripple Down Rules is a way of approaching knowledge acquisition. Knowledge acquisition refers to the transfer knowledge from human experts to knowledge based systems.- Introductory material :...
  
  , a knowledge acquisition methodology
• Symbolic machine learning algorithms
• Subsymbolic machine learning algorithms
• Support vector machine
  Support vector machine
  A support vector machine is a concept in statistics and computer science for a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis...
  
  s
• Random Forests
  Random forest
  Random forest is an ensemble classifier that consists of many decision trees and outputs the class that is the mode of the class's output by individual trees. The algorithm for inducing a random forest was developed by Leo Breiman and Adele Cutler, and "Random Forests" is their trademark...
  
  
• Ensembles of classifiers
  • Bootstrap aggregating
    Bootstrap aggregating
    Bootstrap aggregating is a machine learning ensemble meta-algorithm to improve machine learning of statistical classification and regression models in terms of stability and classification accuracy. It also reduces variance and helps to avoid overfitting. Although it is usually applied to decision...
    
     (bagging)
  • Boosting
    Boosting
    Boosting is a machine learning meta-algorithm for performing supervised learning. Boosting is based on the question posed by Kearns: can a set of weak learners create a single strong learner? A weak learner is defined to be a classifier which is only slightly correlated with the true classification...
    
    
• Ordinal classification
• Regression analysis
  Regression analysis
  In statistics, regression analysis includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables...
  
  
• Information fuzzy networks
  Information Fuzzy Networks
  Info Fuzzy Networks is a greedy machine learning algorithm for supervised learning.The data structure produced by the learning algorithm is also called Info Fuzzy Network.IFN construction is quite similar to decision trees' construction....
  
   (IFN)

Statistical classification

• AODE
  AODE
  Averaged one-dependence estimators is a probabilistic classification learning technique. It was developed to address the attribute-independence problem of the popular naive Bayes classifier...
  
  
• Linear classifier
  Linear classifier
  In the field of machine learning, the goal of statistical classification is to use an object's characteristics to identify which class it belongs to. A linear classifier achieves this by making a classification decision based on the value of a linear combination of the characteristics...
  
  s
  • Fisher's linear discriminant
  • Logistic regression
    Logistic regression
    In statistics, logistic regression is used for prediction of the probability of occurrence of an event by fitting data to a logit function logistic curve. It is a generalized linear model used for binomial regression...
    
    
  • Naive Bayes classifier
    Naive Bayes classifier
    A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions...
    
    
  • Perceptron
    Perceptron
    The perceptron is a type of artificial neural network invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt. It can be seen as the simplest kind of feedforward neural network: a linear classifier.- Definition :...
    
    
  • Support vector machine
    Support vector machine
    A support vector machine is a concept in statistics and computer science for a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis...
    
    s
• Quadratic classifier
  Quadratic classifier
  A quadratic classifier is used in machine learning and statistical classification to separate measurements of two or more classes of objects or events by a quadric surface...
  
  s
• k-nearest neighbor
• Boosting
  Boosting
  Boosting is a machine learning meta-algorithm for performing supervised learning. Boosting is based on the question posed by Kearns: can a set of weak learners create a single strong learner? A weak learner is defined to be a classifier which is only slightly correlated with the true classification...
  
  
• Decision trees
  Decision tree learning
  Decision tree learning, used in statistics, data mining and machine learning, uses a decision tree as a predictive model which maps observations about an item to conclusions about the item's target value. More descriptive names for such tree models are classification trees or regression trees...
  
  
  • C4.5
  • Random forest
    Random forest
    Random forest is an ensemble classifier that consists of many decision trees and outputs the class that is the mode of the class's output by individual trees. The algorithm for inducing a random forest was developed by Leo Breiman and Adele Cutler, and "Random Forests" is their trademark...
    
    s
• Bayesian network
  Bayesian network
  A Bayesian network, Bayes network, belief network or directed acyclic graphical model is a probabilistic graphical model that represents a set of random variables and their conditional dependencies via a directed acyclic graph . For example, a Bayesian network could represent the probabilistic...
  
  s
• Hidden Markov model
  Hidden Markov model
  A hidden Markov model is a statistical Markov model in which the system being modeled is assumed to be a Markov process with unobserved states. An HMM can be considered as the simplest dynamic Bayesian network. The mathematics behind the HMM was developed by L. E...
  
  s

Unsupervised learning

• Artificial neural network
  Artificial neural network
  An artificial neural network , usually called neural network , is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks. A neural network consists of an interconnected group of artificial neurons, and it processes...
  
  
• Data clustering
  Data clustering
  Cluster analysis or clustering is the task of assigning a set of objects into groups so that the objects in the same cluster are more similar to each other than to those in other clusters....
  
  
• Expectation-maximization algorithm
  Expectation-maximization algorithm
  In statistics, an expectation–maximization algorithm is an iterative method for finding maximum likelihood or maximum a posteriori estimates of parameters in statistical models, where the model depends on unobserved latent variables...
  
  
• Self-organizing map
  Self-organizing map
  A self-organizing map or self-organizing feature map is a type of artificial neural network that is trained using unsupervised learning to produce a low-dimensional , discretized representation of the input space of the training samples, called a map...
  
  
• Radial basis function network
  Radial basis function network
  A radial basis function network is an artificial neural network that uses radial basis functions as activation functions. It is a linear combination of radial basis functions...
  
  
• Vector Quantization
  Vector quantization
  Vector quantization is a classical quantization technique from signal processing which allows the modeling of probability density functions by the distribution of prototype vectors. It was originally used for data compression. It works by dividing a large set of points into groups having...
  
  
• Generative topographic map
  Generative Topographic Map
  Generative topographic map is a machine learning method that is a probabilistic counterpart of the self-organizing map , is provably convergent and does not require a shrinking neighborhood or a decreasing step size...
  
  
• Information bottleneck method
  Information bottleneck method
  The information bottleneck method is a technique introduced by Naftali Tishby et al. [1] for finding the best tradeoff between accuracy and complexity when summarizing a random variable X, given a joint probability distribution between X and an observed relevant variable Y...
  
  
• IBSEAD

Association rule learning

• Apriori algorithm
  Apriori algorithm
  In computer science and data mining, Apriori is a classic algorithm for learning association rules. Apriori is designed to operate on databases containing transactions...
  
  
• Eclat algorithm
• FP-growth algorithm

Reinforcement learning

• Temporal difference learning
  Temporal difference learning
  Temporal difference learning is a prediction method. It has been mostly used for solving the reinforcement learning problem. "TD learning is a combination of Monte Carlo ideas and dynamic programming ideas." TD resembles a Monte Carlo method because it learns by sampling the environment according...
  
  
• Q-learning
  Q-learning
  Q-learning is a reinforcement learning technique that works by learning an action-value function that gives the expected utility of taking a given action in a given state and following a fixed policy thereafter. One of the strengths of Q-learning is that it is able to compare the expected utility...
  
  
• Learning Automata
  Learning Automata
  A branch of the theory of adaptive control is devoted to learning automata surveyed by Narendra and Thathachar which were originally described explicitly as finite state automata...