Book:Machine Learning - The Complete

Guide
1 Machine Learning
Introduction and Main Principles Machine learning
Data analysis
Occams razor
Curse of dimensionality
No free lunch theorem
Accuracy paradox
Overtting
Regularization (machine learning)
Inductive bias
Data dredging
Ugly duckling theorem
Uncertain data
Background and Preliminaries
Knowledge discovery in Databases Knowledge
discovery
Data mining
Predictive analytics
Predictive modelling
Business intelligence
Reactive business intelligence
Business analytics
Reactive business intelligence
Pattern recognition
Reasoning Abductive reasoning
Inductive reasoning
First-order logic
Inductive logic programming
Reasoning system
Case-based reasoning
Textual case based reasoning
Causality
Search Methods Nearest neighbor search
Stochastic gradient descent
Beam search
Best-rst search
Breadth-rst search
Hill climbing
Grid search
Brute-force search
Depth-rst search
Tabu search
Anytime algorithm
Statistics Exploratory data analysis
Covariate
Statistical inference
Algorithmic inference
Bayesian inference
Base rate
Bias (statistics)
Gibbs sampling
Cross-entropy method
Latent variable
Maximum likelihood
Maximum a posteriori estimation
Expectationmaximization algorithm
Expectation propagation
KullbackLeibler divergence
Generative model
Main Learning Paradigms Supervised learning
1
2 1 MACHINE LEARNING
Unsupervised learning
Active learning (machine learning)
Reinforcement learning
Multi-task learning
Transduction
Explanation-based learning
Oine learning
Online learning model
Online machine learning
Hyperparameter optimization
Classication Tasks Classication in machine learning
Concept class
Features (pattern recognition)
Feature vector
Feature space
Concept learning
Binary classication
Decision boundary
Multiclass classication
Class membership probabilities
Calibration (statistics)
Concept drift
Prior knowledge for pattern recognition
Iris ower data set (Classic data sets)
Online Learning Margin Infused Relaxed Algorithm
Semi-supervised learning Semi-supervised learning
One-class classication
Coupled pattern learner
Lazy learning and nearest neighbors Lazy learning
Eager learning
Instance-based learning
Cluster assumption
K-nearest neighbor algorithm
IDistance
Large margin nearest neighbor
Decision Trees Decision tree learning
Decision stump
Pruning (decision trees)
Mutual information
Adjusted mutual information
Information gain ratio
Information gain in decision trees
ID3 algorithm
C4.5 algorithm
CHAID
Information Fuzzy Networks
Grafting (decision trees)
Incremental decision tree
Alternating decision tree
Logistic model tree
Random forest
Linear Classiers Linear classier
Margin (machine learning)
Margin classier
Soft independent modelling of class analogies
Statistical classication Statistical classication
Probability matching
Discriminative model
Linear discriminant analysis
Multiclass LDA
Multiple discriminant analysis
Optimal discriminant analysis
Fisher kernel
Discriminant function analysis
Multilinear subspace learning
Quadratic classier
Variable kernel density estimation
Category utility
Evaluation of Classication Models Data classica-
tion (business intelligence)
Training set
3
Test set
Synthetic data
Cross-validation (statistics)
Loss function
Hinge loss
Generalization error
Type I and type II errors
Sensitivity and specicity
Precision and recall
F1 score
Confusion matrix
Matthews correlation coecient
Receiver operating characteristic
Lift (data mining)
Stability in learning
Features Selection and Features Extraction Data
Pre-processing
Discretization of continuous features
Feature selection
Feature extraction
Dimension reduction
Principal component analysis
Multilinear principal-component analysis
Multifactor dimensionality reduction
Targeted projection pursuit
Multidimensional scaling
Nonlinear dimensionality reduction
Kernel principal component analysis
Kernel eigenvoice
Gramian matrix
Gaussian process
Kernel adaptive lter
Isomap
Manifold alignment
Diusion map
Elastic map
Locality-sensitive hashing
Spectral clustering
Minimum redundancy feature selection
Clustering Cluster analysis
K-means clustering
K-means++
K-medians clustering
K-medoids
DBSCAN
Fuzzy clustering
BIRCH (data clustering)
Canopy clustering algorithm
Cluster-weighted modeling
Clustering high-dimensional data
Cobweb (clustering)
Complete-linkage clustering
Constrained clustering
Correlation clustering
CURE data clustering algorithm
Data stream clustering
Dendrogram
Determining the number of clusters in a data set
FLAME clustering
Hierarchical clustering
Information bottleneck method
Lloyds algorithm
Nearest-neighbor chain algorithm
Neighbor joining
OPTICS algorithm
PitmanYor process
Single-linkage clustering
SUBCLU
Thresholding (image processing)
UPGMA
Evaluation of Clustering Methods Rand index
Dunn index
4 1 MACHINE LEARNING
DaviesBouldin index
Jaccard index
MinHash
K q-ats
Rule Induction Decision rules
Rule induction
Classication rule
CN2 algorithm
Decision list
First Order Inductive Learner
Association rules and Frequent Item Sets
Association rule learning
Apriori algorithm
Contrast set learning
Anity analysis
K-optimal pattern discovery
Ensemble Learning Ensemble learning
Ensemble averaging
Consensus clustering
AdaBoost
Boosting
Bootstrap aggregating
BrownBoost
Cascading classiers
Co-training
CoBoosting
Gaussian process emulator
Gradient boosting
LogitBoost
LPBoost
Mixture model
Product of Experts
Random multinomial logit
Random subspace method
Weighted Majority Algorithm
Randomized weighted majority algorithm
Graphical Models Graphical model
State transition network
Bayesian Learning Methods Naive Bayes classier
Averaged one-dependence estimators
Bayesian network
Bayesian additive regression kernels
Variational message passing
Markov Models Markov model
Maximum-entropy Markov model
Hidden Markov model
BaumWelch algorithm
Forwardbackward algorithm
Hierarchical hidden Markov model
Markov logic network
Markov chain Monte Carlo
Markov random eld
Conditional random eld
Predictive state representation
Learning Theory Computational learning theory
Version space
Probably approximately correct learning
VapnikChervonenkis theory
Shattering (machine learning)
VC dimension
Minimum description length
Bondys theorem
Inferential theory of learning
Rademacher complexity
Teaching dimension
Subclass reachability
Sample exclusion dimension
Unique negative dimension
Uniform convergence (combinatorics)
Witness set
Support Vector Machines Kernel methods
Support vector machine
5
Structural risk minimization
Empirical risk minimization
Kernel trick
Least squares support vector machine
Relevance vector machine
Sequential minimal optimization
Structured SVM
Regression analysis Outline of regression analysis
Regression analysis
Dependent and independent variables
Linear model
Linear regression
Least squares
Linear least squares (mathematics)
Local regression
Additive model
Antecedent variable
Autocorrelation
Backtting algorithm
Bayesian linear regression
Bayesian multivariate linear regression
Binomial regression
Canonical analysis
Censored regression model
Coecient of determination
Comparison of general and generalized linear models
Compressed sensing
Conditional change model
Controlling for a variable
Cross-sectional regression
Curve tting
Deming regression
Design matrix
Dierence in dierences
Dummy variable (statistics)
Errors and residuals in statistics
Errors-in-variables models
Explained sum of squares
Explained variation
First-hitting-time model
Fixed eects model
Fraction of variance unexplained
FrischWaughLovell theorem
General linear model
Generalized additive model
Generalized additive model for location, scale and shape
Generalized estimating equation
Generalized least squares
Generalized linear array model
Generalized linear mixed model
Generalized linear model
Growth curve
Guess value
Hat matrix
Heckman correction
Heteroscedasticity-consistent standard errors
HosmerLemeshow test
Instrumental variable
Interaction (statistics)
Isotonic regression
Iteratively reweighted least squares
Kitchen sink regression
Lack-of-t sum of squares
Leverage (statistics)
Limited dependent variable
Linear probability model
Mallowss Cp
Mean and predicted response
Mixed model
Moderation (statistics)
Moving least squares
Multicollinearity
6 1 MACHINE LEARNING
Multiple correlation
Multivariate probit
Multivariate adaptive regression splines
NeweyWest estimator
Non-linear least squares
Nonlinear regression
Logistic Regression Logit
Multinomial logit
Logistic regression
Bio-inspired Methods Bio-inspired computing
Metaheuristic and search algs. there
Swarm intelligence and methods there
Particular algorithms:
Particle_swarm_optimization
Ant colony optimization algorithms
Articial immune system
Firey algorithm, 2008
Cuckoo search, 2009
Bat algorithm, 2010
Evolutionary Algorithms Evolvability (computer sci-
ence)
Evolutionary computation
Evolutionary algorithm
Genetic algorithm
Chromosome (genetic algorithm)
Crossover (genetic algorithm)
Fitness function
Evolutionary data mining
Genetic programming
Learnable Evolution Model
Stochastic diusion search (SDS)
Neural Networks Neural network
Articial neural network
Articial neuron
Types of articial neural networks
Perceptron
Multilayer perceptron
Activation function
Self-organizing map
Attractor network
Diusion Networks
ADALINE
Adaptive Neuro Fuzzy Inference System
Adaptive resonance theory
IPO underpricing algorithm
ALOPEX
Articial Intelligence System
Autoassociative memory
Autoencoder
Backpropagation
Bcpnn
Bidirectional associative memory
Biological neural network
Boltzmann machine
Restricted Boltzmann machine
Cascade correlation algorithm
Cellular neural network
Cerebellar Model Articulation Controller
Committee machine
Competitive learning
Compositional pattern-producing network
Computational cybernetics
Computational neurogenetic modeling
Confabulation (neural networks)
Cortical column
Counterpropagation network
Covers theorem
Cultured neuronal network
Dehaene-Changeux Model
Delta rule
Early stopping
Echo state network
The Emotion Machine
Evolutionary Acquisition of Neural Topologies
7
Extension neural network
Feed-forward
Feedforward neural network
Fuzzy cellular neural networks
Generalized Hebbian Algorithm
Generative topographic map
Group method of data handling
Growing self-organizing map
Memory-prediction framework
Helmholtz machine
Hierarchical temporal memory
Hopeld network
Hybrid neural network
HyperNEAT
Infomax
Instantaneously trained neural networks
Interactive Activation and Competition
Leabra
Learning Vector Quantization
Lernmatrix
LindeBuzoGray algorithm
Liquid state machine
Long short term memory
Madaline
Modular neural networks
MoneyBee
Neocognitron
Nervous system network models
NETtalk (articial neural network)
Neural backpropagation
Neural coding
Neural cryptography
Neural decoding
Neural gas
Neural Information Processing Systems
Neural modeling elds
Neural oscillation
Neurally controlled animat
Neuroevolution of augmenting topologies
Neuroplasticity
Ni1000
Nonspiking neurons
Nonsynaptic plasticity
Ojas rule
Optical neural network
Phase-of-ring code
Promoter based genetic algorithm
Pulse-coupled networks
Quantum neural network
Radial basis function
Radial basis function network
Random neural network
Recurrent neural network
Reentry (neural circuitry)
Reservoir computing
Rprop
Semantic neural network
Sigmoid function
SNARC
Softmax activation function
Spiking neural network
Stochastic neural network
Synaptic plasticity
Synaptic weight
Tensor product network
Time delay neural network
U-Matrix
Universal approximation theorem
Winner-take-all
Winnow (algorithm)
Reinforcement learning Reinforcement learning
Markov decision process
8 1 MACHINE LEARNING
Bellman equation
Q-learning
Temporal dierence learning
SARSA
Multi-armed bandit
Apprenticeship learning
Predictive learning
Text Mining Text mining
Natural language processing
Document classication
Bag of words model
N-gram
Part-of-speech tagging
Sentiment analysis
Information extraction
Topic model
Concept mining
Semantic analysis (machine learning)
Automatic summarization
Automatic distillation of structure
String kernel
Biomedical text mining
Never-Ending Language Learning
Structure Mining Structure mining
Structured learning
Structured prediction
Sequence mining
Sequence labeling
Process mining
Advanced Learning Tasks Multi-label classication
Classier chains
Web mining
Anomaly detection
Anomaly Detection at Multiple Scales
Local outlier factor
Novelty detection
GSP Algorithm
Optimal matching
Record linkage
Meta learning (computer science)
Learning automata
Learning to rank
Multiple-instance learning
Statistical relational learning
Relational classication
Data stream mining
Alpha algorithm
Syntactic pattern recognition
Multispectral pattern recognition
Algorithmic learning theory
Deep learning
Bongard problem
Learning with errors
Parity learning
Inductive transfer
Granular computing
Conceptual clustering
Formal concept analysis
Biclustering
Information visualization
Co-occurrence networks
Applications Problem domain
Recommender system
Collaborative ltering
Proling (information science)
Speech recognition
Stock forecast
Activity recognition
Data Analysis Techniques for Fraud Detection
Molecule mining
Predictive behavioral targeting
9
Proactive Discovery of Insider Threats Using Graph
Analysis and Learning
Robot learning
Computer vision
Facial recognition system
Outlier detection
Anomaly detection
Novelty detection
Software R (programming language)
MapReduce
Oracle Data Mining
Pentaho
Mallet (software project)
Orange (software)
Learning Based Java
Scikit-learn
Waes (machine learning)
Apache Mahout
Data Applied
Data Mining Extensions
ELKI
Feature Selection Toolbox
Monte Carlo Machine Learning Library (MCMLL)
Neural network software
Software mining
10 2 TEXT AND IMAGE SOURCES, CONTRIBUTORS, AND LICENSES
2 Text and image sources, contributors, and licenses
2.1 Text
Book:Machine Learning - The Complete Guide Source: http://en.wikipedia.org/wiki/Book:Machine_Learning_-_The_Complete_
Guide?oldid=613781094 Contributors: The Anome, Cherkash, SchreiberBike, Liorrokach and Anonymous: 3
2.2 Images
File:Kernel_Machine.png Source: http://upload.wikimedia.org/wikipedia/commons/1/1b/Kernel_Machine.png License: CC0 Contribu-
tors: Own work Original artist: Alisneaky
2.3 Content license
Creative Commons Attribution-Share Alike 3.0