I was very confused when i started machine learning because i hadn’t idea where to start,which algorithm should be learn,how to classify the algorithm on what basis so i was frustrated and always stuck in a situations where from you cannot go ahead or back so after lots of research i found a blog article name as “A Tour Of Machine Learning Algorithm” by Jason Brownlee.
We can categorize machine learning algorithm in two parts:(1)Learning Style (2) Similarity in form(function)
Algorithm grouped by Learning Style:
It is popular in machine learning or artificial intelligence we first consider learning style then we consider algorithm and we know there are different way an algorithm can model a problem based on it’s interaction with experience, environment or whatever we call the input data.
This way of organizing machine learning is very useful because it forces you to think about the roles of input data and the model preparations process and select one that is most appropriate for your model.
The three different learning style in algorithm are:
1.Supervised Learning:Input data is called training data and has a known label or result such as span or not span, or a stock price at a time.Example problems are classification and regression. Example algorithms are Logistics, Linear regression,Back propagation etc.
2.Unsupervised Learning:Input data is not labeled and does not have a known result.Example problems are Clustering, Dimensionality reduction,association rule learning. Example algorithms are The Apriori algorithm and k-means.
3.Semi-supervised Learning: Input data is a mixture of labeled and unlabeled examples.Example problems are classification and regression. Example algorithms are extensions to other flexible methods that make assumptions about how to model the unlabeled data. Moreover semi-supervised are used in image classification problems where there are large data-sets and few labeled.
Algorithm grouped by similarity:
This is useful grouping method but it is not perfect.There are still algorithm that could easily fit into multiple categories. There are also categories that have the same name that describe the problem and class of algorithm such as Regression and Clustering.
Algorithm generally grouped by together in terms of their function(how they work). For example tree based method and neural network inspired method.
1.Regression Algorithm: Regression is concerned with modelling the relationship between variables that is iteratively refined using a measure of error in the prediction made by model.
The most popular regressions algorithms are:
- Oridinary Least Square Regression(OLSR).
- Linear Regression
- Logistics Regression
- Stepwise Regression
- Multivariate Adaptive Regression Splines(MARS)
- Locally Estimated Scatterplot Smoothing(LESS)
- Ridge Regression
- Least Absolute Shrinkage and Selection Operators(LASSO)
- Polynomial Regression
- Elasticnet Regression
- Ecological Regression
- Bayesian Regression
- Robust Regression
- Least-angle Regression
2.Instance-based Algorithm:Instance-based learning model is a decision problem with instances or examples of training data that are deemed important or required to the model.
The most popular instance-based algorithms are:
- k-Nearest Neighbor(kNN)
- Learning Vector Quantization (LVQ)
- Self-Organizing Map (SOM)
- Locally Weighted Learning (LWL)
3.Decision Tree Algorithm:Decision tree methods construct a model of decisions made based on actual values of attributes in the data.
The most popular decision tree algorithms are:
- Classification and Regression Tree
- Iterative Dichotomiser 3 (ID3)
- C4.5 and C5.0 (different versions of a powerful approach)
- Chi-squared Automatic Interaction Detection (CHAID)
- Decision Stump
- Conditional Decision Trees
4.Bayesian Algorithm:Bayesian methods are those that explicitly apply Bayes’ Theorem for problems such as classification and regression.
The most popular Bayesian algorithms are:
- Naive Bayes
- Gaussian Naive Bayes
- Multinomial Naive Bayes
- Average One-Dependence Estimator(AODE)
- Bayesian Belief Network(BBN)
- Bayesian Network(BN)
5.Clustring Algorithm:Clustering methods are typically organized by the modeling approaches such as centroid-based and hierarchal.
The most popular clustering algorithms are:
- Expectation Maximization(EM)
- Hierarchical Clustring
6.Associates Rule Learning Algorithm:Association rule learning methods extract rules that best explain observed relationships between variables in data.
The most popular association rule learning algorithms are:
- Apriori algorithm
- Eclat algorithm
7.Artificial Neural Network:Artificial Neural Networks are models that are inspired by the structure and/or function of biological neural networks.
The most popular artificial neural network algorithms are:
- Hopfield Network
- Radial Basis Function Network(RBFN)
8.Deep Learning Network:Deep Learning methods are a modern update to Artificial Neural Networks that exploit abundant cheap computation.
The most popular deep learning algorithms are:
- Deep Boltzmann Machine(DBM)
- Deep Belief Network(DBN)
- Convolutional Neural Network(CNN)
- Stacked Auto-Encoders
9.Dimensionality Reduction Algorithm:Like clustering methods, dimensionality reduction seek and exploit the inherent structure in the data.
The most popular dimensionality Reduction algorithms are:
- Principal Component Analysis(PCA)
- Principal Component Regression(PCR)
- Partial Least Square Regression(PLSR)
- Sammom Mapping
- Multidimensional Scaling
- Projection Pursuit
- Linear Discriminant Analysis (LDA)
- Mixture Discriminant Analysis (MDA)
- Quadratic Discriminant Analysis (QDA)
- Flexible Discriminant Analysis (FDA)
10.Ensemble Algorithm: Ensemble methods are models composed of multiple weaker models that are independently trained and whose predictions are combined in some way to make the overall prediction.
The most popular dimensionality Reduction algorithms are:
- Bootstrapped Aggregation(Bagging)
- Stacked Generalization
- Gradient Boosting Machines (GBM)
- Gradient Boosted Regression Trees (GBRT)
- Random Forests
There are also sub fields of machine learning like:
- Computational Intelligence
- Computer Vision
- Natural Language Processing
- Recommender System
- Reinforcement Learning
- Graphical Learning and more..