#Categories

  • Sheikh Aman

SVM implementation in python. | Proper explanation.



Don’t forget to hit ❤. :)

Topics you will learn

  • History of Support Vector Machine.

  • Introduction to Support Vector Machine.

  • Example of Support Vector Machine.

  • Application of Support Vector Machine.

  • Hyper plane and support vectors in support vector machine algorithm.

  • Tuning parameters for SVM algorithm.

  • SVM implementation in pyhton.

  • Pros and Cons for SVM.

  • Conclusion.



History of Support Vector Machine.


The original and first SVM algorithm was invented by Vladimir N, Vapnik and Alexey Ya, Chervonenkis in 1963. In 1992, Bernhard Boser, Isabelle Guyon and Vladimir Vapnik suggested how to make or create nonlinear classifiers with the help of kernel trick to maximum-margin hyperplanes. The present standard incarnation was proposed by Corinna Cortes and Vapnik in 1993 and published in 1995.


Introduction to Support Vector Machine.


Support Vector Machine or SVM is one among the foremost popular Supervised Learning algorithms, which is employed for Classification also as Regression problems. However, primarily, it's used for Classification problems in Machine Learning.


The goal of the SVM algorithm is to make the simplest line or decision boundary which will segregate n-dimensional space into classes in order that we will easily put the new data point within the correct category in the future. This best decision boundary is termed a hyperplane.


SVM chooses the extreme or acute points/vectors that help in creating the hyperplane. These extreme cases are called support vectors, and hence algorithm is termed as Support Vector Machine.



Example of Support Vector Machine.


Suppose there is a basket of fruits Where there are two types of fruits orange and mango.

Suppose we see a strange round yellow fruit, it might be a ripe mango or an orange. So, If we want a model to detect whether it is an orange or mango, so those types of model can be created by the SVM algorithm.


We will first train our model with lots of images of orange and mango so that it can learn about different features of orange and mango, and then we test it with this strange fruit. So as support vector creates a decision boundary between these two data (orange and mango) and choose extreme cases (support vectors), it will see the extreme case of orange and mango. On the basis of the support vectors, it will classify it as mango or orange.


Application of Support Vector Machine.


SVMs are often used to solve various real-world problems:


  • SVMs are helpful in the text as well as hypertext categorization, as their application can reduce the requirement for labelled training instances in both the quality inductive as well as transductive settings. Some methods for shallow semantic parsing are related to support vector machines.

  • Classification of images also can be performed using SVMs. According to many experimental results, it shows that SVM always achieves significantly higher search accuracy than traditional query refinement schemes after more than two rounds (best practice three to four rounds) of relevance feedback. this can be also true for image segmentation systems, including those employing a modified version SVM that uses the privileged approach as suggested by Vapnik.

  • Classification of satellite data like SAR data with the help of supervised SVM.

  • Hand-written characters are often recognized using SVM.


Hyper plane and support vectors in support vector machine algorithm.


Hyperplane:


There are multiple lines/decision boundaries to segregate the classes in n-dimensional space, but we'd like to search out out the simplest decision boundary that helps to classify the data points. This best boundary is known as hyperplane of SVM.


The dimensions of the hyperplane depend upon the features present within the dataset, which suggests if there are 2 features, then hyperplane is going to be a straight line. And if there are 3 features, then hyperplane is going to be a 2-dimension plane.


We always create a hyperplane that includes a maximum margin, which suggests the utmost distance between the data points.


Support Vectors:


The data points or vectors that are the closest to the hyperplane and which affect the position of the hyperplane are termed as Support Vector. Since these vectors support the hyperplane, hence called a Support vector.


Tuning parameters for SVM algorithm.


Kernel


The learning of the hyperplane in linear SVM is done by transforming the matter using some linear algebra. this is often where the kernel plays a task.

For linear kernel the equation for prediction for a brand new input using the dot product between the input (x) and every support vector (xi) is calculated as follows:


f(x) = B(0) + sum(ai * (x,xi))

This is an equation that involves calculating the inner products of a brand new input vector (x) with all support vectors in training data. The coefficients B0 and ai (for each input) must be estimated from the training data by the learning or training algorithm.


The polynomial kernel are often written as K(x,xi) = 1 + sum(x * xi)^d and exponential as K(x,xi) = exp(-gamma * sum((x — xi²)). [Source for this excerpt : http://machinelearningmastery.com/].
Polynomial and exponential kernels calculate separation line during a higher dimension. this is often called kernel trick

Regularization


The Regularization parameter tells the SVM optimization what proportion you would like to avoid misclassifying each training example.


For large values of C, the optimization will choose a smaller-margin hyperplane if that hyperplane does a far better job of getting all the training points classified correctly. Conversely, a really small value of C will cause the optimizer to seem for a larger-margin separating hyperplane, although that hyperplane misclassifies more points.


The images below (same as image 1 and image 2 in section 2) are an example of two different regularization parameter. The left one has some misclassification thanks to lower regularization value. a better value ends up in results just like the right one.




Gamma


The gamma parameter defines how far the influence of one training example reaches, with low values meaning ‘far’ and high values meaning ‘close’. In other words, with low gamma, points far from plausible separation line are considered within the calculation for the separation line. Whereas high gamma means the points near to plausible line are considered within the calculation.




Margin


And finally last but vital characteristic of SVM classifier. SVM to core tries to realize a decent margin.

A margin is always a separation of the line to the closest class points.

A good margin is one where this separation is larger for both the classes. Images below give to the visual example of excellent and bad margin. a decent or good margin allows the points to be in their respective classes without crossing to other class.


image credit goes machine learning 101


SVM implementation in pyhton.


Here we have used Iris dataset.

Python code snippnets with ouput.









python code for SVM.




Pros and Cons for SVM.


Pros:

  • It works very well with a clear margin of separation

  • It is effective in high dimensional spaces.

  • It is effective in cases where the amount of dimensions is bigger than the amount of samples.

  • It uses a subset of training points within the decision function (called support vectors), so it's also memory efficient.

Cons:

  • It doesn’t perform well once we have a large data set because the specified training time is higher

  • It also doesn’t perform okay, when the data set has more noise i.e. target classes are overlapping

  • SVM doesn’t directly provide probability estimates, these are calculated using a rich five-fold cross-validation. it's included within the related SVC method of Python scikit-learn library.


Conclusion.


Hey buddy, I think this is enough for Support Vector Machine. Here you have learned these things:

  • History of Support Vector Machine.

  • Introduction to Support Vector Machine.

  • Example of Support Vector Machine.

  • Application of Support Vector Machine.

  • Hyper plane and support vectors in support vector machine algorithm.

  • Tuning parameters for SVM algorithm.

  • SVM implementation in pyhton.

  • Pros and Cons for SVM.

If you face any kind of problem then follow me on twitter and just ping me.

Thanks.

Something Interesting

#Some_Interesting_topics

MR. MACHINE

Subscribe to Our Newsletter
Copyright © 2020 MR. Machine. All Rights Reserved
  • Facebook
  • Twitter
  • Pinterest
  • Instagram