Support Vector Machines Part-1

Hi,

Did you try cross-validation?

Thanks.

Hi,

It really depends on the data. Thus we tune these hyperparameters and pick those which yield better performance.

Thanks.

Hi,

You may refer to https://stackoverflow.com/questions/58410187/how-to-plot-predicted-values-vs-the-true-value

Thanks.

Hi,

https://github.com/cloudxlab/ml/blob/master/machine_learning/support_vector_machines.ipynb

Thanks.

Docstring:
Set printing options.

These options determine the way floating point numbers, arrays and
other NumPy objects are displayed.

Parameters
----------
precision : int or None, optional
    Number of digits of precision for floating point output (default 8).
    May be `None` if `floatmode` is not `fixed`, to print as many digits as
    necessary to uniquely specify the value.

You can try running the following in order to get documentation: np.set_printoptions?

At 2:02:00, see the above video (https://youtu.be/jNKbdFHivT4?t=7348). I have added the screenshot below too.

We are basically talking about moons data set. The label, in that case, is 0 & 1. There are two classes which we have to identify based on two feature X1, X2.

Hi,

Thank you for pointing this out.

Thanks.

Hi,

The image that you are referring contains of points that are being classified based on their target variable. This is a form of supervised learning. Clustering on that other hand is an unsupervised learning. Having said that, SVM is being considered for clustering, you can go through the below paper for further information:

http://web.cs.ucla.edu/~qgu/pdf/aistats13b.pdf

Thanks.

Hi,

The StandardScalar function is used to scale data, to predict using that data you need to use a prediction algorithm. Could you please tell me which problem are you trying to solve here.

Thanks.

-- Rajtilak Bhattacharjee

Hi,

If you check the first video, it ends on the slide "Nonlinear SVM Classification: Example", the second video starts with the slide "Linear SVM Classification". If you check the slides on the first video, these are slide# 114, and 116 respectively. Please let us know if you find some other content.

Thanks.

-- Rajtilak Bhattacharjee

Hi Saurabh,

Thanks for your feedback, the videos are updated and the sections are added.

Happy Learning.

Hi,

Would request you to clone our GitHub repository and you would find all the files there.

Thanks.

-- Rajtilak Bhattacharjee

Hi,

gamma is the kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’. If gamma='scale' (default) is passed then it uses 1 / (n_features * X.var()) as value of gamma, if ‘auto’, uses 1 / n_features.

Thanks.

-- Rajtilak Bhattacharjee

Hi,

Depending on the number of training sets (data)/features that you have, you can choose to use either logistic regression or support vector machine.
Lets take these as an example where :

n = number of features,
m = number of training examples

1. If n is large (1–10,000) and m is small (10–1000) : use logistic regression or SVM with a linear kernel.
2. If n is small (1–10 00) and m is intermediate (10–10,000) : use SVM with (Gaussian, polynomial etc) kernel
3. If n is small (1–10 00), m is large (50,000–1,000,000+): first, manually add more features and then use logistic regression or SVM with a linear kernel

Also, you can cross validate as a practice exercise.

Thanks.

-- Rajtilak Bhattacharjee

Hi,

SVM uses the full data and solve a convex optimization problem with respect to these data points, whereas SGD Classifier can treat the data in batches and performs a gradient descent aiming to minimize *expected* loss with respect to the sample distribution. There are no better algorithm, it depends on the data which one you would like to use.

Thanks.

-- Rajtilak Bhattacharjee

We use kernel trick when the classes in the dataset are not linearly separable, using the kernel trick transforms the input space to higher dimensional space, so the point of doing a train test split on the data is to get an unbiased estimate of the model's performance on unseen data. This is important as owing to the non-linearity some data may be misclassified if the model has built a wrong decision boundary.

But when the dataset is simple and the classes are linearly separable SVC uses a subset of the data (support vectors) on which we are training to create the decision boundaries and the unseen data on which we might make our predictions later on will very much resemble the linearly separable training set and hence may not require the test set for evaluating the model's performance.

Please clarify, whether I am correct.

Thank you

The data is generated/loaded by the notebook code.