# 支持向量机（SVM）

## 算法原理

SVM学习的基本想法是求解能够正确划分训练数据集并且几何间隔最大的分离超平面

\gamma_{i}=y_{i}\left(\frac{\boldsymbol{w}}{|\boldsymbol{w}|} \cdot \boldsymbol{x}_{\boldsymbol{i}}+\frac{b}{|\boldsymbol{w}|}\right)

\max _{\boldsymbol{w}, b} \boldsymbol{\gamma}

s.t. \quad y_{i}\left(\frac{\boldsymbol{w}}{|\boldsymbol{w}|} \cdot \boldsymbol{x}_{i}+\frac{b}{|\boldsymbol{w}|}\right) \geq \gamma, i=1,2, \ldots, N

y_{i}\left(\frac{\boldsymbol{w}}{|\boldsymbol{w}| \gamma} \cdot \boldsymbol{x}_{i}+\frac{b}{|\boldsymbol{w}| \gamma}\right) \geq 1

\boldsymbol{w}=\frac{\boldsymbol{w}}{|\boldsymbol{w}| \gamma}
b=\frac{b}{|\boldsymbol{w}| \gamma}

\min_{w, b} \frac{1}{2}|\boldsymbol{w}|^{2}

s.t. y_{i}\left(\boldsymbol{w} \cdot \boldsymbol{x}_{i} + b\right) \geq 1, i=1,2, \ldots, N

\boldsymbol{w}^{}=\sum_{i=1}^{N} \alpha_{i}^{} y_{i} \boldsymbol{x}{i}
b^{*}=y
{j}-\sum_{i=1}^{N} \alpha_{i}^{*} y_{i}\left(\boldsymbol{x}{i} \cdot \boldsymbol{x}{j}\right)

\min {\boldsymbol{w}, b, \xi{i}} \frac{1}{2}|\boldsymbol{w}|^{2}+C \sum_{i=1}^{m} \xi_{i}
s.t. \quad y_{i}\left(\boldsymbol{w} \cdot \boldsymbol{x}{i}+b\right) \geq 1-\xi{i} \xi_{i} \geq 0, i=1,2, \ldots, N

## ski-learn代码

from sklearn import datasets
import numpy as np
import matplotlib.pyplot as plt

x = iris.data
y = iris.target
X = x[y < 2, :2]
y = y[y < 2]

from sklearn.preprocessing import StandardScaler

ss = StandardScaler()
ss.fit(X)
X_s = ss.transform(X)
# 引入线性
from sklearn.svm import LinearSVC
def plot_decision_boundary(model, axis):
x0, x1 = np.meshgrid(
np.linspace(axis[0], axis[1], int((axis[1] - axis[0]) * 100)).reshape(-1, 1),
np.linspace(axis[2], axis[3], int((axis[3] - axis[2]) * 100)).reshape(-1, 1),
)
X_new = np.c_[x0.ravel(), x1.ravel()]
y_predict = model.predict(X_new)
zz = y_predict.reshape(x0.shape)
from matplotlib.colors import ListedColormap
custom_cmap = ListedColormap(['#EF9A9A', '#FFF59D', '#90CAF9'])
plt.contourf(x0, x1, zz, linewidth=5, cmap=custom_cmap)

svc = LinearSVC(C=1e9)
svc.fit(X_s, y)
plot_decision_boundary(svc, axis=[-3, 3, -3, 3])
plt.scatter(X_s[y==0,0], X_s[y==0,1])
plt.scatter(X_s[y==1,0], X_s[y==1,1])
plt.show()

## 非线性SVM算法原理

核函数表示，通过一个非线性转换后的两个实例间的内积

K(x, z)=\phi(x) \cdot \phi(z)

\begin{aligned}
&\text { 多项式核函数 } \quad K(x, y)=(x \cdot y+c)^{d}\
&\text { 线性核函数 } \quad K(x, y)=x \cdot y
\end{aligned}

## 高斯核函数

RBF Radial Basis Function Kernel

K(x, y)=e^{-\gamma|x-y|^{2}}