Machine-Learning-Algorithm: 【Numpy 手写实现】SVM 支持向量机 | KNN K近邻 | Kmeans | Logistic Regression 逻辑回归 | Maximum Entropy 最大熵 | Naive Bayes 朴素贝叶斯 | Perception 感知机 | Decision Tree 决策树 | Random Forest 随机森林 | DBDT | GDA 高斯判别分析 | PCA 主成分分析

# Machine-Learning-Algorithm

**注意: 每个文件只有开始的 class 是模型本身，其它代码都是用来测试的，每个模型的实现都在 100 行以内**

**Note: Only the class at the beginning of each file is the model itself, the rest of the code is for testing, and the implementation of each model is within 100 lines**

---

### 1. Logistic Regression

**File** - [logistic_regression.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/logistic_regression.py)

**Cost Function** -
 
 <img src="https://latex.codecogs.com/gif.latex?H(p,q)=-\sum_{i}^{&space;}[y^{(i)}\&space;log\&space;h_\theta(x^{(i)})&plus;(1-y^{(i)})\&space;log(1-h_\theta(x^{(i)})))]&space;" />

**Optimization Algorithm** - Gradient descent method

---

### 2. Support Vector Machine

**File** - [support_vector_machine.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/support_vector_machine.py)

**Example** -

**Cost Function** -

**Optimization Algorithm** - Sequential minimal optimization (SMO)

---

### 3. Perception

**File** - [perception.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/perception.py)

**Example** -

---

### 4. Naive Bayes

**File** - [naive_bayes.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/naive_bayes.py)

**Example** -

---

### 5. K-Nearest Neighbor

**File** - [k_nearest_neighbor.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/k_nearest_neighbor.py) | [util_kd_tree.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/util_kd_tree.py)

**Example** -

---

### 6. Decision Tree

**File** - [decision_tree.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/decision_tree.py)

**Optimization Algorithm** - Generalized Iterative Scaling (GIS)

**Example** -

---

### 7. Random Forest

**File** - [random_forest.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/random_forest.py) | | [decision_tree.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/decision_tree.py)

**Example** -

---

### 8. Gradient Boosting Decision Tree

**File** - [gradient_boosting_decision_tree.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/gradient_boosting_decision_tree.py) | [decision_tree.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/decision_tree.py)

---

### 9. Linear Discriminant Analysis

**File** - [linear_discriminant_analysis.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/linear_discriminant_analysis.py)

---

### 10. Maximum Entropy

**File** - [maximum_entropy.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/maximum_entropy.py)

**Example** -

---

### 11. Gaussian Discriminant Analysis

**File** - [gaussian_discriminant_analysis.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/gaussian_discriminant_analysis.py)

---

### 12. Principal Component Analysis

**File** - [principal_component_analysis.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/principal_component_analysis.py)

**Example** -

---

### 13. K-means

**File** - [kmeans.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/kmeans.py) | [util_kd_tree.py](https://github.com/KangCai/Machine-Learning-Algorithm/blob/master/util_kd_tree.py)

顾智拼/Machine-Learning-Algorithm

简介

发行版

贡献者

近期动态

顾智拼/Machine-Learning-Algorithm .gitee-modal { width: 500px !important; }

简介

发行版

贡献者

近期动态

搜索帮助

顾智拼/Machine-Learning-Algorithm