# ML_Notes **Repository Path**: mirrors_zhulei227/ML_Notes ## Basic Information - **Project Name**: ML_Notes - **Description**: 机器学习算法的公式推导以及numpy实现 - **Primary Language**: Unknown - **License**: Not specified - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 1 - **Forks**: 1 - **Created**: 2022-01-11 - **Last Updated**: 2026-01-25 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README ### 机器学习笔记 对目前主流的机器学习算法进行公式推导、问题分析以及代码实现（主要基于numpy），持续更新（下面链接如果加载不出来，对应内容可在notebooks文件夹下找到）： [01_线性模型_线性回归](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/01_%E7%BA%BF%E6%80%A7%E6%A8%A1%E5%9E%8B_%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92.ipynb) [01_线性模型_线性回归_正则化(Lasso,Ridge,ElasticNet)](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/01_%E7%BA%BF%E6%80%A7%E6%A8%A1%E5%9E%8B_%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92_%E6%AD%A3%E5%88%99%E5%8C%96(Lasso%2CRidge%2CElasticNet).ipynb) [02_线性模型_逻辑回归](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/02_%E7%BA%BF%E6%80%A7%E6%A8%A1%E5%9E%8B_%E9%80%BB%E8%BE%91%E5%9B%9E%E5%BD%92.ipynb) [03_二分类转多分类的一般实现](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/03_%E4%BA%8C%E5%88%86%E7%B1%BB%E8%BD%AC%E5%A4%9A%E5%88%86%E7%B1%BB%E7%9A%84%E4%B8%80%E8%88%AC%E5%AE%9E%E7%8E%B0.ipynb) [04_线性模型_感知机](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/04_%E7%BA%BF%E6%80%A7%E6%A8%A1%E5%9E%8B_%E6%84%9F%E7%9F%A5%E6%9C%BA.ipynb) [05_线性模型_最大熵模型](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/05_%E7%BA%BF%E6%80%A7%E6%A8%A1%E5%9E%8B_%E6%9C%80%E5%A4%A7%E7%86%B5%E6%A8%A1%E5%9E%8B.ipynb) [06_优化_拟牛顿法实现(DFP,BFGS)](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/06_%E4%BC%98%E5%8C%96_%E6%8B%9F%E7%89%9B%E9%A1%BF%E6%B3%95%E5%AE%9E%E7%8E%B0(DFP%2CBFGS).ipynb) [07_01_svm_硬间隔支持向量机与SMO](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/07_01_svm_%E7%A1%AC%E9%97%B4%E9%9A%94%E6%94%AF%E6%8C%81%E5%90%91%E9%87%8F%E6%9C%BA%E4%B8%8ESMO.ipynb) [07_02_svm_软间隔支持向量机](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/07_02_svm_%E8%BD%AF%E9%97%B4%E9%9A%94%E6%94%AF%E6%8C%81%E5%90%91%E9%87%8F%E6%9C%BA.ipynb) [07_03_svm_核函数与非线性支持向量机](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/07_03_svm_%E6%A0%B8%E5%87%BD%E6%95%B0%E4%B8%8E%E9%9D%9E%E7%BA%BF%E6%80%A7%E6%94%AF%E6%8C%81%E5%90%91%E9%87%8F%E6%9C%BA.ipynb) [08_代价敏感学习_添加sample_weight支持](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/08_%E4%BB%A3%E4%BB%B7%E6%95%8F%E6%84%9F%E5%AD%A6%E4%B9%A0_%E6%B7%BB%E5%8A%A0sample_weight%E6%94%AF%E6%8C%81.ipynb) [09_01_决策树_ID3与C4.5](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/09_01_%E5%86%B3%E7%AD%96%E6%A0%91_ID3%E4%B8%8EC4.5.ipynb) [09_02_决策树_CART](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/09_02_%E5%86%B3%E7%AD%96%E6%A0%91_CART.ipynb) [10_01_集成学习_简介](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_01_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_%E7%AE%80%E4%BB%8B.ipynb) [10_02_集成学习_boosting_adaboost_classifier](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_02_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_boosting_adaboost_classifier.ipynb) [10_03_集成学习_boosting_adaboost_regressor](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_03_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_boosting_adaboost_regressor.ipynb) [10_04_集成学习_boosting_提升树](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_04_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_boosting_%E6%8F%90%E5%8D%87%E6%A0%91.ipynb) [10_05_集成学习_boosting_gbm_regressor](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_05_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_boosting_gbm_regressor.ipynb) [10_06_集成学习_boosting_gbm_classifier](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_06_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_boosting_gbm_classifier.ipynb) [10_07_集成学习_bagging](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_07_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_bagging.ipynb) [10_08_集成学习_bagging_randomforest](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_08_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_bagging_randomforest.ipynb) [10_09_集成学习_bagging_高阶组合_stacking](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_09_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_bagging_%E9%AB%98%E9%98%B6%E7%BB%84%E5%90%88_stacking.ipynb) [10_10_集成学习_xgboost_原理介绍及回归树的简单实现](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_10_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_xgboost_%E5%8E%9F%E7%90%86%E4%BB%8B%E7%BB%8D%E5%8F%8A%E5%9B%9E%E5%BD%92%E6%A0%91%E7%9A%84%E7%AE%80%E5%8D%95%E5%AE%9E%E7%8E%B0.ipynb) [10_11_集成学习_xgboost_回归的简单实现](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_11_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_xgboost_%E5%9B%9E%E5%BD%92%E7%9A%84%E7%AE%80%E5%8D%95%E5%AE%9E%E7%8E%B0.ipynb) [10_12_集成学习_xgboost_回归的更多实现：泊松回归、gamma回归、tweedie回归](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_12_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_xgboost_%E5%9B%9E%E5%BD%92%E7%9A%84%E6%9B%B4%E5%A4%9A%E5%AE%9E%E7%8E%B0%EF%BC%9A%E6%B3%8A%E6%9D%BE%E5%9B%9E%E5%BD%92%E3%80%81gamma%E5%9B%9E%E5%BD%92%E3%80%81tweedie%E5%9B%9E%E5%BD%92.ipynb) [10_13_集成学习_xgboost_分类的简单实现](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_13_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_xgboost_%E5%88%86%E7%B1%BB%E7%9A%84%E7%AE%80%E5%8D%95%E5%AE%9E%E7%8E%B0.ipynb) [10_14_集成学习_xgboost_优化介绍](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_14_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_xgboost_%E4%BC%98%E5%8C%96%E4%BB%8B%E7%BB%8D.ipynb) [10_15_集成学习_lightgbm_进一步优化](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_15_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_lightgbm_%E8%BF%9B%E4%B8%80%E6%AD%A5%E4%BC%98%E5%8C%96.ipynb) [10_16_集成学习_dart_提升树与dropout的碰撞](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_16_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_dart_%E6%8F%90%E5%8D%87%E6%A0%91%E4%B8%8Edropout%E7%9A%84%E7%A2%B0%E6%92%9E.ipynb) [10_17_集成学习_树模型的可解释性_模型的特征重要性及样本的特征重要性(sabaas,shap,tree shap)](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_17_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_%E6%A0%91%E6%A8%A1%E5%9E%8B%E7%9A%84%E5%8F%AF%E8%A7%A3%E9%87%8A%E6%80%A7_%E6%A8%A1%E5%9E%8B%E7%9A%84%E7%89%B9%E5%BE%81%E9%87%8D%E8%A6%81%E6%80%A7%E5%8F%8A%E6%A0%B7%E6%9C%AC%E7%9A%84%E7%89%B9%E5%BE%81%E9%87%8D%E8%A6%81%E6%80%A7(sabaas%2Cshap%2Ctree%20shap).ipynb) [11_01_EM_GMM引入问题](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/11_01_EM_GMM%E5%BC%95%E5%85%A5%E9%97%AE%E9%A2%98.ipynb) [11_02_EM_算法框架](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/11_02_EM_%E7%AE%97%E6%B3%95%E6%A1%86%E6%9E%B6.ipynb) [11_03_EM_GMM聚类实现](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/11_03_EM_GMM%E8%81%9A%E7%B1%BB%E5%AE%9E%E7%8E%B0.ipynb) [11_04_EM_GMM分类实现及其与LogisticRegression的关系](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/11_04_EM_GMM%E5%88%86%E7%B1%BB%E5%AE%9E%E7%8E%B0%E5%8F%8A%E5%85%B6%E4%B8%8ELogisticRegression%E7%9A%84%E5%85%B3%E7%B3%BB.ipynb) [12_01_PGM_贝叶斯网(有向无环图)初探](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_01_PGM_%E8%B4%9D%E5%8F%B6%E6%96%AF%E7%BD%91(%E6%9C%89%E5%90%91%E6%97%A0%E7%8E%AF%E5%9B%BE)%E5%88%9D%E6%8E%A2.ipynb) [12_02_PGM_朴素贝叶斯分类器实现](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_02_PGM_%E6%9C%B4%E7%B4%A0%E8%B4%9D%E5%8F%B6%E6%96%AF%E5%88%86%E7%B1%BB%E5%99%A8%E5%AE%9E%E7%8E%B0.ipynb) [12_03_PGM_半朴素贝叶斯分类器实现](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_03_PGM_%E5%8D%8A%E6%9C%B4%E7%B4%A0%E8%B4%9D%E5%8F%B6%E6%96%AF%E5%88%86%E7%B1%BB%E5%99%A8%E5%AE%9E%E7%8E%B0.ipynb) [12_04_PGM_朴素贝叶斯的聚类实现](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_04_PGM_%E6%9C%B4%E7%B4%A0%E8%B4%9D%E5%8F%B6%E6%96%AF%E7%9A%84%E8%81%9A%E7%B1%BB%E5%AE%9E%E7%8E%B0.ipynb) [12_05_PGM_马尔科夫链_初探及代码实现](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_05_PGM_%E9%A9%AC%E5%B0%94%E7%A7%91%E5%A4%AB%E9%93%BE_%E5%88%9D%E6%8E%A2%E5%8F%8A%E4%BB%A3%E7%A0%81%E5%AE%9E%E7%8E%B0.ipynb) [12_06_PGM_马尔科夫链_语言模型及文本生成](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_06_PGM_%E9%A9%AC%E5%B0%94%E7%A7%91%E5%A4%AB%E9%93%BE_%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B%E5%8F%8A%E6%96%87%E6%9C%AC%E7%94%9F%E6%88%90.ipynb) [12_07_PGM_马尔科夫链_PageRank算法](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_07_PGM_%E9%A9%AC%E5%B0%94%E7%A7%91%E5%A4%AB%E9%93%BE_PageRank%E7%AE%97%E6%B3%95.ipynb) [12_08_PGM_HMM_隐马模型：介绍及概率计算（前向、后向算法）](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_08_PGM_HMM_%E9%9A%90%E9%A9%AC%E6%A8%A1%E5%9E%8B%EF%BC%9A%E4%BB%8B%E7%BB%8D%E5%8F%8A%E6%A6%82%E7%8E%87%E8%AE%A1%E7%AE%97%EF%BC%88%E5%89%8D%E5%90%91%E3%80%81%E5%90%8E%E5%90%91%E7%AE%97%E6%B3%95%EF%BC%89.ipynb) [12_09_PGM_HMM_隐马模型：参数学习（有监督、无监督）](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_09_PGM_HMM_%E9%9A%90%E9%A9%AC%E6%A8%A1%E5%9E%8B%EF%BC%9A%E5%8F%82%E6%95%B0%E5%AD%A6%E4%B9%A0%EF%BC%88%E6%9C%89%E7%9B%91%E7%9D%A3%E3%80%81%E6%97%A0%E7%9B%91%E7%9D%A3%EF%BC%89.ipynb) [12_10_PGM_HMM_隐马模型：隐状态预测](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_10_PGM_HMM_%E9%9A%90%E9%A9%AC%E6%A8%A1%E5%9E%8B%EF%BC%9A%E9%9A%90%E7%8A%B6%E6%80%81%E9%A2%84%E6%B5%8B.ipynb) [12_11_PGM_HMM_隐马模型实战：中文分词](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_11_PGM_HMM_%E9%9A%90%E9%A9%AC%E6%A8%A1%E5%9E%8B%E5%AE%9E%E6%88%98%EF%BC%9A%E4%B8%AD%E6%96%87%E5%88%86%E8%AF%8D.ipynb) [12_12_PGM_马尔科夫随机场（无向图）介绍](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_12_PGM_%E9%A9%AC%E5%B0%94%E7%A7%91%E5%A4%AB%E9%9A%8F%E6%9C%BA%E5%9C%BA%EF%BC%88%E6%97%A0%E5%90%91%E5%9B%BE%EF%BC%89%E4%BB%8B%E7%BB%8D.ipynb) [12_13_PGM_CRF_条件随机场：定义及形式（简化、矩阵形式）](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_13_PGM_CRF_%E6%9D%A1%E4%BB%B6%E9%9A%8F%E6%9C%BA%E5%9C%BA%EF%BC%9A%E5%AE%9A%E4%B9%89%E5%8F%8A%E5%BD%A2%E5%BC%8F%EF%BC%88%E7%AE%80%E5%8C%96%E3%80%81%E7%9F%A9%E9%98%B5%E5%BD%A2%E5%BC%8F%EF%BC%89.ipynb) [12_14_PGM_CRF_条件随机场：如何定义特征函数](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_14_PGM_CRF_%E6%9D%A1%E4%BB%B6%E9%9A%8F%E6%9C%BA%E5%9C%BA%EF%BC%9A%E5%A6%82%E4%BD%95%E5%AE%9A%E4%B9%89%E7%89%B9%E5%BE%81%E5%87%BD%E6%95%B0.ipynb) [12_15_PGM_CRF_条件随机场：概率及期望值计算（前向后向算法）](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_15_PGM_CRF_%E6%9D%A1%E4%BB%B6%E9%9A%8F%E6%9C%BA%E5%9C%BA%EF%BC%9A%E6%A6%82%E7%8E%87%E5%8F%8A%E6%9C%9F%E6%9C%9B%E5%80%BC%E8%AE%A1%E7%AE%97%EF%BC%88%E5%89%8D%E5%90%91%E5%90%8E%E5%90%91%E7%AE%97%E6%B3%95%EF%BC%89.ipynb) [12_16_PGM_CRF_条件随机场：参数学习](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_16_PGM_CRF_%E6%9D%A1%E4%BB%B6%E9%9A%8F%E6%9C%BA%E5%9C%BA%EF%BC%9A%E5%8F%82%E6%95%B0%E5%AD%A6%E4%B9%A0.ipynb) [12_17_PGM_CRF_条件随机场：标签预测](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_17_PGM_CRF_%E6%9D%A1%E4%BB%B6%E9%9A%8F%E6%9C%BA%E5%9C%BA%EF%BC%9A%E6%A0%87%E7%AD%BE%E9%A2%84%E6%B5%8B.ipynb) [12_18_PGM_CRF_代码优化及中文分词实践](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_18_PGM_CRF_%E4%BB%A3%E7%A0%81%E4%BC%98%E5%8C%96%E5%8F%8A%E4%B8%AD%E6%96%87%E5%88%86%E8%AF%8D%E5%AE%9E%E8%B7%B5.ipynb) [12_19_PGM_CRF与HMM之间的区别与联系](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/12_19_PGM_CRF%E4%B8%8EHMM%E4%B9%8B%E9%97%B4%E7%9A%84%E5%8C%BA%E5%88%AB%E4%B8%8E%E8%81%94%E7%B3%BB.ipynb) [13_01_sampling_为什么要采样（求期望、积分等）](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/13_01_sampling_%E4%B8%BA%E4%BB%80%E4%B9%88%E8%A6%81%E9%87%87%E6%A0%B7%EF%BC%88%E6%B1%82%E6%9C%9F%E6%9C%9B%E3%80%81%E7%A7%AF%E5%88%86%E7%AD%89%EF%BC%89.ipynb) [13_02_sampling_MC采样：接受-拒绝采样、重要采样](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/13_02_sampling_MC%E9%87%87%E6%A0%B7%EF%BC%9A%E6%8E%A5%E5%8F%97-%E6%8B%92%E7%BB%9D%E9%87%87%E6%A0%B7%E3%80%81%E9%87%8D%E8%A6%81%E9%87%87%E6%A0%B7.ipynb) [13_03_sampling_MCMC：采样原理（再探马尔可夫链）](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/13_03_sampling_MCMC%EF%BC%9A%E9%87%87%E6%A0%B7%E5%8E%9F%E7%90%86%EF%BC%88%E5%86%8D%E6%8E%A2%E9%A9%AC%E5%B0%94%E5%8F%AF%E5%A4%AB%E9%93%BE%EF%BC%89.ipynb) [13_04_sampling_MCMC：MH采样的算法框架](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/13_04_sampling_MCMC%EF%BC%9AMH%E9%87%87%E6%A0%B7%E7%9A%84%E7%AE%97%E6%B3%95%E6%A1%86%E6%9E%B6.ipynb) [13_05_sampling_MCMC：单分量MH采样算法](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/13_05_sampling_MCMC%EF%BC%9A%E5%8D%95%E5%88%86%E9%87%8FMH%E9%87%87%E6%A0%B7%E7%AE%97%E6%B3%95.ipynb) [13_06_sampling_MCMC：Gibbs采样算法](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/13_06_sampling_MCMC%EF%BC%9AGibbs%E9%87%87%E6%A0%B7%E7%AE%97%E6%B3%95.ipynb) [14_01_概率分布：二项分布及beta分布](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/14_01_%E6%A6%82%E7%8E%87%E5%88%86%E5%B8%83%EF%BC%9A%E4%BA%8C%E9%A1%B9%E5%88%86%E5%B8%83%E5%8F%8Abeta%E5%88%86%E5%B8%83.ipynb) [14_02_概率分布：多项分布及狄利克雷分布](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/14_02_%E6%A6%82%E7%8E%87%E5%88%86%E5%B8%83%EF%BC%9A%E5%A4%9A%E9%A1%B9%E5%88%86%E5%B8%83%E5%8F%8A%E7%8B%84%E5%88%A9%E5%85%8B%E9%9B%B7%E5%88%86%E5%B8%83.ipynb) [14_03_概率分布：高斯分布（正态分布）及其共轭先验](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/14_03_%E6%A6%82%E7%8E%87%E5%88%86%E5%B8%83%EF%BC%9A%E9%AB%98%E6%96%AF%E5%88%86%E5%B8%83%EF%BC%88%E6%AD%A3%E6%80%81%E5%88%86%E5%B8%83%EF%BC%89%E5%8F%8A%E5%85%B6%E5%85%B1%E8%BD%AD%E5%85%88%E9%AA%8C.ipynb) [14_04_概率分布：指数族分布](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/14_04_%E6%A6%82%E7%8E%87%E5%88%86%E5%B8%83%EF%BC%9A%E6%8C%87%E6%95%B0%E6%97%8F%E5%88%86%E5%B8%83.ipynb) [15_01_VI_变分推断的原理推导](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/15_01_VI_%E5%8F%98%E5%88%86%E6%8E%A8%E6%96%AD%E7%9A%84%E5%8E%9F%E7%90%86%E6%8E%A8%E5%AF%BC.ipynb) [15_02_VI_变分推断与EM的关系](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/15_02_VI_%E5%8F%98%E5%88%86%E6%8E%A8%E6%96%AD%E4%B8%8EEM%E7%9A%84%E5%85%B3%E7%B3%BB.ipynb) [15_03_VI_一元高斯分布的变分推断实现](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/15_03_VI_%E4%B8%80%E5%85%83%E9%AB%98%E6%96%AF%E5%88%86%E5%B8%83%E7%9A%84%E5%8F%98%E5%88%86%E6%8E%A8%E6%96%AD%E5%AE%9E%E7%8E%B0.ipynb) [15_04_VI_高斯混合模型（GMM）的变分推断实现](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/15_04_VI_%E9%AB%98%E6%96%AF%E6%B7%B7%E5%90%88%E6%A8%A1%E5%9E%8B%EF%BC%88GMM%EF%BC%89%E7%9A%84%E5%8F%98%E5%88%86%E6%8E%A8%E6%96%AD%E5%AE%9E%E7%8E%B0.ipynb) [15_05_VI_线性回归模型的贝叶斯估计推导](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/15_05_VI_%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92%E6%A8%A1%E5%9E%8B%E7%9A%84%E8%B4%9D%E5%8F%B6%E6%96%AF%E4%BC%B0%E8%AE%A1%E6%8E%A8%E5%AF%BC.ipynb) [15_06_VI_线性回归模型的贝叶斯估计实现：证据近似](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/15_06_VI_%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92%E6%A8%A1%E5%9E%8B%E7%9A%84%E8%B4%9D%E5%8F%B6%E6%96%AF%E4%BC%B0%E8%AE%A1%E5%AE%9E%E7%8E%B0%EF%BC%9A%E8%AF%81%E6%8D%AE%E8%BF%91%E4%BC%BC.ipynb) [15_07_VI_线性回归模型的变分推断实现](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/15_07_VI_%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92%E6%A8%A1%E5%9E%8B%E7%9A%84%E5%8F%98%E5%88%86%E6%8E%A8%E6%96%AD%E5%AE%9E%E7%8E%B0.ipynb) [15_08_VI_线性回归模型的贝叶斯估计实现：进一步扩展VI](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/15_08_VI_%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92%E6%A8%A1%E5%9E%8B%E7%9A%84%E8%B4%9D%E5%8F%B6%E6%96%AF%E4%BC%B0%E8%AE%A1%E5%AE%9E%E7%8E%B0%EF%BC%9A%E8%BF%9B%E4%B8%80%E6%AD%A5%E6%89%A9%E5%B1%95VI.ipynb) [16_01_LDA_主题模型原理](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/16_01_LDA_%E4%B8%BB%E9%A2%98%E6%A8%A1%E5%9E%8B%E5%8E%9F%E7%90%86.ipynb) [16_02_LDA_Gibss采样实现](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/16_02_LDA_Gibss%E9%87%87%E6%A0%B7%E5%AE%9E%E7%8E%B0.ipynb) [16_03_LDA_变分EM实现](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/16_03_LDA_%E5%8F%98%E5%88%86EM%E5%AE%9E%E7%8E%B0.ipynb) [17_01_FM_因子分解机的原理介绍及实现](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/17_01_FM_%E5%9B%A0%E5%AD%90%E5%88%86%E8%A7%A3%E6%9C%BA%E7%9A%84%E5%8E%9F%E7%90%86%E4%BB%8B%E7%BB%8D%E5%8F%8A%E5%AE%9E%E7%8E%B0.ipynb) [17_02_FM_FFM的原理介绍及实现](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/17_02_FM_FFM%E7%9A%84%E5%8E%9F%E7%90%86%E4%BB%8B%E7%BB%8D%E5%8F%8A%E5%AE%9E%E7%8E%B0.ipynb) [17_03_FM_FFM的损失函数扩展(possion,gamma,tweedie回归实现以及分类实现)](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/17_03_FM_FFM%E7%9A%84%E6%8D%9F%E5%A4%B1%E5%87%BD%E6%95%B0%E6%89%A9%E5%B1%95(possion%2Cgamma%2Ctweedie%E5%9B%9E%E5%BD%92%E5%AE%9E%E7%8E%B0%E4%BB%A5%E5%8F%8A%E5%88%86%E7%B1%BB%E5%AE%9E%E7%8E%B0).ipynb) [18_01_聚类_距离度量以及性能评估](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/18_01_%E8%81%9A%E7%B1%BB_%E8%B7%9D%E7%A6%BB%E5%BA%A6%E9%87%8F%E4%BB%A5%E5%8F%8A%E6%80%A7%E8%83%BD%E8%AF%84%E4%BC%B0.ipynb) [18_02_聚类_层次聚类_AGNES](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/18_02_%E8%81%9A%E7%B1%BB_%E5%B1%82%E6%AC%A1%E8%81%9A%E7%B1%BB_AGNES.ipynb) [18_03_聚类_密度聚类_DBSCAN](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/18_03_%E8%81%9A%E7%B1%BB_%E5%AF%86%E5%BA%A6%E8%81%9A%E7%B1%BB_DBSCAN.ipynb) [18_04_聚类_原型聚类_K均值](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/18_04_%E8%81%9A%E7%B1%BB_%E5%8E%9F%E5%9E%8B%E8%81%9A%E7%B1%BB_K%E5%9D%87%E5%80%BC.ipynb) [18_05_聚类_原型聚类_LVQ](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/18_05_%E8%81%9A%E7%B1%BB_%E5%8E%9F%E5%9E%8B%E8%81%9A%E7%B1%BB_LVQ.ipynb) [18_06_聚类_谱聚类](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/18_06_%E8%81%9A%E7%B1%BB_%E8%B0%B1%E8%81%9A%E7%B1%BB.ipynb) [19_01_降维_奇异值分解(SVD)](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/19_01_%E9%99%8D%E7%BB%B4_%E5%A5%87%E5%BC%82%E5%80%BC%E5%88%86%E8%A7%A3(SVD).ipynb) [19_02_降维_主成分分析(PCA)](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/19_02_%E9%99%8D%E7%BB%B4_%E4%B8%BB%E6%88%90%E5%88%86%E5%88%86%E6%9E%90(PCA).ipynb) [19_03_降维_线性判别分析(LDA)](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/19_03_%E9%99%8D%E7%BB%B4_%E7%BA%BF%E6%80%A7%E5%88%A4%E5%88%AB%E5%88%86%E6%9E%90(LDA).ipynb) [19_04_降维_多维缩放(MDS)](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/19_04_%E9%99%8D%E7%BB%B4_%E5%A4%9A%E7%BB%B4%E7%BC%A9%E6%94%BE(MDS).ipynb) [19_05_降维_流形学习_等度量映射(Isomap)](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/19_05_%E9%99%8D%E7%BB%B4_%E6%B5%81%E5%BD%A2%E5%AD%A6%E4%B9%A0_%E7%AD%89%E5%BA%A6%E9%87%8F%E6%98%A0%E5%B0%84(Isomap).ipynb) [19_06_降维_流形学习_局部线性嵌入(LLE)](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/19_06_%E9%99%8D%E7%BB%B4_%E6%B5%81%E5%BD%A2%E5%AD%A6%E4%B9%A0_%E5%B1%80%E9%83%A8%E7%BA%BF%E6%80%A7%E5%B5%8C%E5%85%A5(LLE).ipynb) [19_07_降维_非负矩阵分解_NMF](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/19_07_%E9%99%8D%E7%BB%B4_%E9%9D%9E%E8%B4%9F%E7%9F%A9%E9%98%B5%E5%88%86%E8%A7%A3_NMF.ipynb) [20_01_异常检测_HBOS](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/20_01_%E5%BC%82%E5%B8%B8%E6%A3%80%E6%B5%8B_HBOS.ipynb) [20_01_异常检测_pHBOS](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/20_01_%E5%BC%82%E5%B8%B8%E6%A3%80%E6%B5%8B_pHBOS.ipynb) [20_02_异常检测_iForest](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/20_02_%E5%BC%82%E5%B8%B8%E6%A3%80%E6%B5%8B_iForest.ipynb) [20_03_异常检测_KNN](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/20_03_%E5%BC%82%E5%B8%B8%E6%A3%80%E6%B5%8B_KNN.ipynb) [20_04_异常检测_LOF](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/20_04_%E5%BC%82%E5%B8%B8%E6%A3%80%E6%B5%8B_LOF.ipynb) ### 参考 《统计学习方法》第二版 --李航 《机器学习》 --周志华 《深入理解XGBoost》 --何龙 《模式识别与机器学习》（PRML） 《徐亦达机器学习课程》 [bilibili传送门>>>](https://www.bilibili.com/video/BV1Qx411W7mf) 《机器学习—白板推导系列》 [bilibili传送门>>>](https://www.bilibili.com/video/BV1Qx411W7mf)