前言
符號表
第一部分概述.........................................................................1
第1章緒論.............................................................................2
1.1樣例:自動駕駛..................................................................3
1.2模式識別與機(jī)器學(xué)習(xí).............................................................5
1.2.1一個(gè)典型的模式識別流程.....................................................5
1.2.2模式識別vs.機(jī)器學(xué)習(xí).......................................................8
1.2.3評估、部署和細(xì)化...........................................................9
1.3本書的結(jié)構(gòu)......................................................................9
習(xí)題.................................................................................12
第2章數(shù)學(xué)背景知識..................................................................14
2.1線性代數(shù).......................................................................14
2.1.1內(nèi)積、范數(shù)、距離和正交性..................................................14
2.1.2角度與不等式..............................................................15
2.1.3向量投影..................................................................16
2.1.4矩陣基礎(chǔ)..................................................................17
2.1.5矩陣乘法..................................................................18
2.1.6方陣的行列式與逆..........................................................19
2.1.7方陣的特征值、特征向量、秩和跡............................................20
2.1.8奇異值分解................................................................22
2.1.9(半)正定實(shí)對稱矩陣.......................................................22
2.2概率............................................................................23
2.2.1基礎(chǔ)......................................................................23
2.2.2聯(lián)合分布、條件分布與貝葉斯定理............................................25
2.2.3期望與方差/協(xié)方差矩陣.....................................................26
2.2.4不等式....................................................................27
2.2.5獨(dú)立性與相關(guān)性............................................................28
2.2.6正態(tài)分布..................................................................29
2.3優(yōu)化與矩陣微積分..............................................................30
2.3.1局部極小、必要條件和矩陣微積分............................................30
2.3.2凸優(yōu)化與凹優(yōu)化............................................................31
2.3.3約束優(yōu)化和拉格朗日乘子法.................................................33
2.4算法復(fù)雜度.....................................................................34
2.5閱讀材料.......................................................................35
習(xí)題.................................................................................35
第3章模式識別系統(tǒng)概述.............................................................39
3.1人臉識別.......................................................................39
3.2一個(gè)簡單的最近鄰分類器.......................................................40
3.2.1訓(xùn)練或?qū)W習(xí)................................................................40
3.2.2測試或預(yù)測................................................................40
3.2.3最近鄰分類器..............................................................41
3.2.4k-近鄰....................................................................42
3.3丑陋的細(xì)節(jié).....................................................................43
3.4制定假設(shè)并化簡................................................................46
3.4.1設(shè)計(jì)工作環(huán)境vs.設(shè)計(jì)復(fù)雜算法.............................................46
3.4.2假設(shè)與簡化................................................................47
3.5一種框架.......................................................................51
3.6閱讀材料.......................................................................51
習(xí)題.................................................................................53
第4章評估............................................................................55
4.1簡單情形中的準(zhǔn)確率和錯誤率..................................................55
4.1.1訓(xùn)練與測試誤差............................................................56
4.1.2過擬合與欠擬合............................................................56
4.1.3使用驗(yàn)證集來選擇超參數(shù)...................................................58
4.1.4交叉驗(yàn)證..................................................................59
4.2最小化代價(jià)/損失...............................................................61
4.2.1正則化....................................................................62
4.2.2代價(jià)矩陣..................................................................62
4.2.3貝葉斯決策理論............................................................63
4.3不平衡問題中的評估............................................................64
4.3.1單個(gè)類別內(nèi)的比率..........................................................64
4.3.2ROC曲線下的面積.........................................................65
4.3.3查準(zhǔn)率、查全率和F值.....................................................66
4.4我們能達(dá)到100％的準(zhǔn)確率嗎?..................................................68
4.4.1貝葉斯錯誤率..............................................................68
4.4.2真實(shí)標(biāo)記..................................................................69
4.4.3偏置-方差分解.............................................................70
4.5對評估結(jié)果的信心..............................................................73
4.5.1為什么要取平均?...........................................................73
4.5.2為什么要報(bào)告樣本標(biāo)準(zhǔn)差?..................................................74
4.5.3比較兩個(gè)分類器............................................................75
4.6閱讀材料.......................................................................79
習(xí)題.................................................................................79
第二部分與領(lǐng)域知識無關(guān)的特征提取.............................................83
第5章主成分分析.....................................................................84
5.1動機(jī)............................................................................84
5.1.1維度與內(nèi)在維度............................................................84
5.1.2降維......................................................................86
5.1.3PCA與子空間方法.........................................................86
5.2PCA降維到零維子空間........................................................86
5.2.1想法-形式化-優(yōu)化實(shí)踐......................................................87
5.2.2一個(gè)簡單的優(yōu)化............................................................87
5.2.3一些注釋..................................................................88
5.3PCA降維到一維子空間........................................................88
5.3.1新的形式化................................................................88
5.3.2最優(yōu)性條件與化簡..........................................................89
5.3.3與特征分解的聯(lián)系..........................................................90
5.3.4解........................................................................91
5.4PCA投影到更多維度...........................................................91
5.5完整的PCA算法...............................................................92
5.6方差的分析.....................................................................93
5.6.1從最大化方差出發(fā)的PCA..................................................94
5.6.2一種更簡單的推導(dǎo)..........................................................95
5.6.3我們需要多少維度呢?.......................................................95
5.7什么時(shí)候使用或不用PCA呢?..................................................96
5.7.1高斯數(shù)據(jù)的PCA..........................................................96
5.7.2非高斯數(shù)據(jù)的PCA........................................................96
5.7.3含異常點(diǎn)數(shù)據(jù)的PCA......................................................98
5.8白化變換.......................................................................98
5.9特征分解vs.SVD..............................................................98
5.10閱讀材料......................................................................99
習(xí)題.................................................................................99
第6章Fisher線性判別..............................................................103
6.1用于二分類的FLD...........................................................104
6.1.1想法:什么是隔得很遠(yuǎn)呢?..................................................104
6.1.2翻譯成數(shù)學(xué)語言...........................................................105
6.1.3散度矩陣vs.協(xié)方差矩陣..................................................107
6.1.4兩種散度矩陣以及FLD的目標(biāo)函數(shù)........................................108
6.1.5優(yōu)化.....................................................................108
6.1.6等等,我們有一條捷徑.....................................................109
6.1.7二分類問題的FLD.......................................................109
6.1.8陷阱:要是SW不可逆呢?..................................................110
6.2用于多類的FLD..............................................................111
6.2.1稍加修改的符號和SW....................................................111
6.2.2SB的候選................................................................111
6.2.3三個(gè)散度矩陣的故事.......................................................112
6.2.4解.......................................................................113
6.2.5找到更多投影方向.........................................................113
6.3閱讀材料......................................................................113
習(xí)題................................................................................114
第三部分分類器與其他工具.......................................................119
第7章支持向量機(jī)...................................................................120
7.1SVM的關(guān)鍵思想..............................................................120
7.1.1簡化它!簡化它!簡化它!..................................................120
7.1.2查找最大（或較大）間隔的分類器...........................................121
7.2可視化并計(jì)算間隔.............................................................122
7.2.1幾何的可視化.............................................................123
7.2.2將間隔作為優(yōu)化來計(jì)算....................................................124
7.3最大化間隔....................................................................124
7.3.1形式化...................................................................125
7.3.2各種簡化.................................................................125
7.4優(yōu)化與求解....................................................................127
7.4.1拉格朗日函數(shù)與KKT條件................................................127
7.4.2SVM的對偶形式..........................................................128
7.4.3最優(yōu)的b值與支持向量....................................................129
7.4.4同時(shí)考慮原始形式與對偶形式..............................................131
7.5向線性不可分問題和多類問題的擴(kuò)展..........................................131
7.5.1不可分問題的線性分類器..................................................132
7.5.2多類SVM...............................................................134
7.6核SVM.......................................................................134
7.6.1核技巧...................................................................135
7.6.2Mercer條件與特征映射....................................................136
7.6.3流行的核函數(shù)與超參數(shù)....................................................137
7.6.4SVM的復(fù)雜度、權(quán)衡及其他...............................................138
7.7閱讀材料......................................................................139
習(xí)題................................................................................139
第8章概率方法......................................................................144
8.1思考問題的概率路線..........................................................144
8.1.1術(shù)語.....................................................................144
8.1.2分布與推斷...............................................................145
8.1.3貝葉斯定理...............................................................145
8.2各種選擇......................................................................146
8.2.1生成式模型vs.判別式模型................................................146
8.2.2參數(shù)化vs.非參數(shù)化.......................................................147
8.2.3該如何看待一個(gè)參數(shù)呢?...................................................148
8.3參數(shù)化估計(jì)....................................................................148
8.3.1最大似然.................................................................148
8.3.2最大后驗(yàn).................................................................150
8.3.3貝葉斯...................................................................151
8.4非參數(shù)化估計(jì)..................................................................153
8.4.1一個(gè)一維的例子...........................................................153
8.4.2直方圖近似中存在的問題..................................................155
8.4.3讓你的樣本無遠(yuǎn)弗屆.......................................................156
8.4.4核密度估計(jì)...............................................................157
8.4.5帶寬選擇.................................................................158
8.4.6多變量KDE.............................................................158
8.5做出決策......................................................................159
8.6閱讀材料......................................................................159
習(xí)題................................................................................160
第9章距離度量與數(shù)據(jù)變換..........................................................163
9.1距離度量和相似度度量........................................................163
9.1.1距離度量.................................................................164
9.1.2向量范數(shù)和度量...........................................................164
9.1.3`p范數(shù)和`p度量.........................................................165
9.1.4距離度量學(xué)習(xí).............................................................167
9.1.5均值作為一種相似度度量..................................................168
9.1.6冪平均核.................................................................170
9.2數(shù)據(jù)變換和規(guī)范化.............................................................171
9.2.1線性回歸.................................................................172
9.2.2特征規(guī)范化...............................................................173
9.2.3數(shù)據(jù)變換.................................................................175
9.3閱讀材料......................................................................177
習(xí)題................................................................................177
第10章信息論和決策樹.............................................................182
10.1前綴碼和霍夫曼樹............................................................182
10.2信息論基礎(chǔ)...................................................................183
10.2.1熵和不確定性...........................................................184
10.2.2聯(lián)合和條件熵...........................................................184
10.2.3互信息和相對熵.........................................................185
10.2.4一些不等式.............................................................186
10.2.5離散分布的熵...........................................................187
10.3連續(xù)分布的信息論............................................................187
10.3.1微分熵.................................................................188
10.3.2多元高斯分布的熵......................................................189
10.3.3高斯分布是最大熵分布..................................................191
10.4機(jī)器學(xué)習(xí)和模式識別中的信息論.............................................192
10.4.1最大熵.................................................................192
10.4.2最小交叉熵.............................................................193
10.4.3特征選擇...............................................................194
10.5決策樹........................................................................195
10.5.1異或問題及其決策樹模型................................................195
10.5.2基于信息增益的結(jié)點(diǎn)劃分................................................197
10.6閱讀材料.....................................................................198
習(xí)題................................................................................199
第四部分處理變化多端的數(shù)據(jù)....................................................203
第11章稀疏數(shù)據(jù)和未對齊數(shù)據(jù)......................................................204
11.1稀疏機(jī)器學(xué)習(xí)................................................................204
11.1.1稀疏PCA?............................................................204
11.1.2使用`1范數(shù)誘導(dǎo)稀疏性.................................................205
11.1.3使用過完備的字典......................................................208
11.1.4其他一些相關(guān)的話題....................................................210
11.2動態(tài)時(shí)間規(guī)整................................................................212
11.2.1未對齊的時(shí)序數(shù)據(jù)......................................................212
11.2.2思路（或準(zhǔn)則）.........................................................213
11.2.3可視化和形式化.........................................................214
11.2.4動態(tài)規(guī)劃...............................................................215
11.3閱讀材料.....................................................................218
習(xí)題................................................................................218
第12章隱馬爾可夫模型.............................................................222
12.1時(shí)序數(shù)據(jù)與馬爾可夫性質(zhì).....................................................222
12.1.1各種各樣的時(shí)序數(shù)據(jù)和模型..............................................222
12.1.2馬爾可夫性質(zhì)...........................................................224
12.1.3離散時(shí)間馬爾可夫鏈....................................................225
12.1.4隱馬爾可夫模型.........................................................227
12.2HMM學(xué)習(xí)中的三個(gè)基本問題................................................228
12.3?、ˉ和評估問題.............................................................229
12.3.1前向變量和算法.........................................................230
12.3.2后向變量和算法.........................................................231
12.4°、±、?和解碼問題..........................................................234
12.4.1°和獨(dú)立解碼的最優(yōu)狀態(tài)................................................234
12.4.2±、?和聯(lián)合解碼的最優(yōu)狀態(tài).............................................235
12.5?和HMM參數(shù)的學(xué)習(xí).......................................................237
12.5.1Baum-Welch:以期望比例來更新?.......................................238
12.5.2如何計(jì)算?.............................................................238
12.6閱讀材料.....................................................................240
習(xí)題................................................................................241
第五部分高階課題.................................................................245
第13章正態(tài)分布.....................................................................246
13.1定義..........................................................................246
13.1.1單變量正態(tài)分布.........................................................246
13.1.2多元正態(tài)分布...........................................................247
13.2符號和參數(shù)化形式............................................................248
13.3線性運(yùn)算與求和..............................................................249
13.3.1單變量的情形...........................................................249
13.3.2多變量的情形...........................................................250
13.4幾何和馬氏距離..............................................................251
13.5條件作用.....................................................................252
13.6高斯分布的乘積..............................................................253
13.7應(yīng)用Ⅰ:參數(shù)估計(jì)............................................................254
13.7.1最大似然估計(jì)...........................................................254
13.7.2貝葉斯參數(shù)估計(jì).........................................................255
13.8應(yīng)用Ⅱ:卡爾曼濾波..........................................................256
13.8.1模型...................................................................256
13.8.2估計(jì)...................................................................257
13.9在本章中有用的數(shù)學(xué).........................................................258
13.9.1高斯積分...............................................................258
13.9.2特征函數(shù)...............................................................259
13.9.3舒爾補(bǔ)&矩陣求逆引理.................................................260
13.9.4向量和矩陣導(dǎo)數(shù).........................................................262
習(xí)題................................................................................263
第14章EM算法的基本思想........................................................266
14.1GMM:一個(gè)工作實(shí)例.........................................................266
14.1.1高斯混合模型...........................................................266
14.1.2基于隱變量的詮釋......................................................267
14.1.3假若我們能觀測到隱變量,那會怎樣?......................................268
14.1.4我們可以模仿先知嗎?...................................................269
14.2EM算法的非正式描述.......................................................270
14.3期望最大化算法..............................................................270
14.3.1聯(lián)合非凹的不完整數(shù)據(jù)對數(shù)似然..........................................271
14.3.2(可能是)凹的完整數(shù)據(jù)對數(shù)似然..........................................271
14.3.3通用EM的推導(dǎo)........................................................272
14.3.4E步和M步...........................................................274
14.3.5EM算法...............................................................275
14.3.6EM能收斂嗎?..........................................................275
14.4EM用于GMM..............................................................276
14.5閱讀材料.....................................................................279
習(xí)題................................................................................279
第15章卷積神經(jīng)網(wǎng)絡(luò)................................................................281
15.1預(yù)備知識.....................................................................281
15.1.1張量和向量化...........................................................282
15.1.2向量微積分和鏈?zhǔn)椒▌t..................................................283
15.2CNN概覽....................................................................283
15.2.1結(jié)構(gòu)...................................................................283
15.2.2前向運(yùn)行...............................................................285
15.2.3隨機(jī)梯度下降...........................................................285
15.2.4誤差反向傳播...........................................................286
15.3層的輸入、輸出和符號.......................................................287
15.4ReLU層......................................................................288
15.5卷積層........................................................................290
15.5.1什么是卷積?............................................................290
15.5.2為什么要進(jìn)行卷積?.....................................................291
15.5.3卷積作為矩陣乘法......................................................293
15.5.4克羅內(nèi)克積.............................................................295
15.5.5反向傳播:更新參數(shù).....................................................296
15.5.6更高維的指示矩陣......................................................297
15.5.7反向傳播:為前一層準(zhǔn)備監(jiān)督信號.........................................298
15.5.8用卷積層實(shí)現(xiàn)全連接層..................................................300
15.6匯合層........................................................................301
15.7案例分析:VGG-16網(wǎng)絡(luò)......................................................303
15.7.1VGG-Verydeep-16......................................................303
15.7.2感受野.................................................................304
15.8CNN的親身體驗(yàn).............................................................305
15.9閱讀材料.....................................................................305
習(xí)題................................................................................305
參考文獻(xiàn)................................................................................309
英文索引................................................................................325
中文索引................................................................................332