Python大數(shù)據(jù)分析與應(yīng)用實(shí)戰(zhàn)

第 1 章 Python 語(yǔ)法基礎(chǔ) ··························· 1
1.1 安裝 Anaconda ····································· 1
1.1.1 代碼提示 ······························· 4
1.1.2 變量瀏覽 ······························· 5
1.1.3 安裝第三方庫(kù) ························· 5
1.2 語(yǔ)法基礎(chǔ) ············································ 6
1.2.1 字符串、列表、元組、字典和
集合 ····································· 6
1.2.2 條件判斷、循環(huán)和函數(shù) ··········· 13
1.2.3 異常 ··································· 17
1.2.4 特殊函數(shù) ····························· 20
1.3 Python 基礎(chǔ)庫(kù)應(yīng)用入門(mén) ························ 22
1.3.1 NumPy 庫(kù)應(yīng)用入門(mén) ················ 23
1.3.2 Pandas 庫(kù)應(yīng)用入門(mén) ················· 29
1.3.3 Matplotlib 庫(kù)應(yīng)用入門(mén) ············· 40
1.4 本章小結(jié) ·········································· 45
第 2 章 天氣數(shù)據(jù)的獲取與建模分析 ·········· 52
2.1 準(zhǔn)備工作 ·········································· 52
2.2 利用抓取方法獲取天氣數(shù)據(jù) ·················· 54
2.2.1 網(wǎng)頁(yè)解析 ····························· 54
2.2.2 抓取一個(gè)靜態(tài)頁(yè)面中的天氣
數(shù)據(jù) ··································· 57
2.2.3 抓取歷史天氣數(shù)據(jù) ················· 60
2.3 天氣數(shù)據(jù)可視化 ································· 63
2.3.1 查看數(shù)據(jù)基本信息 ················· 63
2.3.2 變換數(shù)據(jù)格式 ······················· 64
2.3.3 氣溫走勢(shì)的折線圖 ················· 66
2.3.4 歷年氣溫對(duì)比圖 ···················· 67
2.3.5 天氣情況的柱狀圖 ················· 69
2.3.6 使用 Tableau 制作天氣情況的
氣泡云圖 ····························· 70
2.3.7 風(fēng)向占比的餅圖 ···················· 73
2.3.8 使用 windrose 庫(kù)繪制風(fēng)玫瑰圖 ·· 74
2.4 機(jī)器學(xué)習(xí)在天氣預(yù)報(bào)中的應(yīng)用 ··············· 76
2.4.1 線性回歸的基本概念 ·············· 76
2.4.2 使用一元線性回歸預(yù)測(cè)氣溫 ····· 77
2.4.3 使用多元線性回歸預(yù)測(cè)氣溫 ····· 85
2.5 本章小結(jié) ·········································· 91
第 3 章 養(yǎng)成游戲中人物的數(shù)據(jù)搭建 ·········· 92
3.1 準(zhǔn)備工作 ·········································· 92
3.2 利用 Pyecharts 庫(kù)進(jìn)行數(shù)據(jù)基本情況分析 ··· 93
3.2.1 感染人數(shù)分布圖 ···················· 94
3.2.2 病情分布圖 ·························· 96
3.2.3 病癥情況堆疊圖 ···················· 97
3.2.4 繪制出院、死亡情況折線圖 ····· 98
3.2.5 病情熱力圖 ························· 100
3.2.6 病情分布象形圖 ··················· 101
3.2.7 人口流動(dòng)示意圖 ··················· 103
| Python 大數(shù)據(jù)分析與應(yīng)用實(shí)戰(zhàn) |
VI
3.3 感染病例分析 ··································· 105
3.3.1 基本信息統(tǒng)計(jì) ······················ 106
3.3.2 使用直方圖展示感染周期 ······· 108
3.3.3 使用詞云圖展示死亡病例情況 ··· 111
3.4 疫情趨勢(shì)預(yù)測(cè) ··································· 114
3.4.1 利用邏輯方程預(yù)測(cè)感染人數(shù) ···· 115
3.4.2 利用 SIR 模型進(jìn)行疫情預(yù)測(cè) ···· 120
3.4.3 Logistic 模型和 SIR 模型的
對(duì)比 ·································· 128
3.5 本章小結(jié) ········································· 131
第 4 章 航空數(shù)據(jù)分析 ···························· 132
4.1 準(zhǔn)備工作 ········································· 132
4.2 基本情況統(tǒng)計(jì)分析 ····························· 135
4.2.1 查看數(shù)據(jù)的基本信息 ············· 135
4.2.2 航空公司、機(jī)型分布 ············· 137
4.2.3 展示各個(gè)城市航班數(shù)量的 3D
地圖 ·································· 139
4.2.4 從首都機(jī)場(chǎng)出發(fā)的?；鶊D ······· 142
4.2.5 通過(guò)關(guān)系圖展示航線 ············· 145
4.3 利用 Floyd 算法計(jì)算最短飛行時(shí)間 ········· 148
4.3.1 Floyd 算法簡(jiǎn)介 ···················· 148
4.3.2 Floyd 算法的流程 ················· 150
4.3.3 算法程序?qū)崿F(xiàn) ······················ 150
4.3.4 結(jié)果分析 ···························· 154
4.4 本章小結(jié) ········································· 158
第 5 章 市民服務(wù)熱線文本數(shù)據(jù)分析 ········· 160
5.1 準(zhǔn)備工作 ········································· 160
5.2 基本情況分析 ··································· 162
5.2.1 數(shù)據(jù)分布基本信息 ················ 162
5.2.2 每日平均工單量分析 ············· 165
5.2.3 來(lái)電時(shí)間分析 ······················ 166
5.2.4 工單類(lèi)型分析 ······················ 167
5.3 利用詞云圖展示工單內(nèi)容 ···················· 171
5.3.1 工單分詞 ···························· 171
5.3.2 去除停用詞 ························· 172
5.3.3 詞頻統(tǒng)計(jì) ···························· 173
5.3.4 市民反映問(wèn)題詞云圖 ············· 175
5.3.5 保存數(shù)據(jù) ···························· 176
5.4 基于樸素貝葉斯的工單自動(dòng)分類(lèi)轉(zhuǎn)辦 ····· 177
5.4.1 需求概述 ···························· 177
5.4.2 樸素貝葉斯模型的基本概念 ···· 177
5.4.3 樸素貝葉斯文本分類(lèi)算法的
流程 ·································· 181
5.4.4 程序?qū)崿F(xiàn) ···························· 182
5.5 基于 K-Means 算法和 PCA 方法降維的
熱點(diǎn)問(wèn)題挖掘 ··································· 189
5.5.1 應(yīng)用場(chǎng)景 ···························· 189
5.5.2 K-Means 算法和 PCA 方法的
基本原理 ···························· 189
5.5.3 熱點(diǎn)問(wèn)題挖掘算法的流程 ······· 193
5.5.4 程序?qū)崿F(xiàn) ···························· 194
5.6 本章小結(jié) ········································· 205
第 6 章 決策樹(shù)信貸風(fēng)險(xiǎn)控制 ·················· 206
6.1 準(zhǔn)備工作 ········································· 206
6.2 數(shù)據(jù)集基本情況分析 ·························· 209
6.2.1 查看數(shù)據(jù)大小和缺失情況 ······· 209
6.2.2 繪制直方圖查看數(shù)據(jù)的分布
情況 ·································· 211
6.2.3 繪制直方圖的 3 種方法 ·········· 212
| 目錄 |
VII
6.2.4 通過(guò)箱型圖查看異常值的情況 ···· 213
6.2.5 異常值和缺失值的處理 ·········· 217
6.2.6 使用小提琴圖展示預(yù)處理后的
數(shù)據(jù) ·································· 218
6.3 利用決策樹(shù)進(jìn)行信貸數(shù)據(jù)建模 ·············· 219
6.3.1 決策樹(shù)原理簡(jiǎn)介 ··················· 219
6.3.2 決策樹(shù)信貸建模流程 ············· 225
6.3.3 利用 scikit-learn 庫(kù)實(shí)現(xiàn)決策樹(shù)
風(fēng)險(xiǎn)控制算法 ······················ 226
6.3.4 模型優(yōu)化 ···························· 231
6.4 本章小結(jié) ········································· 233
第 7 章 利用深度學(xué)習(xí)進(jìn)行垃圾圖片分類(lèi) ···· 234
7.1 準(zhǔn)備工作 ········································· 234
7.2 深度學(xué)習(xí)的基本原理 ·························· 237
7.2.1 CNN 的基本原理 ·················· 237
7.2.2 Keras 庫(kù)簡(jiǎn)介 ······················· 240
7.3 利用 Keras 庫(kù)實(shí)現(xiàn)基于 CNN 的垃圾
圖片分類(lèi) ········································ 241
7.3.1 算法流程 ···························· 241
7.3.2 數(shù)據(jù)預(yù)處理 ························· 241
7.3.3 CNN 模型實(shí)現(xiàn) ····················· 247
7.4 優(yōu)化 CNN 模型 ································· 252
7.4.1 選擇優(yōu)化器 ························· 252
7.4.2 選擇損失函數(shù) ······················ 254
7.4.3 調(diào)整模型 ···························· 256
7.4.4 圖片增強(qiáng) ···························· 259
7.4.5 改變學(xué)習(xí)率 ························· 263
7.5 模型應(yīng)用 ········································· 265
7.6 本章小結(jié) ········································· 268
第 8 章 協(xié)同過(guò)濾和矩陣分解推薦算法
分析 ········································· 269
8.1 準(zhǔn)備工作 ········································· 269
8.2 基于協(xié)同過(guò)濾算法的短視頻完播情況
分析 ··············································· 271
8.2.1 基于用戶的協(xié)同過(guò)濾算法的
原理 ·································· 271
8.2.2 算法流程 ···························· 274
8.2.3 程序?qū)崿F(xiàn) ···························· 275
8.3 基于矩陣分解算法的短視頻完播情況
預(yù)測(cè) ·············································· 283
8.3.1 算法原理 ···························· 283
8.3.2 利用 Surprise 庫(kù)實(shí)現(xiàn) SVD
算法 ·································· 286
8.4 幾種方法在測(cè)試集中的表現(xiàn) ················· 289
8.5 本章小結(jié) ········································· 291
第 9 章 《紅樓夢(mèng)》文本數(shù)據(jù)分析 ············ 292
9.1 準(zhǔn)備工作 ········································· 292
9.1.1 編程環(huán)境 ···························· 292
9.1.2 數(shù)據(jù)情況簡(jiǎn)介 ······················ 293
9.2 分詞 ··············································· 294
9.2.1 讀取數(shù)據(jù) ···························· 295
9.2.2 數(shù)據(jù)預(yù)處理 ························· 298
9.2.3 分詞及去除停用詞 ················ 306
9.2.4 制作詞云圖 ························· 307
9.3 文本聚類(lèi)分析 ··································· 316
9.3.1 構(gòu)建分詞 TF-IDF 矩陣 ··········· 317
9.3.2 K-Means 聚類(lèi) ······················ 318
9.3.3 MDS 降維 ··························· 320
9.3.4 PCA 降維 ··························· 321
| Python 大數(shù)據(jù)分析與應(yīng)用實(shí)戰(zhàn) |
VIII
9.3.5 HC 聚類(lèi) ····························· 323
9.3.6 t -SNE 高維數(shù)據(jù)可視化 ·········· 325
9.4 LDA 主題模型 ·································· 326
9.5 人物社交網(wǎng)絡(luò)分析 ····························· 332
9.6 本章小結(jié) ········································· 338
附錄 A 抓取數(shù)據(jù)請(qǐng)求頭查詢 ··················· 339
附錄 B GraphViz 庫(kù)的安裝方法 ·············· 341
附錄 C 在 Windows 10 中安裝 TensorFlow
的方法 ······································ 343
參考文獻(xiàn) ··············································· 346
致謝 ····················································· 34