环境 | 变量 |
---|---|
系统 | Linux -Ubuntu 18.04 |
Docker | Docker version 19.03.12, build 48a66213fe |
容器 | jonnyan404/mrdoc-nginx lastest |
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/qrcodejs/qrcode.min.js?version=0.6.6 (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 655] during GET /static/qrcodejs/qrcode.min.js?version=0.6.6 (172.17.0.1)
OSError: write error
[pid: 20|app: 0|req: 16603/31885] 172.17.0.1 () {58 vars in 1945 bytes} [Fri May 14 21:36:42 2021] GET /static/qrcodejs/qrcode.min.js?version=0.6.6 => generated 0 bytes in 11 msecs via sendfile() (HTTP/1.0 200) 6 headers in 213 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/mrdoc/mrdoc-docs.js?version=0.6.6 HTTP/1.0" 200 3883 "https://apisev.cn/project-21/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/mrdoc/mrdoc-docs.js?version=0.6.6 (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/mrdoc/mrdoc-docs.js?version=0.6.6 (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 655] during GET /static/mrdoc/mrdoc-docs.js?version=0.6.6 (172.17.0.1)
OSError: write error
[pid: 19|app: 0|req: 15283/31886] 172.17.0.1 () {58 vars in 1939 bytes} [Fri May 14 21:36:42 2021] GET /static/mrdoc/mrdoc-docs.js?version=0.6.6 => generated 0 bytes in 12 msecs via sendfile() (HTTP/1.0 200) 6 headers in 213 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/iconFont/iconfont.js?version=0.6.6 HTTP/1.0" 200 3882 "https://apisev.cn/project-21/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/iconFont/iconfont.js?version=0.6.6 (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/iconFont/iconfont.js?version=0.6.6 (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 655] during GET /static/iconFont/iconfont.js?version=0.6.6 (172.17.0.1)
OSError: write error
[pid: 20|app: 0|req: 16604/31887] 172.17.0.1 () {58 vars in 1941 bytes} [Fri May 14 21:36:42 2021] GET /static/iconFont/iconfont.js?version=0.6.6 => generated 0 bytes in 12 msecs via sendfile() (HTTP/1.0 200) 6 headers in 214 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/editor.md/lib/marked.min.js?version=0.6.6 HTTP/1.0" 200 3883 "https://apisev.cn/project-21/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/editor.md/lib/marked.min.js?version=0.6.6 (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/editor.md/lib/marked.min.js?version=0.6.6 (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 655] during GET /static/editor.md/lib/marked.min.js?version=0.6.6 (172.17.0.1)
OSError: write error
[pid: 19|app: 0|req: 15284/31888] 172.17.0.1 () {58 vars in 1955 bytes} [Fri May 14 21:36:42 2021] GET /static/editor.md/lib/marked.min.js?version=0.6.6 => generated 0 bytes in 12 msecs via sendfile() (HTTP/1.0 200) 6 headers in 213 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/editor.md/editormd.js?version=0.6.6 HTTP/1.0" 200 3882 "https://apisev.cn/project-21/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/editor.md/editormd.js?version=0.6.6 (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/editor.md/editormd.js?version=0.6.6 (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 655] during GET /static/editor.md/editormd.js?version=0.6.6 (172.17.0.1)
OSError: write error
[pid: 20|app: 0|req: 16605/31889] 172.17.0.1 () {58 vars in 1943 bytes} [Fri May 14 21:36:42 2021] GET /static/editor.md/editormd.js?version=0.6.6 => generated 0 bytes in 12 msecs via sendfile() (HTTP/1.0 200) 6 headers in 214 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/dashang/dashang_wx.webp HTTP/1.0" 200 3895 "https://apisev.cn/project-21/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/dashang/dashang_wx.webp (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/dashang/dashang_wx.webp (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 655] during GET /static/dashang/dashang_wx.webp (172.17.0.1)
OSError: write error
[pid: 19|app: 0|req: 15285/31890] 172.17.0.1 () {58 vars in 1969 bytes} [Fri May 14 21:36:42 2021] GET /static/dashang/dashang_wx.webp => generated 0 bytes in 12 msecs via sendfile() (HTTP/1.0 200) 6 headers in 201 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/dashang/dashang_alipay.webp HTTP/1.0" 200 3895 "https://apisev.cn/project-21/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/dashang/dashang_alipay.webp (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/dashang/dashang_alipay.webp (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 655] during GET /static/dashang/dashang_alipay.webp (172.17.0.1)
OSError: write error
[pid: 20|app: 0|req: 16606/31891] 172.17.0.1 () {58 vars in 1977 bytes} [Fri May 14 21:36:42 2021] GET /static/dashang/dashang_alipay.webp => generated 0 bytes in 11 msecs via sendfile() (HTTP/1.0 200) 6 headers in 201 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/dashang/dashang_qq.webp HTTP/1.0" 200 3895 "https://apisev.cn/project-21/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/dashang/dashang_qq.webp (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/dashang/dashang_qq.webp (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 655] during GET /static/dashang/dashang_qq.webp (172.17.0.1)
OSError: write error
[pid: 19|app: 0|req: 15286/31892] 172.17.0.1 () {58 vars in 1969 bytes} [Fri May 14 21:36:42 2021] GET /static/dashang/dashang_qq.webp => generated 0 bytes in 12 msecs via sendfile() (HTTP/1.0 200) 6 headers in 201 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/dashang/dashang_paypal.png HTTP/1.0" 200 3897 "https://apisev.cn/project-21/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/dashang/dashang_paypal.png (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/dashang/dashang_paypal.png (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 655] during GET /static/dashang/dashang_paypal.png (172.17.0.1)
OSError: write error
[pid: 20|app: 0|req: 16607/31893] 172.17.0.1 () {58 vars in 1975 bytes} [Fri May 14 21:36:42 2021] GET /static/dashang/dashang_paypal.png => generated 0 bytes in 10 msecs via sendfile() (HTTP/1.0 200) 6 headers in 199 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/layui/font/iconfont.woff2?v=256 HTTP/1.0" 200 3895 "https://apisev.cn/static/layui/css/layui.css?version=0.6.6" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/layui/font/iconfont.woff2?v=256 (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/layui/font/iconfont.woff2?v=256 (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 655] during GET /static/layui/font/iconfont.woff2?v=256 (172.17.0.1)
OSError: write error
[pid: 19|app: 0|req: 15287/31894] 172.17.0.1 () {60 vars in 1991 bytes} [Fri May 14 21:36:42 2021] GET /static/layui/font/iconfont.woff2?v=256 => generated 0 bytes in 7 msecs via sendfile() (HTTP/1.0 200) 6 headers in 201 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/layui/css/modules/laydate/default/laydate.css?v=5.0.9 HTTP/1.0" 200 3898 "https://apisev.cn/project-21/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/layui/css/modules/laydate/default/laydate.css?v=5.0.9 (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/layui/css/modules/laydate/default/laydate.css?v=5.0.9 (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 655] during GET /static/layui/css/modules/laydate/default/laydate.css?v=5.0.9 (172.17.0.1)
OSError: write error
[pid: 20|app: 0|req: 16608/31895] 172.17.0.1 () {58 vars in 1993 bytes} [Fri May 14 21:36:42 2021] GET /static/layui/css/modules/laydate/default/laydate.css?v=5.0.9 => generated 0 bytes in 12 msecs via sendfile() (HTTP/1.0 200) 6 headers in 198 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/layui/css/modules/layer/default/layer.css?v=3.1.1 HTTP/1.0" 200 3897 "https://apisev.cn/project-21/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/layui/css/modules/layer/default/layer.css?v=3.1.1 (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/layui/css/modules/layer/default/layer.css?v=3.1.1 (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 655] during GET /static/layui/css/modules/layer/default/layer.css?v=3.1.1 (172.17.0.1)
OSError: write error
[pid: 19|app: 0|req: 15288/31896] 172.17.0.1 () {58 vars in 1985 bytes} [Fri May 14 21:36:42 2021] GET /static/layui/css/modules/layer/default/layer.css?v=3.1.1 => generated 0 bytes in 13 msecs via sendfile() (HTTP/1.0 200) 6 headers in 199 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/layui/css/modules/code.css HTTP/1.0" 200 1063 "https://apisev.cn/project-21/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
[pid: 20|app: 0|req: 16609/31897] 172.17.0.1 () {58 vars in 1940 bytes} [Fri May 14 21:36:42 2021] GET /static/layui/css/modules/code.css => generated 1063 bytes in 5 msecs via sendfile() (HTTP/1.0 200) 6 headers in 198 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "POST /get_pro_doc_tree/ HTTP/1.0" 200 338 "https://apisev.cn/project-21/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
[pid: 19|app: 0|req: 15289/31898] 172.17.0.1 () {66 vars in 2106 bytes} [Fri May 14 21:36:42 2021] POST /get_pro_doc_tree/ => generated 338 bytes in 12 msecs (HTTP/1.0 200) 5 headers in 159 bytes (1 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/editor.md/lib/raphael.min.js HTTP/1.0" 200 3883 "https://apisev.cn/project-21/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/editor.md/lib/raphael.min.js (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/editor.md/lib/raphael.min.js (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 655] during GET /static/editor.md/lib/raphael.min.js (172.17.0.1)
OSError: write error
[pid: 20|app: 0|req: 16610/31899] 172.17.0.1 () {58 vars in 1930 bytes} [Fri May 14 21:36:42 2021] GET /static/editor.md/lib/raphael.min.js => generated 0 bytes in 12 msecs via sendfile() (HTTP/1.0 200) 6 headers in 213 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/editor.md/lib/purify.min.js HTTP/1.0" 200 3883 "https://apisev.cn/project-21/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/editor.md/lib/purify.min.js (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/editor.md/lib/purify.min.js (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 655] during GET /static/editor.md/lib/purify.min.js (172.17.0.1)
OSError: write error
[pid: 19|app: 0|req: 15290/31900] 172.17.0.1 () {58 vars in 1928 bytes} [Fri May 14 21:36:42 2021] GET /static/editor.md/lib/purify.min.js => generated 0 bytes in 9 msecs via sendfile() (HTTP/1.0 200) 6 headers in 213 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/layui/css/modules/layer/default/loading-1.gif HTTP/1.0" 200 701 "https://apisev.cn/static/layui/css/modules/layer/default/layer.css?v=3.1.1" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
[pid: 20|app: 0|req: 16611/31901] 172.17.0.1 () {58 vars in 2058 bytes} [Fri May 14 21:36:42 2021] GET /static/layui/css/modules/layer/default/loading-1.gif => generated 701 bytes in 5 msecs via sendfile() (HTTP/1.0 200) 6 headers in 198 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/editor.md/lib/underscore.min.js HTTP/1.0" 200 3883 "https://apisev.cn/project-21/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/editor.md/lib/underscore.min.js (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/editor.md/lib/underscore.min.js (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 655] during GET /static/editor.md/lib/underscore.min.js (172.17.0.1)
OSError: write error
[pid: 19|app: 0|req: 15291/31902] 172.17.0.1 () {58 vars in 1936 bytes} [Fri May 14 21:36:42 2021] GET /static/editor.md/lib/underscore.min.js => generated 0 bytes in 6 msecs via sendfile() (HTTP/1.0 200) 6 headers in 213 bytes (0 switches on core 0)
172.17.0.1 - - [14/May/2021:21:36:42 +0800] "GET /static/editor.md/lib/prettify.min.js HTTP/1.0" 200 3883 "https://apisev.cn/project-21/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56" "119.186.210.55"
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/editor.md/lib/prettify.min.js (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /static/editor.md/lib/prettify.min.js (ip 172.17.0.1) !!!
Fri May 14 21:36:42 2021 - uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 655] during GET /static/editor.md/lib/prettify.min.js (172.17.0.1)
OSError: write error
[pid: 20|app: 0|req: 16612/31903] 172.17.0.1 () {58 vars in 1932 bytes} [Fri May 14 21:36:42 2021] GET /static/editor.md/lib/prettify.min.js => generated 0 bytes in 7 msecs via sendfile() (HTTP/1.0 200) 6 headers in 213 bytes (0 switches on core 0)
附上导致失败的markdown内容
数据源
爬虫要想爬取数据首先提供爬取数据的路径,在这里我们爬取本课程的课程介绍页面。
url:http://www.hubwiz.com/course/562427361bc20c980538e26f
爬虫目标
课程介绍页面中最重要的信息就是每一章节的标题及其中小节的标题名称。我们的目标就是把它爬取出来,然后输出。
小爬虫
提供了爬取路径之后,我们首先利用 http 模块中get 和 request 爬取路径页面中的内容。
var http = require('http');
var url = 'http://www.hubwiz.com/course/5437538a032c781670afddbe/';
http.get(url, function(res){
var html = '';
res.on('data', function(data){
html += data;
})
res.on('end',function(){
console.log(html);
})
}).on('error', function(){
console.log('爬取页面错误');
});
解析:示例中首先引用了 nodejs 的核心模块 http 和提供了爬取路径,然后通过 http 中的 get 接口给 url 发送 get 请求,最回调函数中对请求回来的数据进行处理。
对 html 进行解析,这个时候就用到了 cheerio 这个模块了。
使用方法:var cheerio = require('cheerio');
现在来看一下我们路径中的 html 目标代码:
分析:上述代码中每一个章节都是被包含在 class 为 panel 的标签中。所以我们要对 class 为 panel 的标签进行遍历。
在上节中我们说要对 class 为 panel 的集合进行遍历,但是我们的目标只是每一章的标题和小节的标题。
分析:在上图中的两个红色圆圈中就是我们要爬取的目标,第一个是我们章的标题,第二个是小节的标题。那么我们应该怎样去获取这些标签中内容呢?
获取章标题:$('.panel').find('h4').text();
获取节标题:$('.panel').find('li').text();
在前面的小节中我们把准备工作和 html 代码分析都完成了。现在开始爬取数据。
把对 html 解析的工作封成一个方法,如下:
function crawlerChapter(html) {
var $ = cheerio.load(html);
var chapters = $('.panel');
var data = [];
chapters.map(function (node) {
var chapters = $(this);
var chapterTitle = chapters.find('h4').text().trim();
var sections = chapters.find('li');
var chapterData = {
chapterTitle: chapterTitle,
section: []
};
sections.map(function (node) {
var section = $(this).text().trim();
chapterData.section.push(section);
});
data.push(chapterData);
});
console.log(data);
};
解析:首先通过cheerio的 load 方法把html加载;然后对*.panel*通过 map 进行遍历。之后我们在 map 中 组装 要数据格式,如上述中 chapterData。再对 小节 li 进行遍历,把 section 通过 push 方法 添加到 chapterData.section 的数组中。再把组装好的数据 push 到 我们创建的空数组 data 中。最后通过console.log进行输出。
输出数据之后,我们看到数据中除了第一章中有section有值,其他的章没有值,不要担心,就是这样的。因为我们汇智网对其进行了限制,所以只能爬取到第一章中的小节标题。
在大多数情况下我们爬取出来的数据,可能不是我们最终想要的东西比如说:数据中空值或者空格等等。
如果你细心的话,不难发现我们在上一个小节中在获取在通过 text 获取内容的后面跟随着一个 trim 的方法。这个方法的作用就是处理数据中空格和换行符。
如果你想验证一下效果你可以把右边练习环境中应用到 trim 方法的地方去掉trim方法。
空值的情况,在上一小节我们输出的数据中存在一个空的数组对象,那么我们通过什么方法去处理它呢?这时我们就用到了 filter 方法。
PS:filter方法我们在cheerio API(二) 过滤DOM小节 中已经讲解到了。
代码:
data.filter(function(obj){
return obj.chapterTitle ? true : false;
});
现在把我们在 crawlerChapter 方法中得到的数据 data 组装,进行输出:
function printInfo(data) {
data = data.filter(function filterByID(obj) {
return obj.chapterTitle ? true : false;
});
data.map(function (item) {
var chapterTitle = item.chapterTitle;
console.log('【' + chapterTitle + '】\n');
item.section.map(function (section) {
console.log(' 【' + section + '】\n');
});
})
};
方法解析:在 printInfo 方法中的参数 data ,这个参数需要 crawlerChapter 方法 return 给printInfo。然后就是 data 参数调用 filter 方法把数据为空的去掉。最后就是把章节拼接字符串进行输出。
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。
登录 后才可以发表评论