python - Can't able to crawl the HTML data(elements view, not page view)? -
i want crawl html data(elements view using chrome developer tools, not page view).
i crawled html data in python.
- following tried code in python.
#-*- coding: utf-8 -*- import requests url = 'http://m.entertain.naver.com/comment/list?page=2&gno=news117%2c0002600716&sort=newest&aid=0002600716&oid=117' response = requests.get(url) print(response.content) but page view crawled.
- following crawled page view using above python code.
... <div id="social-comment"> <ul class="cmt_lst"> <li class="ld"><span>로딩중입니다.</span></li> </ul> </div> ... i want crawl elements view using python.
- following elements view using chrome developer tools(i want crawl html data).
... <div id="social-comment"> <div class="sc_cmt_wrp" queryid="c1431107741291317890" style="display: block;"> <div id="tabarea"> <ul class="cmt_tab"> <li style="width:50%" class="on"> <a href="#" class="_tab(commentlistpage) _nclicks(rpt.list)">댓글 <span class="_count">22</span></a> </li> <li><a href="#" class="_tab(commentwritingpage) _nclicks(rpt.write)">댓글쓰기</a></li> </ul> </div> <div id="sortoptionarea" style="display: block;"> <div class="cmt_choice"> <input type="radio" name="scmt-sort" id="newest" class="_scmt_sort(newest) _nclicks(rpt.rct)"> <label for="newest" title="최신순" class="_scmt_sort(newest) _nclicks(rpt.rct) on" onclick="javascript:;">최신순</label> <input type="radio" name="scmt-sort" id="oldest" class="_scmt_sort(oldest) _nclicks(rpt.old)"> <label for="oldest" title="과거순" class="_scmt_sort(oldest) _nclicks(rpt.old)" onclick="javascript:;">과거순</label> <input type="radio" name="scmt-sort" id="likability" class="_scmt_sort(likability) _nclicks(rpt.rcm)"> <label for="likability" title="호감순" class="_scmt_sort(likability) _nclicks(rpt.rcm)" onclick="javascript:;">호감순</label> <input type="radio" name="scmt-sort" id="replycount" class="_scmt_sort(replycount) _nclicks(rpt.rpl)"> <label for="replycount" title="답글많은순" class="_scmt_sort(replycount) _nclicks(rpt.rpl)" onclick="javascript:;">답글많은순</label> </div> </div> <div id="noticearea"></div> <div id="commentlistpaginationarea"> <div class="_nocomments _refreshable"></div> <div class="_commentclosed _refreshable"></div> <div id="commentitemarea" class="_refreshable"><ul class="cmt_lst"> <li id="scmt-item-1149361" class=""> <a href="#" class="name _scmt_item(link, usercommentlistpage, gno, news117,0002600716, commentno,1149361) _nclicks(rpt.prf)">ldo1****</a> <p> 서인영 아직도 존예...하....자야는데..</p> <div class="func"> <span class="time">2015.04.28 오후 11:49</span> <span class="mobile">모바일에서 작성</span> | <a href="#" class="action _scmt_item(report,commentno,1149361) _nclicks(rpt.report)">신고</a> </div> <div class="btn_area2"> <div> <a href="#" class="sc_btn _scmt_item(reply,parentcommentno,1149361) _nclicks(rpt.reply)">답글 <strong>0</strong></a> </div> <div> <a href="#" id="scmt-good-comment-1149361" class="sc_btn recomm _scmt_item(good,commentno,1149361) _nclicks(rpt.sym)">10</a> <a href="#" id="scmt-bad-comment-1149361" class="sc_btn recomm2 _scmt_item(bad,commentno,1149361) _nclicks(rpt.opp)">7</a> </div> </div> </li> <li id="scmt-item-1149360" class=""> <a href="#" class="name _scmt_item(link, usercommentlistpage, gno, news117,0002600716, commentno,1149360) _nclicks(rpt.prf)">dbgh****</a> <p> 1빠 댓글수채우기.</p> <div class="func"> <span class="time">2015.04.28 오후 11:49</span> <span class="mobile">모바일에서 작성</span> | <a href="#" class="action _scmt_item(report,commentno,1149360) _nclicks(rpt.report)">신고</a> </div> <div class="btn_area2"> <div> <a href="#" class="sc_btn _scmt_item(reply,parentcommentno,1149360) _nclicks(rpt.reply)">답글 <strong>0</strong></a> </div> <div> <a href="#" id="scmt-good-comment-1149360" class="sc_btn recomm _scmt_item(good,commentno,1149360) _nclicks(rpt.sym)">1</a> <a href="#" id="scmt-bad-comment-1149360" class="sc_btn recomm2 _scmt_item(bad,commentno,1149360) _nclicks(rpt.opp)">5</a> </div> </div> </li> </ul></div> <div id="paginationarea" style="display: block;"> <div class="cmt_pg"> <a href="#" class="cmt_pg_btn scmt-page-prev _nclicks(rpt.prev)" style="display: inline-block;"><span class="cmt_pg_prev">이전</span></a> <span class="cmt_pg_btn uc_vh scmt-page-prev-off" style="display: none;"><span class="cmt_pg_prev">이전</span></span> <em class="cmt_pg_pg _pageinfo">21 - 22 <span class="u_vc">페이지 </span><span class="cmt_pg_total">/ <span class="u_vc">총 </span>22<span class="u_vc"> 페이지</span></span></em> <span class="cmt_pg_btn uc_vh scmt-page-next-off" style="display: inline-block;"><span class="cmt_pg_next">다음</span></span> <a href="#" class="cmt_pg_btn scmt-page-next _nclicks(rpt.next)" style="display: none;"><span class="cmt_pg_next">다음</span></a> </div></div> </div></div></div> ... i don't know java script. if possible, give tell me easy way(python, wget & curl in linux).
Comments
Post a Comment