Skip to content

flylzj/weibo-scrapy

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

基于微博搜索的关键词微博和用户信息爬取

项目依赖

  • scrapy
  • openpyxl
  • sqlalchemy
  • BeautifulSoup
  • pymysql

运行前的准备

  • 在settings.py中修改WEIBO_Q(这个变量是要搜索的关键词)

  • 配置START_TIME和END_TIME,这两个分别是筛选的开始时间和结束时间

  • 从浏览器复制cookies给COOKIES变量

  • 配置MYSQL_SETTING

运行

scrapy crawl weibo_keywrod 或 python run.py

About

a spider based on s.weibo.com

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages