专门用于爬取 github trending 的爬虫
- node 越新越好
- redis 2.8+
修改redis配置文件,添加 notify-keyspace-events Ex 选项
git clone https://github.com/xiedacon/spider-trending.git
cd spider-trending
npm install
npm start
[
{
"name": "language",
"href": "language href",
"repos": [
{
"author": "repository author",
"repoName": "repository name",
"content": "repository introduction",
"programmingLanguage": "repository language",
"starTotal": "total star",
"fork": "fork",
"starIncrease": "increase star"
}
]
}
]
- 定时任务控制
- pm2
- 部署树莓派
- 数据分析(一个月后)