Commit Graph

24 Commits

Author SHA1 Message Date
Nan Zhou 0cad36e17b support bilibili level two comment 2024-05-26 14:10:57 +08:00
Relakkes e64df93edd feat: 由于xhs和dy现在检测playwright二维码登录了,大概率会出现滑块或者手机验证,增加登录态检测时间为5min,预留足够的时间手动过验证码。 2024-05-15 23:23:30 +08:00
Relakkes 87eb8aa6a7 fix: #230 2024-04-13 20:18:04 +08:00
Tianci-King 1115b0d90c feat(core): 新增控制爬虫 参数起始页面的页数start_page;perf(argparse): 向命令行解析器添加程序参数起始页面页数和关键字 2024-04-12 00:52:47 +08:00
leantli 68a60faa7f chore: 简化判断方式 2024-04-04 00:11:22 +08:00
leantli 133f978477 fix: 修复爬取视频/帖子最大数设置值较低导致不爬取的问题 2024-04-03 12:18:23 +08:00
Relakkes e950e0d6e3 feat: add abstract api client to all platform 2024-03-30 21:27:25 +08:00
Relakkes 59cd9f67a0 feat: 支持评论模式是否开启爬取选项 2024-03-16 11:52:42 +08:00
Relakkes 894dabcf63 refactor: 数据存储重构,分离不同类型的存储实现 2024-01-14 22:06:31 +08:00
Relakkes e31aebbdfb fix: 修复代理Bug 2024-01-13 15:50:02 +08:00
Relakkes 80ecc53cd3 fix: #107 2024-01-03 23:08:30 +08:00
Relakkes c5b64fdbf5 feat: 微博爬虫帖子搜索完成 2023-12-24 17:57:48 +08:00
Relakkes aba9f14f50 refactor: 规范日志打印
feat: B站指定视频ID爬取(bvid)
2023-12-23 01:04:08 +08:00
Relakkes 273c9a316b fix: 修复日志打印时参数格式错误 2023-12-22 23:10:44 +08:00
peanutsplash f17a85305e 添加功能:(哔哩哔哩,快手,小红书)每个视频/帖子抓取评论最大条数限制,评论关键词筛选 2023-12-13 23:53:12 +08:00
Relakkes 97d7a0c38b feat: Bilibili comment done 2023-12-09 21:10:01 +08:00
Relakkes 1cec23f73d feat: 代理IP功能 Done 2023-12-08 00:10:04 +08:00
Relakkes c530bd4219 feat: 代理IP缓存到redis中 2023-12-06 23:49:56 +08:00
Relakkes f71d086464 fix: B站get_wbi_keys函数类型标注问题 2023-12-05 23:32:35 +08:00
Relakkes a6e877de42 fix: 修复B站搜索Field命名 bug
refactor: ping接口统一更换为pong
2023-12-05 22:54:47 +08:00
Relakkes 8f04943105 feat: B站评论API 2023-12-04 23:16:02 +08:00
Relakkes 94b5030ef0 feat: B站二维码、Cookie登录实现 2023-12-04 00:02:00 +08:00
Relakkes a90b411e68 feat: B站爬虫搜索关键词实现 2023-12-03 23:19:02 +08:00
Relakkes 5aeee93fc5 feat: B站爬虫签名实现 2023-12-03 00:30:10 +08:00