Styunlen
|
40daa8d6f3
|
chore: fix wrong log output when weibo crawler finished
Scripts output "Bilibili crawler finished" when Weibo crawler finished.
|
2024-04-06 00:41:05 +08:00 |
chunpat
|
6422500e32
|
Remove duplication Qrcode Show
|
2024-04-05 21:24:06 +08:00 |
leantli
|
68a60faa7f
|
chore: 简化判断方式
|
2024-04-04 00:11:22 +08:00 |
|
leantli
|
133f978477
|
fix: 修复爬取视频/帖子最大数设置值较低导致不爬取的问题
|
2024-04-03 12:18:23 +08:00 |
Relakkes
|
e950e0d6e3
|
feat: add abstract api client to all platform
|
2024-03-30 21:27:25 +08:00 |
|
Relakkes
|
67ec49498a
|
refactor: rename xhs to xiaohongshu
|
2024-03-30 21:17:33 +08:00 |
|
Relakkes
|
96309dcfee
|
fix: 小红书创作者功能数据获取优化
|
2024-03-17 14:50:10 +08:00 |
|
Relakkes
|
59cd9f67a0
|
feat: 支持评论模式是否开启爬取选项
|
2024-03-16 11:52:42 +08:00 |
|
Relakkes
|
41fee4ff4f
|
feat：小红书支持获取评论中的图片链接 #145
|
2024-03-07 22:30:44 +08:00 |
|
Relakkes
|
149b6bcdc8
|
fix: 修复抖音关键词搜索为中文的情况下，有bug
|
2024-03-03 19:36:36 +08:00 |
jayeeliu
|
9a5460ccca
|
Merge branch 'NanmiCoder:main' into main
|
2024-03-02 01:51:02 +08:00 |
jayeeliu@gmail.com
|
61ba8c5cc7
|
feat: 小红书支持通过博主ID采集笔记和评论，小红书type=search时支持配置按哪种排序方式获取笔记数据，小红书笔记增加视频地址和标签字段
|
2024-03-02 01:49:42 +08:00 |
|
Relakkes
|
384c8f9f7e
|
fix: issue #140
|
2024-02-26 23:47:02 +08:00 |
|
Relakkes
|
e940a41033
|
refactor: 移除评论中指定数量和过滤特定关键词的逻辑
|
2024-01-17 23:02:05 +08:00 |
|
Relakkes
|
e0f9a487e4
|
refactor: 代码优化
|
2024-01-16 00:40:07 +08:00 |
|
Relakkes
|
894dabcf63
|
refactor: 数据存储重构，分离不同类型的存储实现
|
2024-01-14 22:06:31 +08:00 |
|
Relakkes
|
e31aebbdfb
|
fix: 修复代理Bug
|
2024-01-13 15:50:02 +08:00 |
|
Relakkes
|
4de14ad6a8
|
fix: 修复微博PC端登录后COOKIE在手机端无法使用的bug
|
2024-01-06 19:18:07 +08:00 |
|
Relakkes
|
fe073801f8
|
feat: 小红书增加获取图片地址的方法
|
2024-01-04 00:11:05 +08:00 |
|
Relakkes
|
80ecc53cd3
|
fix: #107
|
2024-01-03 23:08:30 +08:00 |
|
Relakkes
|
38d6f10bf0
|
feat: 微博二维码登录done
|
2023-12-30 18:54:21 +08:00 |
|
Relakkes
|
eee81622ac
|
feat: 微博支持评论 & 指定帖子
|
2023-12-25 00:02:11 +08:00 |
|
Relakkes
|
c5b64fdbf5
|
feat: 微博爬虫帖子搜索完成
|
2023-12-24 17:57:48 +08:00 |
|
Relakkes
|
aba9f14f50
|
refactor: 规范日志打印
feat: B站指定视频ID爬取（bvid）
|
2023-12-23 01:04:08 +08:00 |
|
Relakkes
|
273c9a316b
|
fix: 修复日志打印时参数格式错误
|
2023-12-22 23:10:44 +08:00 |
|
Relakkes
|
2e032d09dc
|
refactor: xhs add log
|
2023-12-19 23:05:12 +08:00 |
|
Relakkes
|
f85fd97d25
|
fix: #10
|
2023-12-16 00:12:20 +08:00 |
peanutsplash
|
5c6a636352
|
fix-bug:修复抖音评论筛选
|
2023-12-14 00:55:06 +08:00 |
|
peanutsplash
|
f17a85305e
|
添加功能:(哔哩哔哩,快手,小红书)每个视频/帖子抓取评论最大条数限制,评论关键词筛选
|
2023-12-13 23:53:12 +08:00 |
|
Relakkes
|
97d7a0c38b
|
feat: Bilibili comment done
|
2023-12-09 21:10:01 +08:00 |
|
Relakkes
|
1cec23f73d
|
feat: 代理IP功能 Done
|
2023-12-08 00:10:04 +08:00 |
|
Relakkes
|
c530bd4219
|
feat: 代理IP缓存到redis中
|
2023-12-06 23:49:56 +08:00 |
|
Relakkes
|
f71d086464
|
fix: B站get_wbi_keys函数类型标注问题
|
2023-12-05 23:32:35 +08:00 |
|
Relakkes
|
a6e877de42
|
fix: 修复B站搜索Field命名 bug
refactor: ping接口统一更换为pong
|
2023-12-05 22:54:47 +08:00 |
|
peanutsplash
|
ab1a10bac1
|
添加功能:抖音每个视频抓取评论最大条数限制,抖音评论关键词筛选
|
2023-12-05 11:21:47 +08:00 |
|
Relakkes
|
8f04943105
|
feat: B站评论API
|
2023-12-04 23:16:02 +08:00 |
|
Relakkes
|
a790d72db9
|
fix: 修复小红书笔记获取时，不存在的笔记异常问题
|
2023-12-04 21:54:12 +08:00 |
|
Relakkes
|
94b5030ef0
|
feat: B站二维码、Cookie登录实现
|
2023-12-04 00:02:00 +08:00 |
|
Relakkes
|
a90b411e68
|
feat: B站爬虫搜索关键词实现
|
2023-12-03 23:19:02 +08:00 |
|
Relakkes
|
5aeee93fc5
|
feat: B站爬虫签名实现
|
2023-12-03 00:30:10 +08:00 |
|
Relakkes
|
5c920da288
|
feat: 快手二维码登录
|
2023-12-02 18:22:55 +08:00 |
|
Relakkes
|
986179b9c9
|
feat: 增加 IP 代理的最新实现
|
2023-12-02 16:14:36 +08:00 |
|
Relakkes
|
33721e5fbd
|
feat: 快手支持指定视频列表爬取
|
2023-11-27 23:07:04 +08:00 |
|
Relakkes
|
62534d7ee2
|
fix: 移出快手 client 多余的代码
|
2023-11-26 22:11:06 +08:00 |
|
Relakkes
|
dfb1788141
|
feat: 快手视频评论爬取done；数据保存到DB、CSV done
|
2023-11-26 21:43:39 +08:00 |
|
Relakkes
|
2f8541a351
|
Merge branch 'main' into feat/kuaishou_support
|
2023-11-26 15:44:26 +08:00 |
|
Relakkes
|
bdf36ccb09
|
feat: 快手关键词搜索存储CSV完成
|
2023-11-26 01:05:52 +08:00 |
|
Relakkes
|
512192a93e
|
feat: 搜索接口调试完成
|
2023-11-25 00:02:33 +08:00 |
|
Relakkes
|
f08b2ceb76
|
feat: 1、命令行支持快手 2、快手playwright 代码 done
|
2023-11-24 00:04:33 +08:00 |
|
Relakkes
|
6908da6f70
|
fix: 修复小红书client中get请求中文bug
|
2023-11-23 23:27:35 +08:00 |