qnloft
/
MediaCrawler
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
391 Commits 2 Branches 0 Tags 21 MiB
Commit Graph

391 Commits

Author SHA1 Message Date
程序员阿江-Relakkes 3725128a35
Merge pull request #273 from nelzomal/bili-level-two-comment 
feat: support bilibili level two comment
 2024-05-26 15:34:28 +08:00
Nan Zhou 0cad36e17b support bilibili level two comment 2024-05-26 14:10:57 +08:00
Relakkes 764bafc626 feat: 抖音登录态检测逻辑更新支持 2024-05-23 22:15:14 +08:00
Relakkes 3c4695eed7 docs: update README.md 2024-05-18 17:09:51 +08:00
Relakkes e64df93edd feat: 由于xhs和dy现在检测playwright二维码登录了，大概率会出现滑块或者手机验证，增加登录态检测时间为5min，预留足够的时间手动过验证码。 2024-05-15 23:23:30 +08:00
Relakkes 46f8cc064a feat: 增加爬虫程序运行前的说明 2024-05-13 21:01:54 +08:00
Relakkes 704ee8cf9d docs: update README.md 2024-05-12 23:28:50 +08:00
Relakkes 60182213fa docs: update README.md 2024-05-10 22:42:41 +08:00
程序员阿江-Relakkes 2df6ebe0d4
Merge pull request #265 from leantli/fix/value_error&typo 
fix: 捕捉处理 ValueError 报错以及修改错字
 2024-05-10 22:28:37 +08:00
leantli 43acde240b fix: 捕捉处理 ValueError 报错以及修改错字 2024-05-08 22:26:35 +08:00
Relakkes 57ee8dd63b docs: update README.md 2024-05-03 18:31:18 +08:00
Relakkes 292cceb4a6 docs: update README.md 2024-05-01 16:56:45 +08:00
程序员阿江-Relakkes 411ed2ead9
Merge pull request #253 from kexinoh/main 
新增id，解决同一天内的不同查询写入同一个文件的问题
 2024-04-30 23:10:16 +08:00
KEXNA 9f8ffe1840 Update weibo_store_impl.py 
Update bilibili_store_impl.py

新增id

Update bilibili_store_impl.py

新增id，解决同一天内的不同查询写入同一个文件的问题

Update douyin_store_impl.py

新增id，解决同一天内的不同查询写入同一个文件的问题

Update kuaishou_store_impl.py

Update weibo_store_impl.py

Update xhs_store_impl.py

Update weibo_store_impl.py

Update kuaishou_store_impl.py

Update bilibili_store_impl.py

Update douyin_store_impl.py

Update kuaishou_store_impl.py

Update xhs_store_impl.py
 2024-04-30 21:42:06 +08:00
程序员阿江-Relakkes 237d5be894
Merge pull request #255 from henryhyn/feat/log_config 
feat: 配置日志打印文件及行号
 2024-04-27 13:18:13 +08:00
Henry He 0a95b7d30b feat: 配置日志打印文件及行号 2024-04-27 12:11:42 +08:00
程序员阿江-Relakkes 2d051cc670
Merge pull request #252 from henryhyn/fix/help-f-string 
fix: 修复 f-string 双引号问题
 2024-04-26 23:58:25 +08:00
Henry He a2dca888ac fix: 修复 f-string 双引号问题 2024-04-26 11:02:55 +08:00
程序员阿江-Relakkes abb82076eb
Merge pull request #250 from bigfa/fixed-trace-id 
fix:兼容网页端上传的图片原图获取
 2024-04-25 22:18:40 +08:00
程序员阿江-Relakkes 829974c7df
Merge pull request #251 from leantli/chore/add_frequency_qa 
chore: 增加常见问题[小红书扫码登录成功后如何手动验证]
 2024-04-25 22:18:16 +08:00
leantli c9c3d59567 chore: 增加常见问题[小红书扫码登录成功后如何手动验证] 2024-04-25 16:23:48 +08:00
bigfa 99d53cd945 fix:兼容网页端上传的图片原图获取 2024-04-24 14:16:35 +08:00
Relakkes Yang e41fc05eac docs: update README.md 2024-04-22 21:25:10 +08:00
Relakkes Yang 48943a25c5 fix: add uvicorn to requirements.txt 2024-04-21 21:41:27 +08:00
Relakkes Yang 59c683a057 refactor: remove tornado server add fastapi 2024-04-21 21:11:46 +08:00
Relakkes 5d97ffac98 feat: 支持初始化mysql表结构 2024-04-18 22:34:51 +08:00
Relakkes 5681dd6925 fix: #237 2024-04-17 23:32:17 +08:00
Relakkes 41fb7bb583 fix: remove kdl secret key 2024-04-17 23:19:44 +08:00
Relakkes 487afc8e0c refactor: 修改导报顺心 2024-04-17 23:13:40 +08:00
Relakkes d0efd4e4f0 feat: 快代理实现 2024-04-17 23:12:41 +08:00
程序员阿江-Relakkes 30ae3514c3
Merge pull request #238 from keeper-jie/patch-1 
Update 常见问题.md
 2024-04-17 21:40:06 +08:00
Kermit db466fd4d0
Update 常见问题.md 
添加抖音cookie登录报错解决方案
 2024-04-17 11:08:04 +08:00
Relakkes 9443e9d28d docs: Update README.md 2024-04-14 20:45:49 +08:00
Relakkes 87eb8aa6a7 fix: #230 2024-04-13 20:18:04 +08:00
Relakkes c26270146f fix: 修复ip代理池设计bug 2024-04-13 18:04:33 +08:00
Relakkes e367129b5f fix: 修复pr#229的小bug 2024-04-13 13:48:29 +08:00
程序员阿江-Relakkes a341dc2aff
Merge pull request #229 from Tianci-King/main 
feat(core): 新增控制爬虫参数起始页面的页数start_page;perf(argparse): 向命令行解析器添加程序参数…
 2024-04-13 13:37:35 +08:00
程序员阿江-Relakkes da949f0c7a
Merge pull request #226 from leantli/feat/sub_comments 
feat: 支持爬取小红书二级评论
 2024-04-13 01:17:55 +08:00
leantli 6cabece01a chore: remove redundant line breaks 2024-04-12 18:18:01 +08:00
leantli ad01dfba95 feat: 轻量化支持爬取小红书二级评论 2024-04-12 17:32:20 +08:00
Tianci-King 1115b0d90c feat(core): 新增控制爬虫 参数起始页面的页数start_page;perf(argparse): 向命令行解析器添加程序参数起始页面页数和关键字 2024-04-12 00:52:47 +08:00
leantli 81a9946afd feat: 支持爬取小红书二级评论 2024-04-11 17:16:13 +08:00
程序员阿江-Relakkes bba9841c26
Merge pull request #223 from Ermeng98/main 
新增对微博博客内照片获取的支持 文件存放路径data/weibo/images
 2024-04-11 00:11:04 +08:00
Er_Meng 9cd6efb916 使用isort对引用进行格式化排序 修改微博获取图片默认配置关闭 2024-04-10 09:54:28 +08:00
Ermeng 52c720591f
Merge branch 'NanmiCoder:main' into main 2024-04-10 09:51:14 +08:00
程序员阿江-Relakkes cab91474ad
Merge pull request #224 from Klu5ure/main 
docs: 恢复文档中误删的字
 2024-04-09 23:14:14 +08:00
Relakkes 488acd04df fix: #222 2024-04-09 23:10:51 +08:00
Klu5ure 4ed5d4d73e docs: 恢复文档中误删的字 2024-04-09 23:00:13 +08:00
Er_Meng 16413c3074 新增对微博博客内照片获取的支持 文件存放路径data/weibo/images 2024-04-09 17:21:52 +08:00
Relakkes 5c409c6f0c docs: 群二维码失效，更新 2024-04-08 23:41:53 +08:00