Mediacrawler
Mediacrawler
spider
browser side JavaScript
encryption bypass
Douyin
Kuaishou
Bilibili
Weibo
Zhihu spiders
Captcha cracking tool
Mediacrawler is a powerful spider that employs browser-side JavaScript execution to circumvent encryption and perform valid requests. It can crawl popular short video platforms like Douyin, Kuaishou, Bilibili, and Weibo, as well as extract comments using comment spiders. Additionally, it supports Zhihu spiders with similar logic. The tool also includes a GitHub-sourced Captcha cracking tool to handle any CAPTCHAs encountered during the crawling process.
easyspider
Mediacrawler utilizes browser side javascript execution to circumvent encryption algorithms and issues valid requests.
可以爬取的内容: 小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 ｜ 评论爬虫、微博帖子 ｜ 评论爬虫
知乎爬虫可以按照类似的逻辑进行编写