途家Scraping Api

APi
freelancer
This article describes a scraping API that allows users to retrieve data from various sources with certain restrictions. The API can handle up to 58 pages, each containing 30 items, resulting in a maximum of 1740 items. Users can sort the results by region and price, potentially reducing the total number below 3480 by applying additional filters. If the limit is exceeded, an inspiration-based strategy is used but it might not support pausing and resuming data collection.
Published

June 5, 2022


搜索api 先通过fetch的api获得不同的限制条件

从0开始 最多58页 每页30个 相同条件最多爬1740个 可以倒序排序来翻倍 可合并其他排序来收集更多

用地区做限制 加上价格限制 或者加更多限制 使得总数少于1740*2=3480 用启发式策略 如果实在超过了预设限制不管了直接开始爬 这种策略可能没法暂停继续爬取