2022-12-13
Web Scraping Logic

select targets for scraping. it could be your browsing history, package indexs, social media (dynamic contents, with different accessing methods than web scraping)

if not accessible, access it with proxies, cookies.

finally store the content into compat and usable formats, categorized and linked

Read More

2022-09-10
Gfw Circumvention, Download Youtube Videos, Scrape Banned Websites

binder as colab alternative

apart from kaggle, you can also use github actions, devops and more, if only we can get the results in time with code.

github integrated ci platforms

cirrus graphql spec with artifact info

github actions api: download artifact

circleci: artifact

azure pipelines: artifacts

Read More