Web Crawling

Libraries to automate web scraping.

Newest releases

novitae S T E R R A X C Y L A python program that allows you to put in an excel/csv table followers and/or following of an aimed account, with detailed infos

bhargav794 hacktoberfest-practice-music-playlists Note:-This repo is not included in the Hacktoberfest event, as it is for practice only. If you have any trouble

Irish606 cuii-blocked Dieses Repository enthält eine Liste von Webseiten, die durch die Selbstregulierung Informationswirtschaft e.V. für eine DNS-Sperrung emp

Drovosek01 What is this and for what This is a script that allows you to download portable installers of programs from Adobe for macOS with different versions. T

AcierP discord-account-generator An hcaptcha-solving discord account generator; capable of randomizing names, profile pictures, and verifying phone numbers.

Absolute-Tinkerer CLAPI A basic API to scrape Craigslist. Most useful for viewing posts across a broad geographic area or for viewing posts within a specific timeframe.

youtuberscafe Awesome Youtubers Top 50 most subscribed youtube channels Geographical top YouTubers of Africa top Youtubers of Asia top Youtubers of Europe top Youtu

dchoruzy Welcome to hltv-data 👋 🎮 Data from popular CS:GO website hltv.org Install pip install hltv-data Usage The public methods can be reached using HLTVCl

gautamajay52 UdemyBot - A Simple Udemy Free Courses Scrapper

truroshan Auto Slot Booking when there is a vaccine slot available at your location, by running a script on your phone.

DIGITALCRIMINAL Scrape all the media from an OnlyFans account - Updated regularly

p-ranav saveddit is a bulk media downloader for reddit.

fhamborg news-please is an open source, easy-to-use news crawler that extracts structured information from almost any news website. It can recursively follow internal hyperlinks and read RSS feeds to fetch both most recent and also old, ar

Boris-code feapder 是一款上手简单,功能强大的Python爬虫框架,使用方式类似scrapy,方便由scrapy框架切换过来

srcrs 联通手机营业厅自动完成每日任务,领流量、签到获取积分等,月底流量不发愁。

hu619340515 搜索日志,出现“抢购成功,订单号xxxxx",代表成功抢到了,务必半小时内支付订单!程序暂时不支持自动停止,需要手动STOP!

Sniper970119 本程序可以爬取大众点评搜索页、详情页以及评论页中的相关信息,并将结果写入文件或数据库中。

melo-gonzo This script is useful for downloading stock market data for a wide range of companies specified by their respective tickers.

Threezh1 JSFinder is a tool for quickly extracting URLs and subdomains from JS files on a website.

cocrawler CoCrawler is a versatile web crawler built using modern tools and concurrency.

wkunzhi Python爬虫实战 - 模拟登陆各大网站 包含但不限于:滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝❤️

kurogai An simple script to scrappe deepweb to find pages. Will return if any of those exists and will save on a file. You should specify the name of the file to save. If you wish to estimate the amount of requests, just multiply the numb

meeb TubeSync is a PVR (personal video recorder) for YouTube. Or, like Sonarr but for YouTube (with a built-in download client). It is designed to synchronize channels and playlists from YouTube to local directories and update your med

instaloader Download pictures (or videos) along with their captions and other metadata from Instagram.

con-dog Automate the creation of Python Selenium Scripts by drawing coloured boxes on webpage elements

justinzm 数据接口:百度、谷歌、头条、微博指数,宏观数据,利率数据,货币汇率,千里马、独角兽公司,新闻联播文字稿,影视票房数据,高校名单,疫情数据…

mtimkovich Exports watched movies from Plex to the Letterboxd Import Format.

iliaschalkidis A collection of scripts to crawl English legal corpora 📕 from open public domains.

EricJMarti ⚡️ Get notified as soon as your next CPU, GPU, or game console is in stock

JonasCz The ultimate guide on preventing Website Scraping

dkkocab This is a bot that will buy a PS5 through Walmart. It's written in python and uses selenium to interact with web pages.

enesusta github-trending-crawler is a basic rest API that crawls github.com/trending.

ayoubeddafali A minimal framework to automate web Actions/Plans, and run them in a containerized fashion.