Web Crawling

Libraries to automate web scraping.

Newest releases

ivg3n1 SpotifyToMp3 - downloading spotify playlists from youtube

Alexangelj GM my christmas notes basically my entire process Maybe more than two players have to play against the level. One player stakes, the other bets agains

Gerapy Gerapy Playwright This is a package for supporting Playwright in Scrapy, also this package is a module in Gerapy. Installation pip3 install gerapy-pla

tytydraco ExamineScraper Examine.com supplement research scraper! Why I want to be able to search pages for a specific term. For example, I want to be able to s

grkek Anonymous A minimal web scraper implementation inspired by Crawly Installation Add the dependency to your shard.yml: dependencies: anonymous: gi

guilatrova Google Maps Crawler using Selenium Built as part of the Antifragile Dev Project Selenium crawler that browses Google Maps as a regular user and stores

elia-orsini coinmarketcap-scraper A scraper for CoinMarketCap. It returns the most important data about any cryptocurrency listed. usage With the requirements ins

hrbrmstr cisa-known-exploited-vulns Code with daily GH action to scrape CISA's KEV Catalog (https://www.cisa.gov/known-exploited-vulnerabilities-catalog) Each

gokulapap Autoscraper-n-blogger An Automated udemy coupons scraper which scrapes coupons and autopost the result in blogspot post and notifies via Telegram bot

novitae S T E R R A X C Y L A python program that allows you to put in an excel/csv table followers and/or following of an aimed account, with detailed infos

bhargav794 hacktoberfest-practice-music-playlists Note:-This repo is not included in the Hacktoberfest event, as it is for practice only. If you have any trouble

Irish606 cuii-blocked Dieses Repository enthält eine Liste von Webseiten, die durch die Selbstregulierung Informationswirtschaft e.V. für eine DNS-Sperrung emp

Drovosek01 What is this and for what This is a script that allows you to download portable installers of programs from Adobe for macOS with different versions. T

AcierP discord-account-generator An hcaptcha-solving discord account generator; capable of randomizing names, profile pictures, and verifying phone numbers.

Absolute-Tinkerer CLAPI A basic API to scrape Craigslist. Most useful for viewing posts across a broad geographic area or for viewing posts within a specific timeframe.

youtuberscafe Awesome Youtubers Top 50 most subscribed youtube channels Geographical top YouTubers of Africa top Youtubers of Asia top Youtubers of Europe top Youtu

dchoruzy Welcome to hltv-data 👋 🎮 Data from popular CS:GO website hltv.org Install pip install hltv-data Usage The public methods can be reached using HLTVCl

gautamajay52 UdemyBot - A Simple Udemy Free Courses Scrapper

truroshan Auto Slot Booking when there is a vaccine slot available at your location, by running a script on your phone.

DIGITALCRIMINAL Scrape all the media from an OnlyFans account - Updated regularly

p-ranav saveddit is a bulk media downloader for reddit.

fhamborg news-please is an open source, easy-to-use news crawler that extracts structured information from almost any news website. It can recursively follow internal hyperlinks and read RSS feeds to fetch both most recent and also old, ar

Boris-code feapder 是一款上手简单,功能强大的Python爬虫框架,使用方式类似scrapy,方便由scrapy框架切换过来

srcrs 联通手机营业厅自动完成每日任务,领流量、签到获取积分等,月底流量不发愁。

hu619340515 搜索日志,出现“抢购成功,订单号xxxxx",代表成功抢到了,务必半小时内支付订单!程序暂时不支持自动停止,需要手动STOP!

Sniper970119 本程序可以爬取大众点评搜索页、详情页以及评论页中的相关信息,并将结果写入文件或数据库中。

melo-gonzo This script is useful for downloading stock market data for a wide range of companies specified by their respective tickers.

Threezh1 JSFinder is a tool for quickly extracting URLs and subdomains from JS files on a website.

cocrawler CoCrawler is a versatile web crawler built using modern tools and concurrency.

wkunzhi Python爬虫实战 - 模拟登陆各大网站 包含但不限于:滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝❤️

kurogai An simple script to scrappe deepweb to find pages. Will return if any of those exists and will save on a file. You should specify the name of the file to save. If you wish to estimate the amount of requests, just multiply the numb

meeb TubeSync is a PVR (personal video recorder) for YouTube. Or, like Sonarr but for YouTube (with a built-in download client). It is designed to synchronize channels and playlists from YouTube to local directories and update your med