Scrapy, a fast high-level web crawling & scraping framework for Python.

Scrapy Overview Scrapy is a fast high-level web crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes,

Related Repos

dvingerh Python script to download videos from a TikTok profile without any watermarks.

sameera-madushan SBOX - Subtitle Box SBOX is a python script to download subtitles for your movies from SubDB database using their API. SubDB is a free, centralized subtitle database intended to be used only by opensource and non-commercial softw

chris-hamberg Scraper that downloads Springers Free COVID-19 English books.

mps-youtube Python library to download YouTube content and retrieve metadata

psalias2006 Google2Csv is a simple google scraper that saves the results on a csv/xlsx/jsonl file

knudmoeller This is a scraper for the daily press releases announcing the current Corona/COVID-19 case numbers for Berlin, as issued by the Senatsverwaltung für Gesundheit, Pflege und Gleichstellung (Senate Department for Health, Care and Equality). The output of the scraper is a timeline of data extracted from the individual press releases.

twintproject An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.

adbar Trafilatura is Python package and command-line tool which seamlessly downloads, parses, and scrapes web page data: it can extract metadata, main body text and comments while preserving part of the text formatting and page structure. The output can be converted to different formats.