Web Crawling

github-trending-crawler is a basic rest API that crawls github.com/trending.

A minimal framework to automate web Actions/Plans, and run them in a containerized fashion.

Simple, quick to set up stock notification bot for Nvidia 3080 that I used to get my 3080. Less than 250 lines of code.

🎯 Autonomously buy Nvidia Founders Edition GPUs as soon as they become available

365-Stealer is the tool written in python3 which steals data from victims office365 by using access_token which we get by phishing. It steals outlook mails, attachments, OneDrive files, OneNote notes and injects macros.

Bot for crawling stock availability of RTX 3000 cards and tweeting about it

AutoScraper: A Smart, Fast and Lightweight Automatic Web Scraper for Python

A dungeon crawler designed for a quantum computer as a series of 17000 quantum gates.

支持数据源包括GitHub、QQ邮箱、网易邮箱、阿里邮箱、新浪邮箱、Hotmail邮箱、Outlook邮箱、京东、淘宝、支付宝、中国移动、中国联通、中国电信、知乎、哔哩哔哩、网易云音乐、QQ好友、QQ群、生成朋友圈相册、浏览器浏览历史、12306、博客园、CSDN博客、开源中国博客、简书。

Scrape websites asynchronously with Python 3.8+, Asyncio, & arsenic (aka Selenium for Async).

scans every ticker on the market, gets their last 5 months of volume history, and alerts you when a stock's volume exceeds 10 standard deviations from the mean within the last 3 days

🗃 The open source self-hosted web archive. Takes browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...

Tested environments: Windows, MAC, linux, and windows subsystem for linux (WSL)

A Script which adds all songs from youtube playlist to a new spotify playlist.

Simple Python script, that allow to import favorite tracks, playlists, albums and artists from Yandex.Music to Spotify

This is a package for supporting pyppeteer in Scrapy, also this package is a module in Gerapy.

A collection of useful Korean crawlers (always updated) 🌐

Library to extract realtime and historical data from NSE website.EOD data like bhavcopy and option chain are also saved to directory. First run will create directories for storing the data and will download the index symbols.

A spider coded to get the hot_Comment of WangYiYun

Scrape public filings of the buy + sell orders of U.S. senators and calculate their returns

Download the hottest 100 images from r/wallpaper and set them as your cycling Desktop background.

A simple configurable bot for sending arXiv article alert by mail

获取斗鱼&虎牙&哔哩哔哩&抖音&快手等26个直播平台的真实流媒体地址(直播源)和弹幕，直播源可在PotPlayer、flv.js等播放器中播放。

Trafilatura is Python package and command-line tool which seamlessly downloads, parses, and scrapes web page data: it can extract metadata, main body text and comments while preserving part of the text formatting and page structur

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.

This is a scraper for the daily press releases announcing the current Corona/COVID-19 case numbers for Berlin, as issued by the Senatsverwaltung für Gesundheit, Pflege und Gleichstellung (Senate Department for Health, Care and Equ

Google2Csv is a simple google scraper that saves the results on a csv/xlsx/jsonl file

Python library to download YouTube content and retrieve metadata

Scraper that downloads Springers Free COVID-19 English books.

SBOX - Subtitle Box SBOX is a python script to download subtitles for your movies from SubDB database using their API. SubDB is a free, centralized subtitle database intended to be used only by opensource and non-commer

Python script to download videos from a TikTok profile without any watermarks.

YouTube Downloader V1.1 Requirements python>=3.4 pytube3==9.5.13 Installation Open a terminal and execute git clone https://github.com/Jodagito/YoutubeDownloader on the destination folder. On

Spotify Playlist Downloader Downloads songs as listed on a spotify playlist from youtube. Uses scrapy, selenium (need chrome webdriver) and youtube_dl. It also uses a library called ffmpeg to convert from webm (without