scrapes and downloads over multiple connections using aria2, wget and awk

scrapes and downloads over multiple connections using aria2, wget and awk

Related Repos



aria2 aria2 - The ultra fast download utility Disclaimer This program comes with no warranty. You must use this program at your own risk. Introduction aria2 is a utility for downloading files. The supported p
 
623

ShinCurry Maria A macOS GUI app/widget for aria2 download tool. More details in Wiki. <<<<<<< IMPORTANT Features Native App Aria2 Integrated System Notification Support Today Widget Support Touc
 

benhoyt GoAWK: an AWK interpreter written in Go AWK is a fascinating text-processing language, and somehow after reading the delightfully-terse The AWK Programming Language I was inspired to write an interpreter for it in Go. So
 

huijunchen9260 bib.awk Bibliography manager written in awk Just like shbib, but better. Minimal (only require POSIX compliant awk) Search BibTeX on crossref and goog
 

novoda download-manager A library that handles long-running downloads, handling the network interactions and retrying downloads automatically after failures. Clients can request downloads in batches, receiving a single notification fo
 

novoda download-manager A library that handles long-running downloads, handling the network interactions and retrying downloads automatically after failures. Clients can request downloads in batches, receiving a single notification fo
 

developeranaz URL-TO-MEGA-HEROKU-ARIA2 Still under development Aria2c download url & magnet (currently not available) upload remotely to mega account using rclone.
 

learnbyexample GNU AWK Example based guide to mastering GNU awk. The book also includes exercises to test your understanding, which is presented together as a single file in this repo - Exercises.md See Version_changes.md to keep track of
 

gaowanliang A Telegram Bot that can control your Aria2 server, control server files and also upload to OneDrive.
 

jakelandau XKCD-Archiver Command-line utility that automagically scrapes xkcd.com for every xkcd, ever. Yes, that's 1535 (as of this writing) downloads per full archival. Run this on your USB if you want to slowly whittle away its shelf lif
 
88

remy Snap Like wget -r <url> but specifically designed to support "pretty" URLs. With wget, a URL pointing to /foo would result in /foo.html, but this means the URL has now changed. With snap, it will create the directory /foo
 

robinrodricks FluentFTP is a fully managed FTP and FTPS library for .NET & .NET Standard, optimized for speed. It provides extensive FTP commands, File uploads/downloads, SSL/TLS connections, Automatic directory listing parsing, File hashing/
 

robinrodricks FluentFTP is a fully managed FTP and FTPS library for .NET & .NET Standard, optimized for speed. It provides extensive FTP commands, File uploads/downloads, SSL/TLS connections, Automatic directory listing parsing, File hashing/
 

jellever StreamDivert is a tool to man-in-the-middle or relay in and outgoing network connections on a system. It has the ability to, for example, relay all incoming SMB connections to port 445 to another server, or only relay specific incoming SMB connections from a specific set of source IP's to another server.
 

adbar Trafilatura is Python package and command-line tool which seamlessly downloads, parses, and scrapes web page data: it can extract metadata, main body text and comments while preserving part of the text formatting and page structure. The output can be converted to different formats.
 

A3h1nt Grawler is a tool written in PHP which comes with a web interface that automates the task of using google dorks, scrapes the results, and stores them in a file.
 

juanbrujo CMS.js Album Generator Scrapes a directory of photos and creates markdown files to be used by CMS.js enabled site -- Use: clone / download this files cd to them edit index.js variables as needed: // EDIT AS
 

hnlq715 nginx-vts-exporter Simple server that scrapes Nginx vts stats and exports them via HTTP for Prometheus consumption To support time related histogram metrics, please refer to hnlq715/nginx-prometheus-metrics or #43.
 
API
200

adamtlangley A tool which scrapes public github repositories for common naming conventions in variables, folders and files
 

Yord 🧚 pxi (pixie) is a small, fast, and magical command-line data processor similar to jq, mlr, and awk. Installation Installation is done using npm. $ npm i -g pxi Try pxi --help to see if the installation was
 

theryangeary A human-friendly and fast alternative to cut and (sometimes) awk