Skip to content

A reddit scraper to efficientyl scrape multiple subreddits at the same time

Notifications You must be signed in to change notification settings

CrO2Cl2/BulkRedditScraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

18 Commits
 
 
 
 
 
 

Repository files navigation

This is a highly configurable Reddit scraper to efficiently and continiously scrape reddit for pictures and is fitted with loads of error handling for an exceptionally high stability. It uses Multithreading, and can easily scrape from multiple subredits at the same time. I have gotten a result of about 350 pictures per minute scraped with a internet bandwith of 200Mbps Right now the script is configured to retrieve images only and skip NSFW. the usage of this script is explained in the comments of the code

PLEASE DO NOT USE THIS SCRAPER TO SCRAPE FOR STABLE DIFFUSION ALGORYTHMS AND OTHER ART GENERATING ALGORYTHMS

About

A reddit scraper to efficientyl scrape multiple subreddits at the same time

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages