Concurrency based Reddit Scraper in Go Note: This repo is meant to be used in conjunction reddit-frame
- I wanted to learn Golang and why it's praised a lot in distributed & parallel computing community.
- I hated the fact that reddit sometimes consumes more data than youtube (hats off to youtube engineers!).
- Showcase my skills as an aspiring software engineer.
Make sure you have downloaded and install GoLang from official website
- Clone the project
- CD into the project
- Run
go run main.go *subreddit_names_with_space*
. Eg:go run main.go pics askreddit meme
You will get postData.json
and images
directory containing images scraped from given subreddits
- First, I love reddit!
- Second, Reddit has a very friendly APIs which allowed me to focus on my main objectives.