Crawler with channels written in Go.

• Generates url, link_text, page_title, and page_content for a given list of starting urls.
• Uses Go channels to scale heavy workload.
• Has throttler for each domain.
• Supports filter to narrow crawling scope.