Spiegel Scraper

Script to scrape articles and whole issuses including metadata from the Spiegel magazine archive.

Getting started

Adjust config.yaml to your scraping needs. This project uses pipenv, so make sure it's installed.

Install all dependencies with:

pipenv install

Make sure the virtual environment is activated by checken

pipenv shell

After that you can start the scraper with:

pipenv run scraper

If this does not work, you can run the main.py manually.

Happy scraping!