A simple web scraping tool for recipe sites.
pip install recipe-scrapers
then:
from recipe_scrapers import scrape_me
# give the url as a string, it can be url from any site listed below
scraper = scrape_me('https://www.allrecipes.com/recipe/158968/spinach-and-feta-turkey-burgers/')
scraper.title()
scraper.total_time()
scraper.yields()
scraper.ingredients()
scraper.instructions()
scraper.image()
scraper.links()Note: scraper.links() returns a list of dictionaries containing all of the <a> tag attributes. The attribute names are the dictionary keys.
- https://www.acouplecooks.com
- https://allrecipes.com/
- https://archanaskitchen.com/
- https://averiecooks.com/
- https://bbc.com/
- https://bbc.co.uk/
- https://bbcgoodfood.com/
- https://bettycrocker.com/
- https://bonappetit.com/
- https://bowlofdelicious.com/
- https://budgetbytes.com/
- https://closetcooking.com/
- https://cookieandkate.com/
- https://cookpad.com/
- https://cookstr.com/
- https://copykat.com/
- https://countryliving.com/
- https://cybercook.com.br/
- https://delish.com/
- https://eatsmarter.com/
- https://eatsmarter.de/
- https://epicurious.com/
- https://fifteenspatulas.com/
- https://finedininglovers.com/
- https://fitmencook.com/
- https://food.com/
- https://foodnetwork.com/
- https://foodrepublic.com/
- https://geniuskitchen.com/
- https://giallozafferano.it/
- https://gimmesomeoven.com/
- https://gonnawantseconds.com/
- https://gousto.co.uk/
- https://greatbritishchefs.com/
- https://halfbakedharvest.com/
- https://www.hassanchef.com/
- https://heinzbrasil.com.br/
- https://hellofresh.com/
- https://hellofresh.co.uk/
- https://www.hellofresh.de/
- https://hostthetoast.com/
- https://101cookbooks.com/
- https://receitas.ig.com.br/
- https://inspiralized.com/
- https://jamieoliver.com/
- https://justbento.com/
- https://kennymcgovern.com/
- https://kochbar.de/
- https://lovingitvegan.com/
- https://lecremedelacrumb.com/
- https://marmiton.org/
- https://matprat.no/
- http://mindmegette.hu/
- https://minimalistbaker.com/
- https://misya.info/
- https://momswithcrockpots.com/
- http://motherthyme.com/
- https://mybakingaddiction.com/
- https://myrecipes.com/
- https://healthyeating.nhlbi.nih.gov/
- https://cooking.nytimes.com/
- https://ohsheglows.com/
- https://www.panelinha.com.br/
- https://paninihappy.com/
- https://przepisy.pl/
- https://realsimple.com/
- https://seriouseats.com/
- https://simplyquinoa.com/
- https://simplyrecipes.com/
- https://skinnytaste.com/
- https://southernliving.com/
- https://spendwithpennies.com/
- https://steamykitchen.com/
- https://tastesoflizzyt.com
- https://tasteofhome.com
- https://tasty.co
- https://tastykitchen.com/
- https://thehappyfoodie.co.uk/
- https://thekitchn.com/
- https://thepioneerwoman.com/
- https://thespruceeats.com/
- https://thevintagemixer.com/
- https://thewoksoflife.com/
- https://tine.no/
- https://tudogostoso.com.br/
- https://twopeasandtheirpod.com/
- https://vegolosi.it/
- https://watchwhatueat.com/
- https://whatsgabycooking.com/
- https://en.wikibooks.org/
- https://yummly.com/
Part of the reason I want this open sourced is because if a site makes a design change, the scraper for it should be modified.
If you spot a design change (or something else) that makes the scraper unable to work for a given site - please fire an issue asap.
If you are programmer PRs with fixes are warmly welcomed and acknowledged with a virtual beer.
- Open an Issue providing us the site name, as well as a recipe link from it.
- You are a developer and want to code the scraper on your own:
- If Schema is available on the site - you can do this
- Otherwise, scrape the HTML - like this
Assuming you have python3 installed, navigate to the directory where you want this project to live in and drop these lines
git clone [email protected]:hhursev/recipe-scrapers.git && cd recipe-scrapers && python3 -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt && pre-commit install && python -m coverage run -m unittest && python -m coverage report
- How do I know if a website has a Recipe Schema?
- Go to a recipe on the website you want to be supported.
- Hit
Ctrl - uon your keyboard - Search (
Ctrl -f) forapplication/ld+json. It should be inside ascripttag. - If you found it then it's highly likely your website supports recipe schemas. Otherwise, you'll need to parse the HTML.
All the contributors that helped improving the package. You are awesome!