GitHub - tenforce/ods-scraper: Meta data scraper supporting the creation of dataset descriptions. This has been applied for creating datasets for the European Union Open Data Portal. The work is funded by the Open Data Support project.

tenforce / ods-scraper Public

Notifications You must be signed in to change notification settings
Fork 0
Star 2

Meta data scraper supporting the creation of dataset descriptions. This has been applied for creating datasets for the European Union Open Data Portal. The work is funded by the Open Data Support project.

Apache-2.0 license

2 stars 0 forks Branches Tags Activity

Star

Notifications

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 44 Commits
books_example		books_example
ods		ods
.gitignore		.gitignore
README		README
license		license
package_ods_scraper.sh		package_ods_scraper.sh
scrape-eba.sh		scrape-eba.sh
scrapy.cfg		scrapy.cfg
template.xlsx		template.xlsx

Repository files navigation

= Getting started with the ODS scraper =

This document describes how to install and set up the ODS scraper.


== Installation ==

First install python 2.7.  Then install scrapy. See 
http://scrapy.org/download/ for more information.


== Build your spider ==

In order to build a spider, copy the ods/spiders/spider-example.py file
to a file of your choice in the ods/spiders folder.  The file extension
must be .py.


== Run the spider ==

Each spider has a name.  The name of the spider is supplied in the
spider class.  Assuming your spider is called myOdsSpider, launch
the spider by calling:

```
  scrapy crawl spiderName
```

or add the name of your spider to the relevant list in scrape-eba.sh
and launch all selected spiders.