-
Notifications
You must be signed in to change notification settings - Fork 30
Projects that use seesaw
If you've written something based on seesaw, feel free to add it to this list. Please add your repository and/or Github URL, the relevant Archive Team wiki page (if any), and any interesting use of seesaw that you'd like to mention.
https://github.com/ArchiveTeam/ArchiveBot / http://www.archiveteam.org/index.php?title=ArchiveBot
(pipeline: https://github.com/ArchiveTeam/ArchiveBot/blob/master/pipeline/pipeline.py)
ArchiveBot's downloaders use wget-lua controlled by seesaw. The pipeline is (probably) notable for its large number of custom tasks and its redirect-stdout-to-Redis modification.
Seesaw was chosen for worker management (a) it has a good track record of not falling over under high load, and (b) its standard task library is well-suited to Archive Team work.
https://github.com/ArchiveTeam/patch-grab / http://archiveteam.org/index.php?title=Patch.com
The patch.com grabber is a pretty standard seesaw pipeline; however, it does demonstrate the use of tornado.HTTPClient to expand short item names into longer URL manifests.