Skip to content

Latest commit

 

History

History
28 lines (20 loc) · 728 Bytes

sec_mirror_docker.md

File metadata and controls

28 lines (20 loc) · 728 Bytes
Raw

How to dockerize sec mirror

SEC-XML Site

  • Used to find all company information

SEC-

CIK Sources

CIK_LIST_CSV = '{0}/investment_company_series_class.csv'.format(CONFIG_RES_DIR) CIK_LIST_PRASANNA = '{0}/cik.coleft.c.txt'.format(CONFIG_RES_DIR)

sec_datafabric.py

takes CIK list from sources above retrieves all filings in xmldict form for the CIK Need to savve all this to database, lets go postgres or mongo Need to get CIK list from somewhere else

database

mongo or postgres contains a mirror of all filing records, and dates updated, or downloaded

filing downloaders

slave containers for downloading the content

filing browser GUI

web application to manage scrapes, progress, look at database