-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Washington, DC events scraper #75
Comments
Here is a very good document prepared by @AZIZXlaouiti . https://docs.google.com/document/d/1vFXOAFsGK5AOCbfcvt-LwFR71IUiTkubwa8EcBwgbHQ cc @Shak2000 |
I have hit a road block. Can't quite to come up with a clean way to get events for a given period of time. So far to me, the main candidates for information sources are
I have yet to figure out how to make a query with a period time as parameter(s) into any of the above sources. 1 and 2 can be used to retrieve the most recent N meetings. 3, an API, seemed like a good choice going in, but I am not so warm to it now. It is a good resource to get information about specific bills, laws, etc. It, to me, is almost useless to get agendas, and so on about meetings. |
What about this? https://dccouncil.us/events/2022-01/ You can fill in any year and month and then find the day in the calendar? |
Oh man why didn't I think of this approach! I even saw that calendar before LOL. Yes I think this is, at least to me, the best route I've seen thus far. 👍 Awesome. |
Finally got around to making a first draft. Just getting the minimal now. https://gist.github.com/dphoria/7bea514b1a201f33ade2cf8c8d9fa707 import washington_dc
from datetime import datetime
washington_dc.get_events_on_date(datetime(2022, 2, 1))
|
Foremost question in my head is best way to get votes. I think |
What is highly disappointing is that I thought DC used to have an event's minute items listed in the lower left table on their video player. That seems to be no longer the case? e.g. On http://dc.granicus.com/ViewPublisher.php?view_id=3, click on any "Video" link on the right. The popup is largely empty with just the video. That used to have a lot of useful information we could have used to get |
@dphoria i did notice that along with the absence of pdf document and sometimes captions aren't available |
Nice job!! Can't comment on the PDF document but I wouldn't worry if the captions are optionally available. Seattle has captions for roughly 95% of meetings. If captions aren't available we will roll back to Google. No worries. Excited to see this progress!! |
Any luck in adding to the scraper, @AZIZXlaouiti ? I've been working on other issues recently; probably will be for another couple more weeks. After that I may be able to hop back on this if necessary. Anyway just wanted to check in. |
@dphoria i had some busy weeks (family / interview) related so i wasn't active as i wanted to be but i will resume the work this week . My apologies. |
@dphoria i managed to get the event_minutes added . i parsed the pdf from agenda_uri and managed to get all the legistlation_number after that i'll have to use lims api to get the votes/ votes status /persons. |
https://gist.github.com/AZIZXlaouiti/b3b0ccab24a1fbd0586fb8756fc85c1c
|
No absolutely no need for any apologies. 😄 I was just curious. |
Feature Description
A clear and concise description of the feature you're requesting.
Provide a file in
cdp_scrapers/instances/
likecdp_scrapers/instances/dc.py
or something similar that provides a function that implements API to return Washington, DC city council meetings asList[EventIngestionModel]
for a period of time, e.g.Use Case
Please provide a use case to help us understand your request in context.
Above file and API would be used in deploying a CDP instance for Washington, DC.
The text was updated successfully, but these errors were encountered: