Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate Amin's LODES data code #51

Open
npr99 opened this issue May 21, 2024 · 13 comments
Open

Integrate Amin's LODES data code #51

npr99 opened this issue May 21, 2024 · 13 comments

Comments

@npr99
Copy link
Owner

npr99 commented May 21, 2024

https://github.com/npr99/intersect-community-data/tree/main/pyncoda/99_SandboxCode/SandboxAE

On my VS Code running code for
Employment Inventory.ipynb

copied code from Amin's sandbox into my work folder on VS Code. The gitnore allows work folders to be ingored

@npr99
Copy link
Owner Author

npr99 commented May 21, 2024

code runs and appears to download all 2020 data. But hits an error
FileNotFoundError: [Errno 2] No such file or directory: 'c:\Users\nathanael99\MyProjects\github\intersect-community-data\WorkNPR/output/ks_20057_od_2020_na.csv'

@npr99
Copy link
Owner Author

npr99 commented May 21, 2024

comparing lodes_fullloop.py from Amin and NPR. Looks like Amin added some good try options and error handling.
Added a new function wac_rac_block_to_joblist

@npr99
Copy link
Owner Author

npr99 commented May 21, 2024

line 63 of lodes_datautil.py hard codes the output folder. Try to use this instead of output.
# Make directory to save output
datapath = 'data_LODES'

this seems to be working

@npr99
Copy link
Owner Author

npr99 commented May 21, 2024

comparing Amin's lodes_datautil.py

updates for using LODES8 verses LODES7. This could be added as an input
try except and error handling

cross walk now uses 2020 blocks instead of 2010
good place to use the year variable instead of hard coding.

@npr99
Copy link
Owner Author

npr99 commented May 21, 2024

significant updates to new_jobtypes function.
looks like addition of if statements to streamline the adding of new jobtypes. Ask @aminenderami about the additions to this function.

@npr99
Copy link
Owner Author

npr99 commented May 21, 2024

image
looks like a good bug fix. need to think about why I set expect count to 1 instead of 0

@npr99
Copy link
Owner Author

npr99 commented May 21, 2024

lots of new functions added! need to review them but they look really good. Start at line 1577 of lodes_datautil.py

@npr99
Copy link
Owner Author

npr99 commented May 21, 2024

error for last block in notebook:
FileNotFoundError: [Errno 2] No such file or directory: 'output2/wac_200579616001020_joblist_2020.csv'
looks like same issue with the output folder

change to outputfoldername = "data_LODES",

error persists
output2

ahh - looks like I need to create a directory called output2

fixed issue added code to make a folder in data_LODES to save joblists

Good thing county 20057 is small
https://censusreporter.org/profiles/05000US20057-ford-county-ks/

@npr99
Copy link
Owner Author

npr99 commented May 21, 2024

Looks like @aminenderami code goes up to making the wac and rac job lists.

jobs
wac_200579616001020_joblist_2020 = 11
wac_200579616001046_joblist_2020 = 39
wac_200579616001066_joblist_2020 = 2
wac_200579616001087_joblist_2020 = 4
wac_200579616001153_joblist_2020 = 11
wac_200579616001171_joblist_2020 = 48
wac_200579616001175_joblist_2020 = 3
wac_200579616001197_joblist_2020 = 4
wac_200579616001230_joblist_2020 = 27
wac_200579616001233_joblist_2020 = 130
total jobs in wac = 279

should be 16,064 jobs
image

https://onthemap.ces.census.gov/

looks like jobtypes are just for industry code 1 and jobtypes 3 and 9

ahh - no the issue is that the code is only running for the first 10 blocks.
for block in block_list[0:10]:

@npr99
Copy link
Owner Author

npr99 commented May 21, 2024

code for MCMCSA have not been modified.

@npr99
Copy link
Owner Author

npr99 commented May 21, 2024

@npr99
Copy link
Owner Author

npr99 commented May 21, 2024

next steps - work on running MCMCSA
pyncoda\99_SandboxCode\SandboxNPR\LODES_1av4_CleanLODESdata_2022-02-18.ipynb

check issue with unique id set in loop.

@npr99
Copy link
Owner Author

npr99 commented May 21, 2024

summary - made only minor errors to Employment Inventory.ipynb. Mainly just issues with where the data was being saved.

Need to look at error found in:
#52

Work on MCMCSA code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant