Senior Design Repository for the Statefarm Automotive Fraud Project
Statistics
Database Access
Make a copy of the .env.example
file and make the following changes.
-
remove
.example
from the extension -
Paste the username and password provided in MongoDB Atlas (if you should have access but do not, please contact @waseem-polus)
-
Paste the connection URL provided provided in MongoDB Atlas. Include the password and username fields using
${VARIABLE}
syntax to embed the value of the variable
Run Scrapers locally
Prerequisites
- python3
- pipenv
Installing dependencies
Navigate to scrapers/
and open the virtual environment using
pipenv shell
Then install dependencies using
pipenv install
Scraper Usage
To create build a production-ready Docker Image use
pipenv run build
To create build a development Docker Image use
pipenv run dev
If there is an existing smarecontainer, run the following:
pipenv run stop
To run a docker container "smarecontainer" use (Note: delete any containers with the same name before running)
pipenv run cont
then
# Scrape Craigsist homepage pipenv run craigslist # Scrape Facebook Marketplace homepage pipenv run facebook
00dd2a2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Successfully deployed to the following URLs:
seniordesign – ./
seniordesign-lryanle.vercel.app
seniordesign-git-main-lryanle.vercel.app
smare.vercel.app
smare.lryanle.com