Code

133/clean fb attributes#190

Code

Merged

lryanle merged 130 commits intomainlryanle/SMARE:mainfrom 133/clean-fb-attributeslryanle/SMARE:133/clean-fb-attributesCopy head branch name to clipboardMar 22, 2024

+27,409-2,413

Commits

Commits on Nov 14, 2023

• Add files via upload

  temitayoaderounmu

  authored
  e255ba2

  Copy full SHA for e255ba2

• Update dataAnalysis.ipynb

  temitayoaderounmu

  authored
  77471b8

  Copy full SHA for 77471b8

• Update dataAnalysis.ipynb

  temitayoaderounmu

  authored
  d4f3b25

  Copy full SHA for d4f3b25

• Add files via upload

  temitayoaderounmu

  authored
  e6841e7

  Copy full SHA for e6841e7

Commits on Nov 20, 2023

• Data Preprocessing and Anomaly Detection Implementation

  athiya26

  committed
  ac01bcd

  Copy full SHA for ac01bcd

• Add files via upload

  temitayoaderounmu

  authored
  e5b01d5

  Copy full SHA for e5b01d5

• Add files via upload

  temitayoaderounmu

  authored
  677a9bc

  Copy full SHA for 677a9bc

• Parsing

  

  Yeabsra

  committed
  267d440

  Copy full SHA for 267d440

• Trying to get the data from MongoDb

  

  Yeabsra

  committed
  40e8165

  Copy full SHA for 40e8165

Commits on Nov 26, 2023

• Delete dataAnalysis.ipynb

  temitayoaderounmu

  authored
  8822385

  Copy full SHA for 8822385

• Delete MarketPriceComparison.py

  temitayoaderounmu

  authored
  dd447ef

  Copy full SHA for dd447ef

• Add files via upload

  temitayoaderounmu

  authored
  773cc61

  Copy full SHA for 773cc61

• Add files via upload

  temitayoaderounmu

  authored
  ca6b4e4

  Copy full SHA for ca6b4e4

• Delete price_comparison_results.csv

  temitayoaderounmu

  authored
  186f575

  Copy full SHA for 186f575

• Update MarketPriceComparison.py

  temitayoaderounmu

  authored
  5c58a08

  Copy full SHA for 5c58a08

• Delete dataAnalysis.ipynb

  temitayoaderounmu

  authored
  8ee8887

  Copy full SHA for 8ee8887

• Add files via upload

  temitayoaderounmu

  authored
  51f3473

  Copy full SHA for 51f3473

• Add files via upload

  temitayoaderounmu

  authored
  0da3837

  Copy full SHA for 0da3837

• Delete dataAnalysis.ipynb

  temitayoaderounmu

  authored
  56295e9

  Copy full SHA for 56295e9

• Delete price_comparison_results.csv

  temitayoaderounmu

  authored
  4b70aa3

  Copy full SHA for 4b70aa3

• Delete MarketPriceComparison.py

  temitayoaderounmu

  authored
  ae8e083

  Copy full SHA for ae8e083

• Add files via upload

  temitayoaderounmu

  authored
  b8f955b

  Copy full SHA for b8f955b

• Update MarketPriceComparison.py

  temitayoaderounmu

  authored
  d691651

  Copy full SHA for d691651

Commits on Feb 23, 2024

• Create README.md

  temitayoaderounmu

  authored
  eec9ffc

  Copy full SHA for eec9ffc

• Add files via upload

  temitayoaderounmu

  authored
  c712da6

  Copy full SHA for c712da6

Commits on Mar 4, 2024

• Added facebook attr cleaner script

  waseem-polus

  committed
  6acbce5

  Copy full SHA for 6acbce5

Commits on Mar 13, 2024

• merge changes from main

  waseem-polus

  committed
  23ffe3a

  Copy full SHA for 23ffe3a

• move facebook cleaner to cleaner module

  waseem-polus

  committed
  c0e0e83

  Copy full SHA for c0e0e83

• Create MM.py

  temitayoaderounmu

  authored
  952fcd8

  Copy full SHA for 952fcd8

• Update M3.py

  temitayoaderounmu

  authored
  59a1be0

  Copy full SHA for 59a1be0

Commits on Mar 14, 2024

• started modularizing cleaners

  waseem-polus

  committed
  6ebd39c

  Copy full SHA for 6ebd39c

• added logging to cleaner module

  waseem-polus

  committed
  56c2fd1

  Copy full SHA for 56c2fd1

• increment scraper ver and add cleaner ver

  waseem-polus

  committed
  4d9e84c

  Copy full SHA for 4d9e84c

• added odometer cleaning, and facebook year extraction

  waseem-polus

  committed
  6606cff

  Copy full SHA for 6606cff

• refactoring models (co-authored by ryan)

  waseem-polus

  committed
  d13dd62

  Copy full SHA for d13dd62

• fix(backend): camelCase -> snake_case

  lryanle

  committed
  4399feb

  Copy full SHA for 4399feb

• update requirements.txt

  waseem-polus

  committed
  dd16b5f

  Copy full SHA for dd16b5f

Commits on Mar 15, 2024

• update DB to use snake case

  waseem-polus

  committed
  8601217

  Copy full SHA for 8601217

• removed unused import from m4

  waseem-polus

  committed
  80af198

  Copy full SHA for 80af198

• update cleaners to use snake case for DB operations

  waseem-polus

  committed
  d02f298

  Copy full SHA for d02f298

• extract scraper entrypoint from utils into scraper.py

  waseem-polus

  committed
  ec271c4

  Copy full SHA for ec271c4

• changes from yesterday

  lryanle

  committed
  af564b2

  Copy full SHA for af564b2

• ran linters

  waseem-polus

  committed
  66e91be

  Copy full SHA for 66e91be

• Fix merge conflicts

  waseem-polus

  committed
  345349b

  Copy full SHA for 345349b

• ran isort

  waseem-polus

  committed
  c76c5e5

  Copy full SHA for c76c5e5

• added query to check for missing models

  waseem-polus

  committed
  a7c2964

  Copy full SHA for a7c2964

• Fixed scraper bugs

  waseem-polus

  committed
  44f9f8b

  Copy full SHA for 44f9f8b

• Model 6 - Luxury Brand Detection and Scam Identification

  Show description for 152b015
  athiya26

  authored
  152b015

  Copy full SHA for 152b015

• Model 2 Folder

  athiya26

  authored
  74283c4

  Copy full SHA for 74283c4

• Delete Models/Model 2

  athiya26

  authored
  5760045

  Copy full SHA for 5760045

• added kbbscraper and flask to connect to node server

  athiya26

  authored
  8b7cb88

  Copy full SHA for 8b7cb88

• Add files via upload

  athiya26

  authored
  c83f360

  Copy full SHA for c83f360

• Model 1

  athiya26

  authored
  d481e87

  Copy full SHA for d481e87

• added craigslist cleaners

  waseem-polus

  committed
  2660de5

  Copy full SHA for 2660de5

• Delete Models/Model 2 directory

  athiya26

  authored
  405563f

  Copy full SHA for 405563f

• Create kbbscraper.py

  Show description for 4b2c119
  athiya26

  authored
  4b2c119

  Copy full SHA for 4b2c119

• Delete Models/Model 1

  athiya26

  authored
  ea446a9

  Copy full SHA for ea446a9

• Model 1

  athiya26

  authored
  aa56dfd

  Copy full SHA for aa56dfd

• added kbbscraper and flask to connect to node server

  athiya26

  authored
  dbe6c70

  Copy full SHA for dbe6c70

• Delete Models/Model 1 directory

  athiya26

  authored
  cbf1c21

  Copy full SHA for cbf1c21

• Update model_cleaner.py

  temitayoaderounmu

  authored
  81b7bec

  Copy full SHA for 81b7bec

• Added car make extractor to cleaner module

  waseem-polus

  committed
  01e55fa

  Copy full SHA for 01e55fa

• Started implementing facebook model extractor

  waseem-polus

  committed
  f3915aa

  Copy full SHA for f3915aa

• checkpoint facebook model extractor

  waseem-polus

  committed
  7494e3a

  Copy full SHA for 7494e3a

• Update m3_kbbprice.py

  temitayoaderounmu

  authored
  fdac0c0

  Copy full SHA for fdac0c0

• fix merge conflicts with origin

  waseem-polus

  committed
  8eef18a

  Copy full SHA for 8eef18a

• Merge branch '133/clean-fb-attributes' of https://github.com/lryanle/seniordesign into 133/clean-fb-attributes

  waseem-polus

  committed
  85a3111

  Copy full SHA for 85a3111

• Delete backend/src/models/price_comparison_results.csv

  temitayoaderounmu

  authored
  56450ae

  Copy full SHA for 56450ae

• Add files via upload

  temitayoaderounmu

  authored
  dea4512

  Copy full SHA for dea4512

• added post_log to database.py

  waseem-polus

  committed
  30ad00d

  Copy full SHA for 30ad00d

• Merge branch '133/clean-fb-attributes' of https://github.com/lryanle/seniordesign into 133/clean-fb-attributes

  waseem-polus

  committed
  dd4f710

  Copy full SHA for dd4f710

• renamed kbb_data to car_models

  waseem-polus

  committed
  ec764f9

  Copy full SHA for ec764f9

• logger

  lryanle

  committed
  9cbade1

  Copy full SHA for 9cbade1

• Added model extractor to cleaner

  waseem-polus

  committed
  fd97edb

  Copy full SHA for fd97edb

• Merge branch '133/clean-fb-attributes' of https://github.com/lryanle/seniordesign into 133/clean-fb-attributes

  waseem-polus

  committed
  b4bf68f

  Copy full SHA for b4bf68f

Commits on Mar 16, 2024

• LOGGING IS DONE

  lryanle

  committed
  b2552bb

  Copy full SHA for b2552bb

• changed utils folder -> utilities

  lryanle

  committed
  93d75f1

  Copy full SHA for 93d75f1

• (WIP) added make model parsers to cleaner.py

  waseem-polus

  committed
  0739953

  Copy full SHA for 0739953

• Merge branch '133/clean-fb-attributes' of https://github.com/lryanle/seniordesign into 133/clean-fb-attributes

  waseem-polus

  committed
  e2554dd

  Copy full SHA for e2554dd

• move car_models to utilties

  waseem-polus

  committed
  cb1a866

  Copy full SHA for cb1a866

• added logging to model_manager

  lryanle

  committed
  799272f

  Copy full SHA for 799272f

• craigslist checks location vs website

  waseem-polus

  committed
  6784cf1

  Copy full SHA for 6784cf1

• Merge branch '133/clean-fb-attributes' of https://github.com/lryanle/seniordesign into 133/clean-fb-attributes

  waseem-polus

  committed
  3318a8a

  Copy full SHA for 3318a8a

• edit find analsysis cafs

  waseem-polus

  committed
  6b1e898

  Copy full SHA for 6b1e898

• added logging to database.py

  lryanle

  committed
  04901fc

  Copy full SHA for 04901fc

• absolute path + currency extractor "free" check

  waseem-polus

  committed
  3b20878

  Copy full SHA for 3b20878

• Fixed merge conflicts in database script

  waseem-polus

  committed
  6b1413c

  Copy full SHA for 6b1413c

• added filter function to modmel manager

  waseem-polus

  committed
  4da8fd8

  Copy full SHA for 4da8fd8

• Added logging to all scrapers

  lryanle

  committed
  d2e8dd3

  Copy full SHA for d2e8dd3

• cleaner can push to database again

  waseem-polus

  committed
  9a4cdd9

  Copy full SHA for 9a4cdd9

• Merge branch '133/clean-fb-attributes' of https://github.com/lryanle/seniordesign into 133/clean-fb-attributes

  waseem-polus

  committed
  4709a33

  Copy full SHA for 4709a33

• added logging to cleaners

  lryanle

  committed
  4e421ea

  Copy full SHA for 4e421ea

• fixed logger imports

  waseem-polus

  committed
  fdef454

  Copy full SHA for fdef454

• fix all logger errors

  waseem-polus

  committed
  13f4c93

  Copy full SHA for 13f4c93

• pull and filter data for each model

  waseem-polus

  committed
  2cc16cf

  Copy full SHA for 2cc16cf

• fixed error checking logic

  waseem-polus

  committed
  be40426

  Copy full SHA for be40426

• (WIP) added multiprocessing orchestrator demo

  waseem-polus

  committed
  d4a01f8

  Copy full SHA for d4a01f8

• ran linters

  waseem-polus

  committed
  46429ee

  Copy full SHA for 46429ee

Commits on Mar 17, 2024

• Update m3_kbbprice.py

  Show description for c3a9e96
  temitayoaderounmu

  authored
  c3a9e96

  Copy full SHA for c3a9e96

• Update m4_carfreq.py

  Show description for ff116a5
  temitayoaderounmu

  authored
  ff116a5

  Copy full SHA for ff116a5

Commits on Mar 18, 2024

• Add files via upload

  temitayoaderounmu

  authored
  47ed3c7

  Copy full SHA for 47ed3c7

Commits on Mar 19, 2024

• Fixed OneHotEncoder issue and updated model serialization

  athiya26

  authored
  e1fd72f

  Copy full SHA for e1fd72f

• Serialized preprocessing pipeline

  athiya26

  authored
  8dca73c

  Copy full SHA for 8dca73c

• Serialized trained Isolation Forest model

  athiya26

  authored
  fb1a7e8

  Copy full SHA for fb1a7e8

• Update README.md

  athiya26

  authored
  149bd8c

  Copy full SHA for 149bd8c

• Merge pull request #178 from lryanle/dataAnalysis

  Show description for fe2f208
  athiya26

  authored
  fe2f208

  Copy full SHA for fe2f208

Commits on Mar 20, 2024

• fixed previous merge and cleaned up a lot of the data

  lryanle

  committed
  5b00ca3

  Copy full SHA for 5b00ca3

Commits on Mar 21, 2024

• added model versioning

  waseem-polus

  committed
  c567d47

  Copy full SHA for c567d47

• merge changes from athiya's pr

  waseem-polus

  committed
  03263c9

  Copy full SHA for 03263c9

• Update m5_theftlikelihood.py

  Show description for 0afd40e
  temitayoaderounmu

  authored
  0afd40e

  Copy full SHA for 0afd40e

• Update m5_theftlikelihood.py

  Show description for d5aceb7
  temitayoaderounmu

  authored
  d5aceb7

  Copy full SHA for d5aceb7

• Update m5_theftlikelihood.py

  temitayoaderounmu

  authored
  19ba6e0

  Copy full SHA for 19ba6e0

• tbh idk why this took so long but adding an index to listings on line 31 was the culmination of 6 hours of bugfixing

  lryanle

  committed
  d10ccd0

  Copy full SHA for d10ccd0

• updated formatting and added to model manager

  lryanle

  committed
  ba13880

  Copy full SHA for ba13880

• Update m5_theftlikelihood.py

  Show description for f5446d2
  temitayoaderounmu

  authored
  f5446d2

  Copy full SHA for f5446d2

• Update m3_kbbprice.py

  Show description for 750b125
  temitayoaderounmu

  authored
  750b125

  Copy full SHA for 750b125

• Update m4_carfreq.py

  temitayoaderounmu

  authored
  519be0e

  Copy full SHA for 519be0e

• fixed the model to work with multiple :)

  lryanle

  committed
  f3a8cdc

  Copy full SHA for f3a8cdc

• fixed logger messages to show correct callstack

  lryanle

  committed
  c8a1b5e

  Copy full SHA for c8a1b5e

• orchestrator and db changes

  waseem-polus

  committed
  16d9a5b

  Copy full SHA for 16d9a5b

Commits on Mar 22, 2024

• cope

  lryanle

  committed
  cd601b3

  Copy full SHA for cd601b3

• weeeee model manager works now

  lryanle

  committed
  47cf62a

  Copy full SHA for 47cf62a

• removed debug woo haa

  lryanle

  committed
  5ad7261

  Copy full SHA for 5ad7261

• implemented orchestrator

  waseem-polus

  committed
  2caa8a6

  Copy full SHA for 2caa8a6

• add logging in orchestrator

  waseem-polus

  committed
  c6edd5c

  Copy full SHA for c6edd5c

• cleanning up prod dockerfile

  waseem-polus

  committed
  5d9c843

  Copy full SHA for 5d9c843

• Updated m6_anomaly.py

  athiya26

  authored
  1b8a2f4

  Copy full SHA for 1b8a2f4

• Update model_manager.py

  athiya26

  authored
  8d5e68a

  Copy full SHA for 8d5e68a

• Update model_manager.py

  athiya26

  authored
  93c1dd6

  Copy full SHA for 93c1dd6

• WEEE LETS GO FRONTEND UPDATED

  lryanle

  committed
  4ceb014

  Copy full SHA for 4ceb014