Skip to content

We study the general trend in bias reduction as newer pre-trained models are released. Three recent models ( ELECTRA, DeBERTa and DistilBERT) are chosen and evaluated against two bias benchmarks, StereoSet and CrowS-Pairs. They are compared to the baseline of BERT using the associated metrics

License

Notifications You must be signed in to change notification settings

KranthiGV/bias-evaluation

Repository files navigation

Dataset Setup

  1. cd dataset & wget https://github.com/moinnadeem/StereoSet/blob/master/data/dev.json

Environment Setup

  1. conda env create -f environment-(cpu/gpu).yml
  2. conda activate bias-(cpu/gpu)

Saving Changes

  • conda env export -n bias-(cpu/gpu).yml -f environment-(cpu/gpu).yml --no-builds

Issues Observed

About

We study the general trend in bias reduction as newer pre-trained models are released. Three recent models ( ELECTRA, DeBERTa and DistilBERT) are chosen and evaluated against two bias benchmarks, StereoSet and CrowS-Pairs. They are compared to the baseline of BERT using the associated metrics

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published