Skip to content

Latest commit

 

History

History

get_panlex_data

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 

Get PanLex-BLI Data

  • Download bilingual dictionaries from the PanLex-BLI repo:
    cd /media/data/
    git clone https://github.com/cambridgeltl/panlex-bli.git
    cd panlex-bli/lexicons/
    unzip all-l1-l2.zip
  • Install fastText:
    git clone https://github.com/facebookresearch/fastText.git
    cd fastText
    pip install .
  • Get monolingual embeddings:
    python get_panlex_embs.py

Attention

In PanLex-BLI, source->target and target->source translation tasks have different training and test dictionaries. So, it is needed to train two BLI models respectively for source->target and target->source translations.