Skip to content

software used

Jump to bottom
matthijs van keirsbilck edited this page Apr 30, 2017 · 2 revisions

Set up this repo:

  • ubuntu 16.01, git, download TCDTIMITprocessing.
  • follow instructions there to download, extract and process TCDTIMIT dataset (download ~450GB, processed version ~ 15GB).
  • store all the processed data under a 'TCDTIMIT' directory in your home folder (can be a symlink to another HDD or so). See folder structure shown below.
  • lipreading: faces and mouths are stored in processed/.
    • lip only: use fileDirOps in lipreading/ to get the needed files from processed, restructure and rename them. Then run databaseToPkl in lipraeding/ to convert to usable .pkl files that can be used for training.
  • audio: extractTCDTIMITaudio.py and/or get TIMIT dataset. Then datasetToPkl from audioSR/
  • combinedSR: fileDirOps from combinedSR/ to get the images (will be stored in database/). audio from extractTCDTIMITaudio.py. Copy the audio to where the images are stored (merge the folders so the .wav and .phn files are in the same dir as all the gray_120x120_mouth images, stored in a dir per video).
    Folder structure should then be:
    /home/user/TCDTIMIT / combinedSR / TCDTIMIT / lipspeakers / Lipspkr1 / sa1 / sa1_16_sil.jpg etc
    (combinedSR or audioSR or lipreading)

Installatie Theano/Lasagne op remote GPU server:

Wat ik heb gedaan:

Het grootste deel van het CUDA-stuk moet je niet doen, omdat CUDA al geïnstalleerd is. GPU boost moet je ook niet doen. Kijk wel naar het 'configuration' stukje, die 'export' statements zet je in je ~/.bashrc bij.

cuDNN moet je wel nog zelf installeren voor betere performantie. (zie stap 5) Vergeet ook zeker niet je ~/.theanorc aan te passen.

  • cuDNN: je moet een account aanmaken op Nvidia Developer (https://developer.nvidia.com/cudnn), dan kan je de library downloaden. Ze zeggen daar dat je ze moet installeren naar /usr/local/, maar je kan het ook lokaal installeren: http://deeplearning.net/software/theano/library/sandbox/cuda/dnn.html (zet die exports bij in je ~/.bashrc)

  • Testen:

      - de tests op die installatie- tutorial

  -  http://deeplearning.net/software/theano/tutorial/using_gpu.html#id1 Sla die python code op als bv testGPU.py, en run 'python testGPU.py'.

  - op het eind kan je de MNIST test van het begin van de installatie-tutorial nog eens runnen, dat zou op de GPU moeten werken

Installatie van Caffe

sudo apt-get install gcc-5 g++-5
sudo ln -s  /usr/bin/gcc-5 /usr/bin/gcc -f
sudo ln -s  /usr/bin/g++-5 /usr/bin/g++ -f
  • you could get errors like '/usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.20' not found'. Then do this link:
cd ~/anaconda2/lib
mv -vf libstdc++.so.6 libstdc++.so.6.old
ln -s /usr/lib/x86_64-linux-gnu/libstdc++.so.6 ./libstdc++.so.6
  • after running 'make runtest' with success, run 'make pycaffe' and 'export PYTHONPATH=/home/user/bin/caffe-master/python' (replace by directory where you installed caffe)
  • evt deze tutorial for more instructions

Other software

Caffe model omzetten in Lasagne model:

jupyter nbconvert --to script [YOUR_NOTEBOOK].ipynb

dlib, skikit-image -> face recognition
PIL, ffmpeg -> image pre-processing

Libraries

Theano: http://www.deeplearning.net/software/theano/
TensorFlow: https://github.com/tensorflow/tensorflow
Caffe: http://caffe.berkeleyvision.org/
Comparison: https://medium.com/@sentimentron/faceoff-theano-vs-tensorflow-e25648c31800#.js48hkwqm Lasagne: https://github.com/Lasagne no-learn: https://github.com/dnouri/nolearn

Python: tqdm, urllib, requests, PIL, pickle, scipy, numpy

Nvidia CUDA:

Home
general idea
software used
links
work-overview
thesis-conversations

Clone this wiki locally