topic_coherence_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Demonstration of the topic coherence pipeline in Gensim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will be using the `c_v` coherence for two different LDA models: a \"good\" and a \"bad\" LDA model. The good LDA model will be trained over 50 iterations and the bad one for 1 iteration. Hence in theory, the good LDA model will be able come up with better or more human-understandable topics. Therefore the coherence measure output for the good LDA model should be more (better) than that for the bad LDA model. This is because, simply, the good LDA model usually comes up with better topics that are more human interpretable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from __future__ import print_function\n",
    "\n",
    "import logging\n",
    "import warnings\n",
    "\n",
    "from gensim.models import CoherenceModel, LdaModel\n",
    "from gensim.corpora import Dictionary\n",
    "\n",
    "warnings.filterwarnings('ignore')  # To ignore all warnings that arise here to enhance clarity\n",
    "\n",
    "logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.DEBUG)\n",
    "\n",
    "import load_lee_background_corpus as load_texts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set up corpus"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/home/Desenvolvimento/anaconda3/lib/python3.6/site-packages/gensim/test/test_data\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2017-11-22 13:57:30,794 : INFO : collecting all words and their counts\n",
      "2017-11-22 13:57:30,796 : INFO : PROGRESS: at sentence #0, processed 0 words and 0 word types\n",
      "2017-11-22 13:57:30,846 : INFO : collected 20429 word types from a corpus of 19878 words (unigram + bigrams) and 300 sentences\n",
      "2017-11-22 13:57:30,847 : INFO : using 20429 counts as vocab in Phrases<0 vocab, min_count=5, threshold=10.0, max_vocab_size=40000000>\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Returning 300 training texts\n"
     ]
    }
   ],
   "source": [
    "texts = load_texts.get_train_texts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2017-11-22 13:57:35,193 : INFO : adding document #0 to Dictionary(0 unique tokens: [])\n",
      "2017-11-22 13:57:35,239 : INFO : built Dictionary(3824 unique tokens: ['hundred', 'people', 'home', 'southern', 'highland']...) from 300 documents (total 18861 corpus positions)\n"
     ]
    }
   ],
   "source": [
    "dictionary = Dictionary(texts)\n",
    "corpus = [dictionary.doc2bow(text) for text in texts]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set up two topic models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll be setting up two different LDA Topic models. A good one and bad one. To build a \"good\" topic model, we'll simply train it using more iterations than the bad one. Therefore the coherence should in theory be better for the good model than the bad one since it would be producing more \"human-interpretable\" topics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2017-11-22 13:58:32,114 : INFO : using symmetric alpha at 0.1\n",
      "2017-11-22 13:58:32,117 : INFO : using symmetric eta at 0.0002615062761506276\n",
      "2017-11-22 13:58:32,120 : INFO : using serial LDA version on this node\n",
      "2017-11-22 13:58:32,572 : INFO : running online (multi-pass) LDA training, 10 topics, 5 passes over the supplied corpus of 300 documents, updating model once every 300 documents, evaluating perplexity every 300 documents, iterating 200x with a convergence threshold of 0.001000\n",
      "2017-11-22 13:58:32,574 : WARNING : too few updates, training might not converge; consider increasing the number of passes or iterations to improve accuracy\n",
      "2017-11-22 13:58:33,174 : DEBUG : bound: at document #0\n",
      "2017-11-22 13:58:36,314 : INFO : -11.109 per-word bound, 2209.1 perplexity estimate based on a held-out corpus of 300 documents with 18861 words\n",
      "2017-11-22 13:58:36,316 : INFO : PROGRESS: pass 0, at document #300/300\n",
      "2017-11-22 13:58:36,318 : DEBUG : performing inference on a chunk of 300 documents\n",
      "2017-11-22 13:58:38,543 : DEBUG : 227/300 documents converged within 200 iterations\n",
      "2017-11-22 13:58:38,545 : DEBUG : updating topics\n",
      "2017-11-22 13:58:38,701 : INFO : topic #0 (0.100): 0.008*\"cancer\" + 0.007*\"month\" + 0.007*\"australia\" + 0.007*\"year\" + 0.006*\"day\" + 0.005*\"commission\" + 0.005*\"people\" + 0.005*\"minister\" + 0.004*\"police\" + 0.004*\"attack\"\n",
      "2017-11-22 13:58:38,703 : INFO : topic #6 (0.100): 0.010*\"government\" + 0.008*\"security\" + 0.008*\"president\" + 0.007*\"australia\" + 0.007*\"group\" + 0.006*\"force\" + 0.006*\"afghan\" + 0.006*\"police\" + 0.005*\"agreement\" + 0.005*\"man\"\n",
      "2017-11-22 13:58:38,704 : INFO : topic #9 (0.100): 0.010*\"australia\" + 0.009*\"arafat\" + 0.009*\"people\" + 0.008*\"test\" + 0.008*\"day\" + 0.007*\"government\" + 0.007*\"attack\" + 0.006*\"qantas\" + 0.006*\"group\" + 0.006*\"official\"\n",
      "2017-11-22 13:58:38,707 : INFO : topic #4 (0.100): 0.008*\"government\" + 0.007*\"attack\" + 0.006*\"day\" + 0.006*\"force\" + 0.005*\"test\" + 0.005*\"pakistan\" + 0.005*\"israeli\" + 0.005*\"australia\" + 0.005*\"israel\" + 0.005*\"arafat\"\n",
      "2017-11-22 13:58:38,709 : INFO : topic #7 (0.100): 0.010*\"metre\" + 0.008*\"israeli\" + 0.007*\"security\" + 0.006*\"year\" + 0.006*\"force\" + 0.006*\"people\" + 0.006*\"fire\" + 0.005*\"arafat\" + 0.005*\"event\" + 0.005*\"palestinian\"\n",
      "2017-11-22 13:58:38,711 : INFO : topic diff=6.915563, rho=1.000000\n",
      "2017-11-22 13:58:38,800 : DEBUG : bound: at document #0\n",
      "2017-11-22 13:58:40,182 : INFO : -7.925 per-word bound, 243.0 perplexity estimate based on a held-out corpus of 300 documents with 18861 words\n",
      "2017-11-22 13:58:40,184 : INFO : PROGRESS: pass 1, at document #300/300\n",
      "2017-11-22 13:58:40,186 : DEBUG : performing inference on a chunk of 300 documents\n",
      "2017-11-22 13:58:40,643 : DEBUG : 300/300 documents converged within 200 iterations\n",
      "2017-11-22 13:58:40,645 : DEBUG : updating topics\n",
      "2017-11-22 13:58:40,827 : INFO : topic #2 (0.100): 0.014*\"government\" + 0.009*\"australia\" + 0.009*\"man\" + 0.007*\"cent\" + 0.006*\"year\" + 0.006*\"airline\" + 0.006*\"flight\" + 0.006*\"hollingworth\" + 0.005*\"attack\" + 0.005*\"wicket\"\n",
      "2017-11-22 13:58:40,829 : INFO : topic #5 (0.100): 0.016*\"fire\" + 0.010*\"taliban\" + 0.010*\"firefighter\" + 0.010*\"area\" + 0.008*\"line\" + 0.008*\"afghanistan\" + 0.008*\"force\" + 0.008*\"authority\" + 0.008*\"government\" + 0.007*\"sydney\"\n",
      "2017-11-22 13:58:40,832 : INFO : topic #6 (0.100): 0.011*\"government\" + 0.009*\"security\" + 0.007*\"group\" + 0.007*\"force\" + 0.007*\"afghan\" + 0.007*\"australia\" + 0.006*\"president\" + 0.006*\"agreement\" + 0.006*\"police\" + 0.006*\"man\"\n",
      "2017-11-22 13:58:40,835 : INFO : topic #1 (0.100): 0.010*\"australia\" + 0.009*\"child\" + 0.009*\"year\" + 0.008*\"people\" + 0.007*\"day\" + 0.006*\"company\" + 0.006*\"report\" + 0.005*\"staff\" + 0.005*\"leader\" + 0.005*\"guide\"\n",
      "2017-11-22 13:58:40,838 : INFO : topic #3 (0.100): 0.013*\"afghanistan\" + 0.010*\"force\" + 0.008*\"tora_bora\" + 0.007*\"laden\" + 0.007*\"power\" + 0.007*\"security\" + 0.007*\"union\" + 0.006*\"area\" + 0.006*\"afghan\" + 0.006*\"commission\"\n",
      "2017-11-22 13:58:40,840 : INFO : topic diff=0.165499, rho=0.577350\n",
      "2017-11-22 13:58:40,940 : DEBUG : bound: at document #0\n",
      "2017-11-22 13:58:42,560 : INFO : -7.845 per-word bound, 230.0 perplexity estimate based on a held-out corpus of 300 documents with 18861 words\n",
      "2017-11-22 13:58:42,562 : INFO : PROGRESS: pass 2, at document #300/300\n",
      "2017-11-22 13:58:42,563 : DEBUG : performing inference on a chunk of 300 documents\n",
      "2017-11-22 13:58:42,996 : DEBUG : 300/300 documents converged within 200 iterations\n",
      "2017-11-22 13:58:42,999 : DEBUG : updating topics\n",
      "2017-11-22 13:58:43,154 : INFO : topic #9 (0.100): 0.012*\"arafat\" + 0.010*\"australia\" + 0.008*\"people\" + 0.008*\"test\" + 0.008*\"qantas\" + 0.008*\"day\" + 0.007*\"attack\" + 0.007*\"government\" + 0.007*\"group\" + 0.006*\"match\"\n",
      "2017-11-22 13:58:43,156 : INFO : topic #3 (0.100): 0.014*\"afghanistan\" + 0.010*\"force\" + 0.009*\"tora_bora\" + 0.008*\"laden\" + 0.007*\"union\" + 0.007*\"security\" + 0.007*\"power\" + 0.007*\"commission\" + 0.006*\"afghan\" + 0.006*\"qaeda\"\n",
      "2017-11-22 13:58:43,159 : INFO : topic #4 (0.100): 0.008*\"test\" + 0.008*\"government\" + 0.007*\"day\" + 0.007*\"australia\" + 0.007*\"attack\" + 0.006*\"pakistan\" + 0.006*\"detainee\" + 0.005*\"south_africa\" + 0.005*\"fire\" + 0.005*\"radio\"\n",
      "2017-11-22 13:58:43,163 : INFO : topic #6 (0.100): 0.012*\"government\" + 0.009*\"security\" + 0.008*\"afghan\" + 0.008*\"force\" + 0.008*\"group\" + 0.007*\"agreement\" + 0.007*\"australia\" + 0.006*\"police\" + 0.006*\"president\" + 0.006*\"man\"\n",
      "2017-11-22 13:58:43,167 : INFO : topic #2 (0.100): 0.014*\"government\" + 0.009*\"australia\" + 0.008*\"man\" + 0.007*\"cent\" + 0.007*\"year\" + 0.007*\"hollingworth\" + 0.006*\"airline\" + 0.006*\"flight\" + 0.005*\"wicket\" + 0.005*\"attack\"\n",
      "2017-11-22 13:58:43,171 : INFO : topic diff=0.136776, rho=0.500000\n",
      "2017-11-22 13:58:43,265 : DEBUG : bound: at document #0\n",
      "2017-11-22 13:58:44,710 : INFO : -7.795 per-word bound, 222.2 perplexity estimate based on a held-out corpus of 300 documents with 18861 words\n",
      "2017-11-22 13:58:44,711 : INFO : PROGRESS: pass 3, at document #300/300\n",
      "2017-11-22 13:58:44,712 : DEBUG : performing inference on a chunk of 300 documents\n",
      "2017-11-22 13:58:45,155 : DEBUG : 300/300 documents converged within 200 iterations\n",
      "2017-11-22 13:58:45,156 : DEBUG : updating topics\n",
      "2017-11-22 13:58:45,311 : INFO : topic #8 (0.100): 0.015*\"people\" + 0.010*\"year\" + 0.008*\"report\" + 0.008*\"new_york\" + 0.007*\"australia\" + 0.006*\"city\" + 0.006*\"economy\" + 0.006*\"rate\" + 0.005*\"government\" + 0.005*\"police\"\n",
      "2017-11-22 13:58:45,313 : INFO : topic #6 (0.100): 0.012*\"government\" + 0.009*\"security\" + 0.009*\"afghan\" + 0.009*\"force\" + 0.008*\"group\" + 0.007*\"agreement\" + 0.007*\"australia\" + 0.006*\"afghanistan\" + 0.006*\"man\" + 0.006*\"police\"\n",
      "2017-11-22 13:58:45,315 : INFO : topic #5 (0.100): 0.019*\"fire\" + 0.011*\"firefighter\" + 0.011*\"area\" + 0.010*\"taliban\" + 0.010*\"afghanistan\" + 0.009*\"line\" + 0.009*\"force\" + 0.008*\"authority\" + 0.008*\"sydney\" + 0.008*\"government\"\n",
      "2017-11-22 13:58:45,317 : INFO : topic #7 (0.100): 0.014*\"israeli\" + 0.011*\"metre\" + 0.010*\"arafat\" + 0.009*\"security\" + 0.008*\"palestinian\" + 0.007*\"force\" + 0.007*\"israel\" + 0.007*\"west_bank\" + 0.006*\"militant\" + 0.006*\"year\"\n",
      "2017-11-22 13:58:45,319 : INFO : topic #9 (0.100): 0.012*\"arafat\" + 0.010*\"australia\" + 0.008*\"people\" + 0.008*\"qantas\" + 0.008*\"test\" + 0.008*\"attack\" + 0.007*\"day\" + 0.007*\"government\" + 0.007*\"group\" + 0.007*\"official\"\n",
      "2017-11-22 13:58:45,321 : INFO : topic diff=0.110772, rho=0.447214\n",
      "2017-11-22 13:58:45,410 : DEBUG : bound: at document #0\n",
      "2017-11-22 13:58:46,953 : INFO : -7.761 per-word bound, 216.9 perplexity estimate based on a held-out corpus of 300 documents with 18861 words\n",
      "2017-11-22 13:58:46,954 : INFO : PROGRESS: pass 4, at document #300/300\n",
      "2017-11-22 13:58:46,955 : DEBUG : performing inference on a chunk of 300 documents\n",
      "2017-11-22 13:58:47,332 : DEBUG : 300/300 documents converged within 200 iterations\n",
      "2017-11-22 13:58:47,334 : DEBUG : updating topics\n",
      "2017-11-22 13:58:47,511 : INFO : topic #5 (0.100): 0.019*\"fire\" + 0.012*\"firefighter\" + 0.011*\"area\" + 0.011*\"taliban\" + 0.010*\"afghanistan\" + 0.009*\"line\" + 0.009*\"force\" + 0.009*\"sydney\" + 0.008*\"wind\" + 0.008*\"government\"\n",
      "2017-11-22 13:58:47,514 : INFO : topic #3 (0.100): 0.015*\"afghanistan\" + 0.011*\"force\" + 0.009*\"laden\" + 0.009*\"tora_bora\" + 0.008*\"commission\" + 0.007*\"security\" + 0.007*\"union\" + 0.007*\"united_states\" + 0.007*\"qaeda\" + 0.007*\"power\"\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2017-11-22 13:58:47,515 : INFO : topic #8 (0.100): 0.015*\"people\" + 0.010*\"year\" + 0.008*\"report\" + 0.008*\"new_york\" + 0.007*\"australia\" + 0.006*\"city\" + 0.006*\"economy\" + 0.006*\"rate\" + 0.005*\"world\" + 0.005*\"government\"\n",
      "2017-11-22 13:58:47,518 : INFO : topic #9 (0.100): 0.013*\"arafat\" + 0.010*\"australia\" + 0.009*\"qantas\" + 0.008*\"people\" + 0.008*\"attack\" + 0.007*\"test\" + 0.007*\"day\" + 0.007*\"government\" + 0.007*\"group\" + 0.007*\"official\"\n",
      "2017-11-22 13:58:47,521 : INFO : topic #7 (0.100): 0.015*\"israeli\" + 0.012*\"arafat\" + 0.011*\"metre\" + 0.009*\"security\" + 0.009*\"palestinian\" + 0.008*\"israel\" + 0.007*\"force\" + 0.007*\"west_bank\" + 0.007*\"militant\" + 0.007*\"official\"\n",
      "2017-11-22 13:58:47,524 : INFO : topic diff=0.090895, rho=0.408248\n",
      "2017-11-22 13:58:47,528 : INFO : using symmetric alpha at 0.1\n",
      "2017-11-22 13:58:47,531 : INFO : using symmetric eta at 0.0002615062761506276\n",
      "2017-11-22 13:58:47,534 : INFO : using serial LDA version on this node\n",
      "2017-11-22 13:58:48,031 : INFO : running online (single-pass) LDA training, 10 topics, 1 passes over the supplied corpus of 300 documents, updating model once every 300 documents, evaluating perplexity every 300 documents, iterating 1x with a convergence threshold of 0.001000\n",
      "2017-11-22 13:58:48,033 : WARNING : too few updates, training might not converge; consider increasing the number of passes or iterations to improve accuracy\n",
      "2017-11-22 13:58:48,752 : DEBUG : bound: at document #0\n",
      "2017-11-22 13:58:50,138 : INFO : -11.396 per-word bound, 2695.1 perplexity estimate based on a held-out corpus of 300 documents with 18861 words\n",
      "2017-11-22 13:58:50,140 : INFO : PROGRESS: pass 0, at document #300/300\n",
      "2017-11-22 13:58:50,142 : DEBUG : performing inference on a chunk of 300 documents\n",
      "2017-11-22 13:58:50,238 : DEBUG : 0/300 documents converged within 1 iterations\n",
      "2017-11-22 13:58:50,240 : DEBUG : updating topics\n",
      "2017-11-22 13:58:50,346 : INFO : topic #1 (0.100): 0.008*\"australia\" + 0.007*\"government\" + 0.006*\"people\" + 0.005*\"day\" + 0.005*\"force\" + 0.005*\"attack\" + 0.004*\"year\" + 0.004*\"today\" + 0.004*\"police\" + 0.004*\"arafat\"\n",
      "2017-11-22 13:58:50,348 : INFO : topic #5 (0.100): 0.007*\"year\" + 0.007*\"australia\" + 0.007*\"government\" + 0.006*\"day\" + 0.006*\"people\" + 0.005*\"afghanistan\" + 0.005*\"force\" + 0.004*\"police\" + 0.004*\"security\" + 0.004*\"area\"\n",
      "2017-11-22 13:58:50,349 : INFO : topic #9 (0.100): 0.007*\"government\" + 0.007*\"year\" + 0.006*\"australia\" + 0.005*\"force\" + 0.005*\"day\" + 0.005*\"attack\" + 0.005*\"people\" + 0.004*\"afghanistan\" + 0.004*\"time\" + 0.004*\"man\"\n",
      "2017-11-22 13:58:50,352 : INFO : topic #2 (0.100): 0.006*\"australia\" + 0.005*\"day\" + 0.005*\"people\" + 0.005*\"government\" + 0.005*\"attack\" + 0.004*\"force\" + 0.004*\"area\" + 0.004*\"afghanistan\" + 0.004*\"year\" + 0.004*\"fire\"\n",
      "2017-11-22 13:58:50,354 : INFO : topic #0 (0.100): 0.007*\"australia\" + 0.006*\"year\" + 0.006*\"government\" + 0.006*\"people\" + 0.005*\"arafat\" + 0.005*\"day\" + 0.005*\"force\" + 0.005*\"fire\" + 0.004*\"security\" + 0.004*\"israeli\"\n",
      "2017-11-22 13:58:50,355 : INFO : topic diff=2.626188, rho=1.000000\n"
     ]
    }
   ],
   "source": [
    "goodLdaModel = LdaModel(corpus=corpus, id2word=dictionary, iterations=200, passes=5, num_topics=10)\n",
    "badLdaModel = LdaModel(corpus=corpus, id2word=dictionary, iterations=1, num_topics=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View the pipeline parameters for one coherence model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2017-11-22 13:59:36,510 : DEBUG : Setting topics to those of the model: LdaModel(num_terms=3824, num_topics=10, decay=0.5, chunksize=2000)\n",
      "2017-11-22 13:59:36,515 : DEBUG : Setting topics to those of the model: LdaModel(num_terms=3824, num_topics=10, decay=0.5, chunksize=2000)\n"
     ]
    }
   ],
   "source": [
    "goodcm = CoherenceModel(model=goodLdaModel, texts=texts, dictionary=dictionary, coherence='c_v')\n",
    "badcm = CoherenceModel(model=badLdaModel, texts=texts, dictionary=dictionary, coherence='c_v')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pipeline parameters for C_V coherence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Coherence_Measure(seg=<function s_one_set at 0x1a076a3e18>, prob=<function p_boolean_sliding_window at 0x1a07696d08>, conf=<function cosine_similarity at 0x1a076fde18>, aggr=<function arithmetic_mean at 0x1a076ff400>)\n"
     ]
    }
   ],
   "source": [
    "print(goodcm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Print coherence values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2017-11-22 14:00:42,194 : INFO : using ParallelWordOccurrenceAccumulator(processes=3, batch_size=64) to estimate probabilities from sliding windows\n",
      "2017-11-22 14:00:42,342 : DEBUG : completed batch 0; 64 documents processed (92 virtual)\n",
      "2017-11-22 14:00:42,398 : DEBUG : completed batch 0; 64 documents processed (258 virtual)\n",
      "2017-11-22 14:00:42,422 : DEBUG : completed batch 0; 64 documents processed (305 virtual)\n",
      "2017-11-22 14:00:42,431 : DEBUG : observed sentinel value; terminating\n",
      "2017-11-22 14:00:42,434 : DEBUG : finished all batches; 64 documents processed (305 virtual)\n",
      "2017-11-22 14:00:42,438 : INFO : serializing accumulator to return to master...\n",
      "2017-11-22 14:00:42,442 : INFO : accumulator serialized\n",
      "2017-11-22 14:00:42,503 : DEBUG : completed batch 1; 128 documents processed (332 virtual)\n",
      "2017-11-22 14:00:42,506 : DEBUG : observed sentinel value; terminating\n",
      "2017-11-22 14:00:42,509 : DEBUG : finished all batches; 128 documents processed (332 virtual)\n",
      "2017-11-22 14:00:42,512 : INFO : serializing accumulator to return to master...\n",
      "2017-11-22 14:00:42,516 : INFO : accumulator serialized\n",
      "2017-11-22 14:00:42,561 : DEBUG : completed batch 1; 108 documents processed (485 virtual)\n",
      "2017-11-22 14:00:42,564 : DEBUG : observed sentinel value; terminating\n",
      "2017-11-22 14:00:42,570 : DEBUG : finished all batches; 108 documents processed (485 virtual)\n",
      "2017-11-22 14:00:42,574 : INFO : serializing accumulator to return to master...\n",
      "2017-11-22 14:00:42,578 : INFO : accumulator serialized\n",
      "2017-11-22 14:00:42,609 : INFO : 3 accumulators retrieved from output queue\n",
      "2017-11-22 14:00:42,642 : INFO : accumulated word occurrence stats for 1122 virtual documents\n",
      "2017-11-22 14:00:43,391 : INFO : using ParallelWordOccurrenceAccumulator(processes=3, batch_size=64) to estimate probabilities from sliding windows\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.350333107184\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2017-11-22 14:00:43,446 : DEBUG : completed batch 0; 64 documents processed (92 virtual)\n",
      "2017-11-22 14:00:43,498 : DEBUG : completed batch 0; 64 documents processed (305 virtual)\n",
      "2017-11-22 14:00:43,496 : DEBUG : completed batch 0; 64 documents processed (258 virtual)\n",
      "2017-11-22 14:00:43,503 : DEBUG : observed sentinel value; terminating\n",
      "2017-11-22 14:00:43,506 : DEBUG : finished all batches; 64 documents processed (305 virtual)\n",
      "2017-11-22 14:00:43,511 : INFO : serializing accumulator to return to master...\n",
      "2017-11-22 14:00:43,501 : DEBUG : completed batch 1; 128 documents processed (332 virtual)\n",
      "2017-11-22 14:00:43,514 : INFO : accumulator serialized\n",
      "2017-11-22 14:00:43,535 : DEBUG : observed sentinel value; terminating\n",
      "2017-11-22 14:00:43,538 : DEBUG : finished all batches; 128 documents processed (332 virtual)\n",
      "2017-11-22 14:00:43,541 : INFO : serializing accumulator to return to master...\n",
      "2017-11-22 14:00:43,545 : INFO : accumulator serialized\n",
      "2017-11-22 14:00:43,577 : DEBUG : completed batch 1; 108 documents processed (485 virtual)\n",
      "2017-11-22 14:00:43,581 : DEBUG : observed sentinel value; terminating\n",
      "2017-11-22 14:00:43,586 : DEBUG : finished all batches; 108 documents processed (485 virtual)\n",
      "2017-11-22 14:00:43,591 : INFO : serializing accumulator to return to master...\n",
      "2017-11-22 14:00:43,594 : INFO : accumulator serialized\n",
      "2017-11-22 14:00:43,617 : INFO : 3 accumulators retrieved from output queue\n",
      "2017-11-22 14:00:43,636 : INFO : accumulated word occurrence stats for 1122 virtual documents\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.272100427948\n"
     ]
    }
   ],
   "source": [
    "print(goodcm.get_coherence())\n",
    "print(badcm.get_coherence())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}