diff --git a/docs/notebooks/Tensorboard_doc2vec.ipynb b/docs/notebooks/Tensorboard_doc2vec.ipynb deleted file mode 100644 index 865f9dd56f..0000000000 --- a/docs/notebooks/Tensorboard_doc2vec.ipynb +++ /dev/null @@ -1,905 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Visualizing Doc2Vec with TensorBoard\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "collapsed": true - }, - "source": [ - "\n", - "\n", - "\n", - "\n", - "In this tutorial, I will explain how to visualize Doc2Vec Embeddings aka [Paragraph Vectors]() via TensorBoard. It is a data visualization framework for visualizing and inspecting the TensorFlow runs and graphs. We will use a built-in Tensorboard visualizer called *Embedding Projector* in this tutorial. It lets you interactively visualize and analyze high-dimensional data like embeddings.\n", - "\n", - "For this tutorial, a transformed MovieLens dataset[1] was used from this [repository](https://github.com/RaRe-Technologies/movie-plots-by-genre) and the movie titles were added afterwards. You can download the prepared csv from [here](https://github.com/parulsethi/DocViz/blob/master/movie_plots.csv). The input documents for training are the synopsis of movies, on which Doc2Vec model is trained. \n", - "\n", - "The visualizations will be a scatterplot as seen in the above image, where each datapoint is labelled by the movie title and colored by it's corresponding genre. You can also visit this [Projector link](http://projector.tensorflow.org/?config=https://raw.githubusercontent.com/parulsethi/DocViz/master/movie_plot_config.json) which is configured with my embeddings for the above mentioned dataset. " - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Define a Function to Read and Preprocess Text" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/html": [ - "
\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
MovieIDTitlesPlotsGenres
01Toy Story (1995)A little boy named Andy loves to be in his roo...animation
12Jumanji (1995)When two kids find and play a magical board ga...fantasy
23Grumpier Old Men (1995)Things don't seem to change much in Wabasha Co...comedy
36Heat (1995)Hunters and their prey--Neil and his professio...action
47Sabrina (1995)An ugly duckling having undergone a remarkable...romance
59Sudden Death (1995)Some terrorists kidnap the Vice President of t...action
610GoldenEye (1995)James Bond teams up with the lone survivor of ...action
715Cutthroat Island (1995)Morgan Adams and her slave, William Shaw, are ...action
817Sense and Sensibility (1995)When Mr. Dashwood dies, he must leave the bulk...romance
918Four Rooms (1995)This movie features the collaborative director...comedy
1019Ace Ventura: When Nature Calls (1995)Ace Ventura, emerging from self-imposed exile ...comedy
1129City of Lost Children, The (Cité des enfants p...Krank (Daniel Emilfork), who cannot dream, kid...sci-fi
1232Twelve Monkeys (a.k.a. 12 Monkeys) (1995)In a future world devastated by disease, a con...sci-fi
1334Babe (1995)Farmer Hoggett wins a runt piglet at a local f...fantasy
1439Clueless (1995)A rich high school student tries to boost a ne...romance
1544Mortal Kombat (1995)Based on the popular video game of the same na...action
1648Pocahontas (1995)Capt. John Smith leads a rag-tag band of Engli...animation
1750Usual Suspects, The (1995)Following a truck hijack in New York, five con...comedy
1857Home for the Holidays (1995)After losing her job, making out with her soon...comedy
1969Friday (1995)Two homies, Smokey and Craig, smoke a dope dea...comedy
2070From Dusk Till Dawn (1996)Two criminals and their hostages unknowingly s...action
2176Screamers (1995)(SIRIUS 6B, Year 2078) On a distant mining pla...sci-fi
2282Antonia's Line (Antonia) (1995)In an anonymous Dutch village, a sturdy, stron...fantasy
2388Black Sheep (1996)Comedy about the prospective Washington State ...comedy
2495Broken Arrow (1996)\"Broken Arrow\" is the term used to describe a ...action
25104Happy Gilmore (1996)A rejected hockey player puts his skills to th...comedy
26105Bridges of Madison County, The (1995)Photographer Robert Kincaid wanders into the l...romance
27110Braveheart (1995)When his secret bride is executed for assaulti...action
28141Birdcage, The (1996)Armand Goldman owns a popular drag nightclub i...comedy
29145Bad Boys (1995)Marcus Burnett is a hen-pecked family man. Mik...action
...............
1813122902Fantastic Four (2015)FANTASTIC FOUR, a contemporary re-imagining of...sci-fi
1814127098Louis C.K.: Live at The Comedy Store (2015)Comedian Louis C.K. performs live at the Comed...comedy
1815127158Tig (2015)An intimate, mixed media documentary that foll...comedy
1816127202Me and Earl and the Dying Girl (2015)Seventeen-year-old Greg has managed to become ...comedy
1817129354Focus (2015)In the midst of veteran con man Nicky's latest...action
1818129428The Second Best Exotic Marigold Hotel (2015)The Second Best Exotic Marigold Hotel is the e...comedy
1819129937Run All Night (2015)Professional Brooklyn hitman Jimmy Conlon is m...action
1820130490Insurgent (2015)One choice can transform you-or it can destroy...sci-fi
1821130520Home (2015)An alien on the run from his own people makes ...animation
1822130634Furious 7 (2015)Dominic and his crew thought they'd left the c...action
1823131013Get Hard (2015)Kevin Hart plays the role of Darnell--a family...comedy
1824132046Tomorrowland (2015)Bound by a shared destiny, a bright, optimisti...sci-fi
1825132480The Age of Adaline (2015)A young woman, born at the turn of the 20th ce...romance
1826132488Lovesick (2014)Lovesick is the comic tale of Charlie Darby (M...fantasy
1827132796San Andreas (2015)In San Andreas, California is experiencing a s...action
1828132961Far from the Madding Crowd (2015)In Victorian England, the independent and head...romance
1829133195Hitman: Agent 47 (2015)An assassin teams up with a woman to help her ...action
1830133645Carol (2015)In an adaptation of Patricia Highsmith's semin...romance
1831134130The Martian (2015)During a manned mission to Mars, Astronaut Mar...sci-fi
1832134368Spy (2015)A desk-bound CIA analyst volunteers to go unde...comedy
1833134783Entourage (2015)Movie star Vincent Chase, together with his bo...comedy
1834134853Inside Out (2015)After young Riley is uprooted from her Midwest...comedy
1835135518Self/less (2015)A dying real estate mogul transfers his consci...sci-fi
1836135861Ted 2 (2015)Months after John's divorce, Ted and Tami-Lynn...comedy
1837135887Minions (2015)Ever since the dawn of time, the Minions have ...comedy
1838136016The Good Dinosaur (2015)In a world where dinosaurs and humans live sid...animation
1839139855Anomalisa (2015)Michael Stone, an author that specializes in c...animation
1840142997Hotel Transylvania 2 (2015)The Drac pack is back for an all-new monster c...animation
1841145935Peanuts Movie, The (2015)Charlie Brown, Lucy, Snoopy, and the whole gan...animation
1842149406Kung Fu Panda 3 (2016)Continuing his \"legendary adventures of awesom...comedy
\n", - "

1843 rows × 4 columns

\n", - "
" - ], - "text/plain": [ - " MovieID Titles \\\n", - "0 1 Toy Story (1995) \n", - "1 2 Jumanji (1995) \n", - "2 3 Grumpier Old Men (1995) \n", - "3 6 Heat (1995) \n", - "4 7 Sabrina (1995) \n", - "5 9 Sudden Death (1995) \n", - "6 10 GoldenEye (1995) \n", - "7 15 Cutthroat Island (1995) \n", - "8 17 Sense and Sensibility (1995) \n", - "9 18 Four Rooms (1995) \n", - "10 19 Ace Ventura: When Nature Calls (1995) \n", - "11 29 City of Lost Children, The (Cité des enfants p... \n", - "12 32 Twelve Monkeys (a.k.a. 12 Monkeys) (1995) \n", - "13 34 Babe (1995) \n", - "14 39 Clueless (1995) \n", - "15 44 Mortal Kombat (1995) \n", - "16 48 Pocahontas (1995) \n", - "17 50 Usual Suspects, The (1995) \n", - "18 57 Home for the Holidays (1995) \n", - "19 69 Friday (1995) \n", - "20 70 From Dusk Till Dawn (1996) \n", - "21 76 Screamers (1995) \n", - "22 82 Antonia's Line (Antonia) (1995) \n", - "23 88 Black Sheep (1996) \n", - "24 95 Broken Arrow (1996) \n", - "25 104 Happy Gilmore (1996) \n", - "26 105 Bridges of Madison County, The (1995) \n", - "27 110 Braveheart (1995) \n", - "28 141 Birdcage, The (1996) \n", - "29 145 Bad Boys (1995) \n", - "... ... ... \n", - "1813 122902 Fantastic Four (2015) \n", - "1814 127098 Louis C.K.: Live at The Comedy Store (2015) \n", - "1815 127158 Tig (2015) \n", - "1816 127202 Me and Earl and the Dying Girl (2015) \n", - "1817 129354 Focus (2015) \n", - "1818 129428 The Second Best Exotic Marigold Hotel (2015) \n", - "1819 129937 Run All Night (2015) \n", - "1820 130490 Insurgent (2015) \n", - "1821 130520 Home (2015) \n", - "1822 130634 Furious 7 (2015) \n", - "1823 131013 Get Hard (2015) \n", - "1824 132046 Tomorrowland (2015) \n", - "1825 132480 The Age of Adaline (2015) \n", - "1826 132488 Lovesick (2014) \n", - "1827 132796 San Andreas (2015) \n", - "1828 132961 Far from the Madding Crowd (2015) \n", - "1829 133195 Hitman: Agent 47 (2015) \n", - "1830 133645 Carol (2015) \n", - "1831 134130 The Martian (2015) \n", - "1832 134368 Spy (2015) \n", - "1833 134783 Entourage (2015) \n", - "1834 134853 Inside Out (2015) \n", - "1835 135518 Self/less (2015) \n", - "1836 135861 Ted 2 (2015) \n", - "1837 135887 Minions (2015) \n", - "1838 136016 The Good Dinosaur (2015) \n", - "1839 139855 Anomalisa (2015) \n", - "1840 142997 Hotel Transylvania 2 (2015) \n", - "1841 145935 Peanuts Movie, The (2015) \n", - "1842 149406 Kung Fu Panda 3 (2016) \n", - "\n", - " Plots Genres \n", - "0 A little boy named Andy loves to be in his roo... animation \n", - "1 When two kids find and play a magical board ga... fantasy \n", - "2 Things don't seem to change much in Wabasha Co... comedy \n", - "3 Hunters and their prey--Neil and his professio... action \n", - "4 An ugly duckling having undergone a remarkable... romance \n", - "5 Some terrorists kidnap the Vice President of t... action \n", - "6 James Bond teams up with the lone survivor of ... action \n", - "7 Morgan Adams and her slave, William Shaw, are ... action \n", - "8 When Mr. Dashwood dies, he must leave the bulk... romance \n", - "9 This movie features the collaborative director... comedy \n", - "10 Ace Ventura, emerging from self-imposed exile ... comedy \n", - "11 Krank (Daniel Emilfork), who cannot dream, kid... sci-fi \n", - "12 In a future world devastated by disease, a con... sci-fi \n", - "13 Farmer Hoggett wins a runt piglet at a local f... fantasy \n", - "14 A rich high school student tries to boost a ne... romance \n", - "15 Based on the popular video game of the same na... action \n", - "16 Capt. John Smith leads a rag-tag band of Engli... animation \n", - "17 Following a truck hijack in New York, five con... comedy \n", - "18 After losing her job, making out with her soon... comedy \n", - "19 Two homies, Smokey and Craig, smoke a dope dea... comedy \n", - "20 Two criminals and their hostages unknowingly s... action \n", - "21 (SIRIUS 6B, Year 2078) On a distant mining pla... sci-fi \n", - "22 In an anonymous Dutch village, a sturdy, stron... fantasy \n", - "23 Comedy about the prospective Washington State ... comedy \n", - "24 \"Broken Arrow\" is the term used to describe a ... action \n", - "25 A rejected hockey player puts his skills to th... comedy \n", - "26 Photographer Robert Kincaid wanders into the l... romance \n", - "27 When his secret bride is executed for assaulti... action \n", - "28 Armand Goldman owns a popular drag nightclub i... comedy \n", - "29 Marcus Burnett is a hen-pecked family man. Mik... action \n", - "... ... ... \n", - "1813 FANTASTIC FOUR, a contemporary re-imagining of... sci-fi \n", - "1814 Comedian Louis C.K. performs live at the Comed... comedy \n", - "1815 An intimate, mixed media documentary that foll... comedy \n", - "1816 Seventeen-year-old Greg has managed to become ... comedy \n", - "1817 In the midst of veteran con man Nicky's latest... action \n", - "1818 The Second Best Exotic Marigold Hotel is the e... comedy \n", - "1819 Professional Brooklyn hitman Jimmy Conlon is m... action \n", - "1820 One choice can transform you-or it can destroy... sci-fi \n", - "1821 An alien on the run from his own people makes ... animation \n", - "1822 Dominic and his crew thought they'd left the c... action \n", - "1823 Kevin Hart plays the role of Darnell--a family... comedy \n", - "1824 Bound by a shared destiny, a bright, optimisti... sci-fi \n", - "1825 A young woman, born at the turn of the 20th ce... romance \n", - "1826 Lovesick is the comic tale of Charlie Darby (M... fantasy \n", - "1827 In San Andreas, California is experiencing a s... action \n", - "1828 In Victorian England, the independent and head... romance \n", - "1829 An assassin teams up with a woman to help her ... action \n", - "1830 In an adaptation of Patricia Highsmith's semin... romance \n", - "1831 During a manned mission to Mars, Astronaut Mar... sci-fi \n", - "1832 A desk-bound CIA analyst volunteers to go unde... comedy \n", - "1833 Movie star Vincent Chase, together with his bo... comedy \n", - "1834 After young Riley is uprooted from her Midwest... comedy \n", - "1835 A dying real estate mogul transfers his consci... sci-fi \n", - "1836 Months after John's divorce, Ted and Tami-Lynn... comedy \n", - "1837 Ever since the dawn of time, the Minions have ... comedy \n", - "1838 In a world where dinosaurs and humans live sid... animation \n", - "1839 Michael Stone, an author that specializes in c... animation \n", - "1840 The Drac pack is back for an all-new monster c... animation \n", - "1841 Charlie Brown, Lucy, Snoopy, and the whole gan... animation \n", - "1842 Continuing his \"legendary adventures of awesom... comedy \n", - "\n", - "[1843 rows x 4 columns]" - ] - }, - "execution_count": 2, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "import gensim\n", - "import pandas as pd\n", - "import smart_open\n", - "import random\n", - "\n", - "# read data\n", - "dataframe = pd.read_csv('movie_plots.csv')\n", - "dataframe" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Below, we define a function to read the training documents, pre-process each document using a simple gensim pre-processing tool (i.e., tokenize text into individual words, remove punctuation, set to lowercase, etc), and return a list of words. Also, to train the model, we'll need to associate a tag/number with each document of the training corpus. In our case, the tag is simply the zero-based line number." - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "def read_corpus(documents):\n", - " for i, plot in enumerate(documents):\n", - " yield gensim.models.doc2vec.TaggedDocument(gensim.utils.simple_preprocess(plot, max_len=30), [i])" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "train_corpus = list(read_corpus(dataframe.Plots))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's take a look at the training corpus." - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/plain": [ - "[TaggedDocument(words=[u'little', u'boy', u'named', u'andy', u'loves', u'to', u'be', u'in', u'his', u'room', u'playing', u'with', u'his', u'toys', u'especially', u'his', u'doll', u'named', u'woody', u'but', u'what', u'do', u'the', u'toys', u'do', u'when', u'andy', u'is', u'not', u'with', u'them', u'they', u'come', u'to', u'life', u'woody', u'believes', u'that', u'he', u'has', u'life', u'as', u'toy', u'good', u'however', u'he', u'must', u'worry', u'about', u'andy', u'family', u'moving', u'and', u'what', u'woody', u'does', u'not', u'know', u'is', u'about', u'andy', u'birthday', u'party', u'woody', u'does', u'not', u'realize', u'that', u'andy', u'mother', u'gave', u'him', u'an', u'action', u'figure', u'known', u'as', u'buzz', u'lightyear', u'who', u'does', u'not', u'believe', u'that', u'he', u'is', u'toy', u'and', u'quickly', u'becomes', u'andy', u'new', u'favorite', u'toy', u'woody', u'who', u'is', u'now', u'consumed', u'with', u'jealousy', u'tries', u'to', u'get', u'rid', u'of', u'buzz', u'then', u'both', u'woody', u'and', u'buzz', u'are', u'now', u'lost', u'they', u'must', u'find', u'way', u'to', u'get', u'back', u'to', u'andy', u'before', u'he', u'moves', u'without', u'them', u'but', u'they', u'will', u'have', u'to', u'pass', u'through', u'ruthless', u'toy', u'killer', u'sid', u'phillips'], tags=[0]),\n", - " TaggedDocument(words=[u'when', u'two', u'kids', u'find', u'and', u'play', u'magical', u'board', u'game', u'they', u'release', u'man', u'trapped', u'for', u'decades', u'in', u'it', u'and', u'host', u'of', u'dangers', u'that', u'can', u'only', u'be', u'stopped', u'by', u'finishing', u'the', u'game'], tags=[1])]" - ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "train_corpus[:2]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Training the Doc2Vec Model\n", - "We'll instantiate a Doc2Vec model with a vector size with 50 words and iterating over the training corpus 55 times. We set the minimum word count to 2 in order to give higher frequency words more weighting. Model accuracy can be improved by increasing the number of iterations but this generally increases the training time. Small datasets with short documents, like this one, can benefit from more training passes." - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/plain": [ - "92031" - ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "model = gensim.models.doc2vec.Doc2Vec(size=50, min_count=2, iter=55)\n", - "model.build_vocab(train_corpus)\n", - "model.train(train_corpus, total_examples=model.corpus_count, epochs=model.iter)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now, we'll save the document embedding vectors per doctag." - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": { - "collapsed": false - }, - "outputs": [], - "source": [ - "model.save_word2vec_format('doc_tensor.w2v', doctag_vec=True, word_vec=False) " - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Prepare the Input files for Tensorboard" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Tensorboard takes two Input files. One containing the embedding vectors and the other containing relevant metadata. We'll use a gensim script to directly convert the embedding file saved in word2vec format above to the tsv format required in Tensorboard." - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "2017-04-20 02:23:05,284 : MainThread : INFO : running ../../gensim/scripts/word2vec2tensor.py -i doc_tensor.w2v -o movie_plot\n", - "2017-04-20 02:23:05,286 : MainThread : INFO : loading projection weights from doc_tensor.w2v\n", - "2017-04-20 02:23:05,464 : MainThread : INFO : loaded (1843, 50) matrix from doc_tensor.w2v\n", - "2017-04-20 02:23:05,578 : MainThread : INFO : 2D tensor file saved to movie_plot_tensor.tsv\n", - "2017-04-20 02:23:05,579 : MainThread : INFO : Tensor metadata file saved to movie_plot_metadata.tsv\n", - "2017-04-20 02:23:05,581 : MainThread : INFO : finished running word2vec2tensor.py\n" - ] - } - ], - "source": [ - "%run ../../gensim/scripts/word2vec2tensor.py -i doc_tensor.w2v -o movie_plot" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The script above generates two files, `movie_plot_tensor.tsv` which contain the embedding vectors and `movie_plot_metadata.tsv` containing doctags. But, these doctags are simply the unique index values and hence are not really useful to interpret what the document was while visualizing. So, we will overwrite `movie_plot_metadata.tsv` to have a custom metadata file with two columns. The first column will be for the movie titles and the second for their corresponding genres." - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "with open('movie_plot_metadata.tsv','w') as w:\n", - " w.write('Titles\\tGenres\\n')\n", - " for i,j in zip(dataframe.Titles, dataframe.Genres):\n", - " w.write(\"%s\\t%s\\n\" % (i,j))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "collapsed": true - }, - "source": [ - "Now you can go to http://projector.tensorflow.org/ and upload the two files by clicking on *Load data* in the left panel.\n", - "\n", - "For demo purposes I have uploaded the Doc2Vec embeddings generated from the model trained above [here](https://github.com/parulsethi/DocViz). You can access the Embedding projector configured with these uploaded embeddings at this [link](http://projector.tensorflow.org/?config=https://raw.githubusercontent.com/parulsethi/DocViz/master/movie_plot_config.json)." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "collapsed": true - }, - "source": [ - "# Using Tensorboard" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "For the visualization purpose, the multi-dimensional embeddings that we get from the Doc2Vec model above, needs to be downsized to 2 or 3 dimensions. So that we basically end up with a new 2d or 3d embedding which tries to preserve information from the original multi-dimensional embedding. As these vectors are reduced to a much smaller dimension, the exact cosine/euclidean distances between them are not preserved, but rather relative, and hence as you’ll see below the nearest similarity results may change.\n", - "\n", - "TensorBoard has two popular dimensionality reduction methods for visualizing the embeddings and also provides a custom method based on text searches:\n", - "\n", - "- **Principal Component Analysis**: PCA aims at exploring the global structure in data, and could end up losing the local similarities between neighbours. It maximizes the total variance in the lower dimensional subspace and hence, often preserves the larger pairwise distances better than the smaller ones. See an intuition behind it in this nicely explained [answer](https://stats.stackexchange.com/questions/176672/what-is-meant-by-pca-preserving-only-large-pairwise-distances) on stackexchange.\n", - "\n", - "\n", - "- **T-SNE**: The idea of T-SNE is to place the local neighbours close to each other, and almost completely ignoring the global structure. It is useful for exploring local neighborhoods and finding local clusters. But the global trends are not represented accurately and the separation between different groups is often not preserved (see the t-sne plots of our data below which testify the same).\n", - "\n", - "\n", - "- **Custom Projections**: This is a custom bethod based on the text searches you define for different directions. It could be useful for finding meaningful directions in the vector space, for example, female to male, currency to country etc.\n", - "\n", - "You can refer to this [doc](https://www.tensorflow.org/get_started/embedding_viz) for instructions on how to use and navigate through different panels available in TensorBoard." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Visualize using PCA\n", - "\n", - "The Embedding Projector computes the top 10 principal components. The menu at the left panel lets you project those components onto any combination of two or three. \n", - "\n", - "The above plot was made using the first two principal components with total variance covered being 36.5%." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "## Visualize using T-SNE\n", - "\n", - "Data is visualized by animating through every iteration of the t-sne algorithm. The t-sne menu at the left lets you adjust the value of it's two hyperparameters. The first one is **Perplexity**, which is basically a measure of information. It may be viewed as a knob that sets the number of effective nearest neighbors[2]. The second one is **learning rate** that defines how quickly an algorithm learns on encountering new examples/data points.\n", - "\n", - "\n", - "\n", - "The above plot was generated with perplexity 8, learning rate 10 and iteration 500. Though the results could vary on successive runs, and you may not get the exact plot as above with same hyperparameter settings. But some small clusters will start forming as above, with different orientations." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Conclusion\n", - "\n", - "We learned about visualizing the Document Embeddings through Tensorboard's Embedding Projector. It is a useful tool for visualizing different types of data for example, word embeddings, document embeddings or the gene expressions and biological sequences. It just needs an input of 2D tensors and then you can explore your data using provided algorithms. You can also perform nearest neighbours search to find most similar data points to your query point.\n", - "\n", - "# References\n", - " 1. https://grouplens.org/datasets/movielens/\n", - " 2. https://lvdmaaten.github.io/tsne/\n", - " \n", - "\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 2", - "language": "python", - "name": "python2" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 2 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython2", - "version": "2.7.13" - } - }, - "nbformat": 4, - "nbformat_minor": 2 -} diff --git a/docs/notebooks/Tensorboard_visualizations.ipynb b/docs/notebooks/Tensorboard_visualizations.ipynb new file mode 100644 index 0000000000..58e031a3bd --- /dev/null +++ b/docs/notebooks/Tensorboard_visualizations.ipynb @@ -0,0 +1,1282 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# TensorBoard Visualizations\n", + "\n", + "\n", + "In this tutorial, we will learn how to visualize different types of NLP based Embeddings via TensorBoard. TensorBoard is a data visualization framework for visualizing and inspecting the TensorFlow runs and graphs. We will use a built-in Tensorboard visualizer called *Embedding Projector* in this tutorial. It lets you interactively visualize and analyze high-dimensional data like embeddings.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Read Data \n", + "\n", + "For this tutorial, a transformed MovieLens dataset[1] is used. You can download the final prepared csv from [here](https://github.com/parulsethi/DocViz/blob/master/movie_plots.csv)." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
MovieIDTitlesPlotsGenres
01Toy Story (1995)A little boy named Andy loves to be in his roo...animation
12Jumanji (1995)When two kids find and play a magical board ga...fantasy
23Grumpier Old Men (1995)Things don't seem to change much in Wabasha Co...comedy
36Heat (1995)Hunters and their prey--Neil and his professio...action
47Sabrina (1995)An ugly duckling having undergone a remarkable...romance
59Sudden Death (1995)Some terrorists kidnap the Vice President of t...action
610GoldenEye (1995)James Bond teams up with the lone survivor of ...action
715Cutthroat Island (1995)Morgan Adams and her slave, William Shaw, are ...action
817Sense and Sensibility (1995)When Mr. Dashwood dies, he must leave the bulk...romance
918Four Rooms (1995)This movie features the collaborative director...comedy
1019Ace Ventura: When Nature Calls (1995)Ace Ventura, emerging from self-imposed exile ...comedy
1129City of Lost Children, The (Cité des enfants p...Krank (Daniel Emilfork), who cannot dream, kid...sci-fi
1232Twelve Monkeys (a.k.a. 12 Monkeys) (1995)In a future world devastated by disease, a con...sci-fi
1334Babe (1995)Farmer Hoggett wins a runt piglet at a local f...fantasy
1439Clueless (1995)A rich high school student tries to boost a ne...romance
1544Mortal Kombat (1995)Based on the popular video game of the same na...action
1648Pocahontas (1995)Capt. John Smith leads a rag-tag band of Engli...animation
1750Usual Suspects, The (1995)Following a truck hijack in New York, five con...comedy
1857Home for the Holidays (1995)After losing her job, making out with her soon...comedy
1969Friday (1995)Two homies, Smokey and Craig, smoke a dope dea...comedy
2070From Dusk Till Dawn (1996)Two criminals and their hostages unknowingly s...action
2176Screamers (1995)(SIRIUS 6B, Year 2078) On a distant mining pla...sci-fi
2282Antonia's Line (Antonia) (1995)In an anonymous Dutch village, a sturdy, stron...fantasy
2388Black Sheep (1996)Comedy about the prospective Washington State ...comedy
2495Broken Arrow (1996)\"Broken Arrow\" is the term used to describe a ...action
25104Happy Gilmore (1996)A rejected hockey player puts his skills to th...comedy
26105Bridges of Madison County, The (1995)Photographer Robert Kincaid wanders into the l...romance
27110Braveheart (1995)When his secret bride is executed for assaulti...action
28141Birdcage, The (1996)Armand Goldman owns a popular drag nightclub i...comedy
29145Bad Boys (1995)Marcus Burnett is a hen-pecked family man. Mik...action
...............
1813122902Fantastic Four (2015)FANTASTIC FOUR, a contemporary re-imagining of...sci-fi
1814127098Louis C.K.: Live at The Comedy Store (2015)Comedian Louis C.K. performs live at the Comed...comedy
1815127158Tig (2015)An intimate, mixed media documentary that foll...comedy
1816127202Me and Earl and the Dying Girl (2015)Seventeen-year-old Greg has managed to become ...comedy
1817129354Focus (2015)In the midst of veteran con man Nicky's latest...action
1818129428The Second Best Exotic Marigold Hotel (2015)The Second Best Exotic Marigold Hotel is the e...comedy
1819129937Run All Night (2015)Professional Brooklyn hitman Jimmy Conlon is m...action
1820130490Insurgent (2015)One choice can transform you-or it can destroy...sci-fi
1821130520Home (2015)An alien on the run from his own people makes ...animation
1822130634Furious 7 (2015)Dominic and his crew thought they'd left the c...action
1823131013Get Hard (2015)Kevin Hart plays the role of Darnell--a family...comedy
1824132046Tomorrowland (2015)Bound by a shared destiny, a bright, optimisti...sci-fi
1825132480The Age of Adaline (2015)A young woman, born at the turn of the 20th ce...romance
1826132488Lovesick (2014)Lovesick is the comic tale of Charlie Darby (M...fantasy
1827132796San Andreas (2015)In San Andreas, California is experiencing a s...action
1828132961Far from the Madding Crowd (2015)In Victorian England, the independent and head...romance
1829133195Hitman: Agent 47 (2015)An assassin teams up with a woman to help her ...action
1830133645Carol (2015)In an adaptation of Patricia Highsmith's semin...romance
1831134130The Martian (2015)During a manned mission to Mars, Astronaut Mar...sci-fi
1832134368Spy (2015)A desk-bound CIA analyst volunteers to go unde...comedy
1833134783Entourage (2015)Movie star Vincent Chase, together with his bo...comedy
1834134853Inside Out (2015)After young Riley is uprooted from her Midwest...comedy
1835135518Self/less (2015)A dying real estate mogul transfers his consci...sci-fi
1836135861Ted 2 (2015)Months after John's divorce, Ted and Tami-Lynn...comedy
1837135887Minions (2015)Ever since the dawn of time, the Minions have ...comedy
1838136016The Good Dinosaur (2015)In a world where dinosaurs and humans live sid...animation
1839139855Anomalisa (2015)Michael Stone, an author that specializes in c...animation
1840142997Hotel Transylvania 2 (2015)The Drac pack is back for an all-new monster c...animation
1841145935Peanuts Movie, The (2015)Charlie Brown, Lucy, Snoopy, and the whole gan...animation
1842149406Kung Fu Panda 3 (2016)Continuing his \"legendary adventures of awesom...comedy
\n", + "

1843 rows × 4 columns

\n", + "
" + ], + "text/plain": [ + " MovieID Titles \\\n", + "0 1 Toy Story (1995) \n", + "1 2 Jumanji (1995) \n", + "2 3 Grumpier Old Men (1995) \n", + "3 6 Heat (1995) \n", + "4 7 Sabrina (1995) \n", + "5 9 Sudden Death (1995) \n", + "6 10 GoldenEye (1995) \n", + "7 15 Cutthroat Island (1995) \n", + "8 17 Sense and Sensibility (1995) \n", + "9 18 Four Rooms (1995) \n", + "10 19 Ace Ventura: When Nature Calls (1995) \n", + "11 29 City of Lost Children, The (Cité des enfants p... \n", + "12 32 Twelve Monkeys (a.k.a. 12 Monkeys) (1995) \n", + "13 34 Babe (1995) \n", + "14 39 Clueless (1995) \n", + "15 44 Mortal Kombat (1995) \n", + "16 48 Pocahontas (1995) \n", + "17 50 Usual Suspects, The (1995) \n", + "18 57 Home for the Holidays (1995) \n", + "19 69 Friday (1995) \n", + "20 70 From Dusk Till Dawn (1996) \n", + "21 76 Screamers (1995) \n", + "22 82 Antonia's Line (Antonia) (1995) \n", + "23 88 Black Sheep (1996) \n", + "24 95 Broken Arrow (1996) \n", + "25 104 Happy Gilmore (1996) \n", + "26 105 Bridges of Madison County, The (1995) \n", + "27 110 Braveheart (1995) \n", + "28 141 Birdcage, The (1996) \n", + "29 145 Bad Boys (1995) \n", + "... ... ... \n", + "1813 122902 Fantastic Four (2015) \n", + "1814 127098 Louis C.K.: Live at The Comedy Store (2015) \n", + "1815 127158 Tig (2015) \n", + "1816 127202 Me and Earl and the Dying Girl (2015) \n", + "1817 129354 Focus (2015) \n", + "1818 129428 The Second Best Exotic Marigold Hotel (2015) \n", + "1819 129937 Run All Night (2015) \n", + "1820 130490 Insurgent (2015) \n", + "1821 130520 Home (2015) \n", + "1822 130634 Furious 7 (2015) \n", + "1823 131013 Get Hard (2015) \n", + "1824 132046 Tomorrowland (2015) \n", + "1825 132480 The Age of Adaline (2015) \n", + "1826 132488 Lovesick (2014) \n", + "1827 132796 San Andreas (2015) \n", + "1828 132961 Far from the Madding Crowd (2015) \n", + "1829 133195 Hitman: Agent 47 (2015) \n", + "1830 133645 Carol (2015) \n", + "1831 134130 The Martian (2015) \n", + "1832 134368 Spy (2015) \n", + "1833 134783 Entourage (2015) \n", + "1834 134853 Inside Out (2015) \n", + "1835 135518 Self/less (2015) \n", + "1836 135861 Ted 2 (2015) \n", + "1837 135887 Minions (2015) \n", + "1838 136016 The Good Dinosaur (2015) \n", + "1839 139855 Anomalisa (2015) \n", + "1840 142997 Hotel Transylvania 2 (2015) \n", + "1841 145935 Peanuts Movie, The (2015) \n", + "1842 149406 Kung Fu Panda 3 (2016) \n", + "\n", + " Plots Genres \n", + "0 A little boy named Andy loves to be in his roo... animation \n", + "1 When two kids find and play a magical board ga... fantasy \n", + "2 Things don't seem to change much in Wabasha Co... comedy \n", + "3 Hunters and their prey--Neil and his professio... action \n", + "4 An ugly duckling having undergone a remarkable... romance \n", + "5 Some terrorists kidnap the Vice President of t... action \n", + "6 James Bond teams up with the lone survivor of ... action \n", + "7 Morgan Adams and her slave, William Shaw, are ... action \n", + "8 When Mr. Dashwood dies, he must leave the bulk... romance \n", + "9 This movie features the collaborative director... comedy \n", + "10 Ace Ventura, emerging from self-imposed exile ... comedy \n", + "11 Krank (Daniel Emilfork), who cannot dream, kid... sci-fi \n", + "12 In a future world devastated by disease, a con... sci-fi \n", + "13 Farmer Hoggett wins a runt piglet at a local f... fantasy \n", + "14 A rich high school student tries to boost a ne... romance \n", + "15 Based on the popular video game of the same na... action \n", + "16 Capt. John Smith leads a rag-tag band of Engli... animation \n", + "17 Following a truck hijack in New York, five con... comedy \n", + "18 After losing her job, making out with her soon... comedy \n", + "19 Two homies, Smokey and Craig, smoke a dope dea... comedy \n", + "20 Two criminals and their hostages unknowingly s... action \n", + "21 (SIRIUS 6B, Year 2078) On a distant mining pla... sci-fi \n", + "22 In an anonymous Dutch village, a sturdy, stron... fantasy \n", + "23 Comedy about the prospective Washington State ... comedy \n", + "24 \"Broken Arrow\" is the term used to describe a ... action \n", + "25 A rejected hockey player puts his skills to th... comedy \n", + "26 Photographer Robert Kincaid wanders into the l... romance \n", + "27 When his secret bride is executed for assaulti... action \n", + "28 Armand Goldman owns a popular drag nightclub i... comedy \n", + "29 Marcus Burnett is a hen-pecked family man. Mik... action \n", + "... ... ... \n", + "1813 FANTASTIC FOUR, a contemporary re-imagining of... sci-fi \n", + "1814 Comedian Louis C.K. performs live at the Comed... comedy \n", + "1815 An intimate, mixed media documentary that foll... comedy \n", + "1816 Seventeen-year-old Greg has managed to become ... comedy \n", + "1817 In the midst of veteran con man Nicky's latest... action \n", + "1818 The Second Best Exotic Marigold Hotel is the e... comedy \n", + "1819 Professional Brooklyn hitman Jimmy Conlon is m... action \n", + "1820 One choice can transform you-or it can destroy... sci-fi \n", + "1821 An alien on the run from his own people makes ... animation \n", + "1822 Dominic and his crew thought they'd left the c... action \n", + "1823 Kevin Hart plays the role of Darnell--a family... comedy \n", + "1824 Bound by a shared destiny, a bright, optimisti... sci-fi \n", + "1825 A young woman, born at the turn of the 20th ce... romance \n", + "1826 Lovesick is the comic tale of Charlie Darby (M... fantasy \n", + "1827 In San Andreas, California is experiencing a s... action \n", + "1828 In Victorian England, the independent and head... romance \n", + "1829 An assassin teams up with a woman to help her ... action \n", + "1830 In an adaptation of Patricia Highsmith's semin... romance \n", + "1831 During a manned mission to Mars, Astronaut Mar... sci-fi \n", + "1832 A desk-bound CIA analyst volunteers to go unde... comedy \n", + "1833 Movie star Vincent Chase, together with his bo... comedy \n", + "1834 After young Riley is uprooted from her Midwest... comedy \n", + "1835 A dying real estate mogul transfers his consci... sci-fi \n", + "1836 Months after John's divorce, Ted and Tami-Lynn... comedy \n", + "1837 Ever since the dawn of time, the Minions have ... comedy \n", + "1838 In a world where dinosaurs and humans live sid... animation \n", + "1839 Michael Stone, an author that specializes in c... animation \n", + "1840 The Drac pack is back for an all-new monster c... animation \n", + "1841 Charlie Brown, Lucy, Snoopy, and the whole gan... animation \n", + "1842 Continuing his \"legendary adventures of awesom... comedy \n", + "\n", + "[1843 rows x 4 columns]" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import gensim\n", + "import pandas as pd\n", + "import smart_open\n", + "import random\n", + "\n", + "# read data\n", + "dataframe = pd.read_csv('movie_plots.csv')\n", + "dataframe" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 1. Visualizing Doc2Vec\n", + "In this part, we will learn about visualizing Doc2Vec Embeddings aka [Paragraph Vectors](https://arxiv.org/abs/1405.4053) via TensorBoard. The input documents for training will be the synopsis of movies, on which Doc2Vec model is trained. \n", + "\n", + "\n", + "\n", + "The visualizations will be a scatterplot as seen in the above image, where each datapoint is labelled by the movie title and colored by it's corresponding genre. You can also visit this [Projector link](http://projector.tensorflow.org/?config=https://raw.githubusercontent.com/parulsethi/DocViz/master/movie_plot_config.json) which is configured with my embeddings for the above mentioned dataset. \n", + "\n", + "\n", + "## Preprocess Text" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Below, we define a function to read the training documents, pre-process each document using a simple gensim pre-processing tool (i.e., tokenize text into individual words, remove punctuation, set to lowercase, etc), and return a list of words. Also, to train the model, we'll need to associate a tag/number with each document of the training corpus. In our case, the tag is simply the zero-based line number." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def read_corpus(documents):\n", + " for i, plot in enumerate(documents):\n", + " yield gensim.models.doc2vec.TaggedDocument(gensim.utils.simple_preprocess(plot, max_len=30), [i])" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "train_corpus = list(read_corpus(dataframe.Plots))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's take a look at the training corpus." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[TaggedDocument(words=['little', 'boy', 'named', 'andy', 'loves', 'to', 'be', 'in', 'his', 'room', 'playing', 'with', 'his', 'toys', 'especially', 'his', 'doll', 'named', 'woody', 'but', 'what', 'do', 'the', 'toys', 'do', 'when', 'andy', 'is', 'not', 'with', 'them', 'they', 'come', 'to', 'life', 'woody', 'believes', 'that', 'he', 'has', 'life', 'as', 'toy', 'good', 'however', 'he', 'must', 'worry', 'about', 'andy', 'family', 'moving', 'and', 'what', 'woody', 'does', 'not', 'know', 'is', 'about', 'andy', 'birthday', 'party', 'woody', 'does', 'not', 'realize', 'that', 'andy', 'mother', 'gave', 'him', 'an', 'action', 'figure', 'known', 'as', 'buzz', 'lightyear', 'who', 'does', 'not', 'believe', 'that', 'he', 'is', 'toy', 'and', 'quickly', 'becomes', 'andy', 'new', 'favorite', 'toy', 'woody', 'who', 'is', 'now', 'consumed', 'with', 'jealousy', 'tries', 'to', 'get', 'rid', 'of', 'buzz', 'then', 'both', 'woody', 'and', 'buzz', 'are', 'now', 'lost', 'they', 'must', 'find', 'way', 'to', 'get', 'back', 'to', 'andy', 'before', 'he', 'moves', 'without', 'them', 'but', 'they', 'will', 'have', 'to', 'pass', 'through', 'ruthless', 'toy', 'killer', 'sid', 'phillips'], tags=[0]),\n", + " TaggedDocument(words=['when', 'two', 'kids', 'find', 'and', 'play', 'magical', 'board', 'game', 'they', 'release', 'man', 'trapped', 'for', 'decades', 'in', 'it', 'and', 'host', 'of', 'dangers', 'that', 'can', 'only', 'be', 'stopped', 'by', 'finishing', 'the', 'game'], tags=[1])]" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_corpus[:2]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Training the Doc2Vec Model\n", + "We'll instantiate a Doc2Vec model with a vector size with 50 words and iterating over the training corpus 55 times. We set the minimum word count to 2 in order to give higher frequency words more weighting. Model accuracy can be improved by increasing the number of iterations but this generally increases the training time. Small datasets with short documents, like this one, can benefit from more training passes." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "5168238" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model = gensim.models.doc2vec.Doc2Vec(size=50, min_count=2, iter=55)\n", + "model.build_vocab(train_corpus)\n", + "model.train(train_corpus, total_examples=model.corpus_count, epochs=model.iter)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, we'll save the document embedding vectors per doctag." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "model.save_word2vec_format('doc_tensor.w2v', doctag_vec=True, word_vec=False) " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Prepare the Input files for Tensorboard" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Tensorboard takes two Input files. One containing the embedding vectors and the other containing relevant metadata. We'll use a gensim script to directly convert the embedding file saved in word2vec format above to the tsv format required in Tensorboard." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "2017-04-20 02:23:05,284 : MainThread : INFO : running ../../gensim/scripts/word2vec2tensor.py -i doc_tensor.w2v -o movie_plot\n", + "2017-04-20 02:23:05,286 : MainThread : INFO : loading projection weights from doc_tensor.w2v\n", + "2017-04-20 02:23:05,464 : MainThread : INFO : loaded (1843, 50) matrix from doc_tensor.w2v\n", + "2017-04-20 02:23:05,578 : MainThread : INFO : 2D tensor file saved to movie_plot_tensor.tsv\n", + "2017-04-20 02:23:05,579 : MainThread : INFO : Tensor metadata file saved to movie_plot_metadata.tsv\n", + "2017-04-20 02:23:05,581 : MainThread : INFO : finished running word2vec2tensor.py\n" + ] + } + ], + "source": [ + "%run ../../gensim/scripts/word2vec2tensor.py -i doc_tensor.w2v -o movie_plot" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The script above generates two files, `movie_plot_tensor.tsv` which contain the embedding vectors and `movie_plot_metadata.tsv` containing doctags. But, these doctags are simply the unique index values and hence are not really useful to interpret what the document was while visualizing. So, we will overwrite `movie_plot_metadata.tsv` to have a custom metadata file with two columns. The first column will be for the movie titles and the second for their corresponding genres." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "with open('movie_plot_metadata.tsv','w') as w:\n", + " w.write('Titles\\tGenres\\n')\n", + " for i,j in zip(dataframe.Titles, dataframe.Genres):\n", + " w.write(\"%s\\t%s\\n\" % (i,j))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "Now you can go to http://projector.tensorflow.org/ and upload the two files by clicking on *Load data* in the left panel.\n", + "\n", + "For demo purposes I have uploaded the Doc2Vec embeddings generated from the model trained above [here](https://github.com/parulsethi/DocViz). You can access the Embedding projector configured with these uploaded embeddings at this [link](http://projector.tensorflow.org/?config=https://raw.githubusercontent.com/parulsethi/DocViz/master/movie_plot_config.json)." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "# Using Tensorboard" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For the visualization purpose, the multi-dimensional embeddings that we get from the Doc2Vec model above, needs to be downsized to 2 or 3 dimensions. So that we basically end up with a new 2d or 3d embedding which tries to preserve information from the original multi-dimensional embedding. As these vectors are reduced to a much smaller dimension, the exact cosine/euclidean distances between them are not preserved, but rather relative, and hence as you’ll see below the nearest similarity results may change.\n", + "\n", + "TensorBoard has two popular dimensionality reduction methods for visualizing the embeddings and also provides a custom method based on text searches:\n", + "\n", + "- **Principal Component Analysis**: PCA aims at exploring the global structure in data, and could end up losing the local similarities between neighbours. It maximizes the total variance in the lower dimensional subspace and hence, often preserves the larger pairwise distances better than the smaller ones. See an intuition behind it in this nicely explained [answer](https://stats.stackexchange.com/questions/176672/what-is-meant-by-pca-preserving-only-large-pairwise-distances) on stackexchange.\n", + "\n", + "\n", + "- **T-SNE**: The idea of T-SNE is to place the local neighbours close to each other, and almost completely ignoring the global structure. It is useful for exploring local neighborhoods and finding local clusters. But the global trends are not represented accurately and the separation between different groups is often not preserved (see the t-sne plots of our data below which testify the same).\n", + "\n", + "\n", + "- **Custom Projections**: This is a custom bethod based on the text searches you define for different directions. It could be useful for finding meaningful directions in the vector space, for example, female to male, currency to country etc.\n", + "\n", + "You can refer to this [doc](https://www.tensorflow.org/get_started/embedding_viz) for instructions on how to use and navigate through different panels available in TensorBoard." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Visualize using PCA\n", + "\n", + "The Embedding Projector computes the top 10 principal components. The menu at the left panel lets you project those components onto any combination of two or three. \n", + "\n", + "The above plot was made using the first two principal components with total variance covered being 36.5%." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "## Visualize using T-SNE\n", + "\n", + "Data is visualized by animating through every iteration of the t-sne algorithm. The t-sne menu at the left lets you adjust the value of it's two hyperparameters. The first one is **Perplexity**, which is basically a measure of information. It may be viewed as a knob that sets the number of effective nearest neighbors[2]. The second one is **learning rate** that defines how quickly an algorithm learns on encountering new examples/data points.\n", + "\n", + "\n", + "\n", + "The above plot was generated with perplexity 8, learning rate 10 and iteration 500. Though the results could vary on successive runs, and you may not get the exact plot as above with same hyperparameter settings. But some small clusters will start forming as above, with different orientations." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 2. Visualizing LDA\n", + "\n", + "In this part, we will see how to visualize LDA in Tensorboard. We will be using the Document-topic distribution as the embedding vector of a document. Basically, we treat topics as the dimensions and the value in each dimension represents the topic proportion of that topic in the document.\n", + "\n", + "## Preprocess Text\n", + "\n", + "We use the movie Plots as our documents in corpus and remove rare words and common words based on their document frequency. Below we remove words that appear in less than 2 documents or in more than 30% of the documents." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "import pandas as pd\n", + "import re\n", + "from gensim.parsing.preprocessing import remove_stopwords, strip_punctuation\n", + "from gensim.models import ldamodel\n", + "from gensim.corpora.dictionary import Dictionary\n", + "\n", + "# read data\n", + "dataframe = pd.read_csv('movie_plots.csv')\n", + "\n", + "# remove stopwords and punctuations\n", + "def preprocess(row):\n", + " return strip_punctuation(remove_stopwords(row.lower()))\n", + " \n", + "dataframe['Plots'] = dataframe['Plots'].apply(preprocess)\n", + "\n", + "# Convert data to required input format by LDA\n", + "texts = []\n", + "for line in dataframe.Plots:\n", + " lowered = line.lower()\n", + " words = re.findall(r'\\w+', lowered, flags = re.UNICODE | re.LOCALE)\n", + " texts.append(words)\n", + "# Create a dictionary representation of the documents.\n", + "dictionary = Dictionary(texts)\n", + "\n", + "# Filter out words that occur less than 2 documents, or more than 30% of the documents.\n", + "dictionary.filter_extremes(no_below=2, no_above=0.3)\n", + "# Bag-of-words representation of the documents.\n", + "corpus = [dictionary.doc2bow(text) for text in texts]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Train LDA Model\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "# Set training parameters.\n", + "num_topics = 10\n", + "chunksize = 2000\n", + "passes = 50\n", + "iterations = 200\n", + "eval_every = None\n", + "\n", + "# Train model\n", + "model = ldamodel.LdaModel(corpus=corpus, id2word=dictionary, chunksize=chunksize, alpha='auto', eta='auto', iterations=iterations, num_topics=num_topics, passes=passes, eval_every=eval_every)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can refer to [this notebook](https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/lda_training_tips.ipynb) also before training the LDA model. It contains tips and suggestions for pre-processing the text data, and how to train the LDA model to get good results." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Doc-Topic distribution\n", + "\n", + "Now we will use `get_document_topics` which infers the topic distribution of a document. It basically returns a list of (topic_id, topic_probability) for each document in the input corpus." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[(0, 0.00029626785677659928),\n", + " (1, 0.99734244187457377),\n", + " (2, 0.00031813940693891458),\n", + " (3, 0.00031573036467256674),\n", + " (4, 0.00033277056023999966),\n", + " (5, 0.00023981837072288835),\n", + " (6, 0.00033113374640540293),\n", + " (7, 0.00027953838669809549),\n", + " (8, 0.0002706215262517565),\n", + " (9, 0.00027353790672011199)]" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Get document topics\n", + "all_topics = model.get_document_topics(corpus, minimum_probability=0)\n", + "all_topics[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The above output shows the topic distribution of first document in the corpus as a list of (topic_id, topic_probability).\n", + "\n", + "Now, using the topic distribution of a document as it's vector embedding, we will plot all the documents in our corpus using Tensorboard." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Prepare the Input files for Tensorboard\n", + "\n", + "Tensorboard takes two input files, one containing the embedding vectors and the other containing relevant metadata. As described above we will use the topic distribution of documents as their embedding vector. Metadata file will consist of Movie titles with their genres." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "# create file for tensors\n", + "with open('doc_lda_tensor.tsv','w') as w:\n", + " for doc_topics in all_topics:\n", + " for topics in doc_topics:\n", + " w.write(str(topics[1])+ \"\\t\")\n", + " w.write(\"\\n\")\n", + " \n", + "# create file for metadata\n", + "with open('doc_lda_metadata.tsv','w') as w:\n", + " w.write('Titles\\tGenres\\n')\n", + " for j, k in zip(dataframe.Titles, dataframe.Genres):\n", + " w.write(\"%s\\t%s\\n\" % (j, k))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "Now you can go to http://projector.tensorflow.org/ and upload these two files by clicking on Load data in the left panel.\n", + "\n", + "For demo purposes I have uploaded the LDA doc-topic embeddings generated from the model trained above [here](https://github.com/parulsethi/LdaProjector/). You can also access the Embedding projector configured with these uploaded embeddings at this [link](http://projector.tensorflow.org/?config=https://raw.githubusercontent.com/parulsethi/LdaProjector/master/doc_lda_config.json)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Visualize using PCA\n", + "\n", + "The Embedding Projector computes the top 10 principal components. The menu at the left panel lets you project those components onto any combination of two or three.\n", + "\n", + "From PCA, we get a simplex (tetrahedron in this case) where each data point represent a document. These data points are colored according to their Genres which were given in the Movie dataset. \n", + "\n", + "As we can see there are a lot of points which cluster at the corners of the simplex. This is primarily due to the sparsity of vectors we are using. The documents at the corners primarily belongs to a single topic (hence, large weight in a single dimension and other dimensions have approximately zero weight.) You can modify the metadata file as explained below to see the dimension weights along with the Movie title.\n", + "\n", + "Now, we will append the topics with highest probability (topic_id, topic_probability) to the document's title, in order to explore what topics do the cluster corners or edges dominantly belong to. For this, we just need to overwrite the metadata file as below:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "tensors = []\n", + "for doc_topics in all_topics:\n", + " doc_tensor = []\n", + " for topic in doc_topics:\n", + " if round(topic[1], 3) > 0:\n", + " doc_tensor.append((topic[0], float(round(topic[1], 3))))\n", + " # sort topics according to highest probabilities\n", + " doc_tensor = sorted(doc_tensor, key=lambda x: x[1], reverse=True)\n", + " # store vectors to add in metadata file\n", + " tensors.append(doc_tensor[:5])\n", + "\n", + "# overwrite metadata file\n", + "i=0\n", + "with open('doc_lda_metadata.tsv','w') as w:\n", + " w.write('Titles\\tGenres\\n')\n", + " for j,k in zip(dataframe.Titles, dataframe.Genres):\n", + " w.write(\"%s\\t%s\\n\" % (''.join((str(j), str(tensors[i]))),k))\n", + " i+=1" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, we upload the previous tensor file \"doc_lda_tensor.tsv\" and this new metadata file to http://projector.tensorflow.org/ .\n", + "\n", + "Voila! Now we can click on any point to see it's top topics with their probabilty in that document, along with the title. As we can see in the above example, \"Beverly hill cops\" primarily belongs to the 0th and 1st topic as they have the highest probability amongst all.\n", + "\n", + "\n", + "\n", + "## Visualize using T-SNE\n", + "\n", + "In T-SNE, the data is visualized by animating through every iteration of the t-sne algorithm. The t-sne menu at the left lets you adjust the value of it's two hyperparameters. The first one is Perplexity, which is basically a measure of information. It may be viewed as a knob that sets the number of effective nearest neighbors[2]. The second one is learning rate that defines how quickly an algorithm learns on encountering new examples/data points.\n", + "\n", + "Now, as the topic distribution of a document is used as it’s embedding vector, t-sne ends up forming clusters of documents belonging to same topics. In order to understand and interpret about the theme of those topics, we can use `show_topic()` to explore the terms that the topics consisted of.\n", + "\n", + "\n", + "\n", + "The above plot was generated with perplexity 11, learning rate 10 and iteration 1100. Though the results could vary on successive runs, and you may not get the exact plot as above even with same hyperparameter settings. But some small clusters will start forming as above, with different orientations.\n", + "\n", + "I named some clusters above based on the genre of it's movies and also using the `show_topic()` to see relevant terms of the topic which was most prevelant in a cluster. Most of the clusters had doocumets belonging dominantly to a single topic. For ex. The cluster with movies belonging primarily to topic 0 could be named Fantasy/Romance based on terms displayed below for topic 0. You can play with the visualization yourself on this [link](http://projector.tensorflow.org/?config=https://raw.githubusercontent.com/parulsethi/LdaProjector/master/doc_lda_config.json) and try to conclude a label for clusters based on movies it has and their dominant topic. You can see the top 5 topics of every point by hovering over it.\n", + "\n", + "Now, we can notice that their are more than 10 clusters in the above image, whereas we trained our model for `num_topics=10`. It's because their are few clusters, which has documents belonging to more than one topic with an approximately close topic probability values." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[('life', 0.0069577926389817156),\n", + " ('world', 0.006240163206609986),\n", + " ('man', 0.0058828040298109794),\n", + " ('young', 0.0053747678629860532),\n", + " ('family', 0.005083746467542196),\n", + " ('love', 0.0048691281379952146),\n", + " ('new', 0.004097644507005606),\n", + " ('t', 0.0037446821043766597),\n", + " ('time', 0.0037022423231064822),\n", + " ('finds', 0.0036129806190553109),\n", + " ('woman', 0.0031742920620375422),\n", + " ('earth', 0.0031692677510459484),\n", + " ('help', 0.0031061538189201504),\n", + " ('it', 0.0028658594310878023),\n", + " ('years', 0.00272218005397741)]" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model.show_topic(topicid=0, topn=15)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can even use pyLDAvis to deduce topics more efficiently. It provides a deeper inspection of the terms highly associated with each individual topic. For this, it uses a measure called **relevance** of a term to a topic that allows users to flexibly rank terms best suited for a meaningful topic interpretation. It's weight parameter called λ can be adjusted to display useful terms which could help in differentiating topics efficiently." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/Users/parul/.virtualenvs/gensim3/lib/python3.4/site-packages/pyLDAvis/_prepare.py:387: DeprecationWarning: \n", + ".ix is deprecated. Please use\n", + ".loc for label based indexing or\n", + ".iloc for positional indexing\n", + "\n", + "See the documentation here:\n", + "http://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate_ix\n", + " topic_term_dists = topic_term_dists.ix[topic_order]\n" + ] + }, + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
\n", + "" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import pyLDAvis.gensim\n", + "\n", + "viz = pyLDAvis.gensim.prepare(model, corpus, dictionary)\n", + "pyLDAvis.display(viz)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The weight parameter λ can be viewed as a knob to adjust the ranks of the terms based on whether they are simply ranked according to their probability in the topic (λ=1) or are normalized by their marginal probability across the corpus (λ=0). Setting λ=1 could result in similar ranking of terms for large no. of topics hence making it difficult to differentiate between them, and setting λ=0 ranks terms solely based on their exclusiveness to current topic which could result in such rare terms that occur in only a single topic and hence the topics may remain difficult to interpret. [(Sievert and Shirley 2014)](https://nlp.stanford.edu/events/illvi2014/papers/sievert-illvi2014.pdf) suggested the optimal value of λ=0.6 based on a user study." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Conclusion\n", + "\n", + "We learned about visualizing the Document Embeddings and LDA Doc-topic distributions through Tensorboard's Embedding Projector. It is a useful tool for visualizing different types of data for example, word embeddings, document embeddings or the gene expressions and biological sequences. It just needs an input of 2D tensors and then you can explore your data using provided algorithms. You can also perform nearest neighbours search to find most similar data points to your query point.\n", + "\n", + "# References\n", + " 1. https://grouplens.org/datasets/movielens/\n", + " 2. https://lvdmaaten.github.io/tsne/\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.3" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/docs/notebooks/doc_lda_pca.png b/docs/notebooks/doc_lda_pca.png new file mode 100644 index 0000000000..2f29a3a731 Binary files /dev/null and b/docs/notebooks/doc_lda_pca.png differ diff --git a/docs/notebooks/doc_lda_tsne.png b/docs/notebooks/doc_lda_tsne.png new file mode 100644 index 0000000000..46c36c19de Binary files /dev/null and b/docs/notebooks/doc_lda_tsne.png differ diff --git a/docs/notebooks/topic_with_coordinate.png b/docs/notebooks/topic_with_coordinate.png new file mode 100644 index 0000000000..e4e9b287c8 Binary files /dev/null and b/docs/notebooks/topic_with_coordinate.png differ