The dataset used is the Penn Tree Bank (PTB) language modeling dataset that must be downloaded from Tomas Mikolov's webpage http://www.fit.vutbr.cz/~imikolov/rnnlm/simple-examples.tgz and then extracted in your data folder. This dataset consists of 929k training words, 73k validation words, and 82k test words. It has 10k words in its vocabulary including the end-of-sentence marker and a special symbol <unk> for rare words.

Generally speaking, a treebank is a collection of sentences collected according to a syntactic annotation scheme that makes machine readable information concerning the linguistic text structure.