The distributed-memory model

This particular flavor of doc2vec closely resembles the CBOW model of word2vec, whereby the algorithm tries to predict a focus word given its surrounding context words but with the addition of a paragraph ID. Think of this as another individual contextual word vector that helps with the prediction task but is constant throughout what we consider to be a document. Continuing our previous example, if we have this movie review (we define one document as one movie review) and our focus word is ideas, we will now have the following architecture:

Note that as we move down the document and change the focus word from ideas to regarding, our context words will obviously change; however, Document ID: 456 remains the same. This is a crucial point in doc2vec, as the document ID is used in the prediction task:

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset