• Use the context vector, or a combination of the context vector and the main vector from GloVe. How do the results change? You can also build a meta-model that calculates the appropriate weight you should give to each vector.
• Do the results improve significantly if we use stemming?
• Can you improve the results by playing with the parameters of GloVe? You can tweak skip_grams_window for instance, or the number of iterations.
• You can also try to implement fastText (https://keras.rstudio.com/articles/examples/imdb_fasttext.html) using keras.