The efforts that have been put into the research of reinforcement learning algorithms in recent years has been huge. Especially since the introduction of the deep neural network as a function approximation, the advancement and results have been outstanding. Yet some major issues remain unsolved. These limit the applicability of RL algorithms to more extensive and interesting tasks. We are talking about the issues of stability, reproducibility, efficiency, and generalization, although scalability and the exploration problem could be added to this list.