To learn about an asynchronous version of the actor-critic algorithm, read https://arxiv.org/pdf/1602.01783.pdf.