The role of the expert in imitation learning

The terms expert, teacher, and supervisor refer to the same concept when speaking of imitation learning algorithms. They express a figure from which the new agent (the learner) can learn.

Fundamentally, the expert can be of any form, from a real human expert to an expert system. The first case is more obvious and adopted. What you are doing is teaching an algorithm to perform a task that a human is already able to do. The advantages are evident and it can be employed in a vast number of tasks.

The second case may not be so common. One of the valid motivations behind choosing a new algorithm trained with IL can be attributed to a slow expert system that, due to technical limitations, cannot be improved. For example, the teacher could be an accurate, but slow, tree search algorithm that is not able to perform at a decent speed at inference time. A deep neural network could be employed in its place. The training of the neural network under the supervision of the tree search algorithm could take some time but, once trained, it could perform much faster during runtime. 

By now, it should be clear that the quality of the policy coming from the learner is largely due to the quality of the information provided by the expert. The performance of the teacher is an upper limit to the final performances of the scholar. A poor teacher will always provide bad data to the learner. Thus, the expert is a key component that sets the bar for the quality of the final agent. With a weak teacher, we cannot pretend to obtain good policies.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset