Let's begin with a few simple examples to understand what is going on. 

For some of us, it's very easy to get tempted to try the shiniest algorithms and do hyper-parameter optimization instead of the less glamorous step-by-step understanding.