Early layers are responsible for detecting simple patterns and are building blocks of the neural network and hence it becomes important that early layers are accurate.