Therefore, the overall loss function of the student network incorporated both knowledge distillation and knowledge loss from the student networks.