In the distillation process, knowledge was transferred from the teacher networks to the student network to minimize knowledge loss.