Curriculum Learning Revisited: Incremental Batch Learning with Instance Typicality Ranking
[ 1 ] Instytut Informatyki, Wydział Informatyki i Telekomunikacji, Politechnika Poznańska | [ SzD ] doctoral school student | [ P ] employee
2021
chapter in monograph / paper
english
- curriculum learning
- typicality
- batch training
EN The technique of curriculum learning mimics cognitive mechanisms observed in human learning, where simpler concepts are presented prior to gradual introduction of more difficult concepts. Until now, the major obstacle for curriculum methods was the lack of a reliable method for estimating the difficulty of training instances. In this paper we show that, instead of trying to assess the difficulty of learning instances, a simple graph-based method of computing the typicality of instances can be used in conjunction with curriculum methods. We design new batch schedulers which organize ordered instances into batches of varying size and learning difficulty. Our method does not require any changes to the architecture of trained models, we improve the training merely by manipulating the order and frequency of instance presentation to the model.
279 - 291
20
70