Empowering On-Device Training: Leveraging Inference Accelerators for Enhanced Training Efficiency
[ 1 ] Wydział Automatyki, Robotyki i Elektrotechniki, Politechnika Poznańska | [ 2 ] Instytut Robotyki i Inteligencji Maszynowej, Wydział Automatyki, Robotyki i Elektrotechniki, Politechnika Poznańska | [ SzD ] doktorant ze Szkoły Doktorskiej | [ P ] pracownik
[2.2] Automatyka, elektronika, elektrotechnika i technologie kosmiczne
2024
abstrakt
angielski
- on-device training
- hardware acceleration
- AI accelerator
- continual learning
EN On-device training is essential in practical continual learning (CL) applications. However, the current methods predominantly focus on memory optimization, while computing power and time budget are primary concerns for most real-world systems. Thus, this study leverages a high-performance AI co-processor primarily developed to enhance the inference of deep learning algorithms to accelerate the on-device training. In conducted examinations, layers' parameters of various levels of model architecture were frozen, transferred, and processed on an AI chip. While the model adaptive stage, with a backpropagation algorithm, was computed on the CPU. The proposed approach achieved up to 17$\times$ training time acceleration compared to a CPU-only process without performance degradation. Concerning conducted examinations, the research findings highlight the potential of advanced hardware accelerators in addressing the computational challenges of on-device CL, paving the way for more efficient and practical deployment in resource-constrained settings.
211 - 214