Title
Adaptive cluster-count selection via deep Q-learning for turbofan engine prognostics and health monitoring
Authors
[ 1 ] Instytut Napędów i Lotnictwa, Wydział Inżynierii Lądowej i Transportu, Politechnika Poznańska | [ P ] employee
Scientific discipline (Law 2.0)
Year of publication
2026
Published in
Article type
scientific article
Publication language
english
Keywords
Abstract
EN Clustering engine run-to-failure data is a crucial step in many prognostics and health management pipelines; however, selecting the optimal number of clusters (k) remains a challenging task. Traditional heuristics (e.g., the elbow method, silhouette scores) require recalculating metrics for every candidate k, and often lack adaptability to varying engine regimes. This comprehensive framework illustrates an innovative approach to integrating reinforcement learning with unsupervised clustering, enabling the automatic tuning of clustering algorithm parameters based on performance metrics. Two custom Reinforcement Learning (RL) environments were developed: the original engine data environment, which uses a single-step Deep Q-Network (DQN) combined with k-means, and the improved one, which uses a multi-step DQN and Gaussian mixture models (GMM) with regularization. RL agents are trained using Double Deep Q-Networks (Double-DQN) with ε-greedy exploration, target-network updates, and experience replay, to maximize cluster-quality rewards over episodes, based on silhouette and Calinski–Harabasz (CH) indices. The RL agents automatically learn to pick k in the 2÷10 range that yields high silhouette and CH scores. This approach eliminates the need for exhaustive grid searches, offering a significant reduction in computational cost. Evaluated on Principal Component Analysis (PCA)-reduced NASA Commercial Modular Aero-Propulsion System Simulation (CMAPSS) datasets, the improved RL agent achieves superior clustering performance, with silhouette scores up to 0.98 and CH indices up to 1.32E07, outperforming traditional methods. To validate results, RL-based k-selection was compared to Density-Based Spatial Clustering of Applications with Noise (DBSCAN), hierarchical, spectral, and standard GMM clustering methods. Training progress, cluster-quality metrics, and 3D cluster distributions were visualized to confirm the achievements.
Date of online publication
07.02.2026
Pages (from - to)
132294-1 - 132294-13
Ministry points / journal
140
System created by Poznań University of Technology
and Poznan Supercomputing and Networking Center
Log in through eKonto to add to SIS