FreqCondNorm: Towards Cross-domain Predictive Maintenance through a Frequency-Conditioned Transformer Foundation Model

Problem

The paper addresses the challenge of poor transferability of predictive maintenance models across different machines and operating conditions, particularly in scenarios with limited labeled data and varying sampling frequencies. This issue is critical as it hampers the effectiveness of predictive maintenance strategies in real-world applications. The work is presented as a preprint, indicating it has not yet undergone peer review.

Method

The authors propose a transformer-based architecture that incorporates a frequency-conditioned normalization layer inspired by Feature-wise Linear Modulation (FiLM). This architecture is designed to enhance the model's ability to adapt to different operational contexts by conditioning on the frequency of the input data. The model is pretrained on several datasets, including CWRU, MFPT, UOC18, PRONOSTIA, and CMAPSS. The training methodology employs masked auto-encoding and contrastive learning techniques, with a focus on balanced domain sampling to ensure diverse representation during training.

Results

The proposed model achieves a fault diagnosis accuracy of 99.2% on the CWRU dataset, outperforming a convolutional neural network (CNN) baseline by 6.4 percentage points. Additionally, it demonstrates a zero-shot accuracy of 82.1% on the MFPT dataset, although no baseline for this metric is provided in the paper.

Limitations

The authors note that there is no reported improvement in the prediction of remaining useful life (RUL), suggesting a potential mismatch between the objectives of pretraining and RUL prediction tasks. This limitation indicates that while the model excels in fault diagnosis, its applicability to RUL prediction may require further refinement or additional training strategies.

Why it matters

The implications of this work are significant for the field of predictive maintenance, particularly in enhancing the transferability of models across different operational contexts. By leveraging a frequency-conditioned approach, the proposed method could facilitate more robust predictive maintenance solutions in diverse industrial settings, potentially leading to improved operational efficiency and reduced downtime.