GrainSpeech: Less Context, More Detail for Compact Speech Synthesis

Problem

The paper addresses the quality-capacity trade-off in compact acoustic models for speech synthesis. It highlights the limitations of existing models in effectively utilizing context for accurate predictions, particularly in pitch, energy, and duration. This work is presented as a preprint and has not undergone peer review.

Method

The authors propose a novel approach that includes a receptive-field-scaling study, which demonstrates that expanding self-attention beyond 15 phonemes does not yield consistent improvements in prediction accuracy for pitch, energy, or duration. To mitigate this, they introduce a fixed-receptive-field convolutional encoder that significantly reduces prediction errors: 36.0% for pitch, 17.3% for energy, and 3.4% for duration. Additionally, they employ a mel-specific formulation that leverages axis-specific gradients, overlapping local statistics, and log-domain variance matching to enhance the model's performance.

Results

The model achieves a notable reduction in prediction errors: 36.0% for pitch, 17.3% for energy, and 3.4% for duration compared to prior methods. It operates with a compact architecture of only 264.8K parameters. Furthermore, it demonstrates impressive real-time Mel generation capabilities, achieving a speedup of 17.9x on a microcontroller (MCU). The UTMOS scores indicate that the model's performance is comparable to larger models while utilizing less than 1.5% of their parameters.

Limitations

The authors do not report any limitations in their work, and no obvious limitations are identified in the provided text.

Why it matters

The implications of this research are significant for the development of efficient speech synthesis systems, particularly in resource-constrained environments. By demonstrating that high-quality synthesis can be achieved with a compact model, this work paves the way for deploying advanced speech synthesis in applications where computational resources are limited, such as mobile devices and embedded systems.