Singapore Institute of Technology
wang21o_interspeech.pdf (1.9 MB)

A weight moving average based alternate decoupled learning algorithm for long-tailed language identification

Download (1.9 MB)
conference contribution
posted on 2024-04-03, 02:48 authored by Hui Wang, Lin Liu, Yan Song, Lei Fang, Ian McLoughlinIan McLoughlin, Li-Rong Dai

Language identification (LID) research has made tremendous progress in recent years, especially with the introduction of deep learning techniques. However, for real-world applications where the distribution of different language data is highly imbalanced, the performance of existing LID systems is still far from satisfactory. This raises the challenge of long-tailed LID. In this paper, we propose an effective weight moving average (WMA) based alternate decoupled learning algorithm, termed WADCL, for long-tailed LID. The system is divided into two components, a frontend feature extractor and a backend classifier. These are then alternately learned in an end-to-end manner using different sampling schemes to alleviate the distribution mismatch between training and test datasets. Furthermore, our WMA method aims to mitigate the side-effects of re-sampling schemes, by fusing the model parameters learned along the trajectory of stochastic gradient descent (SGD) optimization. To validate the effectiveness of the proposed WADCL algorithm, we evaluate and compare several systems over a language dataset constructed to match a long-tailed distribution based on real world application [1]. The experimental results from the long-tailed language dataset demonstrate that the proposed algorithm is able to achieve significant performance gains over existing state-of-the-art x-vector based LID methods.


Journal/Conference/Book title

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, 30 August – 3 September, 2021, Brno, Czechia.

Publication date



  • Published

Rights statement

Wang, H., Liu, L., Song, Y., Fang, L., McLoughlin, I., Dai, L.-R. (2021) A Weight Moving Average Based Alternate Decoupled Learning Algorithm for Long-Tailed Language Identification. Proc. Interspeech 2021, 1499-1503, doi: 10.21437/Interspeech.2021-776

Usage metrics



    Ref. manager