LoRA-Enhanced Language Alignments for Robust Code-Mixed Text Representation

The utilization of code-mixed texts allows individuals the opportunity to express sentiments flexibly in international and multilingual contexts. However, the diversity of languages and pragmatic writing styles can lead to semantic shifts at both word and sentence levels, resulting in a degraded comprehension of code-mixed texts by machines. To tackle this issue, we propose a method for language alignments, which aligns both word- and sentence-level semantic representations via a low-rank injection (LoRI) and a data augmentation strategy (DA), dubbed LoRIDA. LoRI integrates linguistic features into textual representations as a feature fusion mechanism. To further bridge the gaps between sentence-level semantics, we augment code-mixed data into individual source languages and apply a knowledge distillation method for joint alignments. We evaluate the performance of the proposed method on four code-mixed sentiment analysis datasets, demonstrating its superiority over existing methods. Our code is publicly available at https://github.com/linsongisgood/LELA.