Siqi Li

4111 Engineering Hall

Irvine, CA 92617

My name is Siqi Li, and I am a PhD student in the Department of Electrical Engineering and Computer Science at the University of California, Irvine, where I work under the supervision of Prof. Yasser Shoukry in the Resilient Cyber-Physical Systems Lab.

My research interests lie in vision–language models (VLMs) and multimodal large language models(MLLMs). I am passionate about enhancing their capabilities and applying them to real-world scenarios where they can meaningfully help people. In particular, I am interested in training these models to learn from human interaction, integrate into robots and AI agents, and develop stronger reasoning abilities to understand and support human needs. I also focus on improving their robustness against real-world challenges and adversarial attacks, ensuring they remain reliable and trustworthy in practical applications.

news

Jan 15, 2016	A simple inline announcement with Markdown emoji!
Nov 07, 2015	A long announcement with details
Oct 22, 2015	A simple inline announcement.

selected publications

EMNLP

Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration Approach

Siqi Li, Danni Liu, and Jan Niehues

In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Nov 2024

Abs DOI

Direct speech translation (ST) models often struggle with rare words. Incorrect translation of these words can have severe consequences, impacting translation quality and user trust. While rare word translation is inherently challenging for neural models due to sparse learning signals, real-world scenarios often allow access to translations of past recordings on similar topics. To leverage these valuable resources, we propose a retrieval-and-demonstration approach to enhance rare word translation accuracy in direct ST models. First, we adapt existing ST models to incorporate retrieved examples for rare word translation, which allows the model to benefit from prepended examples, similar to in-context learning. We then develop a cross-modal (speech-to-speech, speech-to-text, text-to-text) retriever to locate suitable examples. We demonstrate that standard ST models can be effectively adapted to leverage examples for rare word translation, improving rare word translation accuracy over the baseline by 17.6% with gold examples and 8.5% with retrieved examples. Moreover, our speech-to-speech retrieval approach outperforms other modalities and exhibits higher robustness to unseen speakers. Our code is publicly available.