Robust Multimodal Learning

Multimodal systems often struggle when one or more input sources are missing, corrupted, or imbalanced. My work develops learning strategies that preserve performance under these real-world conditions by enabling models to reason effectively with incomplete information. This includes designing projection-based methods that learn to estimate missing modality representations directly from the available ones and training unified models that handle all missing-modality configurations without the need for modality-specific retraining or complex adaptation procedures. I also explore efficient fine-tuning techniques, such as LoRA-based unimodal adaptation, to reduce computational overhead while maintaining strong performance across diverse modalities and tasks. These efforts benefit from my background in NLP, which strengthens my understanding of cross-modal interactions and how language models can support robust reasoning when some modalities are absent.

Niki Nezakati
Niki Nezakati
CS PhD Student