Semi-supervised domain adaptation (SSDA) is a statistical learning problem that involves learning from a small portion of labeled target data and a large portion of unlabeled target data, together with many labeled source data, to achieve strong predictive performance on the target domain. Since the source and target domains exhibit distribution shifts, the effectiveness of SSDA methods relies on assumptions that relate the source and target distributions. In this talk, we develop a theoretical framework based on structural causal models to analyze and compare the performance of SSDA methods. We introduce fine-tuning algorithms under various assumptions about the relationship between source and target distributions and show how these algorithms enable models trained on source and unlabeled target data to perform well on the target domain with low target sample complexity. When such relationships are unknown, as is often the case in practice, we propose the Multi-Start Fine-Tuning (MSFT) algorithm, which selects the best-performing model from fine-tuning with multiple initializations. Our analysis shows that MSFT achieves optimal target prediction performance with significantly fewer labeled target samples compared to target-only approaches, demonstrating its effectiveness in scenarios with limited target labels.
|
