Collapsing Distance: The Curse of Ground Truth in Computational Narrative Understanding

Human label variation (HLV) is slowly gaining attention in natural language processing (NLP) research. It challenges the longstanding machine learning (ML) tradition of having a single ground truth via majority voting label aggregation. It influences all components of an ML pipeline, from data to modeling and evaluation. Albeit its relevance, HLV is rarely discussed in the context of computational narrative understanding. In this position paper, we bridge this gap and provide a distance-focused analysis of the ML pipeline. We examine how the single ground truth assumption shapes the interpretation of distance in annotation reliability, loss computation, and model evaluation metrics, and provide practical implications for the narrative understanding community.