How we ask people to answer questions about visualizations can shape what we measure. In this position paper, we argue that answer modality is a methodological choice that deserves explicit consideration.
Drawing on observations from three studies, we discuss how answer modalities (from multiple-choice options to numerical estimates or spatial responses) may influence reasoning strategies, introduce mental transformations, and reveal or conceal different kinds of errors. We offer practical guidance for aligning response formats with what a visualization evaluation aims to assess.
