Subjective visualization experiences: impact of visual design and experimental design

In contrast to objectively measurable aspects (such as accuracy, reading speed, or memorability), the subjective experience of visualizations has only recently gained importance, and we have less experience how to measure it. We explore how subjective experience is affected by chart design using multiple experimental methods. We measure the effects of changes in color, orientation, and source annotation on the perceived readability and trustworthiness of simple bar charts. Three different experimental designs (single image rating, forced choice comparison, and semi-structured interviews) provide similar but different results. We find that these subjective experiences are different from what prior work on objective dimensions would predict. Seemingly inconsequential choices, like orientation, have large effects for some methods, indicating that study design alters decision-making strategies. Next to insights into the effect of chart design, we provide methodological insights, such as a suggested need to carefully isolate individual elements in charts to study subjective experiences.