REVIEW 7 minor 53 references
A Task Taxonomy for Conformance Checking
T0 review · 0 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Conformance checking analyses reduce to 102 distinct tasks, each expressible as a six-dimensional tuple, giving visualization designers and evaluators a concrete target to anchor their work.
desk verdict A genuine first artifact: a systematically built 102-task taxonomy for conformance checking, with honest limitations that are acknowledged rather than hidden. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying object is the six-dimensional taxonomy itself, with each task coded as a six-tuple; the constraint type dimension—a subset choice over control-flow, data, resource, and time perspectives—was introduced to capture tasks that differ only in process perspective. The construction method follows the iterative taxonomy development method with ending conditions, seeded by the generic design space of visualization tasks as a meta-characteristic. Task means are taken from a standard classification of visualization actions, while data characteristics and data targets were derived inductively from the case studies. To find dependencies among tasks, the authors treat each case study as a trace in an event log whose activities are the tasks, then apply process discovery to the resulting log.
What would settle it
Interview or shadow professional process analysts in industry and collect the conformance-related questions they ask; if a substantial share of those questions cannot be expressed as one of the 102 six-tuples without adding a new characteristic to a dimension, the taxonomy's completeness claim is refuted.
Extended reading notes
Core claim
The central claim is that the analytical purposes of conformance checking form a finite, structured space that can be captured as 102 tasks, each described by a six-tuple along the dimensions task goal (describe, explore, explain, confirm, present), task means (derive, identify, summarize, compare, present, discover, annotate, explore), data characteristics (such as guideline violations, process conformance, reasons for violations), constraint type (control-flow, data, resource, time, or a subset of these), data target (log, trace, event), and data cardinality (single, multiple, all). The taxonomy was derived through an iterative coding of tasks reported in 33 case studies and harmonized over five coding rounds. It also reports frequency patterns and dependencies: conformance analyses are predominantly exploratory rather than confirmatory, and they typically progress from a log-level overview to trace- and event-level detail, with explanatory analyses occurring last. The authors' purpose is to give visualization evaluation and design a task-based reference point, so that a visualization's usefulness can be assessed against concrete analytical purposes rather than generic visual idioms.
Load-bearing premise
The list of 102 tasks is assumed to cover the questions that real conformance-checking analysts actually ask, but it is generated from 33 academic case studies that had to literally contain the term 'conformance checking' and use real-world event data; the authors explicitly state the task list is most likely not complete.
Editorial extensions
If this is right
- A visualization can now be mapped to the specific tasks it supports, such as 'Describe: Derive Process conformance' or 'Explore: Identify Guideline violations', making tool evaluations comparable.
- Empirical user studies can measure effectiveness and efficiency per task rather than judging a visualization in the abstract; Section 7.1 lays out such a study design for identifying guideline violations.
- Taxonomy tasks can guide the design of new visualizations by making explicit which analytical purposes a design must serve, potentially reducing ad-hoc vendor-driven choices.
- The dependency patterns suggest tool workflows should start with log-level overviews, support drilling through traces to events, and place explanatory analysis near the end.
- The taxonomy points to underexplored tasks—confirmatory analyses appear in only two tasks—as candidate areas for future conformance checking research and tooling.
Reading between the lines
- The same case-study-to-taxonomy pipeline could be applied to other process mining subfields, such as process discovery or predictive monitoring, yielding comparable task taxonomies that the visual analytics community could integrate into a unified process mining task model.
- Adopting the six-tuple as a machine-readable task descriptor would allow a visualization to be annotated with the tasks it supports, enabling automated matching between user questions and tool features, and gap analysis across a tool's visualizations.
- The taxonomy predicts testable differences in visualization effectiveness: for example, tasks with data cardinality 'all' (e.g., conformance distributions) plausibly benefit from aggregated idioms, whereas 'single'-trace tasks plausibly benefit from per-trace chart idioms such as chevron diagrams; this can be checked in user experiments.
- The observed workflow pattern—log to trace to event, describe and present early, explain late—could be turned into a prescriptive interaction design pattern for conformance checking tools, including default drill-down paths and placement of explanation views.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper develops a task taxonomy for conformance checking, motivated by the need to give visualization evaluation and design a task-level anchor. The authors generate tasks from 33 conformance checking case studies (screened from 209 publications), code 102 tasks into six dimensions (task goal, task means, data characteristics, constraint type, data target, data cardinality), refine the taxonomy over five iterations using the Nickerson et al. method, analyze the frequency of and dependencies between tasks, and illustrate selected tasks with visualizations from commercial process mining tools. The central contribution is the taxonomy itself, together with the claim that it provides a structured way to determine the analytical purpose of conformance checking visualizations.
Significance. If the taxonomy is accepted, it gives the conformance checking community a structured vocabulary for analysis tasks and provides a bridge between visual analytics and process mining. The paper's method is a genuine strength: the case-study selection uses explicit inclusion criteria, the coding is double-coded with consensus, the taxonomy development is documented across five iterations, and the data are made available online. The dependency analysis via process discovery is a useful addition that goes beyond a static classification. The external-validity concern raised by the reliance on 33 academic case studies is real but appropriately scoped: the paper explicitly frames the task list as a first observation and states in Section 7.2 that it is most likely not complete, and Section 5 presents the commercial-tool mapping as illustrative rather than exhaustive. I therefore do not regard this as a load-bearing flaw for the paper's stated contribution, although a broader mapping of all 102 tasks to tool visualizations would strengthen future work.
minor comments (7)
- [Section 5, first paragraph] The text says 'As a validation of all 101 tasks is out of scope,' but the taxonomy contains 102 tasks everywhere else in the paper (Section 3.2.3, Table 4, Section 4.1). Please correct the number.
- [Section 5.6] The text says 'Despite occurring only two times in our literature review,' but Table 3 lists 'Process conformance over time' with frequency 1 and Table 5 lists 'Describe: Derive Process conformance over time' with frequency 1. Please reconcile the count or explain which two occurrences are meant.
- [Section 3.2 and Section 4.1] The taxonomy requirements state that each dimension has mutually exclusive characteristics and that each task has exactly one characteristic per dimension, but the constraint type is later described as a 'subset choice' and as allowing tasks to be associated with multiple realizations. Please clarify that the subset itself is the single characteristic value, or revise the wording to remove the apparent contradiction.
- [Section 3.1.1] The supplementary literature search is described as limiting results to the '50 most relevant papers' based on a gradual decline in relevance after scanning titles and abstracts. This selection heuristic should be documented more precisely, including the search date and the decision rule, so that the screening is reproducible.
- [Section 3.2.2] The double-coding procedure is described, but no inter-rater reliability statistic (for example, Cohen's kappa) is reported. Reporting agreement would strengthen the reliability of the coding procedure that underpins the taxonomy.
- [Figure 2 caption] The caption contains a stray template fragment, '19.12.2017Beispiel-Fußzeile 2', which appears unrelated to the figure and should be removed.
- [Section 4.3] The dependency analysis uses an encoded task order as the timestamp in the event log. It would be helpful to state how tasks that are described in parallel or without a clear ordering within a case study were handled, since this affects the discovered process models.
Circularity Check
No significant circularity: the taxonomy is induced from externally sourced case studies and validated against commercial tool visualizations not used in task generation.
full rationale
The paper constructs a task taxonomy rather than deriving a prediction from an assumed result. Task generation is grounded in 33 external case studies collected from a prior literature review and a supplementary search (Section 3.1), with inclusion criteria based on the presence of the term 'conformance checking', use of conformance checking techniques, and real-world event data. The categorization dimensions are taken from established external visualization design-space frameworks (Schulz et al., Munzner) and refined through documented iterative coding following Nickerson et al. No equation is fitted, and no quantity is predicted from the taxonomy itself. The only instances of self-citation appear in the related-work discussion (references [15], [16], and [17]) and are not load-bearing for the taxonomy's construction or validation. The external validity check in Section 5 maps seven selected tasks to visualizations from commercial tools that were not part of the original task generation, which is an independent, albeit partial, validation. The authors' explicit limitations in Section 7.2, including the possibility that the task list is incomplete and that subjective judgment played a role, weaken generality but do not constitute circular reasoning. Thus, no circular step can be exhibited from the paper's own text.
Assumptions & free parameters
assumptions (5)
- domain assumption Conformance checking tasks can be identified independently of the underlying conformance checking technique.
- domain assumption The visualization task design space (goal, means, data characteristics, data target, data cardinality) transfers to conformance checking as a meta-characteristic.
- domain assumption Academic case studies that mention conformance checking and use real-world event data are a sufficient proxy for real analyst tasks.
- domain assumption The 33 retained case studies contain enough distributed task occurrences to support the discovered task dependencies.
- domain assumption The coding categories satisfy mutual exclusivity and collective exhaustiveness, the ending conditions of Nickerson et al.'s taxonomy method.
Cite this review
Pith. "Pith review of A Task Taxonomy for Conformance Checking." pith.science (2026). https://pith.science/paper/AQNK35FW
@misc{pith2026250711976,
author = {Pith},
title = {Pith review of: A Task Taxonomy for Conformance Checking},
year = {2026},
howpublished = {\url{https://pith.science/paper/AQNK35FW}},
note = {Machine review of arXiv:2507.11976}
}
read the original abstract
Conformance checking is a sub-discipline of process mining, which compares observed process traces with a process model to analyze whether the process execution conforms with or deviates from the process design. Organizations can leverage this analysis, for example to check whether their processes comply with internal or external regulations or to identify potential improvements. Gaining these insights requires suitable visualizations, which make complex results accessible and actionable. So far, however, the development of conformance checking visualizations has largely been left to tool vendors. As a result, current tools offer a wide variety of visual representations for conformance checking, but the analytical purposes they serve often remain unclear. However, without a systematic understanding of these purposes, it is difficult to evaluate the visualizations' usefulness. Such an evaluation hence requires a deeper understanding of conformance checking as an analysis domain. To this end, we propose a task taxonomy, which categorizes the tasks that can occur when conducting conformance checking analyses. This taxonomy supports researchers in determining the purpose of visualizations, specifying relevant conformance checking tasks in terms of their goal, means, constraint type, data characteristics, data target, and data cardinality. Combining concepts from process mining and visual analytics, we address researchers from both disciplines to enable and support closer collaborations.
Figures
Figures from the paper (20 more)
Reference graph
Works this paper leans on
-
[1]
W. van der Aalst, Process mining: A 360 degree overview, in: Process Mining Handbook, Springer, 2022, pp. 3–34
work page 2022
-
[2]
J. Carmona, B. van Dongen, A. Solti, M. Weidlich, Conformance check- ing, Springer, 2018
work page 2018
- [3]
-
[4]
F. Emamjome, R. Andrews, A. ter Hofstede, A case study lens on process mining in practice, in: On the Move to Meaningful Internet Systems, Springer, 2019, pp. 127–145
work page 2019
-
[5]
F AU, Process Mining Survey (2021). URL https://www.processmining-software.com/wp-content/upl oads/2021/FAU2021_Process_Mining_Survey.pdf
work page 2021
-
[6]
Reinkemeyer, Process mining in action, Springer, 2020
L. Reinkemeyer, Process mining in action, Springer, 2020. 41
work page 2020
-
[7]
T. Voglhofer, S. Rinderle-Ma, Collection and elicitation of business pro- cess compliance patterns with focus on data aspects, Business & infor- mation systems engineering 62 (4) (2020) 361–377
work page 2020
-
[8]
J. Peeperkorn, S. vanden Broucke, J. De Weerdt, Global conformance checking measures using shallow representation and deep learning, En- gineering Applications of Artificial Intelligence 123 (2023) 106393
work page 2023
Show all 53 references
-
[9]
Knuplesch, M
D. Knuplesch, M. Reichert, A. Kumar, A framework for visually moni- toring business process compliance, Information Systems 64 (2017)
2017
-
[10]
M. Gall, S. Rinderle-Ma, Visual modeling of instance-spanning con- straints in process-aware information systems, in: Advanced Informa- tion Systems Engineering, Springer, 2017, pp. 597–611
2017
-
[11]
Garcia-Banuelos, N
L. Garcia-Banuelos, N. Van Beest, M. Dumas, M. La Rosa, W. Mertens, Complete and interpretable conformance checking of business processes, IEEE Transactions on Software Engineering 44 (3) (2017) 262–290
2017
-
[12]
Gschwandtner, Visual analytics meets process mining: Challenges and opportunities, in: Data-Driven Process Discovery and Analysis, Springer, 2017, pp
T. Gschwandtner, Visual analytics meets process mining: Challenges and opportunities, in: Data-Driven Process Discovery and Analysis, Springer, 2017, pp. 142–154
2017
-
[13]
Klinkm¨ uller, R
C. Klinkm¨ uller, R. M¨ uller, I. Weber, Mining process mining practices: An exploratory characterization of information needs in process analyt- ics, in: Business Process Management, Springer, 2019, pp. 322–337
2019
-
[14]
D. Keim, G. Andrienko, J.-D. Fekete, C. G¨ org, J. Kohlhammer, G. Melan¸ con, Visual analytics: Definition, process, and challenges, in: Information Visualization: Human-Centered Issues and Perspectives, Springer, 2008, pp. 154–175
2008
-
[15]
Rehse, L
J. Rehse, L. Pufahl, M. Grohs, L. Klein, Process mining meets visual analytics: The case of conformance checking, in: Hawaii International Conference on System Sciences, ScholarSpace, 2023, pp. 5452–5461
2023
-
[16]
H¨ age, J.-R
M.-C. H¨ age, J.-R. Rehse, A taxonomy for conformance checking visual- izations, in: ICPM Workshops, Springer, 2025, pp. 507–519
2025
-
[17]
Klessascheck, T
F. Klessascheck, T. Knoche, L. Pufahl, Designing and Evaluating a Structural Model for Conformance Checking Visualizations, in: Visual Process Analytics Workshop, The Eurographics Association, 2024. 42
2024
-
[18]
Munzner, A nested model for visualization design and validation, Transactions on visualization and computer graphics 15 (6) (2009) 921– 928
T. Munzner, A nested model for visualization design and validation, Transactions on visualization and computer graphics 15 (6) (2009) 921– 928
2009
-
[19]
Sedlmair, M
M. Sedlmair, M. Meyer, T. Munzner, Design study methodology: Re- flections from the trenches and the stacks, Transactions on visualization and computer graphics 18 (12) (2012) 2431–2440
2012
-
[20]
Kerracher, J
N. Kerracher, J. Kennedy, Constructing and evaluating visualisation task classifications: Process and considerations, in: Computer Graphics Forum, Vol. 36, Wiley Online Library, 2017, pp. 47–59
2017
-
[21]
Weske, Business Process Management - Concepts, Languages, Ar- chitectures, Vol
M. Weske, Business Process Management - Concepts, Languages, Ar- chitectures, Vol. 3, Springer, 2019
2019
-
[22]
Dumas, M
M. Dumas, M. La Rosa, J. Mendling, H. Reijers, Fundamentals of busi- ness process management, Vol. 2, Springer, 2018
2018
-
[23]
Object Management Group, Notation BPMN version 2.0 (2011)
2011
-
[24]
van der Aalst, et al., Process mining manifesto, in: BPM Workshops, 2012, pp
W. van der Aalst, et al., Process mining manifesto, in: BPM Workshops, 2012, pp. 169–194
2012
-
[25]
Andrienko, G
N. Andrienko, G. Andrienko, Exploratory analysis of spatial and tem- poral data: a systematic approach, Springer Science & Business Media, 2006
2006
-
[26]
Kerracher, J
N. Kerracher, J. Kennedy, K. Chalmers, A task taxonomy for temporal graph visualisation, IEEE transactions on visualization and computer graphics 21 (10) (2015) 1160–1172
2015
-
[27]
Munzner, Visualization analysis and design, CRC press, 2014
T. Munzner, Visualization analysis and design, CRC press, 2014
2014
-
[28]
F. Beck, M. Burch, S. Diehl, D. Weiskopf, A taxonomy and survey of dynamic graph visualization, in: Computer Graphics Forum, Vol. 36, Wiley Online Library, 2017, pp. 133–159
2017
-
[29]
McGee, M
F. McGee, M. Ghoniem, B. Otjacques, B. Renoust, D. Archambault, A. Kerren, B. Pinaud, G. Melan¸ con, M. Pohl, T. von Landesberger, Task taxonomy for multilayer networks, in: Visual Analysis of Multilayer Networks, Springer, 2021, pp. 37–44. 43
2021
-
[30]
Par´ e, M.-C
G. Par´ e, M.-C. Trudel, M. Jaana, S. Kitsiou, Synthesizing information systems knowledge: A typology of literature reviews, Information & Management 52 (2) (2015) 183–199
2015
-
[31]
Nickerson, U
R. Nickerson, U. Varshney, J. Muntermann, A method for taxonomy de- velopment and its application in information systems, European Journal of Information Systems 22 (2013) 336–359
2013
-
[32]
Recker, Scientific research in information systems, 2nd Edition, Springer, 2021
J. Recker, Scientific research in information systems, 2nd Edition, Springer, 2021
2021
-
[33]
Schulz, T
H.-J. Schulz, T. Nocke, M. Heitzler, H. Schumann, A design space of visualization tasks, IEEE Transactions on Visualization and Computer Graphics 19 (12) (2013) 2366–2375
2013
-
[34]
Tominski, H
C. Tominski, H. Schumann, Interactive visual data analysis, AK Peter- s/CRC Press, 2020
2020
-
[35]
Gleicher, M
M. Gleicher, M. Riveiro, T. von Landesberger, O. Deussen, R. Chang, C. Gillman, A problem space for designing visualizations, IEEE Com- puter Graphics and Applications 43 (4) (2023) 111–120
2023
-
[36]
Burattin, F
A. Burattin, F. Maggi, A. Sperduti, Conformance checking based on multi-perspective declarative process models, Expert Systems with Ap- plications 65 (2016) 194–211
2016
-
[37]
Janssenswillen, B
G. Janssenswillen, B. Depaire, Towards confirmatory process discovery: Making assertions about the underlying system, Business & Information Systems Engineering 61 (6) (2019) 713–728
2019
-
[38]
Rinner, E
C. Rinner, E. Helm, R. Dunkl, H. Kittler, S. Rinderle-Ma, Process min- ing and conformance checking of long running processes in the context of melanoma surveillance, International journal of environmental research and public health 15 (12) (2018) 2809
2018
-
[39]
de Leoni, W
M. de Leoni, W. van der Aalst, Aligning event logs and process models for multi-perspective conformance checking: An approach based on in- teger linear programming, in: Business Process Management, Springer, 2013, pp. 113–129. 44
2013
-
[40]
Zhang, L
S. Zhang, L. Genga, L. Dekker, H. Nie, X. Lu, H. Duan, U. Kaymak, Fuzzy multi-perspective conformance checking for business processes, Applied Soft Computing 130 (2022) 109710
2022
-
[41]
URL https://www.processmining-software.com/tools/
F AU, Process Mining Software Comparison (2020). URL https://www.processmining-software.com/tools/
2020
-
[42]
URL https://docs.celonis.com/en/analysis---conformance-che cker.html
Celonis, Analysis - Conformance Checker (2024). URL https://docs.celonis.com/en/analysis---conformance-che cker.html
2024
-
[43]
van der Aalst, M
W. van der Aalst, M. de Leoni, A. ter Hofstede, Process mining and vi- sual analytics: Breathing life into business process models, BPM Center Report 17 (2011) 699–730
2011
-
[44]
Kriglstein, M
S. Kriglstein, M. Pohl, S. Rinderle-Ma, M. Stallinger, Visual analytics in process mining: Classification of process mining techniques, in: EuroVis Workshop on Visual Analytics, The Eurographics Association, 2016, pp. 43–47
2016
-
[45]
Miksch, C
S. Miksch, C. Di Ciccio, P. Soffer, B. Weber, Visual analytics meets pro- cess mining: Challenges and opportunities, IEEE Computer Graphics and Applications 44 (6) (2024) 132–141. doi:10.1109/MCG.2024.345 6916
2024 doi
-
[46]
Mendling, D
J. Mendling, D. Djurica, M. Malinova, Cognitive effectiveness of rep- resentations for process mining, in: Business Process Management, Springer, 2021, pp. 17–22
2021
-
[47]
Y. Guo, S. Guo, Z. Jin, S. Kaul, D. Gotz, N. Cao, Survey on visual analysis of event sequence data, IEEE Transactions on Visualization and Computer Graphics 28 (12) (2022) 5091–5112
2022
-
[48]
Yeshchenko, J
A. Yeshchenko, J. Mendling, A survey of approaches for event sequence analysis and visualization, Information Systems 120 (2024) 102283
2024
-
[49]
Kaouni, G
A. Kaouni, G. Theodoropoulou, A. Bousdekis, A. Voulodimos, G. Miaoulis, Visual analytics in process mining for supporting business process improvement, in: Novelties in Intelligent Digital Systems, IOS Press, 2021, pp. 166–175. 45
2021
-
[50]
M. Gall, S. Rinderle-Ma, Evaluating compliance state visualizations for multiple process models and instances, in: Business Process Manage- ment Forum, Springer, 2021, pp. 126–142
2021
-
[51]
Knoblich, J
S. Knoblich, J. Mendling, H. Jambor, Review of Visual Encodings in Common Process Mining Tools, in: Visual Process Analytics Workshop, The Eurographics Association, 2024
2024
-
[52]
Brinkman, F
R. Brinkman, F. Mannhardt, R. Mennens, R. Scheepens, Interpretabil- ity Challenges for Discovered Process Models: A User Study and Proto- type Solution, in: Visual Process Analytics Workshop, The Eurograph- ics Association, 2024
2024
-
[53]
vom Brocke, A
J. vom Brocke, A. Simons, K. Riemer, B. Niehaves, R. Plattfaut, A. Cleven, Standing on the shoulders of giants: Challenges and recom- mendations of literature search in information systems research, Com- munications of the association for information systems 37 (1) (2015) 9. 46
2015
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.