Aggregating empirical evidence from data strategy studies: a case on model quantization

Guilherme Horta Travassos; Paulo S\'ergio Medeiros dos Santos; Santiago del Rey; Silverio Mart\'inez-Fern\'andez; Xavier Franch

REVIEW 3 major objections 3 minor 38 references

Aggregating empirical evidence from data strategy studies: a case on model quantization

T0 review · 3 major / 3 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Aggregated evidence from six studies shows model quantization weakly hurts accuracy while strongly improving storage and GPU energy use.

desk verdict A transparent and useful synthesis whose qualitative conclusions are credible, but the headline belief percentages rest on an unvalidated discount coefficient and a mislabeled forest plot, so treat the numbers as illustrative rather than measured. read the letter →

arxiv 2505.00816 v1 pith:CYNQ52GF submitted 2025-05-01 cs.SE cs.LG

Santiago del Rey , Paulo Sérgio Medeiros dos Santos , Guilherme Horta Travassos , Xavier Franch , Silverio Martínez-Fernández This is my paper · ORCID

classification cs.SEcs.LG

keywords modelquantizationevidenceaggregationStructuredSynthesisMethoddatastrategystudiesdeeplearningresourceefficiencygreenAIDempster-Shafertheory

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Model quantization maps a deep learning model's continuous values into a smaller set of discrete values, shrinking the model and speeding it up. This paper aggregates evidence from six primary studies, yielding 19 evidence models, to characterize what quantization does to correctness and resource efficiency. The authors' proposed theory is that quantization strongly improves storage size and GPU energy consumption, weakly improves inference power draw, leaves GPU power draw and inference latency between indifferent and weakly positive, and weakly hurts accuracy. The paper also claims that the Structured Synthesis Method (SSM) can feasibly aggregate artifact-based 'data strategy' studies, and that the main obstacle to finer conclusions is fragmented evidence per quantization technique.

What carries the argument

The Structured Synthesis Method (SSM) is the aggregation apparatus: it represents each study as a diagrammatic evidence model, converts reported relative improvements into a seven-point Likert intensity (strongly negative to strongly positive) using contextually defined thresholds, and assigns each effect a belief value derived from the GRADE study-type hierarchy refined by quality assessment. A novel discount formula, $\mathrm{discount} = 1 - e^{-0.1 \times |\mathrm{IQR}/\mu|}$, weakens beliefs for highly dispersed measurements. Dempster-Shafer theory then combines the belief distributions over intensity hypotheses via Dempster's rule, producing the aggregated intensities and beliefs that are the paper's main output.

What would settle it

Recompute the aggregated intensities and beliefs directly from the raw measurements of the six primary studies, bypassing the Fig. 3 thresholds and the Eq. 1 discount; if the direct values put storage-size gains below the 'strongly positive' band or accuracy losses beyond 'weakly negative,' the paper's headline trade-off would not survive.

Watch

Extended reading notes

Core claim

Across 19 evidence models extracted from the six selected studies, the aggregated evidence indicates that model quantization weakly negatively affects accuracy (aggregated intensity WN with 90% belief) and has an indifferent effect on F1 score, while strongly positively affecting storage size (SP, 100% belief) and GPU energy consumption ({PO,SP}, 95% belief). Inference latency aggregates at {IF,WP} with 98% belief and inference power draw at WP with 93% belief. The aggregation also shows that belief in a positive effect on inference energy consumption decreased because the evidence spans from {IF,WP} to SP. The authors therefore state that quantization weakly compromises correctness while significantly improving resource efficiency, a manageable trade-off for resource-constrained deployment, and that the evidence remains too fragmented to generalize per quantization technique.

Load-bearing premise

The whole aggregation rests on the assumption that converting each study's quantitative improvements into seven-point intensity levels via the thresholds in Fig. 3, and adjusting beliefs with the authors' new discount formula, faithfully represents the evidence; if either mapping distorts effect sizes, the aggregated intensities and beliefs would be wrong.

Editorial extensions

If this is right

DL engineers in resource-constrained settings can expect quantization to deliver large storage and GPU energy savings while risking only a small accuracy drop, supporting deployment on edge devices and battery-powered systems.
Because storage size aggregated at 100% belief and GPU energy at 95%, these two benefits are the most reliable claims to build on; other resource metrics should be treated as less certain.
The weakly negative accuracy effect is an average: evidence includes cases where accuracy improves, so the trade-off is context-dependent rather than a fixed penalty.
The decreased belief for inference energy consumption indicates that conflicting evidence on this metric remains unresolved, so it should not be cited as a guaranteed benefit.
Future primary studies should report raw data and repeated-run variation, since only two of six selected studies made raw data available; better data would raise the reliability of any re-aggregation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A re-aggregation that treats each primary study as one unit rather than weighting by the number of reported samples would test whether the 28,000 samples from one quasi-experiment dominate the storage-size and GPU-energy beliefs; the paper does not perform this robustness check.
The authors' own observation that smaller precisions and full weight-and-activation quantization yield stronger effects suggests a testable stratification: aggregating separately by precision (e.g., INT8 vs. INT4) should show steeper accuracy degradation and larger efficiency gains at lower precision.
The new discount formula (Eq. 1) is a judgment call; plugging in alternative dispersion penalties (e.g., coefficient of variation or standard deviation) would reveal how sensitive the headline beliefs are to that choice.
The same SSM pipeline could be applied to other optimization tactics, like pruning or knowledge distillation, which would let the field compare efficiency-correctness trade-offs across techniques on a common scale.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The first thing you should know: this paper is a genuinely useful methodological case study, but its headline numbers are softer than they look. The qualitative conclusion—quantization weakly hurts accuracy while improving storage, latency, and energy—is robust and matches what the earlier surveys already said. The specific belief values (90%, 100%, 95%) are method-dependent and should not be quoted without caveats.

What is actually new: it is the first systematic synthesis of model quantization effects using SSM, working from six primary studies and 19 evidence models. The authors ship a replication package, use the Evidence Factory tool, and are unusually candid about their process. The discussion of how to adapt synthesis methods to data strategy studies is the strongest part—the point about quality checklists needing new dimensions for non-human subjects is worth taking seriously.

The soft spots are exactly where the stress-test lands. Equation 1 introduces a discount factor with an arbitrary 0.1 coefficient, and the paper itself admits in Section VII that this is a novel contribution needing validation. Every belief in Table II passes through that formula, so the confidence percentages are not robust until someone shows a sensitivity analysis across a plausible range of that coefficient. The same applies to the intensity thresholds in Fig. 3, which are contextually defined without external justification. And the figure labels those ranges as "95% confidence intervals" when the text later clarifies they are just intensity ranges on the Likert scale—that is misleading and should be fixed before publication.

Two smaller issues. The author overlap with primary study S5 is not disclosed; it is not disqualifying, but a reader should know that two co-authors of this synthesis also authored one of the six studies being aggregated. And the aggregation lumps all quantization methods together despite acknowledging method as a confound; they handle it reasonably in the threats section, but it limits the granularity of any claim per technique.

The direction of every qualitative conclusion is probably right. The problem is only that the quantified beliefs are presented as if they were measured when they are partly constructed by unvalidated choices. That is a fixable presentation and analysis issue, not a fatal flaw.

Who is this for? Researchers working on evidence synthesis in SE, especially anyone trying to aggregate data strategy studies. It deserves a serious referee—the methodological template and transparency are valuable even if the specific belief values need revision. I would accept it for peer review and ask for a sensitivity analysis of Eq. 1 and corrected labels in Fig. 3.

Referee Report

3 major / 3 minor

Summary. This manuscript reports a research synthesis of six primary studies on model quantization in deep learning systems, using the Structured Synthesis Method (SSM) to aggregate evidence. The authors extract 19 evidence models, assign Likert-scale effect intensities and belief values, and combine them with Dempster-Shafer theory. Their headline findings are that model quantization weakly negatively affects accuracy (90% belief), indifferently affects F1 score, and consistently improves resource efficiency metrics such as storage size (100%), GPU energy consumption (95%), and inference latency (98%). The paper also draws methodological lessons for aggregating evidence from data-strategy studies, including challenges in quality assessment, data availability, and the use of adapted forest plots.

Significance. If the aggregation is sound, this is a useful contribution to the empirical software engineering literature on green AI and model deployment. The paper is unusually transparent: it ships a replication package, makes all evidence models publicly available in the Evidence Factory tool, is explicit about using an LLM in study selection and about the manual verification of that step, and candidly acknowledges in Section VII that its discount formula is novel and requires further validation. The qualitative directional findings—quantization tends to improve resource efficiency while slightly hurting accuracy—are plausible and consistent with the broader literature. However, the quantified belief percentages, which are central to the paper's claims, depend on several unvalidated modeling choices. The paper's own limitation statements confirm that these choices are not yet calibrated, and therefore the headline numbers should not be treated as robust until sensitivity analysis is provided.

major comments (3)

[Section IV, Eq. (1); Table II] The discount factor in Eq. (1) uses a coefficient of 0.1 that is not derived from any external criterion, and the manuscript itself states in Section VII that this is a novel contribution requiring further empirical validation. Because this discount is applied to every extracted effect before Dempster-Shafer combination, all aggregated beliefs in Table II (e.g., accuracy 90%, storage size 100%, GPU energy 95%) depend on this uncalibrated constant. A sensitivity analysis over a plausible range of the coefficient, or a calibration against an external benchmark, is necessary to support the reported belief values.
[Section IV, Fig. 3] The thresholds used to convert relative improvements into seven-point Likert intensities are described as 'contextually defined' and are not derived from any external criterion. This is a load-bearing choice because effect intensity determines the hypotheses combined by DST. Moreover, Fig. 3 labels the plotted ranges as '95% confidence intervals,' but the text immediately following the figure clarifies that the aggregated points are placed at the center of the intensity ranges and that the intervals merely span those ranges. This label is statistically misleading and should be corrected to something like 'intensity range' or 'uncertainty range.'
[Section V-A, Table II (Accuracy row)] The accuracy aggregation is reported as having a 90% belief with a 'weakly negative' intensity, yet the same row shows a conflict value of 0.41 and a belief difference of 24%, and the text notes contradictory results in the evidence. The paper should explain how Dempster-Shafer combination yields a 90% belief in a WN effect when the underlying evidence is so conflicted, and clarify whether this belief value represents agreement, evidence strength, or both. Without such an explanation, the reader cannot interpret the 90% as a meaningful confidence level.

minor comments (3)

[Section VII] The threats-to-validity section refers to the discounting mechanism as 'described in Section III-C,' but Eq. (1) actually appears in Section IV; update the cross-reference.
[Section V-C (Proposed theory)] In the proposed theory box, the word 'indiferent' is misspelled; it should read 'indifferent.'
[Section III-A.1, search string] The search string contains the typo 'fundational model' (should be 'foundational model') and the token '?NN' is unclear; if it is intended to match 'DNN' or 'CNN,' please write the intended alternatives explicitly.

Circularity Check

0 steps flagged · score 2.0 of 10

Synthesis is independent of its inputs; minor self-citations and an arbitrary but disclosed discount formula do not make the central claim circular.

full rationale

The paper's derivation chain aggregates quantitative data from six external primary studies and derives aggregated intensities and beliefs through SSM. No step reduces a target result to an input by construction: the reported effect intensities are direct summaries of measured relative improvements mapped through the thresholds in Fig. 3; the belief values in Table II are arithmetic consequences of GRADE-based base beliefs, quality assessment, and the discount formula Eq. 1. The paper itself flags in Section VII (Threats to Validity) that the discount term is a novel contribution proposed by the authors and that 'further empirical validation is required to establish its robustness across broader contexts.' This is an openly disclosed methodological weakness, not a hidden equivalence: the beliefs are not presented as external truths or predictions, and the final intensities are not encoded in the discount. The SSM method and Evidence Factory tool are cited from the authors' own prior work, but the method is applied as a framework, not as evidence for the quantization effects; its assumptions do not include the target result. One primary study (S5) shares co-authors, but five other independent studies contribute evidence, and the direction and magnitude of the central effects are supported without S5 being load-bearing. The self-citations are minor and non-circular; the arbitrary constant in Eq. 1 and the contextual thresholds are correctness and robustness risks, not circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The synthesis introduces no new physical entities. It does introduce a new discounting formula with a hand-chosen coefficient and relies on several domain assumptions about the commensurability and independence of evidence across heterogeneous studies, plus an evidence-grading scheme adapted from medicine.

free parameters (2)

Discount coefficient = 0.1
Chosen by the authors in Eq. 1 to control the strength of belief discounting; no calibration or independent validation is provided.
Intensity mapping thresholds = 2, 5, 10, 15, 20, 25, 50, 75, 100 percent relative improvement
Fig. 3 defines boundaries for converting quantitative improvements into seven-point Likert intensities; these are contextually defined and arbitrary.

assumptions (4)

domain assumption Dempster-Shafer combination of beliefs is appropriate for aggregating evidence models
The synthesis relies on DST as the mathematical foundation for combining beliefs, assuming that belief masses from different studies can be pooled and that independence holds.
domain assumption GRADE evidence hierarchy maps study design to belief value ranges
Belief values are derived from the GRADE hierarchy (unsystematic observations 0.00-0.25, etc.), which is a medical evidence grading scheme, not designed for data strategy studies.
domain assumption Relative improvements from different baselines and metrics are commensurable across studies
Studies compare quantized models against different baselines (FP64, FP32, FP16) and different models or datasets; the synthesis treats relative improvements as directly comparable.
domain assumption Multiple evidence models extracted from one study can be treated as independent evidence
Some studies yield many evidence models (e.g., S3 contributes eight); the aggregation treats these as independent despite likely correlation, and no clustering adjustment is applied.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Aggregating empirical evidence from data strategy studies: a case on model quantization." pith.science (2026). https://pith.science/paper/CYNQ52GF

@misc{pith2026250500816,
  author       = {Pith},
  title        = {Pith review of: Aggregating empirical evidence from data strategy studies: a case on model quantization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CYNQ52GF}},
  note         = {Machine review of arXiv:2505.00816}
}

abstract

Background: As empirical software engineering evolves, more studies adopt data strategies$-$approaches that investigate digital artifacts such as models, source code, or system logs rather than relying on human subjects. Synthesizing results from such studies introduces new methodological challenges. Aims: This study assesses the effects of model quantization on correctness and resource efficiency in deep learning (DL) systems. Additionally, it explores the methodological implications of aggregating evidence from empirical studies that adopt data strategies. Method: We conducted a research synthesis of six primary studies that empirically evaluate model quantization. We applied the Structured Synthesis Method (SSM) to aggregate the findings, which combines qualitative and quantitative evidence through diagrammatic modeling. A total of 19 evidence models were extracted and aggregated. Results: The aggregated evidence indicates that model quantization weakly negatively affects correctness metrics while consistently improving resource efficiency metrics, including storage size, inference latency, and GPU energy consumption$-$a manageable trade-off for many DL deployment contexts. Evidence across quantization techniques remains fragmented, underscoring the need for more focused empirical studies per technique. Conclusions: Model quantization offers substantial efficiency benefits with minor trade-offs in correctness, making it a suitable optimization strategy for resource-constrained environments. This study also demonstrates the feasibility of using SSM to synthesize findings from data strategy-based research.

Figures

Figures reproduced from arXiv: 2505.00816 by the authors.

**Figure 2.** Evidence model representing the results of S1. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Adapted forest plot showing the effects of applying model quantization on dependent variables from all the extracted evidence from primary studies S1 to S6. Links to each evidence diagram are provided in the replication package [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 36 canonical work pages

[1]

The who, what, how of software engineering research: A socio-technical framework,

M.-A. Storey, N. A. Ernst, C. Williams, and E. Kalliamvakou, “The who, what, how of software engineering research: A socio-technical framework,” Empirical Software Engineering , vol. 25, no. 5, pp. 4097– 4129, 2020

work page 2020
[2]

Software Engineering for AI-Based Systems: A Survey,

S. Mart ´ınez-Fern´andez, J. Bogner, X. Franch, M. Oriol, J. Siebert, A. Trendowicz, A. M. V ollmer, and S. Wagner, “Software Engineering for AI-Based Systems: A Survey,” ACM Trans. Softw. Eng. Methodol. , vol. 31, no. 2, pp. 37e:1–37e:59, 2022

work page 2022
[3]

Energy-Efficient Respiratory Anomaly Detection in Premature New- born Infants,

A. Paul, M. A. S. Tajin, A. Das, W. M. Mongan, and K. R. Dandekar, “Energy-Efficient Respiratory Anomaly Detection in Premature New- born Infants,” Electronics, vol. 11, no. 5, p. 682, 2022

work page 2022
[4]

Im- pact of ML optimization tactics on greener pre-trained ML models,

A. Gonz ´alez, J. Casta ˜no, X. Franch, and S. Mart ´ınez-Fern´andez, “Im- pact of ML optimization tactics on greener pre-trained ML models,” Computing, vol. 107, no. 4, pp. 1–25, 2025

work page 2025
[5]

On the Representation and Aggregation of Evidence in Software Engineering: A Theory and Belief-based Perspective,

P. S. Medeiros dos Santos and G. H. Travassos, “On the Representation and Aggregation of Evidence in Software Engineering: A Theory and Belief-based Perspective,” Electronic Notes in Theoretical Computer Science, vol. 292, pp. 95–118, 2013

work page 2013
[6]

A Synthesis of Green Architectural Tactics for ML-Enabled Systems,

H. J ¨arvenp¨a¨a, P. Lago, J. Bogner, G. Lewis, H. Muccini, and I. Ozkaya, “A Synthesis of Green Architectural Tactics for ML-Enabled Systems,” in ICSE-SEIS’24, Jun. 2024, pp. 130–141

work page 2024
[7]

[Online]

Replication package. [Online]. Available: https://figshare.com/s/ f92e01d7548664883abb

work page
[8]

[Online]

Evidence Factory - Model Quantization Synthesis. [Online]. Available: https://evidencefactory.lens-ese.cos.ufrj.br/synthesis/editor/243422

work page

Show all 38 references

[9]

Contemporary Advances in Neural Network Quantization: A Survey,

M. Li, Z. Huang, L. Chen, J. Ren, M. Jiang, F. Li, J. Fu, and C. Gao, “Contemporary Advances in Neural Network Quantization: A Survey,” in IJCNN ’24, 2024, pp. 1–10

2024
[10]

A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification,

B. Rokh, A. Azarpeyvand, and A. Khanteymoori, “A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification,” ACM Trans. Intell. Syst. Technol. , vol. 14, no. 6, pp. 97:1–97:50, 2023

2023
[11]

A Scoping Review on Quantization Methods for Medical Imaging AI,

A. R. Paddo, R. T. Raju, J. W. Gichoya, and S. Purkayastha, “A Scoping Review on Quantization Methods for Medical Imaging AI,” in ISBI ’24, 2024, pp. 1–5

2024
[12]

Research Synthesis in Software Engineering,

P. S. M. dos Santos and G. H. Travassos, “Research Synthesis in Software Engineering,” in Contemporary Empirical Methods in Soft- ware Engineering , M. Felderer and G. H. Travassos, Eds. Springer International Publishing, 2020, pp. 443–474

2020
[13]

Do software models based on the UML aid in source-code comprehensibility? Aggregating evidence from 12 controlled experiments,

G. Scanniello, C. Gravino, M. Genero, J. A. Cruz-Lemus, G. Tortora, M. Risi, and G. Dodero, “Do software models based on the UML aid in source-code comprehensibility? Aggregating evidence from 12 controlled experiments,” Empirical Software Engineering, vol. 23, no. 5, pp. 2695...

2018
[14]

Software Artifact Mining in Software Engineering Conferences: A Meta-Analysis,

Z. Abou Khalil and S. Zacchiroli, “Software Artifact Mining in Software Engineering Conferences: A Meta-Analysis,” in ESEM ’22 , 2022, pp. 227–237

2022
[15]

Towards an evidence-based theoretical framework on factors influencing the software development productivity,

W. A. Chapetta and G. H. Travassos, “Towards an evidence-based theoretical framework on factors influencing the software development productivity,”Empirical Software Engineering, vol. 25, no. 5, pp. 3501– 3543, 2020

2020
[16]

The well-being of software engineers: A systematic literature review and a theory,

P. Godliauskas and D. ˇSmite, “The well-being of software engineers: A systematic literature review and a theory,” Empirical Software Engi- neering, vol. 30, no. 1, pp. 1–42, 2024

2024
[17]

Comparing techniques for aggregating inter- related replications in software engineering,

A. Santos and N. Juristo, “Comparing techniques for aggregating inter- related replications in software engineering,” in ESEM ’18 , 2018, pp. 1–10

2018
[18]

How Many Papers Should You Review? A Research Synthesis of Systematic Literature Reviews in Software Engineering,

X. Wang, H. Edison, D. Khanna, and U. Rafiq, “How Many Papers Should You Review? A Research Synthesis of Systematic Literature Reviews in Software Engineering,” in ESEM ’23, 2023, pp. 1–6

2023
[19]

On the Investigation of Empirical Contradictions - Aggregated Results of Local Studies on Readability and Comprehensibility of Source Code,

T. V . Ribeiro, P. S. M. dos Santos, and G. H. Travassos, “On the Investigation of Empirical Contradictions - Aggregated Results of Local Studies on Readability and Comprehensibility of Source Code,” Empir- ical Software Engineering , vol. 28, no. 6, pp. 1–41, 2023

2023
[20]

https://www.din.de/en/innovation-and-research/climate-change/resource- efficient-software-in-the-focus-of-the-iso-open-consultation

Resource-efficient software in the focus of the ISO Open Consultation. https://www.din.de/en/innovation-and-research/climate-change/resource- efficient-software-in-the-focus-of-the-iso-open-consultation
[21]

Wohlin, P

C. Wohlin, P. Runeson, M. H ¨ost, M. C. Ohlsson, B. Regnell, and A. Wessl ´en, Experimentation in Software Engineering . Springer Science & Business Media, 2012

2012
[22]

Verifiable and Energy Efficient Medical Image Analysis with Quantised Self-attentive Deep Neural Networks,

R. Sathish, S. Khare, and D. Sheet, “Verifiable and Energy Efficient Medical Image Analysis with Quantised Self-attentive Deep Neural Networks,” in Distributed, Collaborative, and Federated Learning, and Affordable AI and Healthcare for Resource Diverse Global Health, 2022, pp...

2022
[23]

Experimental energy consumption analysis of neural network model compression methods on microcontrollers with applications in bird call classification,

Y . Tao, Y . Hioka, and Y . Lu, “Experimental energy consumption analysis of neural network model compression methods on microcontrollers with applications in bird call classification,” in CSDE 2022, 2022, pp. 1–8

2022
[24]

Energy Cost Modelling for Optimizing Large Language Model Inference on Hardware Accelerators,

R. Geens, M. Shi, A. Symons, C. Fang, and M. Verhelst, “Energy Cost Modelling for Optimizing Large Language Model Inference on Hardware Accelerators,” in SOCC 2024, 2024, pp. 1–6

2024
[25]

Language Models in Software Development Tasks: An Experimental Analysis of Energy and Accuracy,

N. Alizadeh, B. Belchev, N. Saurabh, P. Kelbert, and F. Castor, “Language Models in Software Development Tasks: An Experimental Analysis of Energy and Accuracy,” Pre-print, 2025. [Online]. Available: https://arxiv.org/abs/2412.00329

2025 arXiv
[26]

Aggregating Empirical Evidence about the Benefits and Drawbacks of Software Reference Architectures,

S. Martinez-Fernandez, P. S. Medeiros Dos Santos, C. P. Ayala, X. Franch, and G. H. Travassos, “Aggregating Empirical Evidence about the Benefits and Drawbacks of Software Reference Architectures,” in ESEM ’15, 2015, pp. 1–10

2015
[27]

On the benefits and challenges of using kanban in software engineering: A structured synthesis study,

P. S. M. Dos Santos, A. C. Beltr ˜ao, B. P. De Souza, and G. H. Travassos, “On the benefits and challenges of using kanban in software engineering: A structured synthesis study,” Journal of Software Engineering Research and Development, vol. 6, no. 1, p. 13, 2018

2018
[28]

Suc- cessful combination of database search and snowballing for identification of primary studies in systematic literature studies,

C. Wohlin, M. Kalinowski, K. Romero Felizardo, and E. Mendes, “Suc- cessful combination of database search and snowballing for identification of primary studies in systematic literature studies,” Information and Software Technology, vol. 147, p. 106908, 2022

2022
[29]

ChatGPT application in Systematic Literature Reviews in Software Engineering: An evaluation of its accuracy to support the selection activity,

K. R. Felizardo, M. S. Lima, A. Deizepe, T. U. Conte, and I. Stein- macher, “ChatGPT application in Systematic Literature Reviews in Software Engineering: An evaluation of its accuracy to support the selection activity,” in ESEM ’24, 2024, pp. 25–36

2024
[30]

A Framework for Using LLMs for Repository Mining Studies in Empirical Software Engineering,

V . De Martino, J. Casta ˜no, F. Palomba, X. Franch, and S. Mart ´ınez- Fern´andez, “A Framework for Using LLMs for Repository Mining Studies in Empirical Software Engineering,” in WSESE ’25, 2025

2025
[31]

Structured Synthesis Method: The Evidence Factory Tool,

P. S. M. dos Santos and G. H. Travassos, “Structured Synthesis Method: The Evidence Factory Tool,” in ESEM ’17, 2017, pp. 480–481

2017
[32]

Auerbach and L

C. Auerbach and L. B. Silverstein, Qualitative Data: An Introduction to Coding and Analysis . NYU Press, 2003

2003
[33]

Using meta ethnography to synthesise qualitative research: A worked example,

N. Britten, R. Campbell, C. Pope, J. Donovan, M. Morgan, and R. Pill, “Using meta ethnography to synthesise qualitative research: A worked example,” Journal of Health Services Research & Policy , vol. 7, no. 4, pp. 209–215, 2002

2002
[34]

Grading quality of evidence and strength of recommen- dations,

G. W. Group, “Grading quality of evidence and strength of recommen- dations,” BMJ, vol. 328, no. 7454, p. 1490, 2004

2004
[35]

Shafer, A mathematical theory of evidence

G. Shafer, A mathematical theory of evidence . Princeton university press, 1976, vol. 42

1976
[36]

Optimization Strategies for Enhancing Resource Efficiency in Transformers & Large Language Models,

T. Wallace, N. Ezzati-Jivan, and B. Ombuki-Berman, “Optimization Strategies for Enhancing Resource Efficiency in Transformers & Large Language Models,” in ICPE ’25, 2025

2025
[37]

Green My LLM: Studying the key factors affecting the energy consumption of code assistants,

T. Coignion, C. Quinton, and R. Rouvoy, “Green My LLM: Studying the key factors affecting the energy consumption of code assistants,” Pre-print, 2024. [Online]. Available: http://arxiv.org/abs/2411.11892

2024 arXiv
[38]

[Online]

Evidence Factory - Model Quantization Synthesis - FP32 to INT8. [Online]. Available: https://evidencefactory.lens-ese.cos.ufrj.br/ evidenceEditor/290075

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

The who, what, how of software engineering research: A socio-technical framework,

M.-A. Storey, N. A. Ernst, C. Williams, and E. Kalliamvakou, “The who, what, how of software engineering research: A socio-technical framework,” Empirical Software Engineering , vol. 25, no. 5, pp. 4097– 4129, 2020

work page 2020

[2] [2]

Software Engineering for AI-Based Systems: A Survey,

S. Mart ´ınez-Fern´andez, J. Bogner, X. Franch, M. Oriol, J. Siebert, A. Trendowicz, A. M. V ollmer, and S. Wagner, “Software Engineering for AI-Based Systems: A Survey,” ACM Trans. Softw. Eng. Methodol. , vol. 31, no. 2, pp. 37e:1–37e:59, 2022

work page 2022

[3] [3]

Energy-Efficient Respiratory Anomaly Detection in Premature New- born Infants,

A. Paul, M. A. S. Tajin, A. Das, W. M. Mongan, and K. R. Dandekar, “Energy-Efficient Respiratory Anomaly Detection in Premature New- born Infants,” Electronics, vol. 11, no. 5, p. 682, 2022

work page 2022

[4] [4]

Im- pact of ML optimization tactics on greener pre-trained ML models,

A. Gonz ´alez, J. Casta ˜no, X. Franch, and S. Mart ´ınez-Fern´andez, “Im- pact of ML optimization tactics on greener pre-trained ML models,” Computing, vol. 107, no. 4, pp. 1–25, 2025

work page 2025

[5] [5]

On the Representation and Aggregation of Evidence in Software Engineering: A Theory and Belief-based Perspective,

P. S. Medeiros dos Santos and G. H. Travassos, “On the Representation and Aggregation of Evidence in Software Engineering: A Theory and Belief-based Perspective,” Electronic Notes in Theoretical Computer Science, vol. 292, pp. 95–118, 2013

work page 2013

[6] [6]

A Synthesis of Green Architectural Tactics for ML-Enabled Systems,

H. J ¨arvenp¨a¨a, P. Lago, J. Bogner, G. Lewis, H. Muccini, and I. Ozkaya, “A Synthesis of Green Architectural Tactics for ML-Enabled Systems,” in ICSE-SEIS’24, Jun. 2024, pp. 130–141

work page 2024

[7] [7]

[Online]

Replication package. [Online]. Available: https://figshare.com/s/ f92e01d7548664883abb

work page

[8] [8]

[Online]

Evidence Factory - Model Quantization Synthesis. [Online]. Available: https://evidencefactory.lens-ese.cos.ufrj.br/synthesis/editor/243422

work page