Pith. sign in

REVIEW 4 major objections 6 minor 38 references

Runtime Failure Hunting for Physics Engine Based Software Systems: How Far Can We Go?

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PhysiXFails shows physics failures in engine-based software are detectable from video alone, and prompt-tuned multimodal models beat specialized detectors, reaching 89.5% identification accuracy.

desk verdict A genuinely useful benchmark and taxonomy, but the method rankings need label validation and matched controls before they can be trusted. read the letter →

arxiv 2507.22099 v1 pith:QT22UJ5O submitted 2025-07-29 cs.CV cs.AIcs.MMcs.SE

Shuqing Li , Qiang Chen , Xiaoxue Ren , Michael R. Lyu This is my paper
classification cs.CVcs.AIcs.MMcs.SE
keywords physicsenginetestingfailuredetectionruntimevideoanalysislargemultimodalmodelspromptengineeringanomalybenchmarkdatasettaxonomy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that physics failures in software built on physics engines can be studied systematically from runtime video alone, with no access to engine internals, and that current detection technology is better than commonly assumed. The authors build a 1,000-clip benchmark of buggy and non-buggy gameplay, organize the bugs into a 17-category taxonomy of violated physical principles, and pit eight families of detectors against two tasks: saying whether any physics rule is broken, and saying which rule is broken. Their central results are that prompt-tuned large multimodal models outperform specialized computer-vision and fine-tuned baselines, reaching 89.5% identification accuracy, and that videos with simultaneous violations are easier for physics-aware models, not harder. The developer survey adds a practical reason to care: most practitioners still detect these failures by eye, so a working automated oracle would fill a real gap in reliability testing for games, simulators, and safety-critical systems.

What carries the argument

The load-bearing artifact is PhysiXFails itself: a curated, balanced collection of 1,000 runtime video clips (500 buggy, 500 non-buggy) assembled from community bug reports, gameplay videos, and the GlitchBench set, together with a 39-clip multi-violation subset. Around it sits an open-coding taxonomy that sorts the buggy clips into 17 manifestation categories across 10 physics principles, and a two-task evaluation protocol: violation detection (is any rule broken?) and violation identification (which rule?). Prompt design is the third piece of machinery, as most evaluated methods run three progressive prompt rounds that move from context-rich descriptions to a bare general physics-compliance check, and the custom Gemini prompts add freeform, rule-based, and few-shot variants. This three-part structure is what lets the paper attribute performance gaps to model capability versus prompt design rather than to dataset noise.

What would settle it

Ask independent physics experts to re-label a random sample of PhysiXFails clips without seeing the original labels; if agreement is low, or if many buggy clips turn out to be intentional game design, the method rankings collapse. To test the multi-violation effect separately, compare the 39 multi-failure clips against a matched set of 39 single-failure clips from the same games; if the 9.3-point improvement disappears, it was a selection artifact.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that physics failures in PE-based software are a distinct, characterizable class of runtime bugs: they show up as observable deviations from physical laws rather than crashes, and they can be labeled, taxonomized, and detected from video without white-box access. The core evidence is PhysiXFails, a balanced set of 1,000 clips in which the 500 buggy examples fall into 17 manifestation categories dominated by gravity violations (40%) and Newton's-law violations (28.2%). Evaluating adapted baselines and prompt-based systems on it, the paper finds that general multimodal models with custom prompts lead the leaderboard, with DEVIL-Gemini reaching 69.2% accuracy on violation detection and a custom Gemini prompt reaching 89.5% on violation identification, while specialized video-anomaly models often collapse to chance or to always-alarm behavior. The paper further reports that the 39-clip multi-violation subset is easier, not harder, for physics-aware methods, with PhyGenEval improving 9.3 percentage points over its full-dataset accuracy.

Load-bearing premise

The load-bearing premise is that the 500 'buggy' clips really are physics failures and the 500 'non-buggy' clips are clean; if these labels are wrong, every reported accuracy measures label noise, not detection skill.

Editorial extensions

If this is right

  • The benchmark and taxonomy become a common measuring stick for future detectors, and the best 69.2% detection accuracy marks the headroom left to close.
  • Custom-prompted general multimodal models, at 89.5% identification accuracy, can serve as black-box runtime oracles that need neither engine internals nor per-game retraining.
  • Physics-aware methods improve on multi-violation clips while generic anomaly detectors degrade, so detector choice should follow whether a method models physical rules rather than just motion statistics.
  • The developer survey directly motivates putting such detectors into CI/CD pipelines: 84.4% of respondents still rely on visual inspection and 81.3% want real-time detection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same prompt-tuning recipe is likely to transfer to footage from safety-critical simulators, such as autonomous driving or surgical robotics, where physics failures are rarer and more costly, an extension the paper does not test.
  • The multi-violation 'amplification effect' rests on only 39 clips, so it may be a selection artifact; a matched single-versus-multi comparison would settle whether concurrent violations genuinely sharpen the signal.
  • Because gravity and Newton's-law violations dominate the corpus, the taxonomy may capture what players notice and report rather than the true distribution of physics-engine bugs; logs or fuzzer-generated failures would probably look different.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents PhysiXFails, a dataset of 1,000 runtime video clips (500 buggy and 500 non-buggy) drawn from community bug reports and YouTube, along with a 17-category taxonomy of physics failure manifestations, an evaluation of eight detection approaches (video anomaly detection, prompt-based LMMs, and fine-tuned multimodal models), and a developer survey with 32 practitioners. The central claims are that LMM-based methods generally outperform traditional computer vision approaches, that custom Gemini prompts achieve particularly high violation-identification accuracy (up to 89.5%), and that multi-violation videos are sometimes easier to detect than single-violation ones. The authors release the dataset, code, and questionnaire responses.

Significance. If the benchmark and its labels are valid, this is a useful contribution: it is the first large-scale dataset targeted specifically at physics failures in PE-based software, the taxonomy offers a structured vocabulary for future work, and the evaluation provides a broad comparison across several model families. The inclusion of a developer study adds practical context, and the public release of dataset and code supports reproducibility. The main value, however, rests on the validity of the ground-truth labels and on whether the reported method rankings measure physics-failure detection rather than dataset-selection artifacts.

major comments (4)
  1. [Section III.A and Section IV.A] The ground-truth labels for the 500 buggy and 500 non-buggy clips are the basis for every accuracy, F1, and AUC reported in Tables III-VI, yet no inter-annotator agreement is reported. Section IV.A states that 'initial annotations were iteratively refined using a large multimodal model (LMM) such as GPT-4o,' and Section VIII.A concedes that 'subjective judgments may have influenced the classification of physics failures.' This is a load-bearing assumption for the benchmark's validity. Please report quantitative label-reliability statistics (e.g., Cohen's kappa on an independently annotated random subset), and explicitly state the role of GPT-4o in per-video labeling as opposed to taxonomy refinement.
  2. [Section III.A] The negative set is not matched to the positive set. Non-buggy videos are 'all sourced from YouTube,' while 71 buggy videos come from Reddit and GlitchBench in addition to YouTube; moreover, the buggy videos are user-curated bug reports, which may differ systematically from ordinary gameplay footage in editing style, scene selection, and visual content. Consequently, a detector could achieve high discrimination by exploiting these distributional differences rather than by identifying physical inconsistencies. Please either construct a matched negative set (e.g., same games, comparable scene types, similar production characteristics) or provide a controlled analysis, such as same-game pairs or a human-verified matched subset, to rule out this confound.
  3. [Section VI.D] The 'multi-violation amplification effect' is based on only 39 videos (PhysiXMultiFails) compared against a much larger, unmatched set of single-violation videos. The multi-violation subset is not matched on game, video length, scene complexity, or source, so the reported 9.3-percentage-point accuracy improvement for PhyGenEval could be a selection artifact rather than a genuine property of multi-failure detection. Please either match the multi-violation videos to single-violation clips on these dimensions or limit the claim to the current subset and explicitly discuss the mismatch as a threat to the finding.
  4. [Section IV.A and Tables III-VI] Because the taxonomy and per-video labels were iteratively refined with GPT-4o, and the top-performing VI methods use Gemini (another general LMM), the evaluation may reward agreement with GPT-4o's classification biases instead of independently established physical ground truth. This is not an accusation of circularity in the derivation, but a correctness-risk concern. Please clarify whether the LMM was used only to suggest candidate categories or also to assign video-level labels, and provide a human-labeled validation sample (e.g., 100 videos annotated by independent human judges without LMM involvement) to confirm that the taxonomy and method rankings hold under human gold labels.
minor comments (6)
  1. [Section VI.A] The text refers to 'Tables V and VI' when discussing full-dataset violation detection and identification, but the full-dataset results appear in Tables III and IV; the caption of Table IV also says 'PHYSIXMULTIFAILS' even though the table reports full-dataset results. Please renumber and relabel the tables consistently.
  2. [Section V.A and Table I] The model is called 'VideoPhy' in the text and figures (e.g., Figure 2) but 'VideoCon-Physics' in Table I. Please use a single name consistently throughout.
  3. [Figure 11] The subfigures inside Figure 11 are labeled as separate figures (Figure 2 through Figure 10) in the caption, and the caption text says 'This figure presents the density plots' while the main text refers to 'Figure 11.' Please align the subfigure numbering and caption references.
  4. [Appendix] The developer questionnaire contains several typos, including 'sofwtare,' 'Maintainence,' and 'Sparsity.' Please proofread the appendix before publication.
  5. [References] References [7] and [27] are the same PhyFu paper, and reference [2] appears twice in the introduction. Please deduplicate and renumber.
  6. [Section VI.D] In the first paragraph of Section VI.D, '76.3% accuracy on PHYSIXMULTIFAILS compared to 67.0% on PHYSIXFAILS' should be compared against the single-violation accuracy of 66.9% shown in Table V; please use consistent numbers in the text and tables.

Circularity Check

0 steps flagged · score 0.0 of 10

No definitional circularity: detector scores are measured against an externally curated label set, not derived from the detectors' own outputs.

full rationale

The paper's chain of claims is: (1) curate 1,000 videos with buggy/non-buggy labels (Section III); (2) build a 17-category taxonomy via human open coding with GPT-4o assistance and expert review (Section IV.A); (3) measure detectors against those labels on violation detection and violation identification tasks (Sections V-VI); (4) report rankings and developer perceptions. No step makes the predicted quantity equal to an input by construction. The taxonomy supplies category names to some prompts (e.g., Gemini-Custom-Rules in Section V.A uses 'standard rule categories'), but the model still has to map each video to the correct category; the per-video label is not contained in the category set. The headline violation identification result (Gemini-Custom-Freeform-VI, 89.5%) is the variant in which 'the model generates its own rules,' so it does not even depend on the taxonomy. The multi-violation analysis (Section VI.D) compares 39 multi-failure videos against single-failure videos that are not matched, which can make the reported 'amplification effect' a selection artifact; that is a statistical validity threat, not circularity. Section VIII.A explicitly concedes that 'subjective judgments may have influenced the classification of physics failures,' an honest limitation about label reliability, but label noise does not make the detector evaluation circular. Self-citations ([28]-[37]) appear in related-work positioning and are not load-bearing for the empirical rankings. Therefore, no circular step meeting the quote-and-reduction standard was found.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities, particles, forces, or conserved quantities. Its central dependency is on the validity of the benchmark labels and taxonomy, which are treated as ground truth without independent verification. The main hand-chosen elements are decision thresholds and custom prompt designs, both of which directly affect the reported performance numbers.

free parameters (2)
  • Decision thresholds for score-based detectors = not reported
    Binary labels for violation detection and identification are derived from continuous scores (e.g., DEVIL naturalness, S3R anomaly scores, MIST scores), but the paper does not state the threshold values, which determine all reported accuracy, F1, and AUC numbers.
  • Custom Gemini prompt set = not applicable
    The three custom prompt rounds (freeform reasoning, rule-based classification, rule-based with few-shot examples) were designed by the authors and evaluated on the same benchmark, acting as hand-chosen adaptation choices that drive the highest reported accuracy.
assumptions (4)
  • domain assumption Buggy videos obtained from community forums, Reddit, YouTube, and GlitchBench are genuine physics failures rather than intentional game behavior or unrelated glitches.
    Section III.A data collection relies on community reports as ground truth; if many clips are intentional effects or non-physics visual bugs, the benchmark labels are invalid.
  • domain assumption Non-buggy videos collected from YouTube contain no physics failures.
    Section III.B applies quality filtering but no independent verification that the 500 non-buggy clips are truly compliant with physics.
  • domain assumption The 17-category taxonomy, refined by GPT-4o and reviewed by two physics experts, is an accurate and complete labeling of observed physics failures.
    Section IV.A uses this taxonomy as ground truth for both violation detection and violation identification; inaccuracies propagate into all evaluation metrics.
  • domain assumption Visual frames contain sufficient information to determine physics correctness without audio or engine internals.
    Section II.A defines observable states as visual frames and Section III.B removes audio, so the whole detection problem is framed as visual-only inference.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Runtime Failure Hunting for Physics Engine Based Software Systems: How Far Can We Go?." pith.science (2026). https://pith.science/paper/QT22UJ5O

@misc{pith2026250722099,
  author       = {Pith},
  title        = {Pith review of: Runtime Failure Hunting for Physics Engine Based Software Systems: How Far Can We Go?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QT22UJ5O}},
  note         = {Machine review of arXiv:2507.22099}
}
read the original abstract

Physics Engines (PEs) are fundamental software frameworks that simulate physical interactions in applications ranging from entertainment to safety-critical systems. Despite their importance, PEs suffer from physics failures, deviations from expected physical behaviors that can compromise software reliability, degrade user experience, and potentially cause critical failures in autonomous vehicles or medical robotics. Current testing approaches for PE-based software are inadequate, typically requiring white-box access and focusing on crash detection rather than semantically complex physics failures. This paper presents the first large-scale empirical study characterizing physics failures in PE-based software. We investigate three research questions addressing the manifestations of physics failures, the effectiveness of detection techniques, and developer perceptions of current detection practices. Our contributions include: (1) a taxonomy of physics failure manifestations; (2) a comprehensive evaluation of detection methods including deep learning, prompt-based techniques, and large multimodal models; and (3) actionable insights from developer experiences for improving detection approaches. To support future research, we release PhysiXFails, code, and other materials at https://sites.google.com/view/physics-failure-detection.

Figures

Figures reproduced from arXiv: 2507.22099 by the authors.

Figure 1
Figure 1. Diversified manifestation categories of physics failures. Examples are demonstrated with three frames in time order to [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 5
Figure 5. Gemini [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 7
Figure 7. S3R [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figures from the paper (2 more)
Figure 6
Figure 6. Figure 6: MIST [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 12
Figure 12. Figure 12: Performance Summary of Violation Detection (VD) and Violation Identification (VI) on P [PITH_FULL_IMAGE:figures/full_fig_p008_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 32 canonical work pages

  1. [1]

    Unity game development engine: A technical survey,

    A. Hussain, H. Shakeel, F. Hussain, N. Uddin, and T. L. Ghouri, “Unity game development engine: A technical survey,” Univ. Sindh J. Inf. Commun. Technol, vol. 4, no. 2, pp. 73–81, 2020

    work page 2020

  2. [2]

    Game physics: An analysis of physics engines for first- time physics developers,

    R. M. Templet, “Game physics: An analysis of physics engines for first- time physics developers,” Ph.D. dissertation, California State University, Northridge, 2021

    work page 2021

  3. [3]

    Cutting-edge research in physics engines: Exploring parallel technologies, algorithm insights, and upcoming trends,

    C. Che, “Cutting-edge research in physics engines: Exploring parallel technologies, algorithm insights, and upcoming trends,” Applied and Computational Engineering, vol. 37, pp. 191–196, 02 2024

    work page 2024

  4. [4]

    Airsim: High-fidelity visual and physical simulation for autonomous vehicles,

    S. Shah, D. Dey, C. Lovett, and A. Kapoor, “Airsim: High-fidelity visual and physical simulation for autonomous vehicles,” in Field and Service Robotics: Results of the 11th International Conference . Springer, 2018, pp. 621–635

    2018

  5. [5]

    Using the physx engine for physics-based virtual surgery with force feedback,

    A. Maciel, T. Halic, Z. Lu, L. P. Nedel, and S. De, “Using the physx engine for physics-based virtual surgery with force feedback,” The International Journal of Medical Robotics and Computer Assisted Surgery, vol. 5, no. 3, pp. 341–353, 2009

    work page 2009

  6. [6]

    Guest editorial: Physics-based simulation games,

    J. Renz, R. Miikkulainen, N. R. Sturtevant, and M. H. M. Winands, “Guest editorial: Physics-based simulation games,” IEEE Trans. Comput. Intell. AI Games , vol. 8, no. 2, pp. 101–103, 2016. [Online]. Available: https://doi.org/10.1109/TCIAIG.2016.2571560 JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 17

    work page arXiv 2016

  7. [7]

    Phyfu: Fuzzing modern physics simulation engines,

    D. Xiao, Z. Liu, and S. Wang, “Phyfu: Fuzzing modern physics simulation engines,” in 38th IEEE/ACM International Conference on Automated Software Engineering, ASE 2023, Luxembourg, September 11-15, 2023 . IEEE, 2023, pp. 1579–1591. [Online]. Available: https://doi.org/10.1109/ASE56229.2023.00054

    arXiv 2023

  8. [8]

    Autonomous cars: Research results, issues, and future challenges,

    R. Hussain and S. Zeadally, “Autonomous cars: Research results, issues, and future challenges,” IEEE Communications Surveys & Tutorials , vol. 21, no. 2, pp. 1275–1313, 2018

    work page 2018

Show all 38 references
  1. [9]

    Taming interfacial oxygen vacancies of amphiphilic tungsten ox- ide for enhanced catalysis in oxidative desulfurization,

    J. He, P. Wu, Y . Wu, H. Li, W. Jiang, S. Xun, M. Zhang, W. Zhu, and H. Li, “Taming interfacial oxygen vacancies of amphiphilic tungsten ox- ide for enhanced catalysis in oxidative desulfurization,” ACS Sustainable Chemistry & Engineering , vol. 5, no. 10, pp. 8930–8938, 2017

    2017

  2. [10]

    Fuzzing: State of the art,

    H. Liang, X. Pei, X. Jia, W. Shen, and J. Zhang, “Fuzzing: State of the art,” IEEE Transactions on Reliability , vol. 67, no. 3, pp. 1199–1218, 2018

    2018

  3. [11]

    Glib: towards automated test oracle for graphically-rich applications,

    K. Chen, Y . Li, Y . Chen, C. Fan, Z. Hu, and W. Yang, “Glib: towards automated test oracle for graphically-rich applications,” in Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, se...

    2021

  4. [12]

    Mist: Multiple instance self- training framework for video anomaly detection,

    J.-C. Feng, F.-T. Hong, and W.-S. Zheng, “Mist: Multiple instance self- training framework for video anomaly detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 14 009–14 018

    2021

  5. [13]

    Self- supervised sparse representation for video anomaly detection,

    J.-C. Wu, H.-Y . Hsieh, D.-J. Chen, C.-S. Fuh, and T.-L. Liu, “Self- supervised sparse representation for video anomaly detection,” in Euro- pean Conference on Computer Vision . Springer, 2022, pp. 729–745

    2022

  6. [14]

    Holmes-vad: Towards unbiased and explain- able video anomaly detection via multi-modal llm,

    H. Zhang, X. Xu, X. Wang, J. Zuo, C. Han, X. Huang, C. Gao, Y . Wang, and N. Sang, “Holmes-vad: Towards unbiased and explain- able video anomaly detection via multi-modal llm,” arXiv preprint arXiv:2406.12235, 2024

    2024 arXiv

  7. [15]

    Videophy: Evaluating physical commonsense for video generation,

    H. Bansal, Z. Lin, T. Xie, Z. Zong, M. Yarom, Y . Bitton, C. Jiang, Y . Sun, K.-W. Chang, and A. Grover, “Videophy: Evaluating physical commonsense for video generation,” 2024

    2024

  8. [16]

    Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation,

    X. He, D. Jiang, G. Zhang, M. Ku, A. Soni, S. Siu, H. Chen, A. Chandra, Z. Jiang, A. Arulraj et al. , “Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation,” arXiv preprint arXiv:2406.15252, 2024

    2024 arXiv

  9. [17]

    Evaluation of text-to-video generation models: A dynamics per- spective,

    M. Liao, Q. Ye, W. Zuo, F. Wan, T. Wang, Y . Zhao, J. Wang, X. Zhang et al., “Evaluation of text-to-video generation models: A dynamics per- spective,” Advances in Neural Information Processing Systems , vol. 37, pp. 109 790–109 816, 2024

    2024

  10. [18]

    Towards world simulator: Crafting physical commonsense-based benchmark for video generation,

    F. Meng, J. Liao, X. Tan, W. Shao, Q. Lu, K. Zhang, Y . Cheng, D. Li, Y . Qiao, and P. Luo, “Towards world simulator: Crafting physical commonsense-based benchmark for video generation,” 2024

    2024

  11. [19]

    Gemini: A family of highly capable multimodal models,

    Google, “Gemini: A family of highly capable multimodal models,” Google, Tech. Rep., 12 2023. [Online]. Available: https://storage. googleapis.com/deepmind-media/gemini/gemini 1 report.pdf

    2023

  12. [20]

    Towards world simulator: Crafting physical commonsense-based benchmark for video generation,

    F. Meng, J. Liao, X. Tan, W. Shao, Q. Lu, K. Zhang, Y . Cheng, D. Li, Y . Qiao, and P. Luo, “Towards world simulator: Crafting physical commonsense-based benchmark for video generation,” arXiv preprint arXiv:2410.05363, 2024

    2024 arXiv

  13. [21]

    Glitchbench: Can large multimodal models detect video game glitches?

    M. R. Taesiri, T. Feng, C.-P. Bezemer, and A. Nguyen, “Glitchbench: Can large multimodal models detect video game glitches?” in Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 22 444–22 455

    2024

  14. [22]

    yt-dlp: A youtube-dl fork with additional features and fixes,

    yt-dlp Contributors, “yt-dlp: A youtube-dl fork with additional features and fixes,” 2023. [Online]. Available: https://github.com/yt-dlp/yt-dlp

    2023

  15. [23]

    Moviepy: Video editing with python,

    Zulko and MoviePy Contributors, “Moviepy: Video editing with python,” 2023. [Online]. Available: https://github.com/Zulko/moviepy

    2023

  16. [24]

    J. W. Creswell and C. N. Poth, Qualitative inquiry and research design: Choosing among five approaches . Sage publications, 2016

    2016

  17. [25]

    Augmenting automated game testing with deep reinforcement learning,

    J. Bergdahl, C. Gordillo, K. Tollmar, and L. Gissl ´en, “Augmenting automated game testing with deep reinforcement learning,” in2020 IEEE Conference on Games (CoG) . IEEE, 2020, pp. 600–603

    2020

  18. [26]

    Predart: Towards automatic oracle prediction of object placements in augmented reality testing,

    T. Rafi, X. Zhang, and X. Wang, “Predart: Towards automatic oracle prediction of object placements in augmented reality testing,” in Pro- ceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering, 2022, pp. 1–13

    2022

  19. [27]

    Phyfu: Fuzzing modern physics simulation engines,

    D. Xiao, Z. Liu, and S. Wang, “Phyfu: Fuzzing modern physics simulation engines,” in 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) . IEEE, 2023, pp. 1579– 1591

    2023

  20. [28]

    An exploratory study of bugs in extended reality applications on the web,

    S. Li, Y . Wu, Y . Liu, D. Wang, M. Wen, Y . Tao, Y . Sui, and Y . Liu, “An exploratory study of bugs in extended reality applications on the web,” in ISSRE. IEEE, 2020, pp. 172–183

    2020

  21. [29]

    Towards modeling software quality of virtual reality applications from users’ perspectives,

    S. Li, L. Wei, Y . Liu, C. Gao, S.-C. Cheung, and M. R. Lyu, “Towards modeling software quality of virtual reality applications from users’ perspectives,” arXiv preprint arXiv:2308.06783 , 2023

    2023 arXiv

  22. [30]

    Grounded gui understanding for vision based spatial intelligent agent: Exemplified by virtual reality apps,

    S. Li, B. Li, Y . Liu, C. Gao, J. Zhang, S.-C. Cheung, and M. R. Lyu, “Grounded gui understanding for vision based spatial intelligent agent: Exemplified by virtual reality apps,” arXiv preprint arXiv:2409.10811 , 2024

    2024

  23. [31]

    Xrzoo: A large-scale and versatile dataset of extended reality (xr) applications,

    S. Li, C. Zhang, C. Gao, and M. R. Lyu, “Xrzoo: A large-scale and versatile dataset of extended reality (xr) applications,” arXiv preprint arXiv:2412.06759, 2024

    2024

  24. [32]

    Extended reality cybersickness assessment via user review analysis,

    S. Li, Q. Zheng, C. Gao, J. Feng, and M. R. Lyu, “Extended reality cybersickness assessment via user review analysis,” Proceedings of the ACM on Software Engineering, vol. 2, no. ISSTA, pp. 1303–1325, 2025

    2025

  25. [33]

    3d software synthesis guided by constraint-expressive intermediate representation,

    S. Li, A. Y . Lam, Y . Peng, W. Wang, and M. R. Lyu, “3d software synthesis guided by constraint-expressive intermediate representation,”

  26. [34]

    GLIB: Towards Automated Test Oracle for Graphically-Rich Applications,

    K. Chen, Y . Li, Y . Chen, C. Fan, Z. Hu, and W. Yang, “GLIB: Towards Automated Test Oracle for Graphically-Rich Applications,” in ESEC/FSE. ACM, 2021, pp. 1093–1104

    2021

  27. [35]

    Automatically detecting visual bugs in html5 canvas games,

    F. Macklon, M. R. Taesiri, M. Viggiato, S. Antoszko, N. Romanova, D. Paas, and C.-P. Bezemer, “Automatically detecting visual bugs in html5 canvas games,” in Proceedings of the 37th IEEE/ACM Interna- tional Conference on Automated Software Engineering , 2022, pp. 1–11

    2022

  28. [36]

    Clip meets gamephysics: Towards bug identification in gameplay videos using zero-shot transfer learning,

    M. R. Taesiri, F. Macklon, and C.-P. Bezemer, “Clip meets gamephysics: Towards bug identification in gameplay videos using zero-shot transfer learning,” in Proceedings of the 19th International Conference on Mining Software Repositories , 2022, pp. 270–281

    2022

  29. [37]

    Less cybersickness, please: Demystifying and detecting stereoscopic visual inconsistencies in virtual reality apps,

    S. Li, C. Gao, J. Zhang, Y . Zhang, Y . Liu, J. Gu, Y . Peng, and M. R. Lyu, “Less cybersickness, please: Demystifying and detecting stereoscopic visual inconsistencies in virtual reality apps,” Proceedings of the ACM on Software Engineering , vol. 1, no. FSE, pp. 2167–2189, 2024

    2024

  30. [2025]

    Available: https://arxiv.org/abs/2507.18625

    [Online]. Available: https://arxiv.org/abs/2507.18625

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.