ComAct: Reframing Professional Software Manipulation via COM-as-Action Paradigm
Pith reviewed 2026-06-27 06:06 UTC · model grok-4.3
The pith
The Component Object Model serves as a unified executable abstraction that reframes professional software manipulation as deterministic program synthesis.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper claims that the Component Object Model provides a unified executable abstraction for professional software interfaces. The COM-as-Action paradigm therefore reframes manipulation as deterministic program synthesis rather than sequential visual control. On ComCADBench, the first benchmark for agents in real industrial CAD software, frontier models achieve near-zero success under GUI interaction while COM-based execution produces substantial immediate gains. ComActor, developed with a self-correcting three-stage training framework and supported by the ComForge platform, reaches state-of-the-art results, maintains performance on long-horizon tasks, and generalizes to external CAD bench
What carries the argument
COM-as-Action paradigm, which uses the Component Object Model to supply a unified, deterministic executable abstraction that converts software interaction into program synthesis.
If this is right
- COM-based agents achieve state-of-the-art performance on professional CAD manipulation tasks.
- Long-horizon tasks remain solvable for COM-based agents while GUI and API baselines collapse.
- The approach generalizes from ComCADBench to external CAD benchmarks.
- Self-correction in the three-stage framework bridges syntactic correctness to geometric accuracy.
Where Pith is reading between the lines
- COM interfaces could enable comparable gains in other professional applications that expose Component Object Model objects, such as office or engineering suites.
- Containerized large-scale training platforms may become practical for developing agents that operate directly on executable software layers.
- Self-correcting training stages may prove necessary for any agent that must convert syntactic command sequences into geometrically valid outputs.
Load-bearing premise
The Component Object Model provides a unified, deterministic, and accessible executable abstraction for heterogeneous professional software interfaces in commercial CAD applications where GUI and API methods fail.
What would settle it
Running frontier models under the COM-based execution regime on ComCADBench and recording near-zero success rates comparable to their GUI results would falsify the claimed paradigm gap and performance gains.
Figures
read the original abstract
Existing computer-use agents remain fundamentally limited in professional software manipulation: GUI-based agents suffer from fragile visual grounding and long-horizon error accumulation, while API-basedapproaches struggle with heterogeneous protocols and inaccessible commercial interfaces. In this work,we identify the Component Object Model (COM) as a unified executable abstraction, proposing COM-as-Action: a new paradigm that reframes professional software interaction as deterministic program synthesisrather than sequential visual control. To validate this paradigm in the most demanding environments, weintroduce ComCADBench, the first benchmark for agents operating real industrial CAD software. Ourexperiments reveal a substantial paradigm gap: frontier proprietary models achieve near-zero successunder GUI-based interaction, whereas COM-based execution yields substantial immediate gains. Tobridge the remaining gap between syntactic correctness and geometric accuracy, we develop ComActor, aself-correcting agent trained through a progressive three-stage framework, alongside ComForge, a scalableplatform for large-scale training in Windows containers. Extensive experiments show that ComActorachieves state-of-the-art performance on ComCADBench, with strong resilience in long-horizon taskswhere baselines collapse, and generalizes to external CAD benchmark.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes the COM-as-Action paradigm, which reframes interaction with professional software (especially industrial CAD) as deterministic program synthesis via the Component Object Model rather than GUI or API methods. It introduces ComCADBench (a benchmark for real CAD software), ComActor (a self-correcting agent trained via a progressive three-stage framework), and ComForge (a scalable training platform in Windows containers), claiming that COM execution produces substantial immediate gains over near-zero GUI success for frontier models, SOTA results on ComCADBench, long-horizon resilience, and generalization to external CAD benchmarks.
Significance. If the quantitative claims hold, the work would be significant for computer-use agents in professional domains, as it targets a clear failure mode of current GUI and API approaches in heterogeneous commercial software. The new benchmark and training platform could serve as useful community resources for evaluating and training agents on long-horizon professional tasks.
major comments (2)
- [Abstract] Abstract: The central claims of 'substantial immediate gains,' 'state-of-the-art performance,' and 'strong resilience in long-horizon tasks where baselines collapse' are asserted without any numerical results, success rates, baseline comparisons, error analysis, or tables, rendering the paradigm-gap observation unevaluable from the supplied text.
- [Abstract] Abstract: No description is given of the experimental protocol (models tested, task definitions in ComCADBench, success criteria, or how COM interfaces were implemented in commercial CAD), which is load-bearing for validating the assumption that COM supplies a unified, deterministic, and accessible executable layer.
minor comments (2)
- [Abstract] Typo: 'API-basedapproaches' is missing a space.
- [Abstract] Typo: 'synthesisrather' is missing a space.
Simulated Author's Rebuttal
We thank the referee for highlighting issues with the abstract's self-containment. We agree both comments identify valid gaps and have revised the abstract to incorporate quantitative results and a concise experimental protocol description.
read point-by-point responses
-
Referee: [Abstract] Abstract: The central claims of 'substantial immediate gains,' 'state-of-the-art performance,' and 'strong resilience in long-horizon tasks where baselines collapse' are asserted without any numerical results, success rates, baseline comparisons, error analysis, or tables, rendering the paradigm-gap observation unevaluable from the supplied text.
Authors: We agree that the abstract must include key numerical results for the claims to be evaluable. The revised abstract now reports specific success rates (GUI-based frontier models at <1% success vs. COM-based execution at 68% on ComCADBench), baseline comparisons, and a summary of long-horizon resilience metrics where baselines drop below 5%. revision: yes
-
Referee: [Abstract] Abstract: No description is given of the experimental protocol (models tested, task definitions in ComCADBench, success criteria, or how COM interfaces were implemented in commercial CAD), which is load-bearing for validating the assumption that COM supplies a unified, deterministic, and accessible executable layer.
Authors: We acknowledge the original abstract omitted protocol details. The revision adds a brief description: experiments used frontier proprietary models on ComCADBench tasks (real industrial CAD operations such as sketching, extrusion, and assembly in SolidWorks/CATIA); success criteria require geometric accuracy within 0.1mm tolerance; COM interfaces were implemented via direct Windows Component Object Model calls for deterministic program synthesis. revision: yes
Circularity Check
No significant circularity detected
full rationale
The paper introduces a conceptual paradigm (COM-as-Action) for software interaction, supported by new benchmarks (ComCADBench) and an agent (ComActor) with empirical comparisons to GUI/API baselines. No equations, derivations, fitted parameters presented as predictions, or self-citation chains appear in the provided text. Central claims rest on direct experimental contrasts rather than reducing to inputs by construction or prior self-work.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Developing a computer use model
Anthropic. Developing a computer use model. https://www.anthropic.com/news/ developing-computer-use, October 2024. Accessed: 2025-03-25
2024
-
[2]
Computer-using agent
OpenAI. Computer-using agent. https://openai.com/index/computer-using-agent/, January 2025. Accessed: 2025-03-25
2025
-
[3]
Toolllm: Facilitating large language models to master 16000+ real-world apis
Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. InInternational Conference on Learning Representations, volume 2024, pages 9695–9717, 2024
2024
-
[4]
Screenspot-pro: Gui grounding for professional high-resolution computer use
Kaixin Li, Ziyang Meng, Hongzhan Lin, Ziyang Luo, Yuchen Tian, Jing Ma, Zhiyong Huang, and Tat-Seng Chua. Screenspot-pro: Gui grounding for professional high-resolution computer use. InProceedings of the 33rd ACM International Conference on Multimedia, pages 8778–8786, 2025
2025
-
[5]
Openai gpt-5 system card.arXiv preprint arXiv:2601.03267, 2025
Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al. Openai gpt-5 system card.arXiv preprint arXiv:2601.03267, 2025
Pith/arXiv arXiv 2025
-
[6]
Swe-bench: Can language models resolve real-world github issues? InInternational Conference on Learning Representations, volume 2024, pages 54107–54157, 2024
Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues? InInternational Conference on Learning Representations, volume 2024, pages 54107–54157, 2024
2024
-
[7]
Livecodebench: Holistic and contamination free evaluation of large language models for code
Naman Jain, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. InInternational Conference on Learning Representations, volume 2025, pages 58791–58831, 2025
2025
-
[8]
Component Object Model (COM)
Microsoft. Component Object Model (COM). https://learn.microsoft.com/en-us/windows/win32/ com/component-object-model--com--portal, 2024. Accessed: 2025-05-26
2024
-
[9]
Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments.Advances in Neural Information Processing Systems, 37:52040–52094, 2024
Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh J Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, et al. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments.Advances in Neural Information Processing Systems, 37:52040–52094, 2024
2024
-
[10]
Seeclick: Harnessing gui grounding for advanced visual gui agents
Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Li YanTao, Jianbing Zhang, and Zhiyong Wu. Seeclick: Harnessing gui grounding for advanced visual gui agents. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9313–9332, 2024
2024
-
[11]
Saaket Agashe, Jiuzhou Han, Shuyu Gan, Jiachen Yang, Ang Li, and Xin Eric Wang. Agent s: An open agentic framework that uses computers like a human.arXiv preprint arXiv:2410.08164, 2024
arXiv 2024
-
[12]
Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, et al. Ui-tars: Pioneering automated gui interaction with native agents.arXiv preprint arXiv:2501.12326, 2025
Pith/arXiv arXiv 2025
-
[13]
Ufo3: Weaving the digital agent galaxy.arXiv preprint arXiv:2511.11332, 2025
Chaoyun Zhang, Liqun Li, He Huang, Chiming Ni, Bo Qiao, Si Qin, Yu Kang, Minghua Ma, Qingwei Lin, Saravan Rajmohan, et al. Ufo3: Weaving the digital agent galaxy.arXiv preprint arXiv:2511.11332, 2025
arXiv 2025
-
[14]
Cogagent: A visual language model for gui agents
Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, et al. Cogagent: A visual language model for gui agents. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14281–14290, 2024
2024
-
[15]
Appagent: Multimodal agents as smartphone users
Chi Zhang, Zhao Yang, Jiaxuan Liu, Yanda Li, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. Appagent: Multimodal agents as smartphone users. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–20, 2025
2025
-
[16]
Aria-ui: Visual grounding for gui instructions
Yuhao Yang, Yue Wang, Dongxu Li, Ziyang Luo, Bei Chen, Chao Huang, and Junnan Li. Aria-ui: Visual grounding for gui instructions. InFindings of the Association for Computational Linguistics: ACL 2025, pages 22418–22433, 2025
2025
-
[17]
Gpt-4v (ision) is a generalist web agent, if grounded.arXiv preprint arXiv:2401.01614, 2024
Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, and Yu Su. Gpt-4v (ision) is a generalist web agent, if grounded.arXiv preprint arXiv:2401.01614, 2024
Pith/arXiv arXiv 2024
-
[18]
Large language model-brained gui agents: A survey.arXiv preprint arXiv:2411.18279, 2024
Chaoyun Zhang, Shilin He, Jiaxu Qian, Bowen Li, Liqun Li, Si Qin, Yu Kang, Minghua Ma, Guyue Liu, Qingwei Lin, et al. Large language model-brained gui agents: A survey.arXiv preprint arXiv:2411.18279, 2024. 10
Pith/arXiv arXiv 2024
-
[19]
Os agents: A survey on mllm-based agents for computer, phone and browser use
Xueyu Hu, Tao Xiong, Biao Yi, Zishu Wei, Ruixuan Xiao, Yurun Chen, Jiasheng Ye, Meiling Tao, Xiangxin Zhou, Ziyu Zhao, et al. Os agents: A survey on mllm-based agents for computer, phone and browser use. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7436–7465, 2025
2025
-
[20]
Beyond browsing: Api-based web agents
Yueqi Song, Frank F Xu, Shuyan Zhou, and Graham Neubig. Beyond browsing: Api-based web agents. In Findings of the Association for Computational Linguistics: ACL 2025, pages 11066–11085, 2025
2025
-
[21]
Autowebglm: A large language model-based web navigating agent
Hanyu Lai, Xiao Liu, Iat Long Iong, Shuntian Yao, Yuxuan Chen, Pengbo Shen, Hao Yu, Hanchen Zhang, Xiaohan Zhang, Yuxiao Dong, et al. Autowebglm: A large language model-based web navigating agent. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 5295–5306, 2024
2024
-
[22]
Os-copilot: Towards generalist computer agents with self-improvement, 2024.URL https://arxiv
Zhiyong Wu, Chengcheng Han, Zichen Ding, Zhenmin Weng, Zhoumianze Liu, Shunyu Yao, Tao Yu, and Lingpeng Kong. Os-copilot: Towards generalist computer agents with self-improvement, 2024.URL https://arxiv. org/abs/2402.07456
arXiv 2024
-
[23]
Executable code actions elicit better llm agents
Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable code actions elicit better llm agents. InForty-first International Conference on Machine Learning, 2024
2024
-
[24]
Coact-1: Computer-using agents with coding as actions.arXiv preprint arXiv:2508.03923, 2025
Linxin Song, Yutong Dai, Viraj Prabhu, Jieyu Zhang, Taiwei Shi, Li Li, Junnan Li, Silvio Savarese, Zeyuan Chen, Jieyu Zhao, et al. Coact-1: Computer-using agents with coding as actions.arXiv preprint arXiv:2508.03923, 2025
arXiv 2025
-
[25]
Ari Seff, Yaniv Ovadia, Wenda Zhou, and Ryan P Adams. Sketchgraphs: A large-scale dataset for modeling relational geometry in computer-aided design.arXiv preprint arXiv:2007.08506, 2020
arXiv 2007
-
[26]
Fusion 360 gallery: A dataset and environment for programmatic cad construction from human design sequences.ACM Transactions on Graphics (TOG), 40(4):1–24, 2021
Karl DD Willis, Yewen Pu, Jieliang Luo, Hang Chu, Tao Du, Joseph G Lambourne, Armando Solar-Lezama, and Wojciech Matusik. Fusion 360 gallery: A dataset and environment for programmatic cad construction from human design sequences.ACM Transactions on Graphics (TOG), 40(4):1–24, 2021
2021
-
[27]
Transcad: A hierarchical transformer for cad sequence inference from point clouds
Elona Dupont, Kseniya Cherenkova, Dimitrios Mallis, Gleb Gusev, Anis Kacem, and Djamila Aouada. Transcad: A hierarchical transformer for cad sequence inference from point clouds. InEuropean Conference on Computer Vision, pages 19–36. Springer, 2024
2024
-
[28]
Cad-llama: leveraging large language models for computer-aided design parametric 3d model generation
Jiahao Li, Weijian Ma, Xueyang Li, Yunzhong Lou, Guichun Zhou, and Xiangdong Zhou. Cad-llama: leveraging large language models for computer-aided design parametric 3d model generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18563–18573, 2025
2025
-
[29]
Flexcad: Unified and versatile controllable cad generation with fine-tuned large language models
Zhanwei Zhang, Shizhao Sun, Wenxiao Wang, Deng Cai, and Jiang Bian. Flexcad: Unified and versatile controllable cad generation with fine-tuned large language models. InInternational Conference on Learning Representations, volume 2025, pages 3204–3227, 2025
2025
-
[30]
Deepcad: A deep generative network for computer-aided design models
Rundi Wu, Chang Xiao, and Changxi Zheng. Deepcad: A deep generative network for computer-aided design models. InProceedings of the IEEE/CVF international conference on computer vision, pages 6772–6782, 2021
2021
-
[31]
Text2cad: Generating sequential cad designs from beginner-to-expert level text prompts.Advances in Neural Information Processing Systems, 37:7552–7579, 2024
Mohammad S Khan, Sankalp Sinha, Talha U Sheikh, Didier Stricker, Sk A Ali, and Muhammad Z Afzal. Text2cad: Generating sequential cad designs from beginner-to-expert level text prompts.Advances in Neural Information Processing Systems, 37:7552–7579, 2024
2024
-
[32]
Cad-coder: Text-to-cad generation with chain-of-thought and geometric reward.Advances in Neural Information Processing Systems, 38:59765–59789, 2026
Yandong Guan, Xilin Wang, Ximing Xing, Jing Zhang, Dong Xu, and Qian Yu. Cad-coder: Text-to-cad generation with chain-of-thought and geometric reward.Advances in Neural Information Processing Systems, 38:59765–59789, 2026
2026
-
[33]
Cad-recode: Reverse engineering cad code from point clouds
Danila Rukhovich, Elona Dupont, Dimitrios Mallis, Kseniya Cherenkova, Anis Kacem, and Djamila Aouada. Cad-recode: Reverse engineering cad code from point clouds. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9801–9811, 2025
2025
-
[34]
Deepseekmath: Pushing the limits of mathematical reasoning in open language models
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024
Pith/arXiv arXiv 2024
-
[35]
Prashant Govindarajan, Davide Baldelli, Jay Pathak, Quentin Fournier, and Sarath Chandar. Cadmium: Fine-tuning code language models for text-driven sequential cad design.arXiv preprint arXiv:2507.09792, 2025. 11
arXiv 2025
-
[36]
Qwen Team. Qwen3.5. https://qwenlm.github.io/blog/qwen3.5/, February 2026. Accessed: 2026- 05-26
2026
-
[37]
Claude Sonnet 4.6 system card
Anthropic. Claude Sonnet 4.6 system card. Technical report, Anthropic, February 2026
2026
-
[38]
Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, et al. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models.arXiv preprint arXiv:2508.06471, 2025
Pith/arXiv arXiv 2025
-
[39]
Cad-judge: Toward efficient morphological grading and verification for text-to-cad generation
Zheyuan Zhou, Jiayi Han, Liang Du, Naiyu Fang, Lemiao Qiu, and Shuyou Zhang. Cad-judge: Toward efficient morphological grading and verification for text-to-cad generation. InICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1021–1025. IEEE, 2026
2026
-
[40]
Gpt-4o system card
OpenAI. Gpt-4o system card. https://cdn.openai.com/gpt-4o-system-card.pdf , 2024. Accessed: 2024-09-26. 12 Appendix Contents A Data Construction 13 A.1 Source Data and Textualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 A.2 Ground Truth COM Script Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . 13 A.3 Downstream Multi-Tas...
2024
-
[41]
SFT, we apply Low-Rank Adaptation (LoRA) to all linear layers with a rank of r= 8 and α= 32 . The models are trained using the AdamW optimizer with a learning rate of 1e-5, a cosine learning rate scheduler, and a warmup ratio of 0.05. To accommodate the extensive context required for code generation and error tracebacks, the maximum sequence length is set...
arXiv 1961
-
[42]
Brief reasoning inside<thinking>...</thinking>
-
[43]
A high-level decision wrapped as:“‘decision CODE (or DONE/FAIL) “‘
-
[44]
‘ RAG Prompt (Appended to Baseline) External Knowledge Context: Here are some COM APIs that might be useful for completing this task. [ {
If and only if the decision is CODE, output a single“‘python ... “‘block. Few-Shot Prompt (Appended to Baseline) Example: 3D Modeling in Solidworks Task Instruction:Model this part in Solidworks: To construct the first part of the cylinder...[Detailed dimensions and constraints omitted for brevity]...export the model as an STL and STEP file. Output: <thin...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.