Personal AI Agent for Camera Roll VQA
Pith reviewed 2026-06-28 06:17 UTC · model grok-4.3
The pith
A hierarchical-memory agent outperforms baselines when answering questions over a user's personal photo collection.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper claims that camroll-agent, a conversational system equipped with hierarchical memory and a minimal tool set, enables effective navigation and question answering over large personalized visual memory streams, outperforming existing long-context methods and demonstrating that visual personal memory requires approaches distinct from standard textual long-context handling, especially for consistency, visual detail, and user-specific context.
What carries the argument
camroll-agent: a conversational AI agent that uses hierarchical memory plus a minimal set of tools to navigate and reason over a user's long-horizon personal photo stream.
If this is right
- Personalized visual memory streams need specialized memory hierarchies rather than direct application of text-based long-context techniques.
- The camroll dataset serves as a benchmark exposing gaps in current long-context agents when consistency and user-specific visual details are required.
- With hierarchical memory and few tools, agents can support both factual recall and open-ended recommendations drawn from years of personal photos.
- Handling visual personal memory at scale demands different engineering choices than handling textual memory alone.
Where Pith is reading between the lines
- The same hierarchical-memory pattern could be tested on other personal data streams such as location history or saved documents.
- Deploying such agents would require explicit controls on which parts of the camera roll are visible to the model at any time.
- Re-running the evaluation on camera-roll data collected from additional demographic groups would test whether the reported gap holds beyond the current 50 users.
Load-bearing premise
The manually written questions in the camroll dataset reflect the kinds of queries people would actually pose to an assistant with access to their camera roll.
What would settle it
A controlled experiment in which a flat long-context baseline matches or exceeds camroll-agent accuracy on the full set of 2,500 camroll questions would falsify the claim that the hierarchical-memory design is required.
Figures
read the original abstract
We study the personal camera roll visual question answering setting. In this setting, a conversational AI assistant can access a user's personal camera roll and retrieve relevant photos to answer queries, ranging from simple factual questions (e.g., ``Name of the food I tried yesterday?'') to more open-ended ones (e.g., ``Recommend some dishes I have never eaten before''). Given the vast nature of the personal camera roll (i.e., multiple years, hundreds to thousands of photos), a successful AI assistant needs to understand a long-horizon, highly personalized visual content stream in order to navigate and locate the correct and/or relevant information. To support this, we collect and manually annotate questions that mimic real-world usage. The final dataset, camroll, contains 50 users, 31,476 images, and 2,500 QA pairs. We further design camroll-agent, a conversational AI agent equipped with hierarchical memory and a minimal set of tools for efficient navigation over large, personalized visual memory. Experimental results show that camroll-agent outperforms numerous baselines and methods for long-context understanding AI agents system. Together, the camroll dataset and camroll-agent highlight the gap in AI agents' long-context reasoning: personalized visual memory requires different approaches from standard long-context textual memory, especially when consistency, visual details, and user-specific context are present.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces the camroll dataset (50 users, 31,476 images, 2,500 manually annotated QA pairs) for personal camera-roll visual question answering and proposes camroll-agent, a conversational agent using hierarchical memory and a minimal tool set for navigating long-horizon personalized visual streams. It claims experimental results demonstrate that camroll-agent outperforms baselines for long-context understanding and that personalized visual memory requires distinct approaches from standard textual long-context methods, particularly regarding consistency, visual details, and user-specific context.
Significance. If the outperformance holds under a validated evaluation protocol and the dataset reflects organic usage, the work would usefully highlight limitations of existing long-context agents when applied to visual memory and provide a new benchmark for personalized visual agents. The dataset collection and hierarchical-memory design are concrete contributions that could stimulate follow-up on visual vs. textual memory distinctions.
major comments (2)
- [Dataset Construction] Dataset section: the central claim that results demonstrate a gap between personalized visual memory and textual long-context methods rests on the 2,500 QA pairs accurately mimicking real-world usage patterns (consistency, visual details, user-specific context). The manuscript supplies no supporting evidence such as inter-annotator agreement, comparison against logged user queries, or a user validation study, leaving the representativeness assumption unverified and load-bearing for the gap conclusion.
- [Experimental Results] Experimental Results section: the abstract asserts that camroll-agent 'outperforms numerous baselines' yet the provided description contains no quantitative metrics, baseline specifications, or evaluation protocol details. Without these, the outperformance claim cannot be assessed for statistical significance or fairness of comparison.
minor comments (2)
- [Methods] Clarify the exact composition of the hierarchical memory (e.g., how image embeddings are indexed and retrieved) and the minimal tool set in the methods section to allow reproducibility.
- [Discussion] Add a limitations paragraph discussing potential annotator bias in the QA pairs and the scope of the 50-user sample.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback on dataset representativeness and experimental clarity. We address each major comment below with plans for revision where appropriate.
read point-by-point responses
-
Referee: [Dataset Construction] Dataset section: the central claim that results demonstrate a gap between personalized visual memory and textual long-context methods rests on the 2,500 QA pairs accurately mimicking real-world usage patterns (consistency, visual details, user-specific context). The manuscript supplies no supporting evidence such as inter-annotator agreement, comparison against logged user queries, or a user validation study, leaving the representativeness assumption unverified and load-bearing for the gap conclusion.
Authors: We agree that stronger evidence for representativeness would better support the central claim. The manuscript describes manual annotation to mimic real-world usage, but we will revise to add a dedicated subsection on the annotation protocol, including inter-annotator agreement metrics that were computed internally. We will also elaborate on how the QA pairs were designed around the 50 users' actual photo collections to capture consistency, visual details, and personalization. A comparison to logged user queries is not feasible due to the private nature of personal camera rolls, and a dedicated user validation study was outside the original scope; we will explicitly discuss these as limitations while arguing that the manual process by domain-aware annotators provides a reasonable proxy for organic patterns. revision: partial
-
Referee: [Experimental Results] Experimental Results section: the abstract asserts that camroll-agent 'outperforms numerous baselines' yet the provided description contains no quantitative metrics, baseline specifications, or evaluation protocol details. Without these, the outperformance claim cannot be assessed for statistical significance or fairness of comparison.
Authors: The full experimental results section does contain quantitative metrics (accuracy on the 2,500 QA pairs), baseline specifications (including long-context LLMs and agent variants), and the evaluation protocol. However, we acknowledge these elements were not presented with sufficient prominence or detail for easy assessment. In the revised manuscript we will expand the section with explicit result tables, baseline descriptions, the precise evaluation protocol, and statistical significance analysis to enable full verification of the outperformance claims. revision: yes
Circularity Check
No circularity: empirical evaluation on newly collected dataset
full rationale
The paper collects a new dataset (camroll) with manual annotations and introduces camroll-agent with hierarchical memory and tools, then reports empirical outperformance on 2,500 QA pairs against baselines. No equations, derivations, fitted parameters renamed as predictions, or self-citation chains appear in the abstract or described content. The central claim rests on direct experimental comparison rather than any reduction to inputs by construction. This is a standard empirical setup with no load-bearing self-referential steps.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Survey: The states that phlush away the most memories, 2023
Mixbook. Survey: The states that phlush away the most memories, 2023. Accessed: 2026-05-06
2023
-
[2]
Hunters and gatherers of pictures: Why photography has become a human universal.Frontiers in Psychology, 12, 2021
Lisa Kislinger and Kurt Kotrschal. Hunters and gatherers of pictures: Why photography has become a human universal.Frontiers in Psychology, 12, 2021
2021
-
[3]
David Fernández-Pérez, Lucía Ros, and José M. Latorre. The role of the personal relevance of images in retrieving autobiographical memories for emotion regulation: A randomized controlled trial study.Current Psychology, 43:3523–3537, 2024
2024
-
[4]
Mobile photography statistics, 2024
PhotoAid. Mobile photography statistics, 2024
2024
-
[5]
Digital hoarding: Why your phone has 10,000 photos you’ll never look at, 2025
Affenstunde. Digital hoarding: Why your phone has 10,000 photos you’ll never look at, 2025. Accessed: 2026-04-26
2025
-
[6]
https://support.apple.com/guide/iphone/ use-apple-intelligence-in-photos-iphf7de217f0/ios
Use apple intelligence in photos on iphone. https://support.apple.com/guide/iphone/ use-apple-intelligence-in-photos-iphf7de217f0/ios. Apple Inc., Accessed: 2026-05-06
2026
-
[7]
https://techcommunity.microsoft.com/blog/onedriveblog/ copilot--onedrive-intelligence-in-every-click-inspiration-in-every-memory/ 4458882
Copilot + onedrive: Intelligence in every click, inspiration in every memory. https://techcommunity.microsoft.com/blog/onedriveblog/ copilot--onedrive-intelligence-in-every-click-inspiration-in-every-memory/ 4458882. Microsoft, Accessed: 2026-05-06
2026
-
[8]
OpenAI, Accessed: 2026-05-06
Chatgpt.https://chat.openai.com. OpenAI, Accessed: 2026-05-06
2026
-
[9]
Google, Accessed: 2026-05-06
Gemini.https://gemini.google.com. Google, Accessed: 2026-05-06. 10
2026
-
[10]
Anthropic, Accessed: 2026-05-06
Claude.https://claude.ai. Anthropic, Accessed: 2026-05-06
2026
-
[11]
Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang
Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. InACL, 2024
2024
-
[12]
Gonzalez, Trevor Darrell, and David Chan
Tsung-Han Wu, Giscard Biamby, Jerome Quenum, Ritwik Gupta, Joseph E. Gonzalez, Trevor Darrell, and David Chan. Visual haystacks: A vision-centric needle-in-a-haystack benchmark. InICLR, 2025
2025
-
[13]
From RAG to memory: Non-parametric continual learning for large language models
Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi, Sizhe Zhou, and Yu Su. From RAG to memory: Non-parametric continual learning for large language models. InICML, 2025
2025
-
[14]
Self-RAG: Learning to retrieve, generate, and critique through self-reflection
Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-RAG: Learning to retrieve, generate, and critique through self-reflection. InICLR, 2024
2024
-
[15]
Mem0: Building production-ready ai agents with scalable long-term memory
Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production-ready ai agents with scalable long-term memory. InarXiv, 2025
2025
-
[16]
Memos: An operating system for memory-augmented generation (mag) in large language models
Zhiyu Li, Shichao Song, Hanyu Wang, Simin Niu, Ding Chen, Jiawei Yang, Chenyang Xi, Huayi Lai, Jihao Zhao, Yezhaohui Wang, Junpeng Ren, Zehao Lin, Jiahao Huo, Tianyi Chen, Kai Chen, Kehang Li, Zhiqiang Yin, Qingchen Yu, Bo Tang, Hongkang Yang, Zhi-Qin John Xu, and Feiyu Xiong. Memos: An operating system for memory-augmented generation (mag) in large langu...
2025
-
[17]
Simplemem: Efficient lifelong memory for llm agents
Jiaqi Liu, Yaofeng Su, Peng Xia, Siwei Han, Zeyu Zheng, Cihang Xie, Mingyu Ding, and Huaxiu Yao. Simplemem: Efficient lifelong memory for llm agents. InarXiv, 2026
2026
-
[18]
Lightmem: Lightweight and efficient memory-augmented generation
Jizhan Fang, Xinle Deng, Haoming Xu, Ziyan Jiang, Yuqi Tang, Ziwen Xu, Shumin Deng, Yunzhi Yao, Mengru Wang, Shuofei Qiao, Huajun Chen, and Ningyu Zhang. Lightmem: Lightweight and efficient memory-augmented generation. InICLR, 2026
2026
-
[19]
Evaluating very long-term conversational memory of llm agents.arxiv, 2024
Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluating very long-term conversational memory of llm agents.arxiv, 2024
2024
-
[20]
Taylor, and Dan Roth
Bowen Jiang, Zhuoqun Hao, Young-Min Cho, Bryan Li, Yuan Yuan, Sihao Chen, Lyle Ungar, Camillo J. Taylor, and Dan Roth. Know me, respond to me: Benchmarking llms for dynamic user profiling and personalized responses at scale. InarXiv, 2025
2025
-
[21]
Perltqa: A personal long-term memory dataset for memory classification, retrieval, and fusion in question answering
Yiming Du, Hongru Wang, Zhengyi Zhao, Bin Liang, Baojun Wang, Wanjun Zhong, Zezhong Wang, and Kam-Fai Wong. Perltqa: A personal long-term memory dataset for memory classification, retrieval, and fusion in question answering. InProceedings of the 10th SIGHAN Workshop on Chinese Language Processing (SIGHAN-10), 2024
2024
-
[22]
Deepimagesearch: Benchmarking multimodal agents for context-aware image retrieval in visual histories
Chenlong Deng, Mengjie Deng, Junjie Wu, Dun Zeng, Teng Wang, Qingsong Xie, Jiadeng Huang, Shengjie Ma, Changwang Zhang, Zhaoxiang Wang, Jun Wang, Yutao Zhu, and Zhicheng Dou. Deepimagesearch: Benchmarking multimodal agents for context-aware image retrieval in visual histories. InarXiv, 2026
2026
-
[23]
Photobench: Beyond visual matching towards personalized intent-driven photo retrieval
Tianyi Xu, Rong Shan, Junjie Wu, Jiadeng Huang, Teng Wang, Jiachen Zhu, Wenteng Chen, Minxin Tu, Quantao Dou, Zhaoxiang Wang, Changwang Zhang, Weinan Zhang, Jun Wang, and Jianghao Lin. Photobench: Beyond visual matching towards personalized intent-driven photo retrieval. InarXiv, 2026
2026
-
[24]
Ask photos: Search your memories in google photos
Google. Ask photos: Search your memories in google photos. https://blog.google/ products-and-platforms/products/photos/ask-button-ask-photos-tips/ , 2024. Accessed: 2026-05-04
2024
-
[25]
Personal intelligence in gemini app with nano banana
Google. Personal intelligence in gemini app with nano banana. https://blog.google/ innovation-and-ai/products/gemini-app/personal-intelligence-nano-banana/ , 2026. Accessed: 2026-05-04
2026
-
[26]
Introducing muse spark: Msl’s first model, purpose-built to prioritize people
Meta AI. Introducing muse spark: Msl’s first model, purpose-built to prioritize people. https://ai. meta.com/blog/introducing-muse-spark-msl/, 2026. Accessed: 2026-05-04
2026
-
[27]
Cottrell
Yufei Wang, Zhe Lin, Xiaohui Shen, Radomˇir Mˇech, Gavin Miller, and Garrison W. Cottrell. Recognizing and curating photo albums via event-specific image importance. InBMVC, 2017
2017
-
[28]
Luming Tang, Nataniel Ruiz, Chu Qinghao, Yuanzhen Li, Aleksander Holynski, David E Jacobs, Bharath Hariharan, Yael Pritch, Neal Wadhwa, Kfir Aberman, and Michael Rubinstein. Realfill: Reference-driven generation for authentic image completion.arXiv preprint arXiv:2309.16668, 2023
-
[29]
Myvlm: Personal- izing vlms for user-specific queries, 2024
Yuval Alaluf, Elad Richardson, Sergey Tulyakov, Kfir Aberman, and Daniel Cohen-Or. Myvlm: Personal- izing vlms for user-specific queries, 2024. 11
2024
-
[30]
Yo’LLaV A: Your personalized language and vision assistant
Thao Nguyen, Haotian Liu, Yuheng Li, Mu Cai, Utkarsh Ojha, and Yong Jae Lee. Yo’LLaV A: Your personalized language and vision assistant. InNeurIPS, 2024
2024
-
[31]
Yo’chameleon: Personalized vision and language generation.CVPR, 2025
Thao Nguyen, Krishna Kumar Singh, Jing Shi, Trung Bui, Yong Jae Lee, and Yuheng Li. Yo’chameleon: Personalized vision and language generation.CVPR, 2025
2025
-
[32]
Personavlm: Long-term personal- ized multimodal llms
Chang Nie, Chaoyou Fu, Yifan Zhang, Haihua Yang, and Caifeng Shan. Personavlm: Long-term personal- ized multimodal llms. InCVPR, 2026
2026
-
[33]
Patel, and Yuheng Li
Viet Nguyen, Thao Nguyen, Vishal M. Patel, and Yuheng Li. Personal visual memory from explicit and implicit evidence, 2026
2026
-
[34]
Context length alone hurts llm performance despite perfect retrieval
Yufeng Du, Minyang Tian, Srikanth Ronanki, Subendhu Rongali, Sravan Bodapati, Aram Galstyan, Azton Wells, Roy Schwartz, Eliu A Huerta, and Hao Peng. Context length alone hurts llm performance despite perfect retrieval. InEMNLP, 2025
2025
-
[35]
Memorybank: Enhancing large language models with long-term memory
Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. Memorybank: Enhancing large language models with long-term memory. InarXiv, 2023
2023
-
[36]
Re- Act: Synergizing reasoning and acting in language models
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. Re- Act: Synergizing reasoning and acting in language models. InInternational Conference on Learning Representations (ICLR), 2023
2023
-
[37]
Claude Code: Agentic coding
Anthropic. Claude Code: Agentic coding. https://www.anthropic.com/product/claude-code,
-
[38]
Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li
Bart Thomee, David A. Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. Yfcc100m: the new data in multimedia research. 2016
2016
-
[39]
According to me: Long-term personalized referential memory qa
Jingbiao Mei, Jinghong Chen, Guangyu Yang, Xinyu Hou, Margaret Li, and Bill Byrne. According to me: Long-term personalized referential memory qa. InarXiv, 2026
2026
-
[40]
Episodic memory: From mind to brain.Annual review of psychology, 2002
Endel Tulving. Episodic memory: From mind to brain.Annual review of psychology, 2002
2002
-
[41]
Lawrence Zitnick, and Devi Parikh
Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. InICCV, 2015
2015
-
[42]
Visual instruction tuning, 2023
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023
2023
-
[43]
M3-embedding: Multi- linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation
Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. M3-embedding: Multi- linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In ACL 2024, 2024
2024
-
[44]
Improved baselines with visual instruction tuning, 2023
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2023
2023
-
[45]
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024. A Appendix A.1 Broader Impacts This work studies long-horizon reasoning over personal camera rolls, a setting with potential appli- cations in personalize...
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[46]
Choose exactly one operation for the event table: - ADD: create a new event row - UPDATE: update the latest event row - NO OP: do not modify the event table
-
[47]
a man",
If you choose ADD or UPDATE, return a full event row with fields: - event name - description - date - images Rules: - The image caption must always be present. - Use first-person wording when natural. - The image caption must describe image 2 only, not the profile/reference image. - Be careful about person identity; the person in the first image is the us...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.