Pith. sign in

REVIEW 2 major objections 6 minor 89 references

Multi-Dimensional Insights: Benchmarking Real-World Personalization in Large Multimodal Models

T0 review · 2 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a new benchmark of over 500 real-world images with age-tagged, two-level questions can reveal whether large multimodal models truly serve different age groups, and that current best models like GPT-4o still cap at…

desk verdict A useful new multimodal QA resource, but the age-personalization claim outruns the evaluation design; the model is never given the user's age. read the letter →

arxiv 2412.12606 v1 pith:5O3YO44E submitted 2024-12-17 cs.AI cs.CLcs.CV

YiFan Zhang , Shanglin Lei , Runqi Qiao , Zhuoma GongQue , Xiaoshuai Song , Guanting Dong , Qiuna Tan , Zhe Wei
show 5 more authors
Peiqing Yang Ye Tian Yadong Xue Xiaofei Wang Honggang Zhang
This is my paper · ORCID
classification cs.AIcs.CLcs.CV
keywords largemultimodalmodelspersonalizationbenchmarkage-stratifiedevaluationreal-worldvisualquestionansweringcomplexityreasoninghuman-alignedAI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces MDI-Benchmark, a collection of 514 newly gathered real-world images and 1,298 human-authored questions, to test whether large multimodal models can meet people's actual needs in everyday life. Its central claim is that existing benchmarks measure technical skills but miss two things: whether a model understands a real situation at all, and whether it can serve different age groups with different concerns. To capture those, every image carries a simple question and a harder reasoning question, and the questions are tagged as coming from young, middle-aged, or older people. The paper reports that even the strongest model tested, GPT-4o, reaches only 79 percent accuracy on age-related tasks, which it reads as evidence that current models still fall well short of real-world personalization. A sympathetic reader would take the paper's contribution to be a new evaluation instrument: if its age-stratified questions are valid, they expose capability gaps that technical benchmarks miss.

What carries the argument

The central object is the MDI-Benchmark dataset: 514 new images from six life scenarios (architecture, education, housework, social service, sport, transport), each paired with two levels of questions, Level 1 for perceptual extraction (object detection, OCR, color and position recognition) and Level 2 for analysis and reasoning beyond image content. Each question is tagged by the volunteer's age group, defining young as 10-25, middle-aged as 35-50, and old as 60-75. The scoring metric combines the two levels with equal weight, $Score_{final} = 0.5 \cdot Score_{L1} + 0.5 \cdot Score_{L2}$, converting the abstract idea of personalization into accuracy numbers per scenario, per complexity, and per age bracket.

What would settle it

Ask fresh, independent panels from each of the three age groups, people who did not write MDI questions, to judge whether each age-tagged question reflects their own everyday concerns, then rerun the benchmark on questions that pass independent panel agreement; if the panels reject many questions or the model ranking flips, the claim that the benchmark measures age-group alignment collapses.

Watch

Extended reading notes

Core claim

On the paper's own terms, MDI-Benchmark provides a comprehensive, objective, and accurate evaluation of whether large multimodal models align with diverse human needs in real-world scenarios, and the age-stratified design reveals a difficulty ordering of middle-aged > old > young across all fourteen tested models. The headline evidence is that GPT-4o scores 78.46 overall, with 79.74 average accuracy across age groups, beating the best open-source model by about 13 points and the lowest closed-source model by about 35 points; yet no model is close to saturation. The benchmark further documents that every model loses accuracy when moving from Level 1 perceptual questions to Level 2 reasoning questions, with the sharpest drops in sport and transport, and interprets this as a sign that current training data under-serves everyday-life domains.

Load-bearing premise

The load-bearing premise is that questions written by a small group of volunteers in each age bracket faithfully stand in for the needs and perspectives of that whole age group, and that a model picking the single correct multiple-choice answer is the same as meeting those needs.

Editorial extensions

If this is right

  • If MDI-Benchmark measures what it claims, then age-stratified question sets become a standard check for real-world alignment alongside existing technical benchmarks.
  • The 79 percent ceiling for GPT-4o implies that frontier models still leave roughly a fifth of everyday age-group questions unanswered correctly, and open-source models leave far more.
  • The consistent Level-1 to Level-2 accuracy drop, such as GPT-4o's education score falling from 94.12 to 70.59, implies that reasoning beyond image content, not basic perception, is the binding constraint for real-world usefulness.
  • The observed difficulty ordering of middle-aged > old > young implies that training and evaluation should treat age-group coverage as a balance objective rather than a single average.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same volunteer-question protocol could be reused to stratify by occupation, culture, disability status, or other group dimensions, turning MDI's age axis into a template for measuring personalization more broadly.
  • Beyond the paper: a testable prediction follows, that models fine-tuned on age-specific preference data should improve mostly on the age bracket they were tuned for, not uniformly across all groups.
  • Beyond the paper: because the paper's difficulty ordering is computed from model scores rather than human judgments, an independent human-pilot rating of the same questions could reveal whether middle-aged questions are intrinsically harder or merely harder for current models.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The manuscript proposes MDI-Benchmark, a multimodal VQA dataset of 514 newly collected real-world images and 1,298 human-authored multiple-choice questions organized along three dimensions: six everyday-life scenarios (architecture, education, housework, social services, sports, transport), two question-complexity levels (perceptual vs. reasoning), and three author age groups (young, middle-aged, old). The authors evaluate 14 closed- and open-source LMMs and report that GPT-4o achieves the highest final score (78.46; 79.74 on the age dimension), with generally better performance on Level-1 perceptual questions and on education/architecture scenarios. The main contribution claimed is that the benchmark provides a "comprehensive, objective and accurate" evaluation of whether LMMs align with the diverse needs of different age groups in real-world scenarios.

Significance. The dataset itself is a concrete artifact: the images are new, the question/answer pipeline involves three age-stratified volunteer groups, cross-validation by all three groups, and expert screening, and the data and evaluation code are publicly released. Evaluating 14 models is a reasonable survey of the state of the art. If the age-stratified scores were shown to measure age-group alignment, the benchmark would fill a real gap. However, the central construct validity is questionable: the age tag reflects who wrote the question, not a condition the model is asked to satisfy, and the authors' own difficulty interpretation (Section 4.5) indicates that the age gaps are at least partly question-difficulty gaps. The paper also provides no human baseline or uncertainty estimates. With these fixed or the claims appropriately reframed, the benchmark's descriptive value would stand.

major comments (2)
  1. [§3.2, Table 4, §4.5] The evaluation never manipulates the age dimension. Questions are tagged as old/mid/young according to the age of the volunteer who authored them (Section 3.2), but the prompt template in Table 4 contains only the Question and Option fields; no age, user profile, or 'answer for an elderly user' instruction is given to the model. Consequently, the age-stratified accuracies in Table 3 compare question sets written by different volunteer groups, not the model's ability to adapt to users of different ages. The paper's own analysis in Section 4.5 confirms this: it interprets the summed scores as the 'actual difficulty order of questions across age levels: middle-aged > old > young' and attributes it to middle-aged questions requiring 'greater logical reasoning and background knowledge.' This is an admission that age-stratified accuracy is confounded with question difficulty. Without age-conditioned prompting, difficulty calibration, or a human baseline, the abstract's claim that MDI-Benchmark evaluates age-group personalization is not supported. I recommend either (a) re-running evaluation with age-contextualized prompts (e.g., presenting the user's age and asking the model to answer accordingly), or (b) reframing the contributions as a benchmark of age-stratified question performance rather than age alignment.
  2. [§4.1–§4.5, Tables 2, 3, 6] All reported accuracies are point estimates without confidence intervals, significance tests, or a human baseline. From Table 1, the per-scenario age-group cells contain roughly 65–80 questions, and the aggregated age-stratified cells contain 426–436 questions; for such sample sizes, differences of a few percentage points are within the expected sampling error (a 5% gap at n=80 has a standard error of roughly 3–4%). The paper, however, makes fine-grained comparative claims, such as GPT-4o having 'smaller performance gaps across all three age-related categories' (Section 4.5) and a '13-point advantage' over the best open-source model, without any indication of variance. Moreover, no human accuracy is reported, so the statement that 79% 'indicat[es] that existing LMMs still have considerable room for improvement' has no absolute reference point. I ask the authors to add confidence intervals (or bootstrap estimates), a human-baseline experiment, and/or explicit disclaimers about noise in the fine-grained comparisons.
minor comments (6)
  1. [Table 1] The 'Total' row lists the number of images as 86, but the column sums to 514; the correct total should be 514, matching the text '514 images'.
  2. [§3.2] The sentence 'we input the scenario dimension information into open-source models (e.g., GPT-4o, Gemini 1.5 Pro) and closed-source models (e.g., LLaVA-NeXT, MiniCPM)' has the open/closed labels reversed: GPT-4o and Gemini 1.5 Pro are closed-source, while LLaVA-NeXT and MiniCPM are open-source.
  3. [§4.1 / Table 5] The model name is inconsistent: Section 4.1 and the leaderboard list 'LLaVA-NeXT-72B' (also spelled 'LLaV A-NeXT-72B'), while the model list in Section 4.1 says 'LLaV A-NeXT-70B' and Table 5 says 'LLaVA-NeXT-72B'; please unify the name and confirm the correct parameter count.
  4. [Table 4 / §4.1] The strict output-format requirement means that any response not matching the template is counted as incorrect. The paper does not report the fraction of responses that violated the format; if non-negligible, the accuracy scores conflate instruction-following with content correctness. Please report format-failure rates and consider a more tolerant answer-matching procedure.
  5. [Abstract / §4.5] The phrase 'age-related tasks' in the abstract overstates what is measured. The benchmark contains questions written by (and tagged for) different age groups, but the models are not asked to perform any age-specific behavior. Please adjust the wording to avoid implying that the models were prompted with age information.
  6. [Related Work / §2.3] The claim of being 'the first to propose a multi-modal benchmark for real-world personalization' is not fully substantiated; the related-work section cites several personalized-LLM efforts but does not compare directly with any existing multimodal benchmarks that also incorporate user context or demographic tags. Please add a clearer comparison to establish novelty.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: benchmark scores are direct accuracy measurements; no fitted parameter or self-citation chain is load-bearing.

full rationale

The paper's contribution is a new dataset and accuracy evaluation, not a mathematical derivation of a predicted quantity. The only hand-set scalar, alpha in Eq. (1), is an explicit weighting choice (default 0.5) and does not enter the age-stratified accuracy table that supports the 79% claim. Age-stratified scores in Table 3 are direct answer-accuracy percentages on volunteer-authored questions; nothing is fitted to those scores and then reported as a prediction. The paper includes several self-citations (e.g., Dong et al. 2024c, Qiao et al. 2024a, Song et al. 2024), but only in related-work enumerations; none of these citations supplies a load-bearing premise or forbids alternative choices. There is no imported uniqueness theorem and no ansatz smuggled in by citation. The strongest conceptual concern—that age-bracket accuracy differences may be confounded with the difficulty of questions written by different volunteer groups—is a construct-validity or benchmarking concern, not circularity: the claim that MDI measures age-group needs rests on an assumption about the proxy, not on an equation that defines the result in terms of its inputs. Accordingly, no circular step can be quoted and the appropriate score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claims rest on two kinds of unpaid premises: design choices (scoring weight alpha, age bracket definitions) and validity assumptions (age of question author as proxy for group needs, multiple-choice accuracy as proxy for alignment). No invented entities are introduced.

free parameters (2)
  • Scoring weight alpha = 0.5
    Equation 1 defines Final = alpha * Score_L1 + (1-alpha) * Score_L2; alpha is set to 0.5 without justification. It linearly mixes the two complexity levels in the reported final scores.
  • Age group boundaries = 10-25, 35-50, 60-75
    Hand-specified brackets in Section 3.1. Ages 26-34 and 51-59 are excluded, and the paper does not justify the gaps or the bracket widths.
assumptions (4)
  • domain assumption A volunteer's age group is a valid proxy for the real-world needs and perspectives of that age group
    The age dimension is constructed by having volunteers from each age bracket pose questions about their own needs (Section 3.1, Section 3.2). The benchmark assumes these questions represent the target group's needs.
  • domain assumption Accuracy on the benchmark's multiple-choice questions is a valid measure of how well an LMM aligns with human needs
    The evaluation protocol measures whether the model selects the correct multiple-choice option (Sections 3.1, 4.1). This operationalizes 'alignment with needs' as answer accuracy, ignoring open-ended generation and adaptation.
  • domain assumption The expert screening in Section 3.2 ensures that the annotated correct answers are correct
    Question correctness relies on expert review of volunteer-submitted questions, without reporting inter-annotator agreement.
  • domain assumption The survey of 2374 respondents and the 30-person construction team is representative of the three age groups' real-world needs
    The scenario selection and question topics are derived from a survey and volunteer elicitation described in Sections 3.1 and 3.2; representativeness is asserted, not measured.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-Dimensional Insights: Benchmarking Real-World Personalization in Large Multimodal Models." pith.science (2026). https://pith.science/paper/5O3YO44E

@misc{pith2026241212606,
  author       = {Pith},
  title        = {Pith review of: Multi-Dimensional Insights: Benchmarking Real-World Personalization in Large Multimodal Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5O3YO44E}},
  note         = {Machine review of arXiv:2412.12606}
}
read the original abstract

The rapidly developing field of large multimodal models (LMMs) has led to the emergence of diverse models with remarkable capabilities. However, existing benchmarks fail to comprehensively, objectively and accurately evaluate whether LMMs align with the diverse needs of humans in real-world scenarios. To bridge this gap, we propose the Multi-Dimensional Insights (MDI) benchmark, which includes over 500 images covering six common scenarios of human life. Notably, the MDI-Benchmark offers two significant advantages over existing evaluations: (1) Each image is accompanied by two types of questions: simple questions to assess the model's understanding of the image, and complex questions to evaluate the model's ability to analyze and reason beyond basic content. (2) Recognizing that people of different age groups have varying needs and perspectives when faced with the same scenario, our benchmark stratifies questions into three age categories: young people, middle-aged people, and older people. This design allows for a detailed assessment of LMMs' capabilities in meeting the preferences and needs of different age groups. With MDI-Benchmark, the strong model like GPT-4o achieve 79% accuracy on age-related tasks, indicating that existing LMMs still have considerable room for improvement in addressing real-world applications. Looking ahead, we anticipate that the MDI-Benchmark will open new pathways for aligning real-world personalization in LMMs. The MDI-Benchmark data and evaluation code are available at https://mdi-benchmark.github.io/

Figures

Figures reproduced from arXiv: 2412.12606 by the authors.

Figure 1
Figure 1. The MDI-Benchmark includes real needs of different age groups in six major real-world [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The overview of the MDI Benchmark’s six real-world multimodal scenarios, each com [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The average performance of different LMMs on different difficulty levels of the MDI [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (30 more)
Figure 4
Figure 4. Figure 4: Performance of the model at different difficulty levels and the overall performance results [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: The average accuracy and variance of LLMs across six domains at Level 1 and Level 2 [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Performance of different LMMs across the age dimension. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 8
Figure 8. Figure 8: 3. Housework: Including home arrangements, housework activities, household appliances, etc. As shown in [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 7
Figure 7. Figure 7: Examples of Architecture Scenario [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 9
Figure 9. Figure 9: Examples of Housework Scenario [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: Examples of Social Service [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Examples of Sport Scenario. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: Examples of Transport Scenario. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 13
Figure 13. Figure 13: Examples of Architecture Scenario Questions. [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]
Figure 14
Figure 14. Figure 14: Examples of Education Scenario Questions. [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]
Figure 15
Figure 15. Figure 15: Examples of Housework Scenario Questions. [PITH_FULL_IMAGE:figures/full_fig_p022_15.png]
Figure 16
Figure 16. Figure 16: Examples of Social Service Scenario Questions. [PITH_FULL_IMAGE:figures/full_fig_p022_16.png]
Figure 17
Figure 17. Figure 17: Examples of Sport Scenario Questions [PITH_FULL_IMAGE:figures/full_fig_p023_17.png]
Figure 18
Figure 18. Figure 18: Examples of Transport Scenario Question. [PITH_FULL_IMAGE:figures/full_fig_p023_18.png]
Figure 19
Figure 19. Figure 19: Example of Architecture Scenario Age Questions. [PITH_FULL_IMAGE:figures/full_fig_p024_19.png]
Figure 20
Figure 20. Figure 20: Example of Education Scenario Age Questions. [PITH_FULL_IMAGE:figures/full_fig_p024_20.png]
Figure 21
Figure 21. Figure 21: Example of Housework Scenario Age Questions. [PITH_FULL_IMAGE:figures/full_fig_p025_21.png]
Figure 22
Figure 22. Figure 22: Example of Social Service Scenario Age Questions. [PITH_FULL_IMAGE:figures/full_fig_p025_22.png]
Figure 23
Figure 23. Figure 23: Example of Sport Scenario Age Questions. [PITH_FULL_IMAGE:figures/full_fig_p026_23.png]
Figure 24
Figure 24. Figure 24: Example of Transport Scenario Age Questions. [PITH_FULL_IMAGE:figures/full_fig_p026_24.png]
Figure 25
Figure 25. Figure 25: Example of GPT-4o Architecture Scenario Correct Answers. [PITH_FULL_IMAGE:figures/full_fig_p027_25.png]
Figure 26
Figure 26. Figure 26: Example of GPT-4o Education Scenario Correct Answers. [PITH_FULL_IMAGE:figures/full_fig_p028_26.png]
Figure 27
Figure 27. Figure 27: Example of GPT-4o Housework Scenario Correct Answers. [PITH_FULL_IMAGE:figures/full_fig_p028_27.png]
Figure 28
Figure 28. Figure 28: Example of GPT-4o Social Service Scenario Correct Answers. [PITH_FULL_IMAGE:figures/full_fig_p029_28.png]
Figure 29
Figure 29. Figure 29: Example of GPT-4o Sport Scenario Correct Answers. [PITH_FULL_IMAGE:figures/full_fig_p029_29.png]
Figure 30
Figure 30. Figure 30: Example of GPT-4o Trans Scenario Correct Answers. [PITH_FULL_IMAGE:figures/full_fig_p030_30.png]
Figure 31
Figure 31. Figure 31: Example of Information Extraction Error. [PITH_FULL_IMAGE:figures/full_fig_p031_31.png]
Figure 32
Figure 32. Figure 32: Example of Knowledge Deficiency Error. 32 [PITH_FULL_IMAGE:figures/full_fig_p032_32.png]
Figure 33
Figure 33. Figure 33: Example of Reasoning Error. 33 [PITH_FULL_IMAGE:figures/full_fig_p033_33.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

89 extracted references · 14 canonical work pages

  1. [1]

    Phi-3 technical report: A highly capable language model locally on your phone

    Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219, 2024

    arXiv 2024

  2. [2]

    Nocaps: Novel object captioning at scale

    Harsh Agrawal, Karan Desai, Yufei Wang, Xinlei Chen, Rishabh Jain, Mark Johnson, Dhruv Batra, Devi Parikh, Stefan Lee, and Peter Anderson. Nocaps: Novel object captioning at scale. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 8948--8957, 2019

    2019

  3. [3]

    Knowledge-augmented large language models for personalized contextual query suggestion

    Jinheon Baek, Nirupama Chandrasekaran, Silviu Cucerzan, Allen Herring, and Sujay Kumar Jauhar. Knowledge-augmented large language models for personalized contextual query suggestion. In Proceedings of the ACM on Web Conference 2024, pp.\ 3355--3366, 2024

    2024

  4. [4]

    Qwen-vl: A frontier large vision-language model with versatile abilities

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966, 2023

    arXiv 2023

  5. [5]

    Vizwiz: nearly real-time answers to visual questions

    Jeffrey P Bigham, Chandrika Jayant, Hanjie Ji, Greg Little, Andrew Miller, Robert C Miller, Robin Miller, Aubrey Tatarowicz, Brandyn White, Samual White, et al. Vizwiz: nearly real-time answers to visual questions. In Proceedings of the 23nd annual ACM symposium on User interface software and technology, pp.\ 333--342, 2010

    2010

  6. [6]

    Social attention and real-world scenes: The roles of action, competition and social content

    Elina Birmingham, Walter F Bischof, and Alan Kingstone. Social attention and real-world scenes: The roles of action, competition and social content. Quarterly journal of experimental psychology, 61 0 (7): 0 986--998, 2008

    2008

  7. [7]

    Scene text visual question answering

    Ali Furkan Biten, Ruben Tito, Andres Mafla, Lluis Gomez, Mar c al Rusinol, Ernest Valveny, CV Jawahar, and Dimosthenis Karatzas. Scene text visual question answering. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 4291--4301, 2019

    2019

  8. [8]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 0 1877--1901, 2020

    1901

Show all 89 references
  1. [9]

    Scaling synthetic data creation with 1,000,000,000 personas

    Xin Chan, Xiaoyang Wang, Dian Yu, Haitao Mi, and Dong Yu. Scaling synthetic data creation with 1,000,000,000 personas. arXiv preprint arXiv:2406.20094, 2024

    2024 arXiv

  2. [10]

    Microsoft coco captions: Data collection and evaluation server

    Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Doll \'a r, and C Lawrence Zitnick. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325, 2015

    2015 arXiv

  3. [11]

    Vicuna: An open-source chatbot impressing gpt-4 with 90\ See https://vicuna

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90\ See https://vicuna. lmsys. org (accessed 14 April 2023), 2 0 (3): 0 6, 2023

    2023

  4. [12]

    PSSAT: A perturbed semantic structure awareness transferring method for perturbation-robust slot filling

    Guanting Dong, Daichi Guo, Liwen Wang, Xuefeng Li, Zechen Wang, Chen Zeng, Keqing He, Jinzheng Zhao, Hao Lei, Xinyue Cui, Yi Huang, Junlan Feng, and Weiran Xu. PSSAT: A perturbed semantic structure awareness transferring method for perturbation-robust slot filling. In Nicolett...

    2022

  5. [13]

    Bridging the kb-text gap: Leveraging structured knowledge-aware pre-training for KBQA

    Guanting Dong, Rumei Li, Sirui Wang, Yupeng Zhang, Yunsen Xian, and Weiran Xu. Bridging the kb-text gap: Leveraging structured knowledge-aware pre-training for KBQA . In Ingo Frommholz, Frank Hopfgartner, Mark Lee, Michael Oakes, Mounia Lalmas, Min Zhang, and Rodrygo L. T. San...

    2023

  6. [14]

    Self-play with execution feedback: Improving instruction-following capabilities of large language models

    Guanting Dong, Keming Lu, Chengpeng Li, Tingyu Xia, Bowen Yu, Chang Zhou, and Jingren Zhou. Self-play with execution feedback: Improving instruction-following capabilities of large language models. CoRR, abs/2406.13542, 2024 a . doi:10.48550/ARXIV.2406.13542. URL https://doi.o...

    2024 doi arXiv

  7. [15]

    Toward general instruction-following alignment for retrieval-augmented generation

    Guanting Dong, Xiaoshuai Song, Yutao Zhu, Runqi Qiao, Zhicheng Dou, and Ji-Rong Wen. Toward general instruction-following alignment for retrieval-augmented generation. arXiv preprint arXiv:2410.09584, 2024 b

    2024 arXiv

  8. [16]

    How abilities in large language models are affected by supervised fine-tuning data composition

    Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, and Jingren Zhou. How abilities in large language models are affected by supervised fine-tuning data composition. In Lun - Wei Ku, Andre Martins, and Vivek Srikum...

    2024

  9. [17]

    Understand what LLM needs: Dual preference alignment for retrieval-augmented generation

    Guanting Dong, Yutao Zhu, Chenghao Zhang, Zechen Wang, Zhicheng Dou, and Ji - Rong Wen. Understand what LLM needs: Dual preference alignment for retrieval-augmented generation. CoRR, abs/2406.18676, 2024 d . doi:10.48550/ARXIV.2406.18676. URL https://doi.org/10.48550/arXiv.2406.18676

    2024 doi arXiv

  10. [18]

    Mme: A comprehensive evaluation benchmark for multimodal large language models

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, and Rongrong Ji. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306.13394, 2024 a

    2024 arXiv

  11. [19]

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis

    Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. arXiv preprint arXiv:2405.21075, 2024 b

    2024 arXiv

  12. [20]

    Retrieval-augmented generation for large language models: A survey

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997, 2023

    2023 arXiv

  13. [21]

    Tora: A tool-integrated reasoning agent for mathematical problem solving

    Zhibin Gou, Zhihong Shao, Yeyun Gong, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen, et al. Tora: A tool-integrated reasoning agent for mathematical problem solving. arXiv preprint arXiv:2309.17452, 2023

    2023 arXiv

  14. [22]

    Making the v in vqa matter: Elevating the role of image understanding in visual question answering

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 6904--6913, 2017

    2017

  15. [23]

    Retrieval augmented language model pre-training

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. Retrieval augmented language model pre-training. In International conference on machine learning, pp.\ 3929--3938. PMLR, 2020

    2020

  16. [24]

    Measuring massive multitask language understanding, 2021

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding, 2021

    2021

  17. [25]

    Cogagent: A visual language model for gui agents

    Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, et al. Cogagent: A visual language model for gui agents. arXiv preprint arXiv:2312.08914, 2023

    2023 arXiv

  18. [26]

    Minicpm: Unveiling the potential of small language models with scalable training strategies

    Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, et al. Minicpm: Unveiling the potential of small language models with scalable training strategies. arXiv preprint arXiv:2404.06395, 2024

    2024 arXiv

  19. [27]

    Gqa: A new dataset for real-world visual reasoning and compositional question answering

    Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 6700--6709, 2019

    2019

  20. [28]

    Privacy through pseudonymity in user-adaptive systems

    Alfred Kobsa and J \"o rg Schreck. Privacy through pseudonymity in user-adaptive systems. ACM Transactions on Internet Technology (TOIT), 3 0 (2): 0 149--183, 2003

    2003

  21. [29]

    The personalization of conversational agents in health care: systematic review

    Ahmet Baki Kocaballi, Shlomo Berkovsky, Juan C Quiroz, Liliana Laranjo, Huong Ly Tong, Dana Rezazadegan, Agustina Briatore, and Enrico Coiera. The personalization of conversational agents in health care: systematic review. Journal of medical Internet research, 21 0 (11): 0 e15...

    2019

  22. [30]

    Instructerc: Reforming emotion recognition in conversation with a retrieval multi-task llms framework

    Shanglin Lei, Guanting Dong, Xiaoping Wang, Keheng Wang, and Sirui Wang. Instructerc: Reforming emotion recognition in conversation with a retrieval multi-task llms framework. arXiv preprint arXiv:2309.11911, 2023

    2023 arXiv

  23. [31]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \"u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \"a schel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing...

    2020

  24. [32]

    Seed-bench: Benchmarking multimodal llms with generative comprehension

    Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. Seed-bench: Benchmarking multimodal llms with generative comprehension. arXiv preprint arXiv:2307.16125, 2023

    2023 arXiv

  25. [33]

    Llava-med: Training a large language-and-vision assistant for biomedicine in one day

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems, 36, 2024

    2024

  26. [34]

    Llava-next: Improved reasoning, ocr, and world knowledge, 2024 a

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, 2024 a

    2024

  27. [35]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024 b

    2024

  28. [36]

    Lost in the middle: How language models use long contexts

    Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12: 0 157--173, 2024 c

    2024

  29. [37]

    Mmbench: Is your multi-modal model an all-around player? arXiv preprint arXiv:2307.06281, 2023

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? arXiv preprint arXiv:2307.06281, 2023

    2023 arXiv

  30. [38]

    Deepseek-vl: towards real-world vision-language understanding

    Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Yaofeng Sun, et al. Deepseek-vl: towards real-world vision-language understanding. arXiv preprint arXiv:2403.05525, 2024 a

    2024 arXiv

  31. [39]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representat...

    2024

  32. [40]

    Mmevol: Empowering multimodal large language models with evol-instruct

    Run Luo, Haonan Zhang, Longze Chen, Ting-En Lin, Xiong Liu, Yuchuan Wu, Min Yang, Minzheng Wang, Pengpeng Zeng, Lianli Gao, et al. Mmevol: Empowering multimodal large language models with evol-instruct. arXiv preprint arXiv:2409.05840, 2024

    2024 arXiv

  33. [41]

    Ok-vqa: A visual question answering benchmark requiring external knowledge

    Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: A visual question answering benchmark requiring external knowledge. In Proceedings of the IEEE/cvf conference on computer vision and pattern recognition, pp.\ 3195--3204, 2019

    2019

  34. [42]

    Docvqa: A dataset for vqa on document images

    Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp.\ 2200--2209, 2021

    2021

  35. [43]

    Image segmentation using deep learning: A survey

    Shervin Minaee, Yuri Boykov, Fatih Porikli, Antonio Plaza, Nasser Kehtarnavaz, and Demetri Terzopoulos. Image segmentation using deep learning: A survey. IEEE transactions on pattern analysis and machine intelligence, 44 0 (7): 0 3523--3542, 2021

    2021

  36. [44]

    Ocr-vqa: Visual question answering by reading text in images

    Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. Ocr-vqa: Visual question answering by reading text in images. In 2019 international conference on document analysis and recognition (ICDAR), pp.\ 947--952. IEEE, 2019

    2019

  37. [45]

    Hello gpt-4o, 2024

    OpenAI. Hello gpt-4o, 2024. URL https://openai.com/index/hello-gpt-4o/

    2024

  38. [46]

    Gpt-4v (ision) system card

    R OpenAI. Gpt-4v (ision) system card. Citekey: gptvision, 2023

    2023

  39. [47]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 2...

    2022

  40. [48]

    Generative ai for customizable learning experiences

    Ivica Pesovski, Ricardo Santos, Roberto Henriques, and Vladimir Trajkovik. Generative ai for customizable learning experiences. Sustainability, 16 0 (7): 0 3034, 2024

    2024

  41. [49]

    We-math: Does your large multimodal model achieve human-like mathematical reasoning? arXiv preprint arXiv:2407.01284, 2024 a

    Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma GongQue, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, et al. We-math: Does your large multimodal model achieve human-like mathematical reasoning? arXiv preprint arXiv:2407.01284, 2024 a

    2024 arXiv

  42. [50]

    Making visual sense of oracle bones for you and me

    Runqi Qiao, Lan Yang, Kaiyue Pang, and Honggang Zhang. Making visual sense of oracle bones for you and me. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 12656--12665, 2024 b

    2024

  43. [51]

    Ai and personalization

    Omid Rafieian and Hema Yoganarasimhan. Ai and personalization. Artificial Intelligence in Marketing, pp.\ 77--102, 2023

    2023

  44. [52]

    Deep convolutional neural networks for image classification: A comprehensive review

    Waseem Rawat and Zenghui Wang. Deep convolutional neural networks for image classification: A comprehensive review. Neural computation, 29 0 (9): 0 2352--2449, 2017

    2017

  45. [53]

    Machine translation using deep learning: An overview

    Shashi Pal Singh, Ajai Kumar, Hemant Darbari, Lenali Singh, Anshika Rastogi, and Shikha Jain. Machine translation using deep learning: An overview. In 2017 international conference on computer, communications and electronics (comptelix), pp.\ 162--167. IEEE, 2017

    2017

  46. [54]

    Cs-bench: A comprehensive benchmark for large language models towards computer science mastery

    Xiaoshuai Song, Muxi Diao, Guanting Dong, Zhengyang Wang, Yujia Fu, Runqi Qiao, Zhexu Wang, Dayuan Fu, Huangxuan Wu, Bin Liang, Weihao Zeng, Yejie Wang, Zhuoma Gongque, Jianing Yu, Qiuna Tan, and Weiran Xu. Cs-bench: A comprehensive benchmark for large language models towards ...

    2024 doi arXiv

  47. [55]

    Social influence and group identity

    Russell Spears. Social influence and group identity. Annual review of psychology, 72 0 (1): 0 367--390, 2021

    2021

  48. [56]

    Pandagpt: One model to instruction-follow them all

    Yixuan Su, Tian Lan, Huayang Li, Jialu Xu, Yan Wang, and Deng Cai. Pandagpt: One model to instruction-follow them all. arXiv preprint arXiv:2305.16355, 2023

    2023 arXiv

  49. [57]

    Individuals and groups in social psychology

    Henri Tajfel. Individuals and groups in social psychology. British Journal of social and clinical psychology, 18 0 (2): 0 183--190, 1979

    1979

  50. [58]

    Democratizing large language models via personalized parameter-efficient fine-tuning

    Zhaoxuan Tan, Qingkai Zeng, Yijun Tian, Zheyuan Liu, Bing Yin, and Meng Jiang. Democratizing large language models via personalized parameter-efficient fine-tuning. arXiv preprint arXiv:2402.04401, 2024

    2024 arXiv

  51. [59]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  52. [60]

    Lamda: Language models for dialog applications

    Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239, 2022

    2022 arXiv

  53. [61]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  54. [62]

    Cogvlm: Visual expert for pretrained language models

    Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, et al. Cogvlm: Visual expert for pretrained language models. arXiv preprint arXiv:2311.03079, 2023

    2023 arXiv

  55. [63]

    General ocr theory: Towards ocr-2.0 via a unified end-to-end model

    Haoran Wei, Chenglong Liu, Jinyue Chen, Jia Wang, Lingyu Kong, Yanming Xu, Zheng Ge, Liang Zhao, Jianjian Sun, Yuang Peng, et al. General ocr theory: Towards ocr-2.0 via a unified end-to-end model. arXiv preprint arXiv:2409.01704, 2024

    2024 arXiv

  56. [64]

    Personalized large language models

    Stanis aw Wo \'z niak, Bart omiej Koptyra, Arkadiusz Janz, Przemys aw Kazienko, and Jan Koco \'n . Personalized large language models. arXiv preprint arXiv:2402.09269, 2024

    2024 arXiv

  57. [65]

    Semantic parsing by large language models for intricate updating strategies of zero-shot dialogue state tracking

    Yuxiang Wu, Guanting Dong, and Weiran Xu. Semantic parsing by large language models for intricate updating strategies of zero-shot dialogue state tracking. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 202...

    2023 doi

  58. [66]

    A personalized recommendation system with combinational algorithm for online learning

    Jun Xiao, Minjuan Wang, Bingqian Jiang, and Junli Li. A personalized recommendation system with combinational algorithm for online learning. Journal of ambient intelligence and humanized computing, 9: 0 667--677, 2018

    2018

  59. [67]

    Lvlm-ehub: A comprehensive evaluation benchmark for large vision-language models

    Peng Xu, Wenqi Shao, Kaipeng Zhang, Peng Gao, Shuo Liu, Meng Lei, Fanqing Meng, Siyuan Huang, Yu Qiao, and Ping Luo. Lvlm-ehub: A comprehensive evaluation benchmark for large vision-language models. arXiv preprint arXiv:2306.09265, 2023

    2023 arXiv

  60. [68]

    mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration

    Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Haowei Liu, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration. arXiv preprint arXiv:2311.04257, 2023

    2023 arXiv

  61. [69]

    Mmt-bench: A comprehensive multimodal benchmark for evaluating large vision-language models towards multitask agi

    Kaining Ying, Fanqing Meng, Jin Wang, Zhiqian Li, Han Lin, Yue Yang, Hao Zhang, Wenbo Zhang, Yuqi Lin, Shuo Liu, et al. Mmt-bench: A comprehensive multimodal benchmark for evaluating large vision-language models towards multitask agi. arXiv preprint arXiv:2404.16006, 2024

    2024 arXiv

  62. [70]

    From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions

    Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2: 0 67--78, 2014

    2014

  63. [71]

    Metamath: Bootstrap your own mathematical questions for large language models

    Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. arXiv preprint arXiv:2309.12284, 2023 a

    2023 arXiv

  64. [72]

    Mm-vet: Evaluating large multimodal models for integrated capabilities

    Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02490, 2023 b

    2023 arXiv

  65. [73]

    Scaling relationship on learning mathematical reasoning with large language models

    Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Chuanqi Tan, and Chang Zhou. Scaling relationship on learning mathematical reasoning with large language models. CoRR, abs/2308.01825, 2023. doi:10.48550/ARXIV.2308.01825. URL https://doi.org/10.48550/arXiv.2308.01825

    2023 doi arXiv

  66. [74]

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. arXiv preprint arXiv:2311.16502, 2023

    2023 arXiv

  67. [75]

    Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark

    Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Ming Yin, Botao Yu, Ge Zhang, et al. Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark. arXiv preprint arXiv:2409.02813, 2024 a

    2024 arXiv

  68. [76]

    Mammoth2: Scaling instructions from the web, 2024 b

    Xiang Yue, Tuney Zheng, Ge Zhang, and Wenhu Chen. Mammoth2: Scaling instructions from the web, 2024 b . URL https://arxiv.org/abs/2405.03548

    2024 arXiv

  69. [77]

    Evaluating large language models at evaluating instruction following

    Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. Evaluating large language models at evaluating instruction following. arXiv preprint arXiv:2310.07641, 2023

    2023 arXiv

  70. [78]

    Mm-llms: Recent advances in multimodal large language models

    Duzhen Zhang, Yahan Yu, Chenxing Li, Jiahua Dong, Dan Su, Chenhui Chu, and Dong Yu. Mm-llms: Recent advances in multimodal large language models. arXiv preprint arXiv:2401.13601, 2024

    2024 arXiv

  71. [79]

    Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? In European Conference on Computer Vision, pp.\ 169--186

    Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, et al. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? In European Conference on Computer Vision, pp.\ 169--186. Sprin...

    2025

  72. [80]

    A survey of large language models

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023

    2023 arXiv

  73. [81]

    Object detection with deep learning: A review

    Zhong-Qiu Zhao, Peng Zheng, Shou-tao Xu, and Xindong Wu. Object detection with deep learning: A review. IEEE transactions on neural networks and learning systems, 30 0 (11): 0 3212--3232, 2019

    2019

  74. [82]

    Judging llm-as-a-judge with mt-bench and chatbot arena

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36, 2024

    2024

  75. [83]

    Instruction-following evaluation for large language models

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911, 2023

    2023 arXiv

  76. [84]

    Math-puma: Progressive upward multimodal alignment to enhance mathematical reasoning, 2024 a

    Wenwen Zhuang, Xin Huang, Xiantao Zhang, and Jin Zeng. Math-puma: Progressive upward multimodal alignment to enhance mathematical reasoning, 2024 a . URL https://arxiv.org/abs/2408.08640

    2024 arXiv

  77. [85]

    Hydra: Model factorization framework for black-box llm personalization

    Yuchen Zhuang, Haotian Sun, Yue Yu, Qifan Wang, Chao Zhang, and Bo Dai. Hydra: Model factorization framework for black-box llm personalization. arXiv preprint arXiv:2406.02888, 2024 b

    2024 arXiv

  78. [86]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  79. [87]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  80. [88]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  81. [89]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.