Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

PerPO: Perceptual Preference Optimization via Discriminative Rewarding

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PerPO claims that listwise preference optimization weighted by deterministic visual rewards aligns multimodal LLMs with perception, improving visual discrimination and suppressing image-unconditional reward hacking.

desk verdict Useful empirical recipe for MLLM alignment; the ERM theory is a bridge too far. read the letter →

arxiv 2502.04371 v1 pith:M7TM3UQ2 submitted 2025-02-05 cs.AI cs.CLcs.LG

Zining Zhu , Liang Zhao , Kangheng Lin , Jinze Yang , En Yu , Chenglong Liu , Haoran Wei , Jianjian Sun
show 2 more authors
Zheng Ge Xiangyu Zhang
This is my paper · ORCID
classification cs.AIcs.CLcs.LG
keywords perceptualpreferenceoptimizationmultimodallargelanguagemodelslistwisediscriminativerewardempiricalriskminimizationvisualdiscriminationhackingobjectgrounding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PerPO claims that the visual discrimination failures of generative multimodal large language models can be repaired by preference optimization driven by deterministic task rewards, rather than by more data or larger backbones. The method samples many candidate answers for an image, scores each with a cheap automatic metric (intersection-over-union for grounding, edit distance for OCR), and applies a listwise ranking loss in which the reward difference between two candidates acts as a quantitative margin. The authors argue that this bridges generative preference optimization and discriminative empirical risk minimization, because under a simplifying assumption the loss becomes a weighted sum of the rewards, so the model is effectively trained to minimize prediction error while learning a preference ranking. If correct, PerPO offers a human-annotation-free way to improve perception-heavy tasks, and the reported experiments show consistent gains over SFT and DPO on grounding, OCR, hallucination, and general image understanding benchmarks.

What carries the argument

The load-bearing object is the reward-margin-weighted listwise loss of Eq 6, $L_{\mathrm{PerPO}}(\theta)=-\mathbb{E}_{(x,Y)\sim D}[\sum_{\hat R_i>\hat R_j}w_{ij}\log\sigma(\beta(R_i-R_j))]$, with weights $w_{ij}$ defined as a normalized power of the reward gap (Eq 5). The discriminative reward $\hat R=f(x,y)$ is the deterministic error between a rollout and ground truth. The loss inherits the pairwise logistic ranking terms of LiPO, but the margin weight makes the ranking sensitive to the size of the reward gap, and the paper's theoretical section rewrites the objective into a weighted sum over rewards (Eq 8) whose coefficients $\varphi(R_i)$ are sums of log-sigmoid preference terms — this rewrite is what carries the argument that PerPO is a listwise ERM. The $\gamma$ parameter interpolates between plain LiPO (at $\gamma=0$) and reward-magnitude-sensitive ranking.

What would settle it

Compute the gradients of Eq 7 and Eq 8 without the constant-denominator simplification; if they differ, the claimed identity between PerPO and a listwise ERM fails. Separately, replace the margin weights with a constant on the same sampled lists; if PerPO's advantage over DPO persists, the reward-as-margin mechanism is not what drives the gains.

Watch

Extended reading notes

Core claim

The paper's central claim is that a deterministic discriminative reward — how far a generated answer is from visual ground truth, measured by IoU for grounding and edit distance for OCR — can power the entire preference alignment loop. The model samples $N$ candidate answers per image, scores them, and optimizes a listwise ranking loss in which the normalized reward difference $w_{ij}=(\hat R_i-\hat R_j)^\gamma/\sum_{\hat R_i>\hat R_j}(\hat R_i-\hat R_j)^\gamma$ reweights each pairwise logistic term. After reordering terms and treating the normalization as constant, the paper argues, this same loss is a weighted listwise empirical risk minimization over the reward, with the preference terms acting as a difficulty-adaptive focal weight. The claimed upshot is that one objective simultaneously trains the model to prefer better answers and to minimize prediction error, which is why PerPO can improve grounding and OCR accuracy without sacrificing generative strength, and why optimization shifts toward image-conditioned rather than text-only shortcuts.

Load-bearing premise

The argument that PerPO is just a weighted empirical risk minimization depends on treating a normalization term that is actually variable as a constant; if that step fails, the theoretical unification stands on shaky ground, even if the method still works empirically.

Editorial extensions

If this is right

  • If PerPO works as claimed, diverse negative samples for visual alignment can be mined at scale from automatic metrics, removing the need for human preference labels on grounding and OCR tasks.
  • Listwise ordering with reward margins should tie optimization to the image content, which is exactly what the paper's with-image versus without-image comparison shows when PerPO runs ahead of DPO.
  • Because the authors report gains on general benchmarks (LLaVA-W, MMHalBench, POPE, MM-Vet, MM-Bench, MMMU, VQAv2), the method should transfer from two discriminative tasks to broader image understanding.
  • The claimed equivalence implies that generative preference optimization and discriminative risk minimization are not separate regimes but coincide whenever a deterministic reward is available.
  • The paper's capability-saturation finding predicts that PerPO's advantage will appear mainly in models that already have basic competence, so it should be applied after, not instead of, supervised instruction tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the method needs only a cheap deterministic score, a natural extension is to other dense visual tasks with automatic metrics, such as depth estimation, segmentation, or keypoint matching; the paper itself only demonstrates grounding and OCR.
  • The 'ERM equivalence' in Eq 8 depends on a constant-denominator approximation that is not generally true, so until the normalization is handled the theoretical framing is better read as a design rationale than a proof.
  • The observed dependence on base-model maturity suggests a testable curriculum: use standard instruction tuning to build competence, then apply PerPO as a final perceptual polish; the paper does not explore this scheduling directly.
  • If margin weighting is the active ingredient, the same idea should transfer to non-visual preference optimization whenever a quantitative reward exists, which would generalise the mechanism beyond multimodal perception.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Perceptual Preference Optimization (PerPO), a listwise preference optimization method for multimodal large language models (MLLMs). Instead of using human annotations or a learned reward model, PerPO uses deterministic discriminative rewards (IoU for object grounding, edit distance for dense OCR) to score a list of N model-generated rollouts, then optimizes a listwise logistic ranking loss in which each pair is weighted by the normalized reward difference. The authors claim that this reward-as-margin design bridges generative preference optimization and discriminative empirical risk minimization (ERM), mitigates image-unconditional reward hacking, and improves performance on object grounding, dense OCR, and general image understanding benchmarks on top of several LLaVA-family base models. Experiments compare PerPO with SFT and DPO on three base models and two tasks, with ablations over sampling size N, loss temperature β, margin exponent γ, data margin thresholds, and LoRA rank.

Significance. If the empirical results hold, PerPO is a practically useful method: it turns cheap, annotation-free deterministic metrics into preference signals and shows consistent gains over SFT and DPO on multiple benchmarks and base models. The paper includes useful analyses (Best-of-N reward consistency, effect of negative sample count, γ ablation) and ships code, which supports reproducibility of the grounding experiments. The central theoretical claim, however, is currently not established: the derivation of the ERM equivalence relies on an unjustified constant-normalization assumption, so the paper's headline contribution is only partially supported. The empirical contribution alone would still be valuable, but the theoretical framing needs repair before the manuscript reaches the standard of the journal.

major comments (4)
  1. [Section 3, Eqs. (7)-(9)] The derivation of the claim that PerPO is a listwise ERM is not valid as written. Between Eqs. (7) and (8), the paper assumes that γ=1 and that the normalization term Σ_{R̂_i>R̂_j}(R̂_i−R̂_j)^γ is constant. This term is data-dependent: it varies across prompts and across the model's sampled rollouts, so it cannot be dropped without changing the per-example weighting of the loss. Moreover, the resulting weights φ(R_i) in Eq. (9) are functions of the model's own implicit rewards R_i=log(π_θ(y_i|x)/π_ref(y_i|x)), i.e., of the parameters being optimized, so Eq. (8) is a self-referential weighted loss rather than a fixed ERM objective over the discriminative rewards. The paper itself calls this a 'simplified scenario,' but the abstract and contribution list state the ERM connection as a proven theoretical result. As a consequence, the claimed theoretical bridge is not established for the actual objective used in the experiments, and it is particularly problematic that the derivation only covers γ=1 while Figure 4 recommends γ=0.5. I recommend either removing the theoretical claim and rephrasing the ERM link as an interpretation or intuitive connection, or providing a rigorous derivation with error bounds on the constant-normalization approximation.
  2. [Section 4, Tables 1-3 and Figures 3-4] All experimental results are reported from a single run, with no seeds, variances, or significance tests. Many of the reported differences are small (e.g., Table 2 F1 0.54 vs 0.51; Table 1 MMHalBench 0.46 vs 0.48), and some results are non-monotonic (e.g., PerPO LLaVAW in Table 3: 65.9, 61.2, 62.1, 63.1, 64.7 for N=2,4,8,12,20), which makes it difficult to judge which improvements are robust. Please provide means and standard deviations over at least three random seeds, or otherwise justify that the training/evaluation procedure is deterministic enough that single runs are meaningful.
  3. [Section 4.1 and Table 2] The dense OCR evaluation uses a proprietary dataset, described as 'Fox's proprietary dataset' (Section 4.1) and 'Page-ocr in Fox' (Section 5.1). Because the test set is not public, the reported edit distance, F1, BLEU, and METEOR numbers cannot be independently verified or compared by other researchers. Please either release the evaluation set, describe its construction in enough detail to be reproducible, or add results on a public benchmark such as FUNSD, CORD, or DocVQA for the dense OCR scenario.
  4. [Section 5.2 and Figure 1c] The claim that PerPO mitigates image-unconditional reward hacking is supported by the 'w/o image' vs 'w image' comparison in Figure 1c, but the figure does not report how the 'w/o image' condition was constructed (e.g., blank/zero image vs. text-only prompt) or whether the same prompts and sampling procedures were used for both DPO and PerPO. Without this detail and without variance estimates, the reported 'greater performance gap' is not yet convincingly established. Please clarify the protocol and report the underlying numbers for both conditions.
minor comments (6)
  1. [Abstract and Section 1, 'first time' claim] The contribution list states 'we highlight, for the first time, the capability dilemma of generative MLLMs in visual discrimination tasks.' This is a strong novelty claim that is hard to support given existing work on MLLM limitations in perception; consider softening it to 'we highlight' or 'we focus on.'
  2. [Table 1 header] The column header 'RefCOCO testA testB val testA testB val test Score ↑ HalRate ↓' is ambiguous: it is not clear which columns correspond to RefCOCOg and which to LLaVAW/MMHalBench/POPE. Please split the header into clearly labeled column groups.
  3. [Figure 2b] The text says 'Optimal performance occurs at 3k samples,' but Figure 2b shows RefCOCO+ improving with data size while LLaVAW declines; the statement is only true for the aggregate or for LLaVAW. Please clarify which metric the 'optimal' refers to.
  4. [Throughout] There are several typos and inconsistent notations, e.g., 'LLaV AW' for LLaVAW, 'POPEval' in the Table 1 header, and 'arounding' in Section 5.4. Please proofread carefully.
  5. [Section 2, Eq. (4)] In the definition of the LiPO loss, the variables ψ_i are used but not formally defined as ranking levels before the equation; please define them in the text preceding Eq. (4).
  6. [Section 5.1 and Figure 1a] The Best-of-N comparison in Figure 1a uses test set ground truth for Best-of-N while SFT/DPO/PerPO use train set, so the curves are not directly comparable. Since this is presented as an oracle upper bound, please state this explicitly in the figure caption or text.

Circularity Check

1 steps flagged · score 4.0 of 10

The 'PerPO is a listwise ERM' result is a definitional rearrangement of the PerPO loss under an unproven constant-normalization assumption; the empirical benchmark results are externally validated and not circular.

  1. self definitional [Section 3, 'Theoretically, PerPO is a listwise ERM', Eqs. 7-9]
    "We can consider a simplified scenario where γ equals 1 and P ˆRi> ˆRj ( ˆRi − ˆRj) γ is treated as a constant. Formally, this can be expressed as: LPerPO(θ) = −E(x,Y )∼D [ P ˆRi ϕ(Ri) · ˆRi ] , where ϕ(Ri) = P ˆRi > ˆRm log σ(β(Ri −Rm))− P ˆRi < ˆRn log σ(β(Rn −Ri))."

    Eq. 8 is an algebraic rearrangement of Eq. 6 by construction, not an independent derivation: it requires setting γ=1 and treating the data-dependent normalization Z=Σ_{R̂_i>R̂_j}(R̂_i−R̂_j)^γ as a constant, a step the paper itself flags as a 'simplified scenario.' The resulting coefficients φ(R_i) are functions of the same log-probability rewards R_i being optimized, so the claimed equivalence to listwise ERM is a self-referential weighted restatement of the PerPO loss rather than a theorem about ERM. Moreover, the derivation covers only γ=1, while the recommended setting in Figure 4 is γ=0.5, so the abstract's theoretical bridge is not established for the configuration actually used. This does not affect the empirical comparisons, which are evaluated on external benchmarks.

full rationale

The empirical contribution of PerPO is not circular: the method's gains are measured on held-out external benchmarks (RefCOCO/+/g, LLaVA-W, MMHalBench, POPE, MM-Vet, MMBench, MMMU, VQAv2), and no fitted parameter is later renamed as a prediction. The only load-bearing step that reduces to its own input is the theoretical framing: after rearranging Eq. 6 under the stated simplifications, Eq. 8 presents the same loss as a weighted sum of rewards with weights φ(R_i) that depend on the model being optimized, so 'PerPO is a listwise ERM' is a definitional characterization rather than a proved bridge. I considered whether the use of the authors' Fox OCR dataset (Liu et al., 2024a) constitutes load-bearing self-citation; it is a dataset rather than a cited result, so it does not raise the circularity score, though it may be a separate correctness concern. Score 4 reflects partial circularity in the theoretical claim while the central empirical content remains independent.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method adds a small number of hyperparameters (beta, gamma, margin thresholds, N) and relies on the assumption that deterministic visual metrics can serve as preference rewards. It introduces no new architectural entities, forces, or conserved quantities. All assumptions are domain-level or standard math, except the normalization simplification, which is ad hoc to the paper.

free parameters (5)
  • beta (loss temperature) = 0.1
    Tuned via ablation in Figure 2c; controls sensitivity to preference margins in the log-ratio reward.
  • gamma (margin exponent) = 0.5
    Tuned via ablation in Figure 4; controls the exponent on reward differences in the weight w_ij.
  • Data margin threshold (grounding) = 0.8
    Filtering threshold for keeping preference lists in object grounding data; affects dataset composition.
  • Data margin threshold (dense OCR) = 0.04
    Filtering threshold for keeping preference lists in dense OCR data; affects dataset composition.
  • Number of rollouts N per prompt = 20
    Number of samples per instruction used to build the preference list; studied in Table 3.
assumptions (4)
  • domain assumption IoU and edit distance are valid, well-calibrated rewards that rank visual responses by quality.
    Central to the method; all training lists and margins are derived from these rewards, and the paper's Best-of-N analysis is the only evidence for calibration.
  • domain assumption Listwise preference optimization with margin weighting improves alignment better than pairwise DPO.
    The paper's core hypothesis, supported empirically by Tables 1-3 but not proven.
  • ad hoc to paper The normalization term in Eq 7 can be treated as a constant when deriving the ERM equivalence.
    Section 3: 'We can consider a simplified scenario where gamma equals 1 and the sum is treated as a constant.' This simplification is not justified.
  • standard math Bradley-Terry model and the DPO implicit reward formulation apply to multimodal responses.
    Taken from Rafailov et al. 2024; used for the reward parameterization R_i = log pi/pi_ref.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PerPO: Perceptual Preference Optimization via Discriminative Rewarding." pith.science (2026). https://pith.science/paper/M7TM3UQ2

@misc{pith2026250204371,
  author       = {Pith},
  title        = {Pith review of: PerPO: Perceptual Preference Optimization via Discriminative Rewarding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M7TM3UQ2}},
  note         = {Machine review of arXiv:2502.04371}
}
read the original abstract

This paper presents Perceptual Preference Optimization (PerPO), a perception alignment method aimed at addressing the visual discrimination challenges in generative pre-trained multimodal large language models (MLLMs). To align MLLMs with human visual perception process, PerPO employs discriminative rewarding to gather diverse negative samples, followed by listwise preference optimization to rank them.By utilizing the reward as a quantitative margin for ranking, our method effectively bridges generative preference optimization and discriminative empirical risk minimization. PerPO significantly enhances MLLMs' visual discrimination capabilities while maintaining their generative strengths, mitigates image-unconditional reward hacking, and ensures consistent performance across visual tasks. This work marks a crucial step towards more perceptually aligned and versatile MLLMs. We also hope that PerPO will encourage the community to rethink MLLM alignment strategies.

Figures

Figures reproduced from arXiv: 2502.04371 by the authors.

Figure 1
Figure 1. (a) Examples of visual generative and discriminative tasks. (b) Performance comparison in RefCOCOg ( [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Analysis of training data quality, quantity, and hyperparameter [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Relative performance (Left, Human users as judge) and comparative showcases (Right) with and without PerPO [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Performance of different γ values in PerPO loss. we compared the preference optimization results of DPO and PerPO with and without image input on RefCOCOg and LLaVAW. PerPO shows significant performance gains over DPO with image input, demonstrating that PerPO’s optimi…
Figure 5
Figure 5. Figure 5: Comparison of PerPO and SFT across different [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: The prompt for comparing the responses before and after applying PerPO. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multimodal Reward Hacking in Reinforcement Learning

    cs.AI 2026-07 conditional novelty 6.5 of 10

    Imperfect multimodal RL rewards systematically create new failures (NRFR > RHR); scaling and answer-aware rewards help but do not eliminate hacking, and unreliable visual verifiers actively increase it.

Reference graph

Works this paper leans on

87 extracted references · 30 canonical work pages · cited by 1 Pith paper

  1. [1]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Understanding alignment in multimodal llms: A comprehensive study

    Amirloo, E., Fauconnier, J.-P., Roesmann, C., Kerl, C., Boney, R., Qian, Y., Wang, Z., Dehghan, A., Yang, Y., Gan, Z., et al. Understanding alignment in multimodal llms: A comprehensive study. arXiv preprint arXiv:2407.02477, 2024

    arXiv 2024

  3. [3]

    Claude 3.5 sonnet

    Anthropic. Claude 3.5 sonnet. https://anthropic.com/news/claude-3-5-sonnet, 2024

    2024

  4. [4]

    Feedback-generation for programming exercises with GPT-4

    Azaiz, I., Kiesler, N., and Strickroth, S. Feedback-generation for programming exercises with GPT-4 . In ITiCSE (1) . ACM , 2024

    2024

  5. [5]

    G., Guo, Z

    Azar, M. G., Guo, Z. D., Piot, B., Munos, R., Rowland, M., Valko, M., and Calandriello, D. A general theoretical paradigm to understand learning from human preferences. In International Conference on Artificial Intelligence and Statistics, pp.\ 4447--4455. PMLR, 2024

    2024

  6. [6]

    Constitutional ai: Harmlessness from ai feedback

    Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKinnon, C., et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073, 2022

    arXiv 2022

  7. [7]

    V., and Timofte, R

    Bhat, G., Danelljan, M., Gool, L. V., and Timofte, R. Learning discriminative model prediction for tracking. In ICCV , pp.\ 6181--6190. IEEE , 2019

    2019

  8. [8]

    Bradley, R. A. and Terry, M. E. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39 0 (3/4), 1952

    1952

Show all 87 references
  1. [9]

    Brown, T. B. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020

    2005 arXiv

  2. [10]

    Learning to rank using gradient descent

    Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., and Hullender, G. Learning to rank using gradient descent. ACM, pp.\ 89--96, 2005

    2005

  3. [11]

    Burges, C. J. C., Ragno, R., and Le, Q. V. Learning to rank with nonsmooth cost functions. In NIPS , pp.\ 193--200. MIT Press, 2006

    2006

  4. [12]

    and Johnson, M

    Charniak, E. and Johnson, M. Coarse-to-fine n-best parsing and maxent discriminative reranking. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pp.\ 173--180, 2005

    2005

  5. [13]

    E., et al

    Chiang, W.-L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J. E., et al. Vicuna: An open-source chatbot impressing gpt-4 with 90\ See https://vicuna. lmsys. org (accessed 14 April 2023), 2 0 (3): 0 6, 2023

    2023

  6. [14]

    F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D

    Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017

    2017

  7. [15]

    Training verifiers to solve math word problems

    Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training verifiers to solve math word problems. CoRR, abs/2110.14168, 2021

    2021 arXiv

  8. [16]

    Dreamllm: Synergistic multimodal comprehension and creation

    Dong, R., Han, C., Peng, Y., Qi, Z., Ge, Z., Yang, J., Zhao, L., Sun, J., Zhou, H., Wei, H., Kong, X., Zhang, X., Ma, K., and Yi, L. Dreamllm: Synergistic multimodal comprehension and creation. In ICLR . OpenReview.net, 2024

    2024

  9. [17]

    Kto: Model alignment as prospect theoretic optimization

    Ethayarajh, K., Xu, W., Muennighoff, N., Jurafsky, D., and Kiela, D. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306, 2024

    2024 arXiv

  10. [18]

    Mathematical capabilities of chatgpt

    Frieder, S., Pinchetti, L., Chevalier, A., Griffiths, R., Salvatori, T., Lukasiewicz, T., Petersen, P., and Berner, J. Mathematical capabilities of chatgpt. In NeurIPS, 2023

    2023

  11. [19]

    Text-to-audio generation using instruction-tuned LLM and latent diffusion model

    Ghosal, D., Majumder, N., Mehrish, A., and Poria, S. Text-to-audio generation using instruction-tuned LLM and latent diffusion model. CoRR, abs/2304.13731, 2023

    2023 arXiv

  12. [20]

    and Sarawagi, S

    Godbole, S. and Sarawagi, S. Discriminative methods for multi-labeled classification. In PAKDD , volume 3056 of Lecture Notes in Computer Science, pp.\ 22--30. Springer, 2004

    2004

  13. [21]

    Golubev, G. K. On a method of empirical risk minimization. Probl. Inf. Transm., 40 0 (3): 0 202--211, 2004

    2004

  14. [22]

    Making the V in VQA matter: Elevating the role of image understanding in visual question answering

    Goyal, Y., Khot, T., Summers - Stay, D., Batra, D., and Parikh, D. Making the V in VQA matter: Elevating the role of image understanding in visual question answering. In CVPR , pp.\ 6325--6334. IEEE Computer Society, 2017

    2017

  15. [23]

    K., Luo, F., Xiong, Y., and Liang, W

    Guo, D., Zhu, Q., Yang, D., Xie, Z., Dong, K., Zhang, W., Chen, G., Bi, X., Wu, Y., Li, Y. K., Luo, F., Xiong, Y., and Liang, W. Deepseek-coder: When the large language model meets programming - the rise of code intelligence. CoRR, abs/2401.14196, 2024

    2024 arXiv

  16. [24]

    Time course of visual perception: coarse-to-fine processing and beyond

    Hegd \'e , J. Time course of visual perception: coarse-to-fine processing and beyond. Progress in neurobiology, 84 0 (4): 0 405--439, 2008

    2008

  17. [25]

    Orpo: Monolithic preference optimization without reference model, 2024

    Hong, J., Lee, N., and Thorne, J. Orpo: Monolithic preference optimization without reference model, 2024. URL https://arxiv. org/abs/2403.07691, 2403

    2024 arXiv

  18. [26]

    J., Shen, Y., Wallis, P., Allen - Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y., Wallis, P., Allen - Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. In ICLR . OpenReview.net, 2022

    2022

  19. [27]

    Computer vision for autonomous vehicles: Problems, datasets and state of the art

    Janai, J., G \" u ney, F., Behl, A., and Geiger, A. Computer vision for autonomous vehicles: Problems, datasets and state of the art. Found. Trends Comput. Graph. Vis., 12 0 (1-3): 0 1--308, 2020

    2020

  20. [28]

    sdpo: Don't use your data all at once

    Kim, D., Kim, Y., Song, W., Kim, H., Kim, Y., Kim, S., and Park, C. sdpo: Don't use your data all at once. arXiv preprint arXiv:2403.19270, 2024

    2024 arXiv

  21. [29]

    Llava-onevision: Easy visual task transfer

    Li, B., Zhang, Y., Guo, D., Zhang, R., Li, F., Zhang, H., Zhang, K., Li, Y., Liu, Z., and Li, C. Llava-onevision: Easy visual task transfer. CoRR, abs/2408.03326, 2024 a

    2024 arXiv

  22. [30]

    X., and Wen, J.-R

    Li, Y., Du, Y., Zhou, K., Wang, J., Zhao, W. X., and Wen, J.-R. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355, 2023

    2023 arXiv

  23. [31]

    Groundinggpt: Language enhanced multi-modal grounding model

    Li, Z., Xu, Q., Zhang, D., Song, H., Cai, Y., Qi, Q., Zhou, R., Pan, J., Li, Z., Tu, V., et al. Groundinggpt: Language enhanced multi-modal grounding model. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\...

    2024

  24. [32]

    Video-llava: Learning united visual representation by alignment before projection

    Lin, B., Ye, Y., Zhu, B., Cui, J., Ning, M., Jin, P., and Yuan, L. Video-llava: Learning united visual representation by alignment before projection. CoRR, abs/2311.10122, 2023

    2023 arXiv

  25. [33]

    Focal loss for dense object detection

    Lin, T. Focal loss for dense object detection. arXiv preprint arXiv:1708.02002, 2017

    2017 arXiv

  26. [34]

    Focus anywhere for fine-grained multi-page document understanding

    Liu, C., Wei, H., Chen, J., Kong, L., Ge, Z., Zhu, Z., Zhao, L., Sun, J., Han, C., and Zhang, X. Focus anywhere for fine-grained multi-page document understanding. arXiv preprint arXiv:2405.14295, 2024 a

    2024 arXiv

  27. [35]

    Liu, H., Li, C., Li, Y., and Lee, Y. J. Improved baselines with visual instruction tuning, 2023 a

    2023

  28. [36]

    Liu, H., Li, C., Li, Y., and Lee, Y. J. Improved baselines with visual instruction tuning. CoRR, abs/2310.03744, 2023 b

    2023 arXiv

  29. [37]

    Liu, H., Li, C., Li, Y., Li, B., Zhang, Y., Shen, S., and Lee, Y. J. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024 b . URL https://llava-vl.github.io/blog/2024-01-30-llava-next/

    2024

  30. [38]

    Liu, H., Li, C., Wu, Q., and Lee, Y. J. Visual instruction tuning. Advances in neural information processing systems, 36, 2024 c

    2024

  31. [39]

    Lipo: Listwise preference optimization through learning-to-rank

    Liu, T., Qin, Z., Wu, J., Shen, J., Khalman, M., Joshi, R., Zhao, Y., Saleh, M., Baumgartner, S., Liu, J., et al. Lipo: Listwise preference optimization through learning-to-rank. arXiv preprint arXiv:2402.01878, 2024 d

    2024 arXiv

  32. [40]

    Mmbench: Is your multi-modal model an all-around player? In ECCV (6) , volume 15064 of Lecture Notes in Computer Science, pp.\ 216--233

    Liu, Y., Duan, H., Zhang, Y., Li, B., Zhang, S., Zhao, W., Yuan, Y., Wang, J., He, C., Liu, Z., Chen, K., and Lin, D. Mmbench: Is your multi-modal model an all-around player? In ECCV (6) , volume 15064 of Lecture Notes in Computer Science, pp.\ 216--233. Springer, 2024 e

    2024

  33. [41]

    Generation and comprehension of unambiguous object descriptions

    Mao, J., Huang, J., Toshev, A., Camburu, O., and Murphy, K. Generation and comprehension of unambiguous object descriptions. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016

    2016

  34. [42]

    Simpo: Simple preference optimization with a reference-free reward

    Meng, Y., Xia, M., and Chen, D. Simpo: Simple preference optimization with a reference-free reward. arXiv preprint arXiv:2405.14734, 2024

    2024 arXiv

  35. [43]

    Webgpt: Browser-assisted question-answering with human feedback

    Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332, 2021

    2021 arXiv

  36. [44]

    Role design considerations of conversational agents to facilitate discussion and systems thinking

    Nguyen, H. Role design considerations of conversational agents to facilitate discussion and systems thinking. Comput. Educ., 192: 0 104661, 2023

    2023

  37. [45]

    Hello gpt-4o

    OpenAI. Hello gpt-4o. https://openai.com/index/hello-gpt-4o, 2024

    2024

  38. [46]

    L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P. F., Leike, J., and Lowe, R. Training language models t...

    2022

  39. [47]

    Smaug: Fixing failure modes of preference optimisation with dpo-positive

    Pal, A., Karkhanis, D., Dooley, S., Roberts, M., Naidu, S., and White, C. Smaug: Fixing failure modes of preference optimisation with dpo-positive. arXiv preprint arXiv:2402.13228, 2024

    2024 arXiv

  40. [48]

    Papineni, K., Roukos, S., Ward, T., and Zhu, W. J. Bleu: a method for automatic evaluation of machine translation. 2002

    2002

  41. [49]

    R., and Art \' e s - Rodr \' guez, A

    P \' e rez - Cruz, F., Navia - V \' a zquez, \' A ., Figueiras - Vidal, A. R., and Art \' e s - Rodr \' guez, A. Empirical risk minimization for support vector classifiers. IEEE Trans. Neural Networks , 14 0 (2): 0 296--303, 2003

    2003

  42. [50]

    Rio: A benchmark for reasoning intention-oriented objects in open environments

    Qu, M., Wu, Y., Liu, W., Liang, X., Song, J., Zhao, Y., and Wei, Y. Rio: A benchmark for reasoning intention-oriented objects in open environments. Advances in Neural Information Processing Systems, 36, 2024

    2024

  43. [51]

    Improving language understanding by generative pre-training

    Radford, A. Improving language understanding by generative pre-training. 2018

    2018

  44. [52]

    Language models are unsupervised multitask learners

    Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAI blog, 1 0 (8): 0 9, 2019

    2019

  45. [53]

    W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision. In ICML , volume 139 of Proceedings of Machine Learning...

    2021

  46. [54]

    D., Ermon, S., and Finn, C

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024

    2024

  47. [55]

    and Guven, E

    Renze, M. and Guven, E. The effect of sampling temperature on problem solving in large language models. CoRR, abs/2402.05201, 2024

    2024 arXiv

  48. [56]

    Meteor: An automatic metric for mt evaluation with improved correlation with human judgments

    Satanjeev, B. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. ACL-2005, pp.\ 228--231, 2005

    2005

  49. [57]

    Proximal policy optimization algorithms

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  50. [58]

    Planning and decision-making for autonomous vehicles

    Schwarting, W., Alonso - Mora, J., and Rus, D. Planning and decision-making for autonomous vehicles. Annu. Rev. Control. Robotics Auton. Syst., 1: 0 187--210, 2018

    2018

  51. [59]

    Shi, W., Hu, Z., Bin, Y., Liu, J., Yang, Y., Ng, S., Bing, L., and Lee, R. K. Math-llava: Bootstrapping mathematical reasoning for multimodal large language models. CoRR, abs/2406.17294, 2024

    2024 arXiv

  52. [60]

    Skalse, J., Howe, N. H. R., Krasheninnikov, D., and Krueger, D. Defining and characterizing reward hacking. CoRR, abs/2209.13085, 2022

    2022 arXiv

  53. [61]

    Preference ranking optimization for human alignment

    Song, F., Yu, B., Li, M., Yu, H., Huang, F., Li, Y., and Wang, H. Preference ranking optimization for human alignment. In AAAI , pp.\ 18990--18998. AAAI Press, 2024

    2024

  54. [62]

    M., and Christiano, P

    Stiennon, N., Ouyang, L., Wu, J., Ziegler, D. M., and Christiano, P. Learning to summarize from human feedback. 2020

    2020

  55. [63]

    Aligning large multimodal models with factually augmented rlhf

    Sun, Z., Shen, S., Cao, S., Liu, H., Li, C., Shen, Y., Gan, C., Gui, L.-Y., Wang, Y.-X., Yang, Y., et al. Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525, 2023

    2023 arXiv

  56. [64]

    Y., Xu, N., Zhang, S., Poon, H., and Chen, M

    Wang, F., Zhou, W., Huang, J. Y., Xu, N., Zhang, S., Poon, H., and Chen, M. mdpo: Conditional preference optimization for multimodal large language models. arXiv preprint arXiv:2406.11839, 2024 a

    2024 arXiv

  57. [65]

    Decision-making and planning method for autonomous vehicles based on motivation and risk assessment

    Wang, Y., Wang, C., Zhao, W., and Xu, C. Decision-making and planning method for autonomous vehicles based on motivation and risk assessment. IEEE Trans. Veh. Technol. , 70 0 (1): 0 107--120, 2021

    2021

  58. [66]

    Exploring the reasoning abilities of multimodal large language models (mllms): A comprehensive survey on emerging trends in multimodal reasoning

    Wang, Y., Chen, W., Han, X., Lin, X., Zhao, H., Liu, Y., Zhai, B., Yuan, J., You, Q., and Yang, H. Exploring the reasoning abilities of multimodal large language models (mllms): A comprehensive survey on emerging trends in multimodal reasoning. CoRR, abs/2401.06805, 2024 b

    2024 arXiv

  59. [67]

    Vary: Scaling up the vision vocabulary for large vision-language models

    Wei, H., Kong, L., Chen, J., Zhao, L., Ge, Z., Yang, J., Sun, J., Han, C., and Zhang, X. Vary: Scaling up the vision vocabulary for large vision-language models. arXiv preprint arXiv:2312.06109, 2023

    2023 arXiv

  60. [68]

    General ocr theory: Towards ocr-2.0 via a unified end-to-end model

    Wei, H., Liu, C., Chen, J., Wang, J., Kong, L., Xu, Y., Ge, Z., Zhao, L., Sun, J., Peng, Y., et al. General ocr theory: Towards ocr-2.0 via a unified end-to-end model. arXiv preprint arXiv:2409.01704, 2024 a

    2024 arXiv

  61. [69]

    Slow perception: Let's perceive geometric figures step-by-step

    Wei, H., Yin, Y., Li, Y., Wang, J., Zhao, L., Sun, J., Ge, Z., and Zhang, X. Slow perception: Let's perceive geometric figures step-by-step. arXiv preprint arXiv:2412.20631, 2024 b

    2024 arXiv

  62. [70]

    On the road with gpt-4v (ision): Explorations of utilizing visual-language model as autonomous driving agent

    Wen, L., Yang, X., Fu, D., Wang, X., Cai, P., Li, X., Tao, M., Li, Y., Linran, X., Shang, D., et al. On the road with gpt-4v (ision): Explorations of utilizing visual-language model as autonomous driving agent. In ICLR 2024 Workshop on Large Language Model (LLM) Agents, 2024

    2024

  63. [71]

    B., Reich, A., and Anderer, K

    W \" o lfel, M., Shirzad, M. B., Reich, A., and Anderer, K. Knowledge-based and generative-ai-driven pedagogical conversational agents: A comparative study of grice's cooperative principles and trust. Big Data Cogn. Comput., 8 0 (1): 0 2, 2024

    2024

  64. [72]

    A survey on multilingual large language models: Corpora, alignment, and bias

    Xu, Y., Hu, L., Zhao, J., Qiu, Z., Ye, Y., and Gu, H. A survey on multilingual large language models: Corpora, alignment, and bias. arXiv preprint arXiv:2404.00929, 2024

    2024 arXiv

  65. [73]

    Qwen2 technical report

    Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., Huang, F., Dong, G., Wei, H., Lin, H., Tang, J., Wang, J., Yang, J., Tu, J., Zhang, J., Ma, J., Yang, J., Xu, J., Zhou, J., Bai, J., He, J., Lin, J., Dang, K., Lu, K., Chen, K., Yang, K., Li, M....

    2024 arXiv

  66. [74]

    GPT can solve mathematical problems without a calculator

    Yang, Z., Ding, M., Lv, Q., Jiang, Z., He, Z., Guo, Y., Bai, J., and Tang, J. GPT can solve mathematical problems without a calculator. CoRR, abs/2309.03241, 2023 a

    2023 arXiv

  67. [75]

    The dawn of lmms: Preliminary explorations with gpt-4v(ision)

    Yang, Z., Li, L., Lin, K., Wang, J., Lin, C., Liu, Z., and Wang, L. The dawn of lmms: Preliminary explorations with gpt-4v(ision). CoRR, abs/2309.17421, 2023 b

    2023 arXiv

  68. [76]

    Merlin: Empowering multimodal llms with foresight minds

    Yu, E., Zhao, L., Wei, Y., Yang, J., Wu, D., Kong, L., Wei, H., Wang, T., Ge, Z., Zhang, X., et al. Merlin: Empowering multimodal llms with foresight minds. arXiv preprint arXiv:2312.00589, 2023

    2023 arXiv

  69. [77]

    C., and Berg, T

    Yu, L., Poirson, P., Yang, S., Berg, A. C., and Berg, T. L. Modeling context in referring expressions. In Springer International Publishing, 2016

    2016

  70. [78]

    Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback

    Yu, T., Yao, Y., Zhang, H., He, T., Han, Y., Cui, G., Hu, J., Liu, Z., Zheng, H.-T., Sun, M., et al. Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

    2024

  71. [79]

    Mm-vet: Evaluating large multimodal models for integrated capabilities

    Yu, W., Yang, Z., Li, L., Wang, J., Lin, K., Liu, Z., Wang, X., and Wang, L. Mm-vet: Evaluating large multimodal models for integrated capabilities. In ICML . OpenReview.net, 2024 b

    2024

  72. [80]

    MMMU: A massive multi-discipline multimodal understanding and reasoning benchmark for expert AGI

    Yue, X., Ni, Y., Zheng, T., Zhang, K., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., Wei, C., Yu, B., Yuan, R., Sun, R., Yin, M., Zheng, B., Yang, Z., Liu, Y., Huang, W., Sun, H., Su, Y., and Chen, W. MMMU: A massive multi-discipline multimodal understanding an...

    2024

  73. [81]

    Sigmoid loss for language image pre-training

    Zhai, X., Mustafa, B., Kolesnikov, A., and Beyer, L. Sigmoid loss for language image pre-training. In ICCV , pp.\ 11941--11952. IEEE , 2023

    2023

  74. [82]

    Cobra: Extending mamba to multi-modal large language model for efficient inference

    Zhao, H., Zhang, M., Zhao, W., Ding, P., Huang, S., and Wang, D. Cobra: Extending mamba to multi-modal large language model for efficient inference. CoRR, abs/2403.14520, 2024 a

    2024 arXiv

  75. [83]

    Assessing and understanding creativity in large language models

    Zhao, Y., Zhang, R., Li, W., Huang, D., Guo, J., Peng, S., Hao, Y., Wen, Y., Hu, X., Du, Z., Guo, Q., Li, L., and Chen, Y. Assessing and understanding creativity in large language models. CoRR, abs/2401.12491, 2024 b

    2024 arXiv

  76. [84]

    Beyond hallucinations: Enhancing lvlms through hallucination-aware direct preference optimization

    Zhao, Z., Wang, B., Ouyang, L., Dong, X., Wang, J., and He, C. Beyond hallucinations: Enhancing lvlms through hallucination-aware direct preference optimization. arXiv preprint arXiv:2311.16839, 2023

    2023 arXiv

  77. [85]

    Self-supervised visual preference alignment

    Zhu, K., Zhao, L., Ge, Z., and Zhang, X. Self-supervised visual preference alignment. arXiv preprint arXiv:2404.10501, 2024

    2024 arXiv

  78. [86]

    Deformable DETR: deformable transformers for end-to-end object detection

    Zhu, X., Su, W., Lu, L., Li, B., Wang, X., and Dai, J. Deformable DETR: deformable transformers for end-to-end object detection. In ICLR . OpenReview.net, 2021

    2021

  79. [87]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.