Pith. sign in

REVIEW 5 major objections 8 minor 35 references

Vela: Scalable Embeddings with Voice Large Language Models for Multimodal Retrieval

T0 review · 5 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Vela shows that an MLLM prompted to summarize either text or audio 'in one word' and fine-tuned on text pairs alone can outperform CLAP models in standard, long-text, and instruction-conditioned audio retrieval.

desk verdict Text-only contrastive training transfers to audio-text retrieval surprisingly well, but the paper's mechanism evidence is under-specified and the results need error bars before the strongest claims can be trusted. read the letter →

arxiv 2506.14445 v1 pith:XRXQKAHZ submitted 2025-06-17 cs.IR

Ruofan Hu , Yan Xia , Minjie Hong , Jieming Zhu , Bo Chen , Xiaoda Yang , Minghui Fang , Tao Jin This is my paper · ORCID
classification cs.IR
keywords audio-textretrievalmultimodalembeddingscontrastivelearningin-contextmodalitygaplong-textinstruction-conditionedMLLM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vela claims that a multimodal large language model can be converted into a universal audio-text embedding model without any paired audio-text training data. The method prompts the model to summarize either a text caption or an audio clip 'in one word,' adds a small set of in-context exemplars to bring the two modalities into the same embedding region, and then contrastively fine-tunes only the language model on text sentence pairs. On standard text-audio retrieval benchmarks Vela surpasses conventional CLAP models, and it retains accuracy on long-caption and instruction-conditioned retrieval where those models degrade. If the claim holds, paired audio-text collections would no longer be the bottleneck for building audio-language retrieval systems.

What carries the argument

The load-bearing mechanism is the 'in one word' representation: the MLLM is instructed to compress either a text caption or an audio clip into a single word, and the last token's hidden state becomes the embedding. In-context learning examples inject modality-alignment priors into this compression, counteracting what the paper calls lexical salience bias, where the model over-weights syntactically salient words (like 'mimicking') instead of semantically central ones (like 'meow'). Once audio and text share one space, the model is trained only on text pairs; the modality encoder and projector stay frozen while the LLM receives LoRA-based contrastive fine-tuning.

What would settle it

Measure the actual embedding distance between audio clips and their matching text captions on a held-out set of known correspondences, with and without the in-context exemplars. If the audio and text distributions do not substantially overlap after adding the exemplars, the claimed unification that makes text-only training transfer does not occur.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the modality gap in an MLLM can be closed by prompt engineering and in-context learning, so that single-modality contrastive training on text transfers to audio. Concretely, using Qwen2-Audio with the prompt 'Summarize the caption sentence/audio in one word' and 65 text plus 35 audio teaching exemplars produces embeddings whose audio and text distributions overlap. With the audio encoder and projector frozen and only the LLM fine-tuned with a SimCSE-style contrastive loss on roughly 273k text pairs from an NLI dataset, Vela achieves higher Recall@K than CLAP baselines on Clotho and AudioCaps and larger gains on the introduced Vela-long and Vela-conditional benchmarks.

Load-bearing premise

The hand-selected 65 text sentences and 35 audio clips, together with the 'in one word' prompt, are enough to make the frozen audio encoder's outputs land in the same embedding region as the LLM's text embeddings, so that text-only contrastive training transfers to audio.

Editorial extensions

If this is right

  • Text-pair-only training suffices to build competitive audio-text retrieval models, removing the need for large paired audio-caption datasets.
  • MLLM text encoders handle long captions naturally, so retrieval quality improves when captions are lengthened, reversing the degradation seen in CLAP models.
  • Instruction-conditioned retrieval, such as 'ignore the speech and find the violin,' becomes feasible without task-specific paired training data.
  • The same prompt-and-ICL recipe may extend to other modalities processed by the same MLLM backbone, yielding one universal embedding model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same 'in one word' unification might transfer to vision-language MLLMs, allowing a single frozen multimodal backbone to serve as a universal embedding model with text-only fine-tuning; the paper does not test this.
  • The relative importance of in-context learning over contrastive training, suggested by the ablation, implies that the exemplar set is the main lever; a natural experiment is to vary the exemplars randomly and measure retrieval change.
  • For production use, the main scaling question is whether larger MLLM backbones improve retrieval further, since Vela's gains come from the LLM's language understanding rather than from larger audio encoders.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. The paper introduces Vela, a framework that adapts the Qwen2-Audio multimodal LLM into a universal audio-text embedding model. The method consists of three components: (i) a crafted 'in one word' prompt that guides the model to compress a caption or audio input into a single-token representation, (ii) a small set of hand-selected in-context learning exemplars intended to close the modality gap between audio and text embeddings, and (iii) contrastive fine-tuning of only the LLM component on text pairs from the NLI dataset, with the audio encoder and projector frozen. The authors report Recall@K results on AudioCaps and Clotho, claiming that Vela outperforms CLAP-based baselines in standard text-audio retrieval, and they introduce two new LLM-generated benchmarks, Vela-long and Vela-conditional, to demonstrate advantages in long-text and instruction-conditioned retrieval. The paper concludes with an ablation study that attributes the main gain to in-context learning rather than contrastive training.

Significance. If the claims hold, the paper makes a useful contribution: it shows that a frozen multimodal LLM can be turned into a retrieval embedding model using only text-pair training, avoiding the need for large paired audio-text datasets. The standard-dataset comparisons provide a concrete sanity check, and the two new benchmarks target a real limitation of CLAP-style models on long and instruction-like queries. The idea of using prompt engineering and a tiny ICL exemplar set to unify modality representations is interesting and potentially transferable to other modalities. However, the central mechanism is supported almost entirely by an undefined 'Gap' metric, and all results are single runs without uncertainty quantification. The paper does not currently ship code, benchmark data, or the exact prompts used to construct the benchmarks, which limits reproducibility. The contribution is therefore promising but not yet fully verified.

major comments (5)
  1. [Section 4.3, Table 5] The 'Gap' score introduced in Table 5 is never defined. There is no formula, no specification of the embedding space, no aggregation rule, and no indication of which samples are used. Because Table 5 is the only quantitative evidence for the central claim that the crafted prompt and in-context examples close the modality gap, and because the ablation ordering (ICL contributes more than contrastive training) rests entirely on this metric, the mechanism is currently not checkable. Please define the Gap metric precisely and describe how it is computed for each configuration.
  2. [Section 3.2, Tables 2-4] All retrieval results are reported as single-run Recall@K values with no error bars, confidence intervals, or significance tests. Several margins over the strongest baselines are small (e.g., Clotho audio R@10 53.1 versus 52.6 for Microsoft-CLAP; AudioCaps text R@10 87.6 versus 86.4 for LAION-CLAP). To support the 'outperforms' headline and the ablation conclusions, the authors should report multiple seeds or bootstrap confidence intervals and state which differences are statistically significant.
  3. [Section 2.3, Tables 3-4] The two new benchmarks are constructed entirely by LLM generation with no human validation, no sample outputs, and no analysis of possible distributional bias. Since Vela uses an LLM text encoder, the long-text and conditional-instruction benchmarks may inadvertently favor LLM-based models over CLAP's shorter text encoders. Please provide example queries, human evaluation or post-editing statistics, and an analysis of whether the generated instructions preserve the intended semantics and do not introduce artifacts that make retrieval artificially easier for LLM-based methods.
  4. [Section 2.1 and Section 2.2] The central premise of the paper is that 65 text sentences and 35 audio ICL exemplars, together with the 'in one word' prompt, unify audio and text embeddings sufficiently that contrastive training on text pairs transfers to audio retrieval. This premise is supported only by the undefined Gap score in Table 5 and a PCA visualization. Please provide a concrete quantification of the modality gap in the actual embedding space (e.g., distribution distances before and after each component) and a sensitivity analysis over the number and selection of in-context exemplars.
  5. [Section 3.2, Eq. (1)] Training details needed for reproducibility are missing. The temperature parameter tau in Eq. (1) is not given, and the experimental setup omits the learning rate, number of epochs or steps, and optimization details beyond QLoRA hyperparameters. Additionally, the ICL selection thresholds (top 200, then final 65/35) are stated without any ablation or justification. Please report these values and analyze their effect on the results.
minor comments (8)
  1. [Section 4.2] The text says 'As shown in the Table 5' when referring to the Vela-Conditional results; this should be Table 4.
  2. [Table 3] The caption says 'text retrieval under Vela-Long' but the surrounding text describes long-text-to-audio retrieval; please clarify the retrieval direction and the candidate set used for evaluation.
  3. [Section 4.1] The abstract claims Vela 'outperforms traditional CLAP models,' while Section 4.1 says it achieves 'competitive performance'; please harmonize these statements, especially given the one lower R@10 on AudioCaps.
  4. [Section 2.1] The selection of in-context exemplars uses GPT-4 for summary generation and Gemini for semantic fidelity scoring, but the prompts used for these LLM calls are not provided. Please include them for reproducibility.
  5. [Section 2.3] The benchmark construction mentions 'FSD50 and Wavcaps' in Section 3.1, while Section 2.3 uses 'FSD50K' and 'WavCaps'; please make the dataset names consistent.
  6. [Figure 1] The PCA visualization lacks details on how many samples were projected, which embeddings were used, and whether the projection is from the same model configuration as Table 5. Please expand the caption.
  7. [References] Reference [14] is cited for in-context learning but points to DefSent, a sentence-embedding paper; please cite a standard in-context learning reference such as Brown et al. (2020) or a relevant survey.
  8. [General] The code and benchmark data are only promised as 'soon available.' Given that the paper introduces new benchmarks, releasing the benchmark data and evaluation scripts is important for independent verification.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Vela's central retrieval claims rest on external standard benchmarks and independent baselines, and no load-bearing step reduces by construction to its inputs.

full rationale

The paper's headline claim is that Vela outperforms traditional CLAP models on text-audio retrieval. This is measured on AudioCaps and Clotho, both external standard datasets, against independent published CLAP baselines (WavCaps, Auto-ACD, AudioSetCaps, LAION-CLAP, Microsoft-CLAP). The training objective uses only text pairs from NLI, and the test sets are not used in training or prompt selection in any way stated in the paper, so the retrieval numbers are not fitted outputs. The 'in one word' prompt is adopted from E5-V (ref [11]), a non-overlapping prior work, and the modality-gap discussion cites external CLIP/audio studies; there is no load-bearing self-citation chain. The Vela-long and Vela-conditional benchmarks are generated with GPT-4 from Clotho, FSD50K, and WavCaps; these are fixed test artifacts, not functions of Vela's parameters, so they introduce a possible LLM-favorable bias but do not make the evaluation circular. Two caveats are worth flagging but do not constitute circularity: the 'Gap' metric in Table 5 is never defined, so the claimed modality-gap reduction is unverifiable, and all Recall@K numbers are reported as single runs without error bars, so margins such as Clotho R@10 53.1 vs 52.6 may not be robust. These are evidence-quality limitations, not cases where a prediction is equivalent to its inputs by construction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central method introduces no new physical or architectural entities, but it depends on several domain assumptions about LLM prompt behavior and transfer learning, plus several hand-chosen hyperparameters. The undefined 'Gap' metric is a notable ad hoc construct.

free parameters (5)
  • Number of in-context examples = 65 text sentences and 35 audio clips
    Chosen by hand to balance input length with generation performance, as stated in Section 2.1.
  • Top-200 selection threshold = 200
    Retained the top 200 examples after Gemini scoring, a hand-chosen cutoff in Section 2.1.
  • Prompt wording = Summarize the caption sentence in one word: / Summarize the caption of the audio in one word:
    Selected 'through extensive experimentation' according to Section 2.1, but no experiments are shown.
  • LoRA hyperparameters = r=32, alpha=8, dropout=0.05
    Standard choices stated in Section 3.2.
  • Batch size = 168
    Stated in Section 3.2 as a hardware constraint, not tuned for the central claim.
assumptions (4)
  • domain assumption The 'in one word' prompt induces the MLLM to produce a last-token hidden state that summarizes the semantic content of any modality (text or audio).
    The method relies on this to generate embeddings; the paper cites E5-V [11] for vision and assumes it extends to audio via Qwen2-Audio.
  • ad hoc to paper The selected in-context exemplars reduce the modality gap sufficiently to make audio and text embeddings comparable.
    The number and content of exemplars is tuned by the authors using LLM scoring; the generalization to unseen audio is assumed in Section 2.1.
  • domain assumption Contrastive training on text-only pairs transfers to audio-text retrieval.
    This is the core premise of single-modality training; the paper provides empirical support but no theoretical justification.
  • ad hoc to paper The 'Gap' metric (difference between audio and text embedding distributions) is a valid proxy for retrieval alignment.
    The metric is introduced in Table 5 without a formal definition, so its validity cannot be assessed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vela: Scalable Embeddings with Voice Large Language Models for Multimodal Retrieval." pith.science (2026). https://pith.science/paper/XRXQKAHZ

@misc{pith2026250614445,
  author       = {Pith},
  title        = {Pith review of: Vela: Scalable Embeddings with Voice Large Language Models for Multimodal Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XRXQKAHZ}},
  note         = {Machine review of arXiv:2506.14445}
}
read the original abstract

Multimodal large language models (MLLMs) have seen substantial progress in recent years. However, their ability to represent multimodal information in the acoustic domain remains underexplored. In this work, we introduce Vela, a novel framework designed to adapt MLLMs for the generation of universal multimodal embeddings. By leveraging MLLMs with specially crafted prompts and selected in-context learning examples, Vela effectively bridges the modality gap across various modalities. We then propose a single-modality training approach, where the model is trained exclusively on text pairs. Our experiments show that Vela outperforms traditional CLAP models in standard text-audio retrieval tasks. Furthermore, we introduce new benchmarks that expose CLAP models' limitations in handling long texts and complex retrieval tasks. In contrast, Vela, by harnessing the capabilities of MLLMs, demonstrates robust performance in these scenarios. Our code will soon be available.

Figures

Figures reproduced from arXiv: 2506.14445 by the authors.

Figure 1
Figure 1. Distribution of audio embeddings and text embed￾dings from the last token in MLLMs. We compare the results with the ordinary prompt and with our representation method using the crafted prompt and in context learning design. vanced text encoders and external knowledge, enabling them to handle more complex and extended contextual sequences beyond the input constraints of CLAP. Third, MLLMs trans￾fer their grounding an… view at source ↗
Figure 2
Figure 2. The whole framework of Vela. (1) In the training stage, by unifying multimodal representations into the same embedding space, Vela improves multimodal embeddings using only contrastive learning on text pairs. (2) In the inference stage, we freeze the modality encoder and projector in MLLM and replace the LLM with the pertaining one. With our representation method, embeddings are extracted. (3) Through the representa… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 20 canonical work pages

  1. [1]

    Introduction With the rapid advancement of MLLMs, there is an increasing need for embedding models that facilitate cross-modal retrieval. In the acoustic domain, while Contrastive Language Audio Pre- training (CLAP) models [1, 2, 3, 4, 5] have demonstrated im- pressive performance in text-audio retrieval by aligning acous- tic and linguistic representatio...

    arXiv 2025

  2. [2]

    Contrastive Pretraining on Only Text Pairs Person mimicking a cat

    work page

  3. [3]

    Transfer Learned Representation with In Context Learning Summarize the caption sentence in one word: Summarize the caption of the audio in one word: LLM Audio Encoder Demonstration Set MLLM Text Modality Prompt Audio Modality Prompt

    work page

  4. [4]

    (1) In thetrainingstage, by unifying multimodal representations into the same embedding space, Vela improves multimodal embeddings usingonly contrastive learning on text pairs

    Modality Embedding Visualization After Vela Method Brake Rumble Beep Car Rumble Brake Beep Car Sound of Car Sound of Car Δ Modality Gap Decrement Process Audio Embedding Text Embedding 𝑇𝑇𝑁𝑁𝑇𝑇2 𝑇𝑇3 1 1 1 1 1 𝑇𝑇1 𝑇𝑇1 + 𝑇𝑇1 − 𝑇𝑇𝑁𝑁 + 𝑇𝑇𝑁𝑁 − … … Figure 2:The whole framework of Vela. (1) In thetrainingstage, by unifying multimodal representations into the same ...

    work page

  5. [5]

    in one word

    Method 2.1. Unifying Multimodal Embeddings Inspired by previous work [11], we adopt the “in one word” method with MLLMs to unify multimodal embeddings. The core idea behind is to guide MLLMs in compressing the infor- mation, and then leveraging the embeddings of the last token as the representation. Through extensive experimentation, we find that using pr...

    work page

  6. [6]

    Datasets For the training data, we use NLI [18], which contains approx- imately 273k sentence pairs

    Experiment 3.1. Datasets For the training data, we use NLI [18], which contains approx- imately 273k sentence pairs. For the test data, we conduct text- audio retrieval on two open-source datasets, AudioCaps [21] and Clotho [19]. To show the limitations of the CLAP structure in long-text and fine-grained retrieval tasks with instructions, we employ Vela-c...

    work page

  7. [7]

    in one word

    Result 4.1. Text-Audio Retrieval Text-audio retrieval involves searching for an audio clip or a caption in a database based on a query from another modal- ity. In our experiments, we selected several strong baselines based on the CLAP paradigm, including Microsoft-CLAP [1], LAION-CLAP [2], WavCaps [3], AudioSetCaps [4] and Audio- ACD [5]. We select thezer...

    work page

  8. [8]

    Conclusion In this work, we propose Vela, an MLLM-based universal multi- modal framework capable of representing both acoustic and lin- guistic inputs. Technically, it employs a prompt-based represen- tation design and an in-context learning approach to unify mul- timodal representations into a shared embedding space with- out fine-tuning. Through single-...

    work page

Show all 35 references
  1. [9]

    Pioneer” and “Lead- ing Goose

    Acknowledgements This work was supported by the “Pioneer” and “Lead- ing Goose” R&D Program of Zhejiang under (Grant No. 2025C02110), Public Welfare Research Program of Ningbo un- der (Grant No. 2024S062), and Yongjiang Talent Project of Ningbo under (Grant No. 2024A-161-G)

  2. [10]

    Natural language supervision for general-purpose audio representations,

    B. Elizalde, S. Deshmukh, and H. Wang, “Natural language supervision for general-purpose audio representations,” 2023. [Online]. Available: https://arxiv.org/abs/2309.05767

    2023 arXiv

  3. [11]

    Large-scale contrastive language-audio pretrain- ing with feature fusion and keyword-to-caption augmentation,

    Y . Wu*, K. Chen*, T. Zhang*, Y . Hui*, T. Berg-Kirkpatrick, and S. Dubnov, “Large-scale contrastive language-audio pretrain- ing with feature fusion and keyword-to-caption augmentation,” in IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP, 2023

    2023

  4. [12]

    Wavcaps: A chatgpt-assisted weakly- labelled audio captioning dataset for audio-language multimodal research,

    X. Mei, C. Meng, H. Liu, Q. Kong, T. Ko, C. Zhao, M. D. Plumb- ley, Y . Zou, and W. Wang, “Wavcaps: A chatgpt-assisted weakly- labelled audio captioning dataset for audio-language multimodal research,”IEEE/ACM Transactions on Audio, Speech, and Lan- guage Processing, 2024

    2024

  5. [13]

    Audiosetcaps: An enriched audio-caption dataset using automated generation pipeline with large audio and language models,

    J. Bai, H. Liu, M. Wang, D. Shi, W. Wang, M. D. Plumbley, W.- S. Gan, and J. Chen, “Audiosetcaps: An enriched audio-caption dataset using automated generation pipeline with large audio and language models,”arXiv preprint arXiv:2411.18953, 2024

    2024 arXiv

  6. [14]

    Auto-acd: A large-scale dataset for audio-language representation learning,

    L. Sun, X. Xu, M. Wu, and W. Xie, “Auto-acd: A large-scale dataset for audio-language representation learning,” inACM Mul- timedia 2024

    2024

  7. [15]

    Grounding language models for visual entity recog- nition,

    Z. Xiao, M. Gong, P. Cascante-Bonilla, X. Zhang, J. Wu, and V . Ordonez, “Grounding language models for visual entity recog- nition,” inEuropean Conference on Computer Vision. Springer, 2025, pp. 393–411

    2025

  8. [16]

    Improving clip training with language rewrites,

    L. Fan, D. Krishnan, P. Isola, D. Katabi, and Y . Tian, “Improving clip training with language rewrites,”Advances in Neural Infor- mation Processing Systems, vol. 36, pp. 35 544–35 575, 2023

    2023

  9. [17]

    Mate: Meet at the embedding–connecting images with long texts,

    Y . K. Jang, J. Kang, Y . J. Lee, and D. Kim, “Mate: Meet at the embedding–connecting images with long texts,”arXiv preprint arXiv:2407.09541, 2024

    2024 arXiv

  10. [18]

    Metamorph: Multimodal un- derstanding and generation via instruction tuning,

    S. Tong, D. Fan, J. Zhu, Y . Xiong, X. Chen, K. Sinha, M. Rab- bat, Y . LeCun, S. Xie, and Z. Liu, “Metamorph: Multimodal un- derstanding and generation via instruction tuning,”arXiv preprint arXiv:2412.14164, 2024

    2024 arXiv

  11. [19]

    Its not a modality gap: Characterizing and addressing the contrastive gap,

    A. Fahim, A. Murphy, and A. Fyshe, “Its not a modality gap: Characterizing and addressing the contrastive gap,”arXiv preprint arXiv:2405.18570, 2024

    2024 arXiv

  12. [20]

    E5-v: Universal embed- dings with multimodal large language models,

    T. Jiang, M. Song, Z. Zhang, H. Huang, W. Deng, F. Sun, Q. Zhang, D. Wang, and F. Zhuang, “E5-v: Universal embed- dings with multimodal large language models,”arXiv preprint arXiv:2407.12580, 2024

    2024 arXiv

  13. [21]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agar- wal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning. PMLR, 2021, pp. 8748–8763

    2021

  14. [22]

    Mind the gap: Understanding the modality gap in multi-modal con- trastive representation learning,

    V . W. Liang, Y . Zhang, Y . Kwon, S. Yeung, and J. Y . Zou, “Mind the gap: Understanding the modality gap in multi-modal con- trastive representation learning,”Advances in Neural Information Processing Systems, vol. 35, pp. 17 612–17 625, 2022

    2022

  15. [23]

    Defsent: Sen- tence embeddings using definition sentences,

    H. Tsukagoshi, R. Sasano, and K. Takeda, “Defsent: Sen- tence embeddings using definition sentences,”arXiv preprint arXiv:2105.04339, 2021

    2021 arXiv

  16. [24]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Ale- man, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkatet al., “Gpt-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  17. [25]

    Gemini: a family of highly capable multimodal models,

    G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millicanet al., “Gemini: a family of highly capable multimodal models,”arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  18. [26]

    Breaking the length barrier: Llm-enhanced ctr pre- diction in long textual user behaviors,

    B. Geng, Z. Huan, X. Zhang, Y . He, L. Zhang, F. Yuan, J. Zhou, and L. Mo, “Breaking the length barrier: Llm-enhanced ctr pre- diction in long textual user behaviors,” inProceedings of the 47th International ACM SIGIR Conference on Research and Develop- ment in Information Ret...

    2024

  19. [27]

    Simcse: Simple contrastive learn- ing of sentence embeddings,

    T. Gao, X. Yao, and D. Chen, “Simcse: Simple contrastive learn- ing of sentence embeddings,”arXiv preprint arXiv:2104.08821, 2021

    2021 arXiv

  20. [28]

    Clotho: An audio cap- tioning dataset,

    K. Drossos, S. Lipping, and T. Virtanen, “Clotho: An audio cap- tioning dataset,” inICASSP 2020-2020 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 736–740

    2020

  21. [29]

    Fsd50k: an open dataset of human-labeled sound events,

    E. Fonseca, X. Favory, J. Pons, F. Font, and X. Serra, “Fsd50k: an open dataset of human-labeled sound events,”IEEE/ACM Trans- actions on Audio, Speech, and Language Processing, vol. 30, pp. 829–852, 2021

    2021

  22. [30]

    Audiocaps: Generat- ing captions for audios in the wild,

    C. D. Kim, B. Kim, H. Lee, and G. Kim, “Audiocaps: Generat- ing captions for audios in the wild,” inProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol- ume 1 (Long and Short Papers...

    2019

  23. [31]

    Qwen2-audio technical report,

    Y . Chu, J. Xu, Q. Yang, H. Wei, X. Wei, Z. Guo, Y . Leng, Y . Lv, J. He, J. Linet al., “Qwen2-audio technical report,”arXiv preprint arXiv:2407.10759, 2024

    2024 arXiv

  24. [32]

    Qwen technical report,

    J. Bai, S. Bai, Y . Chu, Z. Cui, K. Dang, X. Deng, Y . Fan, W. Ge, Y . Han, F. Huanget al., “Qwen technical report,”arXiv preprint arXiv:2309.16609, 2023

    2023 arXiv

  25. [33]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” inInternational conference on machine learning. PMLR, 2023, pp. 28 492–28 518

    2023

  26. [34]

    Qlora: Efficient finetuning of quantized llms,

    T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “Qlora: Efficient finetuning of quantized llms,”Advances in Neu- ral Information Processing Systems, vol. 36, 2024

    2024

  27. [35]

    From matching to generation: A survey on generative informa- tion retrieval,

    X. Li, J. Jin, Y . Zhou, Y . Zhang, P. Zhang, Y . Zhu, and Z. Dou, “From matching to generation: A survey on generative informa- tion retrieval,”arXiv preprint arXiv:2404.14851, 2024

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.