Pith. sign in

REVIEW 3 major objections 5 minor 123 references

Testing chatbots on the creation of encoders for audio conditioned image generation

T0 review · 3 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Four of five chatbots produced valid audio encoders for Stable Diffusion 1.5, but none could replace its text encoder after supervised training on 2.24 million observations.

desk verdict A well-documented negative result about audio-conditioned image generation, but the inference/training mismatch and missing baselines mean the claimed 'chatbot coding gap' is not actually established. read the letter →

arxiv 2509.09717 v1 pith:FWPIXZ6G submitted 2025-09-09 cs.SD cs.LGeess.AS

Jorge E. León , Miguel Carrasco This is my paper
classification cs.SDcs.LGeess.AS
keywords audio-to-imagegenerationaudioencoderschatbotcodeCLIPembeddingalignmentStableDiffusioncontrastivelearningmodelsmultimodal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether state-of-the-art chatbots can design a neural audio encoder that replaces the text encoder in Stable Diffusion 1.5, enabling image generation directly from sound. Four of the five tested chatbots produced valid encoder designs; after training each on 2.24 million audio-image-text observations with a CLIP-style contrastive loss, none produced embeddings that aligned reliably with the original CLIP text encoder. Quantitative tests showed negative R² values across the board, and audio-only generated images were mostly incoherent, although Grok's encoder produced more recognizable images when combined with text. The authors interpret this as evidence of a remaining coding gap in chatbots and point to a shared architectural bias: all proposals converged on transformer-based encoder stacks. The result matters because it marks audio-to-image conditioning as still unsolved and offers a concrete, reusable stress test for chatbot code-generation ability.

What carries the argument

The load-bearing object is the audio encoder itself, trained to imitate CLIP's text and image embeddings through the TCEOCS loss, a symmetric cross-entropy over matrices of cosine similarities between audio and text, and audio and image projections. The encoder receives raw waveform samples and must output 77×768 matrices, matching the shape of Stable Diffusion 1.5's text-encoder output. The shared prompt, code scaffold, fixed hyperparameters, and training budget isolate the architectural choice as the only variable controlled by the chatbots. Generated images are produced by swapping the audio encoder into the Stable Diffusion 1.5 denoising loop, optionally averaging its guidance embedding

What would settle it

Train a well-established, human-designed audio encoder from the literature under the exact same dataset, 32-epoch budget, loss, and evaluation protocol. If that baseline also fails to align with the CLIP text encoder, the study's negative result is explained by task difficulty or undertraining rather than by the chatbots' architecture proposals; if it succeeds, the chatbots' architectures are directly implicated.

Watch

Extended reading notes

Core claim

The paper reports that, under a shared protocol, five chatbots were asked to write an audio encoder that maps 1-second, 16 kHz, monophonic audio to the 77×768 matrices produced by Stable Diffusion 1.5's CLIP text encoder. Four returned valid architectures and one did not. Each valid encoder was trained identically on over 2.24 million context-linked audio-image-text observations using a symmetric cross-entropy loss over cosine similarities, then evaluated on held-out metrics and on generated images. The central finding is that none of the trained audio encoders is a good replacement for the original text encoder: all average R² values were negative, audio-only generations were mostly incoher

Load-bearing premise

The paper treats the negative result as a chatbot coding gap, but this assumes the failure comes from the proposed architectures rather than from the fixed task setup—one-second audio, noisy generated captions, 32 training epochs, and a contrastive-only loss—especially since the authors' own human-designed encoder fails under the same conditions.

Editorial extensions

If this is right

  • Direct substitution of a trained-from-scratch audio encoder for the frozen CLIP text encoder does not work under the tested conditions: the audio embeddings do not land in the text-embedding space after 32 epochs of contrastive training.
  • Embedding-similarity metrics and image-generation quality are not interchangeable: Gemini had the best quantitative scores, while Grok produced the more coherent images, so reliable evaluation requires both.
  • Current chatbots show a shared architectural bias—every valid proposal was a transformer encoder stack, with two proposals nearly identical—suggesting limited architectural creativity rather than task-driven exploration.
  • A cleaner dataset, more training epochs, and possibly longer-training effects could change the outcome; the authors explicitly leave these as open questions for future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The experiment does not yet separate 'chatbot architecture designs are bad' from 'this alignment task is extremely hard under the fixed budget,' because the authors' own human-designed encoder also failed; a successful human baseline trained under identical conditions would be needed.
  • The shared transformer bias may not be a chatbot-specific flaw: models trained on similar coding corpora might converge to the same familiar pattern, so a more informative test would vary the loss function, input representation, or architectural constraints.
  • A natural next step is to keep the diffusion denoiser trainable or add auxiliary alignment losses, since forcing audio into a frozen text-embedding space with a single global contrastive loss may be the bottleneck rather than the encoder architecture.
  • If this protocol is reused as a benchmark, public exposure may let future chatbots memorize or approximate these solutions, eroding the test's ability to probe genuine creativity and reasoning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper asks whether five publicly available chatbots can design a PyTorch audio encoder that replaces the CLIP ViT-L/14 text encoder of Stable Diffusion 1.5. The authors give ChatGPT o3-mini, Claude 3.7 Sonnet, DeepSeek-R1, Gemini 2.5 Pro, and Grok 3 a shared prompt with a fixed input/output specification (1s, 16 kHz mono audio to a 77×768 matrix), a symmetric cross-entropy objective over cosine similarities (Eq. (2)), and a fixed training protocol. Four chatbots produce valid architectures; Claude does not. All four, plus a manually designed encoder by the authors, are trained on 2,240,231 audio–image–text triples for 32 epochs and evaluated with TCEOCS, μ(MSE), μ(R²), inference time, generated-image quality, and a qualitative element-presence breakdown. None of the encoders yields coherent or semantically aligned images. Gemini has the best aggregate metrics, while Grok produces the most coherent images when mixed with the original text encoder. The paper concludes that chatbots exhibit a shared architectural bias and that a coding gap remains.

Significance. The study is a well-structured empirical probe with several genuine strengths: a shared prompt and training protocol, a held-out test set of 23,524 items, multiple complementary metrics, qualitative element-level evaluation, a public demo, and reliance on a companion dataset. The negative result is reported transparently, including the poor performance of the authors' own encoder. However, the interpretation as a chatbot coding gap rests on two load-bearing assumptions that the paper does not establish: that the inference-time raw outputs are comparable to the training-time normalized projections, and that a competent human-designed encoder trained under the same conditions would succeed. Because the authors' own encoder also fails and no calibrated or positive-control baseline is provided, the evidence supports 'this task setup is hard and the tested architectures fail' more strongly than 'chatbots are poor architecture designers.'

major comments (3)
  1. [§3.2.1, Eq. (2); §3.2.2, Fig. 5; Table 4] The training objective is scale-invariant: Eq. (2) is computed on normalized M×768 projections, so a model can minimize it while its raw 77×768 outputs are arbitrarily far from the CLIP text-encoder distribution. At inference, however, the raw outputs are fed directly to the denoising U-Net (Fig. 5) with no calibration or learned projection. Table 4 shows the consequence: every encoder has astronomically negative raw-output R² values (Ours −1.84E16, ChatGPT −5.71E11, DeepSeek −3.27E11, Gemini −3.17E11, Grok −3.36E11). The paper notes the missing normalizer for Ours, but the same issue applies to all encoders. The failure to generate coherent images is therefore consistent with uncalibrated conditioning rather than architectural inadequacy. To support the stated conclusion, the authors need either an inference-time calibration step (e.g., matching the mean/variance of CLIP text embeddings
  2. [§4, Table 3, Figs. 11–12] The paper lacks a positive control. The authors' own human-designed encoder is trained under identical conditions and also fails, producing 'colorful and indistinguishable noise' (Figs. 11–12). Without a known successful encoder trained under the same data, loss, input length, and epoch budget, the experiment cannot distinguish 'chatbots are bad at this coding task' from 'this alignment task is very hard, undertrained, or hampered by noisy captions.' The abstract's claim that the findings 'reveal a shared architectural bias across chatbots and underscore the remaining coding gap' overreaches. The safest conclusion supported by the data is that none of the tested architectures, including the authors' manual one, works in this setup; the chatbot-specific conclusion requires a successful baseline or an explicit demonstration that the task is feasible under the same conditions.
  3. [Table 3; Table 4; §3.2.2] The headline TCEOCS numbers are not calibrated against a random baseline. For text alignment, the validation TCEOCSt before training is 16.47290–16.47362 and after training 16.47286–16.47289, i.e., essentially unchanged; the test TCEOCSt is about 20.13 for all encoders. Without reporting the TCEOCS of random embeddings, an untrained encoder, or a shuffled-label model, these values are hard to interpret as 'near random' or as evidence of specific failure modes. The reported μ(R²) values are already strongly negative, so this does not change the overall negative verdict, but the TCEOCS framing should be supported by a chance-level reference or omitted.
minor comments (5)
  1. [Table 4 caption] The caption says 'Same subindexes as Table 4' but should refer to Table 3.
  2. [Eq. (2)] The loss has four cross-entropy terms but is divided by 6, described only as a scale factor from [34]. Please explain why 6 rather than 4, or clarify that this is an arbitrary hyperparameter.
  3. [Table 4] The entry for σ(R²)rt for Ours is marked 'invalid' because the value was too close to ±∞. Please report the actual computation and why it is not representable; this is likely a consequence of the raw-output scale issue discussed in the major comments.
  4. [§3.2.2 and Table 3] All metrics come from a single training run and a single prompt attempt per chatbot. The 'Gemini best metrics' vs. 'Grok best images' ranking may be unstable; adding multiple runs or at least acknowledging the lack of variance information would strengthen the comparison.
  5. [§4] The statement that R² ≥ 0.4 is 'usually considered slightly positive' is not standard for a coefficient of determination in regression; consider rephrasing or citing a regression-specific convention.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central negative result is an empirical finding on held-out data against an external CLIP benchmark, not a construction of the training objective.

full rationale

The paper's derivation chain is: (i) chatbots propose audio encoder architectures; (ii) the encoders are trained with the TCEOCS loss in Eq. (2) to align normalized projections with the CLIP text and image encoders; (iii) the trained encoders are evaluated on a held-out test set using several metrics and by generating images with Stable Diffusion 1.5. The evaluation targets are the same CLIP embeddings used for training, but the test split is not used for fitting, so the reported failure to match the text encoder is an empirical outcome rather than a logical consequence of the training objective. No fitted parameter is renamed as a prediction, and the paper does not invoke a uniqueness theorem or rely on a self-citation to justify its central claim. The only self-citation is the companion dataset [53], which is a data resource, not a load-bearing theoretical premise. The paper itself flags the raw-output/normalizer mismatch for its own encoder, acknowledging that the projection includes normalization while raw outputs are uncalibrated; this is a validity concern about the raw-output R2 metric, not circularity. The image-generation evidence is likewise an independent empirical test. Overall, the derivation is self-contained with respect to an external CLIP benchmark, so no significant circularity is present.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new theoretical or physical entities are introduced. The ledger lists the experimental choices and domain assumptions that the central negative result depends on.

free parameters (3)
  • training_epochs = 32
    Fixed budget for every encoder; if the models are undertrained, the comparison conflates architecture quality with training budget.
  • audio_duration_seconds = 1
    Deliberately short input meant to challenge chatbots; determines how much semantic information the encoder can access.
  • guidance_scale = 7.5 (audio-only) / 10 (with image)
    Generation-time parameter chosen by the authors; affects qualitative image outcomes.
assumptions (4)
  • domain assumption Imitating CLIP text embeddings with an audio encoder is a viable route to audio-conditioned image generation with Stable Diffusion 1.5.
    The entire benchmark evaluates encoders by how well they replace the CLIP text encoder; if this premise is false, all encoders could fail regardless of architecture.
  • domain assumption A 1-second, 16 kHz monophonic audio clip carries enough information to predict the CLIP text embedding of the associated caption.
    Implicit in the choice of input format; if audio is too ambiguous, no encoder can align well.
  • standard math The symmetric cross-entropy loss on cosine similarities, with the 1/6 scale from AudioCLIP, is an appropriate objective for measuring semantic alignment.
    The loss is a standard CLIP-style contrastive objective, but the specific scale factor is taken from [34].
  • ad hoc to paper 32 epochs of training is sufficient to fairly compare the proposed architectures.
    The authors repeatedly suspect undertraining for the largest models, yet use the 32-epoch results to conclude that all chatbot designs fail.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Testing chatbots on the creation of encoders for audio conditioned image generation." pith.science (2026). https://pith.science/paper/FWPIXZ6G

@misc{pith2026250909717,
  author       = {Pith},
  title        = {Pith review of: Testing chatbots on the creation of encoders for audio conditioned image generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FWPIXZ6G}},
  note         = {Machine review of arXiv:2509.09717}
}
read the original abstract

On one hand, recent advances in chatbots has led to a rising popularity in using these models for coding tasks. On the other hand, modern generative image models primarily rely on text encoders to translate semantic concepts into visual representations, even when there is clear evidence that audio can be employed as input as well. Given the previous, in this work, we explore whether state-of-the-art conversational agents can design effective audio encoders to replace the CLIP text encoder from Stable Diffusion 1.5, enabling image synthesis directly from sound. We prompted five publicly available chatbots to propose neural architectures to work as these audio encoders, with a set of well-explained shared conditions. Each valid suggested encoder was trained on over two million context related audio-image-text observations, and evaluated on held-out validation and test sets using various metrics, together with a qualitative analysis of their generated images. Although almost all chatbots generated valid model designs, none achieved satisfactory results, indicating that their audio embeddings failed to align reliably with those of the original text encoder. Among the proposals, the Gemini audio encoder showed the best quantitative metrics, while the Grok audio encoder produced more coherent images (particularly, when paired with the text encoder). Our findings reveal a shared architectural bias across chatbots and underscore the remaining coding gap that needs to be bridged in future versions of these models. We also created a public demo so everyone could study and try out these audio encoders. Finally, we propose research questions that should be tackled in the future, and encourage other researchers to perform more focused and highly specialized tasks like this one, so the respective chatbots cannot make use of well-known solutions and their creativity/reasoning is fully tested.

Figures

Figures reproduced from arXiv: 2509.09717 by the authors.

Figure 1
Figure 1. Text-to-image generation example, created with Stable Diffusion 1.5. Text-to [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Types of data modalities. each.2 An example of the use of multiple data modalities tends to be seen in image-to-image generation, where an image is taken as a reference to generate a new image, since the input image is usually accompanied by a text or a label to better condition/guide the final result [69]. In contrast, audio conditioned image-to-image generation has not been explored as much as text conditioned ima… view at source ↗
Figure 3
Figure 3. Summary of our methodology. 1. Generation of the audio encoders: This initial phase involves the design of all the audio encoders to be tested and compared (some chatbots may be unable to come up with a suitable architecture, and thus they would be ruled out of the following tests). 2. Analysis & comparison of the metrics: In this step, we measure and compare several metrics based on the encodings obtained from each… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Inner workings of Stable Diffusion 1.5. As we can see, both an input text and [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: The main concept behind our experiments. The only difference with Figure 4 is [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Workflow for a chatbot to create an audio encoder. This is repeated on each [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Example of generation based on multiple encoders for guidance. Consider that [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Methods of generation we intend to use for each audio encoder. First we will gen [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Benchmark sample of generations for the input text of “Children talking and [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: Benchmark sample of generations for the input text of “The interior of a coffee [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]
Figure 11
Figure 11. Figure 11: Sample of generations for the input image shown in Figure 9, the text of “Children [PITH_FULL_IMAGE:figures/full_fig_p023_11.png]
Figure 12
Figure 12. Figure 12: Sample of generations for the input image shown in Figure 10, the text of “The [PITH_FULL_IMAGE:figures/full_fig_p025_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

123 extracted references · 42 canonical work pages

  1. [1]

    Freesound.https://freesound.org/, 2025

    2025

  2. [2]

    Pexels.https://pexels.com/, 2025

    2025

  3. [3]

    Picryl.https://picryl.com/, 2025

    2025

  4. [4]

    Pixabay.https://pixabay.com/, 2025

    2025

  5. [5]

    Rawpixel.https://rawpixel.com/, 2025

    2025

  6. [6]

    Denk, Zal´ an Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, Matt Sharifi, Neil Zeghidour, and Christian Frank

    Andrea Agostinelli, Timo I. Denk, Zal´ an Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, Matt Sharifi, Neil Zeghidour, and Christian Frank. MusicLM: Generating Music From Text. ArXiv, 2301.11325, 2023

    arXiv 2023

  7. [7]

    Don’t Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering

    Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Aniruddha Kembhavi. Don’t Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering. InPro- ceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, pages 4971–4980, 2018

    2018

  8. [8]

    Mistral Models, 2024

    Mistral AI. Mistral Models, 2024

    2024

Show all 123 references
  1. [9]

    Transcripter- Generation of the transcript from audio to text using Deep Learning.International Journal of Computer Sciences and Engineering, 7(1):770–773, 2019

    Fatima Ansari, Ramsakal Gupta, Uday Singh, and Fahimur Shaikh. Transcripter- Generation of the transcript from audio to text using Deep Learning.International Journal of Computer Sciences and Engineering, 7(1):770–773, 2019

    2019

  2. [10]

    The Claude 3 Model Family: Opus, Sonnet, Haiku, 2024

    Anthropic. The Claude 3 Model Family: Opus, Sonnet, Haiku, 2024

    2024

  3. [11]

    Claude 3.7 Sonnet and Claude Code, 2025

    Anthropic. Claude 3.7 Sonnet and Claude Code, 2025

    2025

  4. [12]

    AudioSetCaps: An Enriched Audio-Caption Dataset using Auto- mated Generation Pipeline with Large Audio and Language Models

    Jisheng Bai, Haohe Liu, Mou Wang, Dongyuan Shi, Mark Plumbley, Woon-Seng Gan, and Jianfeng Chen. AudioSetCaps: An Enriched Audio-Caption Dataset using Auto- mated Generation Pipeline with Large Audio and Language Models. InAudio Imag- ination: NeurIPS 2024 Workshop AI-Driven S...

    2024

  5. [13]

    Are Mod- els Biased on Text without Gender-related Language? InProceedings of the 12th International Conference on Learning Representations, 2024

    Catarina G Bel´ em, Preethi Seshadri, Yasaman Razeghi, and Sameer Singh. Are Mod- els Biased on Text without Gender-related Language? InProceedings of the 12th International Conference on Learning Representations, 2024

    2024

  6. [14]

    Ballester

    Marcelo Bertalm ´ ıo, Guillermo Sapiro, Vicent Caselles, and C. Ballester. Image in- painting. InProceedings of the 27th Internationl Conference on Computer Graphics and Interactive Techniques Conference, pages 417–424, 2000. 29

    2000

  7. [15]

    Improving Image Generation with Better Captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, Wesam Manassra, Prafulla Dhariwal, Casey Chu, Yunxin Jiao, and Aditya Ramesh. Improving Image Generation with Better Captions. 2023

    2023

  8. [16]

    Clifton, Yuxiong He, Dacheng Tao, and Shuaiwen Leon Song

    Fengxiang Bie, Yibo Yang, Zhongzhu Zhou, Adam Ghanem, Minjia Zhang, Zhewei Yao, Xiaoxia Wu, Connor Holmes, Pareesa Golnari, David A. Clifton, Yuxiong He, Dacheng Tao, and Shuaiwen Leon Song. RenAIssance: A Survey into AI Text-to- Image Generation in the Era of Large Model.ArXi...

    2023 arXiv

  9. [17]

    Hanqun Cao, Cheng Tan, Zhangyang Gao, Yilun Xu, Guangyong Chen, Pheng-Ann Heng, and Stan Z. Li. A Survey on Generative Diffusion Models.IEEE Transactions on Knowledge and Data Engineering, 36(7):2814–2830, 2024

    2024

  10. [18]

    A contemporary review on chatbots, AI-powered virtual conversa- tional agents, ChatGPT: Applications, open challenges and future research directions

    Avyay Casheekar, Archit Lahiri, Kanishk Rath, Kaushik Sanjay Prabhakar, and Kathi- ravan Srinivasan. A contemporary review on chatbots, AI-powered virtual conversa- tional agents, ChatGPT: Applications, open challenges and future research directions. Computer Science Review, 52, 2024

    2024

  11. [19]

    Wynne Chin and G. A. Marcoulides. The Partial Least Squares Approach to Structural Equation Modeling.Modern Methods for Business Research, 8:295–358, 1998

    1998

  12. [20]

    Veo, 2024

    Google DeepMind. Veo, 2024

    2024

  13. [21]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Rein- forcement Learning.ArXiv, 2501.12948, 2025

    DeepSeek-AI. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Rein- forcement Learning.ArXiv, 2501.12948, 2025

    2025 arXiv

  14. [22]

    A Survey of On-Device Machine Learning: An Algorithms and Learning Theory Perspective.ACM Transactions on Internet of Things, 2(3), 2021

    Sauptik Dhar, Junyao Guo, Jiayi (Jason) Liu, Samarth Tripathi, Unmesh Kurup, and Mohak Shah. A Survey of On-Device Machine Learning: An Algorithms and Learning Theory Perspective.ACM Transactions on Internet of Things, 2(3), 2021

    2021

  15. [23]

    Jukebox: A Generative Model for Music.ArXiv, 2005.00341, 2020

    Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, and Ilya Sutskever. Jukebox: A Generative Model for Music.ArXiv, 2005.00341, 2020

    2005 arXiv

  16. [24]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zh...

    2024 arXiv

  17. [25]

    Grok, Gemini, ChatGPT and DeepSeek: Comparison and Applications in Conversational Artificial Intelligence

    Murillo Edson de Carvalho Souza and Li Weigang. Grok, Gemini, ChatGPT and DeepSeek: Comparison and Applications in Conversational Artificial Intelligence. 2025

    2025

  18. [26]

    Image Generation: A Review.Neural Processing Letters, 54(5):4609–4646, 2022

    Mohamed Elasri, Omar Elharrouss, Somaya Al-Maadeed, and Hamid Tairi. Image Generation: A Review.Neural Processing Letters, 54(5):4609–4646, 2022

    2022

  19. [27]

    Scaling Rectified Flow Transformers for High-Resolution Image Synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M¨ uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling Rectified Flow Tran...

    2024 arXiv

  20. [28]

    Learning From Noisy Correspondence With Tri-Partition for Cross-Modal Matching.IEEE Transactions on Multimedia, 26:3884–3896, 2024

    Zerun Feng, Zhimin Zeng, Caili Guo, Zheng Li, and Lin Hu. Learning From Noisy Correspondence With Tri-Partition for Cross-Modal Matching.IEEE Transactions on Multimedia, 26:3884–3896, 2024

    2024

  21. [29]

    Line Goes Up? Inherent Limitations of Benchmarks for Evaluating Large Language Models.ArXiv, 2502.14318, 2025

    James Fodor. Line Goes Up? Inherent Limitations of Benchmarks for Evaluating Large Language Models.ArXiv, 2502.14318, 2025

    2025 arXiv

  22. [30]

    Creativity and Machine Learning: A Survey

    Giorgio Franceschelli and Mirco Musolesi. Creativity and Machine Learning: A Survey. ArXiv, 2104.02726, 2022

    2022 arXiv

  23. [31]

    The Pile: An 800GB Dataset of Diverse Text for Language Modeling.ArXiv, 2101.00027, 2020

    Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The Pile: An 800GB Dataset of Diverse Text for Language Modeling.ArXiv, 2101.00027, 2020

    2020 arXiv

  24. [32]

    ImageBind: One Embedding Space To Bind Them All.ArXiv, 2305.05665, 2023

    Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Al- wala, Armand Joulin, and Ishan Misra. ImageBind: One Embedding Space To Bind Them All.ArXiv, 2305.05665, 2023

    2023 arXiv

  25. [33]

    Mamba: Linear-Time Sequence Modeling with Selective State Spaces.ArXiv, 2312.00752, 2024

    Albert Gu and Tri Dao. Mamba: Linear-Time Sequence Modeling with Selective State Spaces.ArXiv, 2312.00752, 2024

    2024 arXiv

  26. [34]

    AudioCLIP: Extend- ing CLIP to Image, Text and Audio.ArXiv, 2106.13043, 2021

    Andrey Guzhov, Federico Raue, J¨ orn Hees, and Andreas Dengel. AudioCLIP: Extend- ing CLIP to Image, Text and Audio.ArXiv, 2106.13043, 2021

    2021 arXiv

  27. [35]

    Deep Residual Learning for Image Recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. InProceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016. 33

    2016

  28. [36]

    Ringle, and Rudolf R

    J¨ org Henseler, Christian M. Ringle, and Rudolf R. Sinkovics. The Use of Partial Least Squares Path Modeling in International Marketing.Advances in International Marketing, 20:277–319, 2009

    2009

  29. [37]

    Intuitive Multilingual Audio-Visual Speech Recognition with a Single-Trained Model

    Joanna Hong, Se Park, and Yong Ro. Intuitive Multilingual Audio-Visual Speech Recognition with a Single-Trained Model. InFindings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4886–4890, 2023

    2023

  30. [38]

    Make-an-audio: text-to-audio generation with prompt-enhanced diffusion models

    Rongjie Huang, Jiawei Huang, Dongchao Yang, Yi Ren, Luping liu, Mingze Li, Zhenhui Ye, Jinglin Liu, Xiang Yin, and Zhou Zhao. Make-an-audio: text-to-audio generation with prompt-enhanced diffusion models. InProceedings of the 40th International Con- ference on Machine Learning...

    2023

  31. [39]

    NLIP: Noise-Robust Language-Image Pre-training

    Runhui Huang, Yanxin Long, Jianhua Han, Hang Xu, Xiwen Liang, Chunjing Xu, and Xiaodan Liang. NLIP: Noise-Robust Language-Image Pre-training. InProceedings of the 37th AAAI Conference on Artificial Intelligence, pages 926–934, 2023

    2023

  32. [40]

    Large Language Models for Code Generation: A Com- prehensive Survey of Challenges, Techniques, Evaluation, and Applications.ArXiv, 2503.01245, 2025

    Nam Huynh and Beiyu Lin. Large Language Models for Code Generation: A Com- prehensive Survey of Challenges, Techniques, Evaluation, and Applications.ArXiv, 2503.01245, 2025

    2025 arXiv

  33. [41]

    Imagen-Team-Google, :, Jason Baldridge, Jakob Bauer, Mukul Bhutani, Nicole Brich- tova, Andrew Bunner, Kelvin Chan, Yichang Chen, Sander Dieleman, Yuqing Du, Zach Eaton-Rosen, Hongliang Fei, Nando de Freitas, Yilin Gao, Evgeny Gladchenko, Sergio G´ omez Colmenarejo, Mandy Guo,...

    2024

  34. [42]

    A Survey on Large Language Models for Code Generation.ArXiv, 2406.00515, 2024

    Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. A Survey on Large Language Models for Code Generation.ArXiv, 2406.00515, 2024

    2024 arXiv

  35. [43]

    Nicolas Jonason and Bob L. T. Sturm. TimbreCLIP: Connecting Timbre to Text and Images.ArXiv, 2211.11225, 2022

    2022 arXiv

  36. [44]

    Noise-Aware Learning from Web-Crawled Image-Text Data for Image Captioning

    Wooyoung Kang, Jonghwan Mun, Sungjun Lee, and Byungseok Roh. Noise-Aware Learning from Web-Crawled Image-Text Data for Image Captioning. InProceedings of the 2023 IEEE International Conference on Computer Vision, pages 2942–2952, 2023

    2023

  37. [45]

    Gemini 2.5: Our most intelligent AI model, 2025

    Koray Kavukcuoglu. Gemini 2.5: Our most intelligent AI model, 2025

    2025

  38. [46]

    Zahra Khanjani, Gabrielle Watson, and Vandana P. Janeja. Audio deepfakes: A survey. Frontiers in Big Data, 5, 2023

    2023

  39. [47]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-Encoding Variational Bayes. InProceed- ings of the 2nd International Conference on Learning Representations, 2014

    2014

  40. [48]

    Benchmarking Cognitive Biases in Large Language Models as Evaluators.ArXiv, 2309.17012, 2023

    Ryan Koo, Minhwa Lee, Vipul Raheja, Jong Inn Park, Zae Myung Kim, and Dongyeop Kang. Benchmarking Cognitive Biases in Large Language Models as Evaluators.ArXiv, 2309.17012, 2023. 35

    2023 arXiv

  41. [49]

    Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code?Proceedings of the ACM on Software Engineering, 1(FSE):2261–2284, 2024

    Bonan Kou, Shengmai Chen, Zhijie Wang, Lei Ma, and Tianyi Zhang. Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code?Proceedings of the ACM on Software Engineering, 1(FSE):2261–2284, 2024

    2024

  42. [50]

    AudioGen: Textually Guided Audio Generation.ArXiv, 2209.15352, 2023

    Felix Kreuk, Gabriel Synnaeve, Adam Polyak, Uriel Singer, Alexandre D´ efossez, Jade Copet, Devi Parikh, Yaniv Taigman, and Yossi Adi. AudioGen: Textually Guided Audio Generation.ArXiv, 2209.15352, 2023

    2023 arXiv

  43. [51]

    BindDiffusion: One Diffusion Model to Bind Them All, 2024

    Sea AI Lab. BindDiffusion: One Diffusion Model to Bind Them All, 2024

    2024

  44. [52]

    FLUX, 2024

    Black Forest Labs. FLUX, 2024

    2024

  45. [53]

    Le´ on and Miguel Carrasco

    Jorge E. Le´ on and Miguel Carrasco. Effectively obtaining acoustic, visual and textual data from videos.ArXiv, 2509.05786, 2025

    2025 arXiv

  46. [54]

    BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Mod- els.ArXiv, 2301.12597, 2023

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Mod- els.ArXiv, 2301.12597, 2023

    2023 arXiv

  47. [55]

    BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Gen- eration.ArXiv, 2201.12086, 2022

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Gen- eration.ArXiv, 2201.12086, 2022

    2022 arXiv

  48. [56]

    Word-Level Explanations for Analyzing Bias in Text-to-Image Models.ArXiv, 2306.05500, 2023

    Alexander Lin, Lucas Monteiro Paes, Sree Harsha Tanneru, Suraj Srinivas, and Himabindu Lakkaraju. Word-Level Explanations for Analyzing Bias in Text-to-Image Models.ArXiv, 2306.05500, 2023

    2023 arXiv

  49. [57]

    Plumbley

    Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D. Plumbley. AudioLDM: Text-to-Audio Generation with Latent Diffusion Models. InProceedings of the 40th International Conference on Machine Learning, pages 21450–21474, 2023

    2023

  50. [58]

    Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models.ArXiv, 2402.17177, 2024

    Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jianfeng Gao, Lifang He, and Lichao Sun. Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models.ArXiv, 2402.17177, 2024

    2024 arXiv

  51. [59]

    Michaud, Max Tegmark, and Mike Williams

    Ziming Liu, Ouail Kitouni, Niklas Nolte, Eric J. Michaud, Max Tegmark, and Mike Williams. Towards understanding grokking: an effective theory of representation learn- ing. InProceedings of the 36th International Conference on Neural Information Pro- cessing Systems, pages 3465...

    2024

  52. [60]

    BLAP: Bootstrapping Language-Audio Pre-training for Music Captioning

    Nathana¨ el Perraudin Luca A Lanzend¨ orfer, Constantin Pinkl and Roger Wattenhofer. BLAP: Bootstrapping Language-Audio Pre-training for Music Captioning. InAudio Imagination: NeurIPS 2024 Workshop AI-Driven Speech, Music, and Sound Genera- tion, 2024. 36

    2024

  53. [61]

    Look, Listen, and Answer: Overcoming Biases for Audio-Visual Question Answering.ArXiv, 2404.12020, 2024

    Jie Ma, Min Hu, Pinghui Wang, Wangchun Sun, Lingyun Song, Hongbin Pei, Jun Liu, and Youtian Du. Look, Listen, and Answer: Overcoming Biases for Audio-Visual Question Answering.ArXiv, 2404.12020, 2024

    2024 arXiv

  54. [62]

    Stable Diffusion Akashic Records, 2023

    Maks-s. Stable Diffusion Akashic Records, 2023

    2023

  55. [63]

    Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence.IEEE Transactions on Artificial Intelli- gence, pages 1–18, 2025

    Timothy R McIntosh, Teo Susnjak, Nalin Arachchilage, Tong Liu, Dan Xu, Paul Wat- ters, and Malka N Halgamuge. Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence.IEEE Transactions on Artificial Intelli- gence, pages 1–18, 2025

    2025

  56. [64]

    Mustango: Toward Controllable Text-to-Music Generation

    Jan Melechovsky, Zixun Guo, Deepanway Ghosal, Navonil Majumder, Dorien Herre- mans, and Soujanya Poria. Mustango: Toward Controllable Text-to-Music Generation. InProceedings of the 2024 North American Chapter of the Association for Computa- tional Linguistics, page 8293–8316, 2024

    2024

  57. [65]

    Mukhamediev, Adilkhan Symagulov, Yan Kuchin, Kirill Yakunin, and Ma- rina Yelis

    Ravil I. Mukhamediev, Adilkhan Symagulov, Yan Kuchin, Kirill Yakunin, and Ma- rina Yelis. From Classical Machine Learning to Deep Neural Networks: A Simplified Scientometric Review.Applied Sciences, 11(12), 2021

    2021

  58. [66]

    DALL·E 3 System Card, 2023

    OpenAI. DALL·E 3 System Card, 2023

    2023

  59. [67]

    Video generation models as world simulators, 2024

    OpenAI. Video generation models as world simulators, 2024

    2024

  60. [68]

    OpenAI o3-mini, 2025

    OpenAI. OpenAI o3-mini, 2025

    2025

  61. [69]

    Image-to-Image Translation: Methods and Applications.IEEE Transactions on Multimedia, 24:3859–3881, 2022

    Yingxue Pang, Jianxin Lin, Tao Qin, and Zhibo Chen. Image-to-Image Translation: Methods and Applications.IEEE Transactions on Multimedia, 24:3859–3881, 2022

    2022

  62. [70]

    SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis.ArXiv, 2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M¨ uller, Joe Penna, and Robin Rombach. SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis.ArXiv, 2307.01952, 2023

    2023 arXiv

  63. [71]

    Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

    Alethea Power, Yuri Burda, Harrison Edwards, Igor Babuschkin, and Vedant Misra. Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets. InPro- ceedings of the 1st Mathematical Reasoning in General Artificial Intelligence Workshop, 2021

    2021

  64. [72]

    MirrorGAN: Learning Text-To-Image Generation by Redescription

    Tingting Qiao, Jing Zhang, Duanqing Xu, and Dacheng Tao. MirrorGAN: Learning Text-To-Image Generation by Redescription. InProceedings of the 2019 IEEE Con- ference on Computer Vision and Pattern Recognition, pages 1505–1514, 2019

    2019

  65. [74]

    Learning Transferable Visual Models From Natural Lan- guage Supervision.ArXiv, 2103.00020, 2024

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sand- hini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Lan- guage Supervision.ArXiv, 2103.00020, 2024

    2024 arXiv

  66. [75]

    Robust Speech Recognition via Large-Scale Weak Supervision

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust Speech Recognition via Large-Scale Weak Supervision. In Proceedings of the 40th International Conference on Machine Learning, pages 28492– 28518, 2023

    2023

  67. [76]

    Zero-Shot Text-to-Image Generation.ArXiv, 2102.12092, 2021

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Rad- ford, Mark Chen, and Ilya Sutskever. Zero-Shot Text-to-Image Generation.ArXiv, 2102.12092, 2021

    2021 arXiv

  68. [77]

    Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrit- twieser, Ioannis Antonoglou, Rohan Anil, Sebastian Borgeaud, Andrew Dai, Katie Mil- lican, Ethan Dyer, Mia G...

    2024 arXiv

  69. [78]

    Stable Diffusion, 2021

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨ orn Om- mer. Stable Diffusion, 2021

    2021

  70. [79]

    High-Resolution Image Synthesis with Latent Diffusion Models.ArXiv, 2112.10752, 2022

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨ orn Ommer. High-Resolution Image Synthesis with Latent Diffusion Models.ArXiv, 2112.10752, 2022

    2022 arXiv

  71. [80]

    Stable Diffusion v1-5 Model Card, 2024

    Robin Rombach and Patrick Esser. Stable Diffusion v1-5 Model Card, 2024

    2024

  72. [81]

    U-Net: Convolutional Net- works for Biomedical Image Segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional Net- works for Biomedical Image Segmentation. InProceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 234–241, 2015

    2015

  73. [82]

    Introducing Gen-3 Alpha: A New Frontier for Video Generation, 2024

    Runway. Introducing Gen-3 Alpha: A New Frontier for Video Generation, 2024

    2024

  74. [83]

    Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Raphael Gontijo-Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic Text-to-Image Diffusion Mo...

    2024

  75. [84]

    A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications.ArXiv, 2402.07927, 2025

    Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications.ArXiv, 2402.07927, 2025

    2025 arXiv

  76. [85]

    Comparison and Analysis of Image-to- Image Generative Adversarial Networks: A Survey.ArXiv, 2112.12625, 2022

    Sagar Saxena and Mohammad Nayeem Teli. Comparison and Analysis of Image-to- Image Generative Adversarial Networks: A Survey.ArXiv, 2112.12625, 2022

    2022 arXiv

  77. [86]

    What is noise?Geophysics, 63(4):1122–1124, 1998

    John Scales and Roel Snieder. What is noise?Geophysics, 63(4):1122–1124, 1998. 41

    1998

  78. [87]

    Large pre-trained language models contain human- like biases of what is right and wrong to do.Nature Machine Intelligence, 4:258–268, 2022

    Patrick Schramowski, Cigdem Turan-Schwiewager, Nico Andersen, Constantin Rothkopf, and Kristian Kersting. Large pre-trained language models contain human- like biases of what is right and wrong to do.Nature Machine Intelligence, 4:258–268, 2022

    2022

  79. [88]

    A comprehensive review of large language models: issues and solutions in learning environments.Discover Sustainability, 6, 2025

    Tariq Shahzad, Tehseen Mazhar, Muhammad Usman Tariq, Wasim Ahmad, Khmaies Ouahada, and Habib Hamam. A comprehensive review of large language models: issues and solutions in learning environments.Discover Sustainability, 6, 2025

    2025

  80. [89]

    I Hear Your True Colors: Image Guided Audio Generation

    Roy Sheffer and Yossi Adi. I Hear Your True Colors: Image Guided Audio Generation. ArXiv, 2211.03089, 2023

    2023 arXiv

  81. [90]

    A Survey on Audio Synthesis and Audio-Visual Multimodal Processing

    Zhaofeng Shi. A Survey on Audio Synthesis and Audio-Visual Multimodal Processing. ArXiv, 2108.00443, 2021

    2021 arXiv

  82. [91]

    Audio-to-Visual Cross-Modal Generation of Birds.IEEE Access, 11:27719–27729, 2023

    Joo Yong Shim, Joongheon Kim, and Jong-Kook Kim. Audio-to-Visual Cross-Modal Generation of Birds.IEEE Access, 11:27719–27729, 2023

    2023

  83. [92]

    Outpainting Images and Videos using GANs.International Journal of Computer Trends and Tech- nology, 68(5):24–29, 2020

    Shailendra Singh, Nainish Aggarwal, Udit Jain, and Hrithik Jaiswal. Outpainting Images and Videos using GANs.International Journal of Computer Trends and Tech- nology, 68(5):24–29, 2020

    2020

  84. [93]

    Pre-trained Speech Processing Models Contain Human-Like Biases that Propagate to Speech Emo- tion Recognition

    Isaac Slaughter, Craig Greenberg, Reva Schwartz, and Aylin Caliskan. Pre-trained Speech Processing Models Contain Human-Like Biases that Propagate to Speech Emo- tion Recognition. InFindings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8967...

    2023

  85. [94]

    A survey of multimodal deep generative models

    Masahiro Suzuki and Yutaka Matsuo. A survey of multimodal deep generative models. Advanced Robotics, 36(5-6):261–278, 2022

    2022

  86. [95]

    CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generation

    Zineng Tang, Ziyi Yang, Mahmoud Khademi, Yang Liu, Chenguang Zhu, and Mohit Bansal. CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generation. ArXiv, 2311.18775, 2023

    2023 arXiv

  87. [96]

    Any- to-any generation via composable diffusion

    Zineng Tang, Ziyi Yang, Chenguang Zhu, Michael Zeng, and Mohit Bansal. Any- to-any generation via composable diffusion. InProceedings of the 37th International Conference on Neural Information Processing Systems, pages 16083–16099, 2024

    2024

  88. [97]

    Movie Gen: A Cast of Media Foundation Models, 2024

    The Movie Gen team. Movie Gen: A Cast of Media Foundation Models, 2024

    2024

  89. [98]

    LLaMA: Open and Efficient Foundation Language Models.ArXiv, 2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ ee Lacroix, Baptiste Rozi` ere, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. LLaMA: Open and Efficient Foundation Lang...

    2023 arXiv

  90. [99]

    Structural Equation Modeling in Information Systems Research Using Partial Least Squares.Journal of Information Technology Theory and Application, 11(2):5–40, 2010

    Nils Urbach and Frederik Ahlemann. Structural Equation Modeling in Information Systems Research Using Partial Least Squares.Journal of Information Technology Theory and Application, 11(2):5–40, 2010

    2010

  91. [100]

    Fugatto 1 - Foundational Genera- tive Audio Transformer Opus 1, 2024

    Rafael Valle, Rohan Badlani, Zhifeng Kong, Sang gil Lee, Arushi Goel, Sungwon Kim, Joao Felipe Santos, Shuqi Dai, Siddharth Gururani, Aya AIJa’fari, Alex Liu, Kevin Shih, Wei Ping, Huck Yang, and Bryan Catanzaro. Fugatto 1 - Foundational Genera- tive Audio Transformer Opus 1, 2024

    2024

  92. [101]

    Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N

    Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is All you Need. InProceedings of the 31st International Conference on Neural Information Processing Systems, page 6000–6010, 2017

    2017

  93. [102]

    Audio Describing Sound – What Sounds are Described and How?: Results from a Flemish case study.Journal of Audiovisual Translation, 5(2):114–133, 2022

    Gert Vercauteren and Nina Reviers. Audio Describing Sound – What Sounds are Described and How?: Results from a Flemish case study.Journal of Audiovisual Translation, 5(2):114–133, 2022

    2022

  94. [103]

    Audiobox: Unified Audio Generation with Natural Language Prompts.ArXiv, 2312.15821, 2023

    Apoorv Vyas, Bowen Shi, Matthew Le, Andros Tjandra, Yi-Chiao Wu, Baishan Guo, Jiemin Zhang, Xinyue Zhang, Robert Adkins, William Ngan, Jeff Wang, Ivan Cruz, Bapi Akula, Akinniyi Akinyemi, Brian Ellis, Rashel Moritz, Yael Yungster, Alice Rako- toarison, Liang Tan, Chris Summers...

    2023 arXiv

  95. [104]

    Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers.ArXiv, 2301.02111, 2023

    Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, and Furu Wei. Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers.ArXiv, 2301.02111, 2023

    2023 arXiv

  96. [105]

    From Association to Generation: Text-only Captioning by Unsupervised Cross-modal Mapping

    Junyang Wang, Ming Yan, Yi Zhang, and Jitao Sang. From Association to Generation: Text-only Captioning by Unsupervised Cross-modal Mapping. InProceedings of the 32nd International Joint Conference on Artificial Intelligence, pages 4326–4334, 2023

    2023

  97. [106]

    A Systematic Review and Comprehensive Analysis of Pioneering AI Chatbot Models from Education to Healthcare: ChatGPT, Bard, Llama, Ernie and Grok.Future Internet, 16(7), 2024

    Ketmanto Wangsa, Shakir Karim, Ergun Gide, and Mahmoud Elkhodr. A Systematic Review and Comprehensive Analysis of Pioneering AI Chatbot Models from Education to Healthcare: ChatGPT, Bard, Llama, Ernie and Grok.Future Internet, 16(7), 2024

    2024

  98. [107]

    Liu, and Hung yi Lee

    Haibin Wu, Xuanjun Chen, Yi-Cheng Lin, Kai wei Chang, Ho-Lam Chung, Alexan- der H. Liu, and Hung yi Lee. Towards audio language modeling – an overview.ArXiv, 2402.13236, 2024

    2024 arXiv

  99. [108]

    Audio-Text Models Do Not Yet Leverage Natural Language

    Ho-Hsiang Wu, Oriol Nieto, Juan Pablo Bello, and Justin Salamon. Audio-Text Models Do Not Yet Leverage Natural Language. InProceedings of the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 1–5, 2023. 43

    2023

  100. [109]

    Wav2CLIP: Learning Robust Audio Representations from Clip

    Ho-Hsiang Wu, Prem Seetharaman, Kundan Kumar, and Juan Pablo Bello. Wav2CLIP: Learning Robust Audio Representations from Clip. InProceedings of the 2022 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 4563–4567, 2022

    2022

  101. [110]

    NExT-GPT: Any- to-Any Multimodal LLM.ArXiv, 2309.05519, 2024

    Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. NExT-GPT: Any- to-Any Multimodal LLM.ArXiv, 2309.05519, 2024

    2024 arXiv

  102. [111]

    Grok 3 Beta — The Age of Reasoning Agents, 2025

    xAI. Grok 3 Beta — The Age of Reasoning Agents, 2025

    2025

  103. [112]

    Web-Bench: A LLM Code Benchmark Based on Web Standards and Frameworks.ArXiv, 2505.07473, 2025

    Kai Xu, YiWei Mao, XinYi Guan, and ZiLong Feng. Web-Bench: A LLM Code Benchmark Based on Web Standards and Frameworks.ArXiv, 2505.07473, 2025

    2025 arXiv

  104. [113]

    Peng Xu, Xiatian Zhu, and David A. Clifton. Multimodal Learning With Transformers: A Survey.IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1–20, 2023

    2023

  105. [114]

    Xuenan Xu, Zhiling Zhang, Zelin Zhou, Pingyue Zhang, Zeyu Xie, Mengyue Wu, and Kenny Q. Zhu. BLAT: Bootstrapping Language-Audio Pre-training based on AudioSet Tag-guided Synthetic Data. InProceedings of the 31st ACM International Conference on Multimedia, page 2756–2764, 2023

    2023

  106. [115]

    BiCro: Noisy Correspondence Rectification for Multi-modality Data via Bi-directional Cross-modal Similarity Consistency

    Shuo Yang, Zhaopan Xu, Kai Wang, Yang You, Hongxun Yao, Tongliang Liu, and Min Xu. BiCro: Noisy Correspondence Rectification for Multi-modality Data via Bi-directional Cross-modal Similarity Consistency. InProceedings of the 2023 IEEE Conference on Computer Vision and Pattern ...

    2023

  107. [116]

    The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)

    Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision). ArXiv, 2309.17421, 2023

    2023 arXiv

  108. [117]

    AudioToken: Adap- tation of Text-Conditioned Diffusion Models for Audio-to-Image Generation.ArXiv, 2305.13050, 2023

    Guy Yariv, Itai Gat, Lior Wolf, Yossi Adi, and Idan Schwartz. AudioToken: Adap- tation of Text-Conditioned Diffusion Models for Audio-to-Image Generation.ArXiv, 2305.13050, 2023

    2023 arXiv

  109. [118]

    Multimodal Image Synthesis and Editing: The Generative AI Era.ArXiv, 2112.13592, 2023

    Fangneng Zhan, Yingchen Yu, Rongliang Wu, Jiahui Zhang, Shijian Lu, Lingjie Liu, Adam Kortylewski, Christian Theobalt, and Eric Xing. Multimodal Image Synthesis and Editing: The Generative AI Era.ArXiv, 2112.13592, 2023

    2023 arXiv

  110. [119]

    Text-to- image Diffusion Models in Generative AI: A Survey.ArXiv, 2303.07909, 2023

    Chenshuang Zhang, Chaoning Zhang, Mengchun Zhang, and In So Kweon. Text-to- image Diffusion Models in Generative AI: A Survey.ArXiv, 2303.07909, 2023

    2023 arXiv

  111. [120]

    Remote Sensing Image Generation From Audio.IEEE Geoscience and Remote Sensing Letters, 18(6):994–998, 2021

    Zhiyuan Zheng, Jun Chen, Xiangtao Zheng, and Xiaoqiang Lu. Remote Sensing Image Generation From Audio.IEEE Geoscience and Remote Sensing Letters, 18(6):994–998, 2021. 44

    2021

  112. [121]

    Cacophony: An Improved Contrastive Audio-Text Model

    Ge Zhu and Zhiyao Duan. Cacophony: An Improved Contrastive Audio-Text Model. ArXiv, 2402.06986, 2024

    2024 arXiv

  113. [122]

    Deep Audio-visual Learning: A Survey.International Journal of Automation and Computing, 18:351–376, 2021

    Hao Zhu, Man-Di Luo, Rui Wang, Ai-Hua Zheng, and Ran He. Deep Audio-visual Learning: A Survey.International Journal of Automation and Computing, 18:351–376, 2021

    2021

  114. [123]

    On Some Biases Encountered in Modern Audio Quality Listening Tests - A Review.Journal of the Audio Engineering Society, 56(6):427–451, 2008

    S lawomir Zieli´ nski, Francis Rumsey, and Søren Bech. On Some Biases Encountered in Modern Audio Quality Listening Tests - A Review.Journal of the Audio Engineering Society, 56(6):427–451, 2008

    2008

  115. [124]

    Audio-to-Image Cross-Modal Generation

    Maciej ˙Zelaszczyk and Jacek Ma´ ndziuk. Audio-to-Image Cross-Modal Generation. ArXiv, 2109.13354, 2021. 45

    2021 arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.