Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

A Survey of RWKV

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This survey argues that RWKV, a recurrent-architecture family, is now a credible Transformer alternative and provides the first systematic map of its design, applications, and open problems.

desk verdict Useful catalog of the RWKV ecosystem, but the RWKV-5 recurrence in Eq. 29 is wrong and should be corrected before anyone relies on the survey. read the letter →

arxiv 2412.14847 v2 pith:M22XFNMT submitted 2024-12-19 cs.CL cs.AI

Zhiyuan Li , Tingyu Xia , Yi Chang , Yuan Wu This is my paper · ORCID
classification cs.CLcs.AI
keywords RWKVlinearattentionrecurrentneuralnetworkTransformeralternativelong-sequencemodelingsurveyefficiencystate-spacemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is the first systematic review of RWKV, a family of sequence models that mixes a recurrent state with a Transformer-style attention framing. The survey's organizing claim is that RWKV captures long-range dependencies with $O(T d)$ time and $O(d)$ memory instead of the Transformer's quadratic cost, and that this efficiency has let it expand from language modeling into vision, audio, time series, and web tools. Its contribution is organizational: it distills the RWKV-4, RWKV-5 (Eagle), and RWKV-6 (Finch) updates, catalogs dozens of downstream systems and evaluation benchmarks, and names open problems in long-context retention, multimodal learning, safety, and hardware acceleration. A reader who wants to know what RWKV is, how it works, and where it has been tried can use this paper as orientation.

What carries the argument

The central object is the WKV (Weighted Key Value) recurrence, the linear-attention core that replaces the softmax query-key product. In RWKV-4 the attention weight for each past token is an exponential decay sum with a learned decay vector $w$ and a separate learned bonus $u$ for the current token; Eagle upgrades the state from a vector to a multi-head matrix-valued state with contraction decay $w = \exp(-\exp(\omega))$; Finch makes the decay input-dependent through a low-rank shift. This recurrence carries the efficiency claim because it compresses the whole context into a fixed-size state, so inference cost no longer grows with sequence length.

What would settle it

Compare Section 3.2's recurrent update, printed as $s' = \operatorname{diag}(u) \cdot s + k^T \cdot \nu$, against the cited Eagle/Finch paper and its released code: if the learned decay is applied to the state differently, the survey's technical core misdescribes the model.

Watch

Extended reading notes

Core claim

The paper's central claim is that RWKV is a viable third path in sequence modeling: it keeps the Transformer's ability to mix information across distant tokens while replacing quadratic self-attention with a recurrent weighted key-value state. The survey presents RWKV-4's time-mixing and channel-mixing blocks, the matrix-valued states and learned contraction decay of Eagle, and the data-dependent decay of Finch, then documents the model's spread across natural language generation, understanding, computer vision, audio, and web applications. A central part of the claim is that these variants preserve linear-time training and inference while approaching Transformer-level expressivity, so the architecture deserves a consolidated reference.

Load-bearing premise

The survey's usefulness rests on the accuracy of the equations it copies from the primary RWKV papers, because a faulty recurrence would misrepresent the architecture it claims to review.

Editorial extensions

If this is right

  • If RWKV's linear recurrence matches Transformer-level quality on the surveyed tasks, long-sequence applications that are memory-bound under attention become feasible at the same model scale.
  • The catalog of implementations in C, C++, Rust, Go, JAX, and edge runtimes implies the architecture can be deployed beyond data-center GPUs, including on-device inference.
  • Benchmarks such as LooGLE, RULER, and S3EVAL show that current RWKV versions lose accuracy on very long or complex in-context tasks, so the survey's own evidence bounds the efficiency claim.
  • RWKV's recurrent core makes it a natural host for parameter-efficient finetuning with LoRA, adapters, and prompt tuning, a direction the paper singles out for future work.
  • The same recurrent core is already being reused for vision, audio, and multimodal tasks, so future gains in long-context retention should transfer across modalities.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The survey groups applications by domain but rarely reports relative performance; a maintained table of RWKV variants versus Transformer baselines on shared benchmarks would turn the catalog into a quantitative comparison.
  • If RWKV is to compete on very long contexts, the decay mechanism itself is the place to watch: static exponential decay limits memory of old tokens, so a content-addressed or learnable memory that escapes pure decay would be a natural next step after Finch.
  • The hardware section implies RWKV's fixed-size state favors edge deployment; a concrete test would be measuring RWKV, Mamba, and a Transformer on the same accelerator under identical long-context workloads.
  • Because several vision papers already adapt RWKV to high-resolution images, a testable extension is that a RWKV-based vision-language model would scale to higher resolutions than a ViT at similar inference cost.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This manuscript is a survey of the RWKV architecture family (RWKV-4, RWKV-5/Eagle, RWKV-6/Finch) and its ecosystem. It provides background on RNNs, Transformers, and Attention-Free Transformers; derives the core time-mixing and channel-mixing equations; compares RWKV with linear Transformers, Mamba, RetNet, and Hyena; catalogs a large number of third-party implementations and applications in NLP, computer vision, audio, web, and evaluation benchmarks; and closes with future directions and challenges. The stated aim is to be 'the first comprehensive review' of RWKV, with a companion GitHub repository of open-source materials.

Significance. The survey addresses a timely and useful goal: RWKV is a prominent linear-complexity architecture family, and a reliable orientation to its design and application landscape would be valuable to practitioners and researchers. The manuscript compiles an extensive catalog of implementations, downstream tasks, and evaluation benchmarks that are not collected elsewhere. It also reproduces several core architectural formulas and clearly identifies the official versions and their relationships. The strength of the survey therefore depends on the correctness of its technical transcription and the credibility of its 'comprehensive' claim; both currently have gaps.

major comments (3)
  1. [Section 3.2, Eqs. (26)-(29)] The recurrent state update for RWKV-5 is stated incorrectly. Eq. (29) reads s' = diag(u)·s + k^T·ν, but the parallel form in Eq. (26) uses diag(w)^{t-1-i} as the decay on historical terms. The correct recurrent update should apply the learned per-channel decay to the previous state: s' = diag(w)·s + k^T·ν, with diag(u) applied only in the output wkv' (Eq. (28)). As written, the recurrence double-counts u, omits the contraction matrix diag(w) entirely from the state path, and thus produces a state evolution different from the parallel form it is supposed to transcribe. This is a load-bearing error: a practitioner implementing RWKV-5 from this survey would build a different model, and the internal inconsistency with the surrounding text (which correctly states that diag(w) is a contraction matrix) makes it a definite mistake rather than a benign notational variant.
  2. [Abstract and Section 6 (Conclusion)] The claim of being 'the first comprehensive review' of RWKV is not substantiated by any search methodology. The paper does not state the databases searched, the inclusion/exclusion criteria, the time window, or any systematic process for identifying the reviewed works. Without such information, 'comprehensive' is an unverified assertion, and the manuscript also risks overlooking prior surveys or structured overviews of the same architecture. The authors should either add a methodology section describing how the literature was gathered and filtered, or temper the claim to 'a broad survey' of the RWKV ecosystem.
  3. [Section 3.4 (Compare RWKV with Other improved models based on Transformer)] The comparative claims are stated qualitatively without supporting data or citations. For example, the sentence 'RWKV excels by offering greater efficiency, quicker inference times, and reduced memory usage during operation' is not tied to any experiment, table, or reference; the same applies to the statement that RetNet 'leads to better performance' and to the Hyena comparison. Since the paper's stated purpose is to assess how RWKV compares with Transformer-based alternatives, these claims should be supported by quantitative evidence from the cited sources (e.g., measured latency, throughput, memory use, benchmark scores) or explicitly marked as qualitative observations. As it stands, Section 3.4 does not provide the reader with a reliable basis for the comparison.
minor comments (4)
  1. [Section 3.2 (RWKV-5)] The notation for the value vector is inconsistent: Eqs. (24), (26), and (27) use v, while Eqs. (28) and (29) use the Greek letter nu (ν). This makes the recurrence equations harder to read and should be unified.
  2. [Section 3.1 (RWKV-4), Eq. (18)] The equation for wkv_t is not typeset with clear spacing between the exponent terms and the plus sign; the expression exp(-(t-1-i)w+ki) should presumably be exp(-(t-1-i)w + k_i), and the summand uses a mix of subscript and inline notation. Please clean up the formatting for readability.
  3. [Section 3.4 (Compare RWKV with Other improved models)] The comparison with Mamba, RetNet, and Hyena would be more informative with a summary table listing architecture type, computational complexity, and representative results. The current prose repeats generic statements and does not give the reader a concrete basis for distinguishing the models.
  4. [Section 5 (Future Directions and Challenges)] The future directions are reasonable but generic; they largely repeat standard LLM challenges (safety, bias, privacy, PEFT) without tying them to RWKV-specific properties. The authors could strengthen this section by grounding each direction in the architectural features of RWKV (e.g., the recurrent state, token shift, and decay mechanism).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: this is a survey that derives no result and fits no parameter, so its conclusions do not reduce to their own inputs.

full rationale

This paper is a survey: it fits no parameters, trains no model, and makes no quantitative prediction. Its load-bearing content is a transcription of architectural definitions and an application catalog drawn from cited primary sources, so there is no derivation chain whose conclusion is equivalent to its premises. The only concrete technical issue is the apparent inconsistency between the RWKV-5 parallel form in Eq. 26, which decays past contributions by diag(w), and the recurrent form in Eqs. 28-29, which applies diag(u) to the old state and drops diag(w) from the state update. That is a transcription or correctness risk, not circularity: the survey does not use Eq. 29 to define or justify Eq. 26, and no quantity is fitted to make the two forms agree. The claim of being the 'first comprehensive review' is an unverified assertion of novelty, which is a scope and verification concern rather than circular reasoning. There are no load-bearing self-citations by the present authors, no uniqueness theorems imported from the authors' own prior work, and no ansatz smuggled in through citation. The survey is self-contained as a literature review, and its correctness should be judged against the primary sources it transcribes, not against itself.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The survey does not fit parameters or postulate new entities. Its burden is instead factual fidelity: it assumes the primary-source formulas are transcribed correctly, that the cited applications and benchmarks are represented accurately, and that the repository list is representative. The first assumption is weakened by the Eq. 29 error.

assumptions (3)
  • domain assumption The cited RWKV-4 and Eagle/Finch formulas are faithfully transcribed from the primary sources [40], [271].
    The survey's technical sections are built on these transcriptions; Section 3.2 Eq. 29 appears to violate this assumption.
  • domain assumption The listed benchmarks and applications are accurately described and the cited repositories exist as claimed.
    The survey's comprehensive claim depends on the reliability of hundreds of citations, most of which are GitHub repositories without independent verification in the paper.
  • ad hoc to paper The set of selected topics reasonably covers the RWKV ecosystem.
    No systematic search protocol is given, so the coverage claim rests on the authors' implicit selection criteria.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Survey of RWKV." pith.science (2026). https://pith.science/paper/M22XFNMT

@misc{pith2026241214847,
  author       = {Pith},
  title        = {Pith review of: A Survey of RWKV},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M22XFNMT}},
  note         = {Machine review of arXiv:2412.14847}
}
read the original abstract

The Receptance Weighted Key Value (RWKV) model offers a novel alternative to the Transformer architecture, merging the benefits of recurrent and attention-based systems. Unlike conventional Transformers, which depend heavily on self-attention, RWKV adeptly captures long-range dependencies with minimal computational demands. By utilizing a recurrent framework, RWKV addresses some computational inefficiencies found in Transformers, particularly in tasks with long sequences. RWKV has recently drawn considerable attention for its robust performance across multiple domains. Despite its growing popularity, no systematic review of the RWKV model exists. This paper seeks to fill this gap as the first comprehensive review of the RWKV architecture, its core principles, and its varied applications, such as natural language generation, natural language understanding, and computer vision. We assess how RWKV compares to traditional Transformer models, highlighting its capability to manage long sequences efficiently and lower computational costs. Furthermore, we explore the challenges RWKV encounters and propose potential directions for future research and advancement. We consistently maintain the related open-source materials at: https://github.com/MLGroupJLU/RWKV-Survey.

Figures

Figures reproduced from arXiv: 2412.14847 by the authors.

Figure 1
Figure 1. Structure of this paper [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The structure of the RWKV model consists of stacked residual [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Examples of downstream tasks utilizing RWKV-based models. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Survey of Retentive Network

    cs.CL 2025-06 conditional novelty 2.0 of 10

    A review that describes the RetNet architecture and enumerates its applications across many domains, without presenting new experimental results.

Reference graph

Works this paper leans on

277 extracted references · 28 canonical work pages · cited by 1 Pith paper

  1. [1]

    The perceptron: a probabilistic model for informa- tion storage and organization in the brain

    F. Rosenblatt, “The perceptron: a probabilistic model for informa- tion storage and organization in the brain.” Psychological review, vol. 65, no. 6, p. 386, 1958

    1958

  2. [2]

    Gradient-based learning applied to document recognition,

    Y. LeCun, L. Bottou, Y. Bengio, and P . Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998

    1998

  3. [3]

    Deep subdomain adaptation network for image classification,

    Y. Zhu, F. Zhuang, J. Wang, G. Ke, J. Chen, J. Bian, H. Xiong, and Q. He, “Deep subdomain adaptation network for image classification,” IEEE transactions on neural networks and learning systems, vol. 32, no. 4, pp. 1713–1722, 2020

    2020

  4. [4]

    Resmlp: Feedforward networks for image classification with data-efficient training,

    H. Touvron, P . Bojanowski, M. Caron, M. Cord, A. El-Nouby, E. Grave, G. Izacard, A. Joulin, G. Synnaeve, J. Verbeek et al. , “Resmlp: Feedforward networks for image classification with data-efficient training,” IEEE transactions on pattern analysis and machine intelligence, vol. 45, no. 4, pp. 5314–5321, 2022

    2022

  5. [5]

    Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups,

    G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V . Vanhoucke, P . Nguyen, T. N. Sainath et al. , “Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups,” IEEE Signal processing magazine, vol. 29, no. 6, pp. 82–97, 2012

    2012

  6. [6]

    Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition,

    G. E. Dahl, D. Yu, L. Deng, and A. Acero, “Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition,” IEEE Transactions on audio, speech, and language processing, vol. 20, no. 1, pp. 30–42, 2011

    2011

  7. [7]

    Transformer transducer: A streamable speech recognition model with transformer encoders and rnn-t loss,

    Q. Zhang, H. Lu, H. Sak, A. Tripathi, E. McDermott, S. Koo, and S. Kumar, “Transformer transducer: A streamable speech recognition model with transformer encoders and rnn-t loss,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 7829–7833

    2020

  8. [8]

    Jumping nlp curves: A review of natu- ral language processing research,

    E. Cambria and B. White, “Jumping nlp curves: A review of natu- ral language processing research,” IEEE Computational intelligence magazine, vol. 9, no. 2, pp. 48–57, 2014

    2014

Show all 277 references
  1. [9]

    Beyond word embeddings: A survey,

    F. Incitti, F. Urli, and L. Snidaro, “Beyond word embeddings: A survey,” Information Fusion, vol. 89, pp. 418–436, 2023

    2023

  2. [10]

    Imagenet classi- fication with deep convolutional neural networks,

    A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classi- fication with deep convolutional neural networks,” Communica- tions of the ACM, vol. 60, no. 6, pp. 84–90, 2017

    2017

  3. [11]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

    2016

  4. [12]

    Dual mixup regularized learning for adversarial domain adaptation,

    Y. Wu, D. Inkpen, and A. El-Roby, “Dual mixup regularized learning for adversarial domain adaptation,” in Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIX 16. Springer, 2020, pp. 540–555

    2020

  5. [13]

    Self- driving cars: A survey,

    C. Badue, R. Guidolini, R. V . Carneiro, P . Azevedo, V . B. Cardoso, A. Forechi, L. Jesus, R. Berriel, T. M. Paixao, F. Mutz et al., “Self- driving cars: A survey,” Expert systems with applications , vol. 165, p. 113816, 2021

    2021

  6. [14]

    About face: A survey of facial recognition evaluation,

    I. D. Raji and G. Fried, “About face: A survey of facial recognition evaluation,” arXiv preprint arXiv:2102.00813, 2021

    2021 arXiv

  7. [15]

    A survey of autonomous driving: Common practices and emerging tech- nologies,

    E. Yurtsever, J. Lambert, A. Carballo, and K. Takeda, “A survey of autonomous driving: Common practices and emerging tech- nologies,” IEEE access, vol. 8, pp. 58 443–58 469, 2020

    2020

  8. [16]

    Long short-term memory,

    S. Hochreiter, “Long short-term memory,” Neural Computation MIT-Press, 1997

    1997

  9. [17]

    Neural machine translation by jointly learning to align and translate,

    D. Bahdanau, “Neural machine translation by jointly learning to align and translate,” arXiv preprint arXiv:1409.0473, 2014

    2014 arXiv

  10. [18]

    A survey of mul- tilingual neural machine translation,

    R. Dabre, C. Chu, and A. Kunchukuttan, “A survey of mul- tilingual neural machine translation,” ACM Computing Surveys (CSUR), vol. 53, no. 5, pp. 1–38, 2020

    2020

  11. [19]

    Auto- matic speech recognition: a survey,

    M. Malik, M. K. Malik, K. Mehmood, and I. Makhdoom, “Auto- matic speech recognition: a survey,” Multimedia Tools and Applica- tions, vol. 80, pp. 9411–9457, 2021

    2021

  12. [20]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems, 2017

    2017

  13. [21]

    Efficient content- based sparse attention with routing transformers,

    A. Roy, M. Saffar, A. Vaswani, and D. Grangier, “Efficient content- based sparse attention with routing transformers,” Transactions of the Association for Computational Linguistics, vol. 9, pp. 53–68, 2021

    2021

  14. [22]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P . Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P . Welinder, P . Christiano, J. Leike, and R. Lowe, “Training language models to follow instructi...

    2022 arXiv

  15. [23]

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,

    W. Fedus, B. Zoph, and N. Shazeer, “Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,”

  16. [24]

    ibot: Image bert pre-training with online tokenizer,

    J. Zhou, C. Wei, H. Wang, W. Shen, C. Xie, A. Yuille, and T. Kong, “ibot: Image bert pre-training with online tokenizer,”

  17. [25]

    Sigt: An efficient end-to-end mimo-ofdm receiver framework based on transformer,

    Z. Ren, N. Cheng, R. Sun, X. Wang, N. Lu, and W. Xu, “Sigt: An efficient end-to-end mimo-ofdm receiver framework based on transformer,” in 2022 5th International Conference on Communi- cations, Signal Processing, and their Applications (ICCSP A). IEEE, 2022, pp. 1–6

    2022

  18. [26]

    Available: https://arxiv.org/abs/2111.07832 JOURNAL OF LATEX CLASS FILES, VOL

    [Online]. Available: https://arxiv.org/abs/2111.07832 JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 15

    2015 arXiv

  19. [27]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018

    2018 arXiv

  20. [28]

    Bigssl: Exploring the frontier of large-scale semi-supervised learning for automatic speech recognition,

    Y. Zhang, D. S. Park, W. Han, J. Qin, A. Gulati, J. Shor, A. Jansen, Y. Xu, Y. Huang, S. Wang et al. , “Bigssl: Exploring the frontier of large-scale semi-supervised learning for automatic speech recognition,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no....

    2022

  21. [29]

    Text classification using im- proved bidirectional transformer,

    M. Tezgider, B. Yildiz, and G. Aydin, “Text classification using im- proved bidirectional transformer,” Concurrency and Computation: Practice and Experience, vol. 34, no. 9, p. e6486, 2022

    2022

  22. [30]

    Improving language understanding by generative pre-training,

    A. Radford, “Improving language understanding by generative pre-training,” 2018

    2018

  23. [31]

    Transformer-based neu- ral network for answer selection in question answering,

    T. Shao, Y. Guo, H. Chen, and Z. Hao, “Transformer-based neu- ral network for answer selection in question answering,” IEEE Access, vol. 7, pp. 26 146–26 156, 2019

    2019

  24. [32]

    A survey of controllable text generation using transformer-based pre-trained language models,

    H. Zhang, H. Song, S. Li, M. Zhou, and D. Song, “A survey of controllable text generation using transformer-based pre-trained language models,” ACM Computing Surveys , vol. 56, no. 3, pp. 1–37, 2023

    2023

  25. [33]

    Attention is not all you need: Pure attention loses rank doubly exponentially with depth,

    Y. Dong, J.-B. Cordonnier, and A. Loukas, “Attention is not all you need: Pure attention loses rank doubly exponentially with depth,” in International Conference on Machine Learning . PMLR, 2021, pp. 2793–2803

    2021

  26. [34]

    Transformer-xl: Attentive language models beyond a fixed-length context,

    Z. Dai, “Transformer-xl: Attentive language models beyond a fixed-length context,” arXiv preprint arXiv:1901.02860, 2019

    1901 arXiv

  27. [35]

    Trans- formers are rnns: Fast autoregressive transformers with linear attention,

    A. Katharopoulos, A. Vyas, N. Pappas, and F. Fleuret, “Trans- formers are rnns: Fast autoregressive transformers with linear attention,” in International conference on machine learning. PMLR, 2020, pp. 5156–5165

    2020

  28. [36]

    Rethinking attention with performers,

    K. Choromanski, V . Likhosherstov, D. Dohan, X. Song, A. Gane, T. Sarlos, P . Hawkins, J. Davis, A. Mohiuddin, L. Kaiser et al. , “Rethinking attention with performers,” arXiv preprint arXiv:2009.14794, 2020

    2009 arXiv

  29. [37]

    Efficiently modeling long sequences with structured state spaces,

    A. Gu, K. Goel, and C. R ´e, “Efficiently modeling long sequences with structured state spaces,” arXiv preprint arXiv:2111.00396 , 2021

    2021 arXiv

  30. [38]

    Lin- former: Self-attention with linear complexity,

    S. Wang, B. Z. Li, M. Khabsa, H. Fang, and H. Ma, “Lin- former: Self-attention with linear complexity,” arXiv preprint arXiv:2006.04768, 2020

    2006 arXiv

  31. [39]

    Retentive network: A successor to transformer for large language models,

    Y. Sun, L. Dong, S. Huang, S. Ma, Y. Xia, J. Xue, J. Wang, and F. Wei, “Retentive network: A successor to transformer for large language models,” arXiv preprint arXiv:2307.08621, 2023

    2023 arXiv

  32. [41]

    Ai-writer,

    P . Bo, “Ai-writer,” https://github.com/BlinkDL/AI-Writer, 2022

    2022

  33. [42]

    Rwkv: Reinventing rnns for the transformer era,

    B. Peng, E. Alcaide, Q. Anthony, A. Albalak, S. Arcadinho, S. Biderman, H. Cao, X. Cheng, M. Chung, M. Grella et al. , “Rwkv: Reinventing rnns for the transformer era,” arXiv preprint arXiv:2305.13048, 2023

    2023 arXiv

  34. [43]

    Multi-scale rwkv with 2- dimensional temporal convolutional network for short-term pho- tovoltaic power forecasting,

    J. Hao, F. Liu, and W. Zhang, “Multi-scale rwkv with 2- dimensional temporal convolutional network for short-term pho- tovoltaic power forecasting,” Energy, vol. 309, p. 133068, 2024

    2024

  35. [44]

    An approach to mongolian neural machine translation based on rwkv language model and contrastive learning,

    X. Liu, Y. Su, W. Nier, Y. Ji, R. Qing Dao Er Ji, and M. Lu, “An approach to mongolian neural machine translation based on rwkv language model and contrastive learning,” in International Conference on Neural Information Processing . Springer, 2023, pp. 327–340

    2023

  36. [45]

    Chatrwkv,

    ——, “Chatrwkv,” https://github.com/BlinkDL/ChatRWKV, 2023

    2023

  37. [46]

    Rwkv-lm,

    B. PENG, “Rwkv-lm,” https://github.com/BlinkDL/ RWKV-LM, 2021

    2021

  38. [47]

    Kong, “rwkv.c,” https://github.com/kong13661/rwkv.c, 2023

    F. Kong, “rwkv.c,” https://github.com/kong13661/rwkv.c, 2023

    2023

  39. [48]

    rwkv.cpp,

    RWKV , “rwkv.cpp,” https://github.com/RWKV/rwkv.cpp, 2023

    2023

  40. [49]

    rwkv-cpp-server,

    Z. Huang, “rwkv-cpp-server,” https://github.com/ ZeldaHuang/rwkv-cpp-server, 2023

    2023

  41. [50]

    rwkv-cpp-accelerated,

    H. Vanderbyl, “rwkv-cpp-accelerated,” https://github.com/ harrisonvanderbyl/rwkv-cpp-accelerated, 2023

    2023

  42. [51]

    web-rwkv,

    cryscan, “web-rwkv,” https://github.com/cryscan/web-rwkv, 2023

    2023

  43. [52]

    rwkv tokenizer cpp,

    szxysdt, “rwkv tokenizer cpp,” https://github.com/szxysdt/ rwkv tokenizer cpp, 2024

    2024

  44. [53]

    rwkv-tokenizer,

    C. Wirawan, “rwkv-tokenizer,” https://github.com/ cahya-wirawan/rwkv-tokenizer, 2024

    2024

  45. [54]

    rwkvk-rs,

    mrsteyk, “rwkvk-rs,” https://github.com/mrsteyk/rwkvk-rs, 2023

    2023

  46. [55]

    rwkv-rust,

    Y. Zero, “rwkv-rust,” https://github.com/yorkzero831/ rwkv-rust, 2023

    2023

  47. [56]

    smolrsrwkv,

    Kerfuffle, “smolrsrwkv,” https://github.com/KerfuffleV2/ smolrsrwkv, 2023

    2023

  48. [57]

    Rwkv5-tensorflow2.0,

    123mbcz123, “Rwkv5-tensorflow2.0,” https://github.com/ 123mbcz123/RWKV5-Tensorflow2.0, 2023

    2023

  49. [58]

    rwkv-tensorflow,

    RWKV , “rwkv-tensorflow,” https://github.com/RWKV/ rwkv-tensorflow, 2023

    2023

  50. [59]

    Tensor-rwkv,

    A. Videla, “Tensor-rwkv,” https://github.com/andvid72/ Tensor-RWKV, 2024

    2024

  51. [60]

    Rwkv-tensorflow2.0,

    ——, “Rwkv-tensorflow2.0,” https://github.com/123mbcz123/ RWKV-Tensorflow2.0, 2023

    2023

  52. [61]

    tensorflow-rwkv,

    P . Rouast, “tensorflow-rwkv,” https://github.com/prouast/ tensorflow-rwkv, 2023

    2023

  53. [62]

    rwkv-cuda,

    rj42, “rwkv-cuda,” https://github.com/rj42/rwkv-cuda, 2023

    2023

  54. [63]

    rwkv.f90,

    nlpodyssey, “rwkv.f90,” https://github.com/nlpodyssey/rwkv. f90, 2023

    2023

  55. [64]

    Rwkv-tf2,

    breadbread1984, “Rwkv-tf2,” https://github.com/ breadbread1984/RWKV-tf2, 2024

    2024

  56. [65]

    rwkv.zig,

    M. Whale, “rwkv.zig,” https://github.com/metalwhale/rwkv. zig, 2023

    2023

  57. [66]

    rwkv.jl,

    Y. Guan, “rwkv.jl,” https://github.com/guanyilun/rwkv.jl, 2023

    2023

  58. [67]

    seasonjs, “rwkv,” https://github.com/seasonjs/rwkv, 2023

    2023

  59. [68]

    nlpodyssey, “rwkv,” https://github.com/nlpodyssey/rwkv, 2023

    2023

  60. [69]

    verbaflow,

    nlpodyssey, “verbaflow,” https://github.com/nlpodyssey/ verbaflow, 2023

    2023

  61. [70]

    go-rwkv.cpp,

    J. Price, “go-rwkv.cpp,” https://github.com/donomii/go-rwkv. cpp, 2023

    2023

  62. [71]

    rwkv-tokenizer-go,

    Ronsor, “rwkv-tokenizer-go,” https://github.com/Ronsor/ rwkv-tokenizer-go, 2024

    2024

  63. [72]

    rwkvalgorithm,

    ZaraK0, “rwkvalgorithm,” https://github.com/ZaraK0/ rwkvalgorithm, 2024

    2024

  64. [73]

    keras-rwkv,

    D. Jack, “keras-rwkv,” https://github.com/jackd/keras-rwkv, 2023

    2023

  65. [74]

    Rwkv6-keras,

    pass lin, “Rwkv6-keras,” https://github.com/pass-lin/ RWKV6-Keras, 2024

    2024

  66. [75]

    Rwkv-v2-rnn-paddle,

    JunnYu, “Rwkv-v2-rnn-paddle,” https://github.com/JunnYu/ RWKV-v2-RNN-paddle, 2022

    2022

  67. [76]

    rwkv-paddle,

    HighCWu, “rwkv-paddle,” https://github.com/HighCWu/ rwkv-paddle, 2023

    2023

  68. [77]

    Rwkv-lm-jax,

    mrsteyk, “Rwkv-lm-jax,” https://github.com/mrsteyk/ RWKV-LM-jax, 2023

    2023

  69. [78]

    tpu rwkv,

    tensorpro, “tpu rwkv,” https://github.com/tensorpro/tpu rwkv, 2023

    2023

  70. [79]

    Rwkv-cpp-node,

    RWKV , “Rwkv-cpp-node,” https://github.com/RWKV/ RWKV-cpp-node, 2023

    2023

  71. [80]

    rwkv-jax,

    saran, “rwkv-jax,” https://github.com/saran-gangster/ rwkv-jax, 2024

    2024

  72. [81]

    mlx-rwkv,

    dc-dc dc, “mlx-rwkv,” https://github.com/dc-dc-dc/mlx-rwkv, 2024

    2024

  73. [82]

    Rwkv-tokenizer-node,

    ——, “Rwkv-tokenizer-node,” https://github.com/RWKV/ RWKV-tokenizer-node, 2023

    2023

  74. [83]

    Rwkv-infctx-trainer-lora,

    ——, “Rwkv-infctx-trainer-lora,” https://github.com/ OpenMOSE/RWKV-infctx-trainer-LoRA, 2024

    2024

  75. [84]

    Rwkv5-lm-lora,

    OpenMOSE, “Rwkv5-lm-lora,” https://github.com/ OpenMOSE/RWKV5-LM-LoRA, 2024

    2024

  76. [85]

    Rwkv-infer,

    OpenMOSE, “Rwkv-infer,” https://github.com/OpenMOSE/ RWKV-Infer, 2024

    2024

  77. [86]

    Rwkv-infctx-trainer,

    RWKV , “Rwkv-infctx-trainer,” https://github.com/RWKV/ RWKV-infctx-trainer, 2023

    2023

  78. [87]

    rwkv-ncnn,

    M. Sophia, “rwkv-ncnn,” https://github.com/MollySophia/ rwkv-ncnn, 2023

    2023

  79. [88]

    Rwkv-android,

    T. Zhang, “Rwkv-android,” https://github.com/ZTMIDGO/ RWKV-Android, 2023

    2023

  80. [89]

    rwkv.jni,

    V . Crana, “rwkv.jni,” https://github.com/vaccovecrana/rwkv. jni, 2023

    2023

  81. [90]

    rwkv-qualcomm,

    ——, “rwkv-qualcomm,” https://github.com/MollySophia/ rwkv-qualcomm, 2024

    2024

  82. [91]

    Rwkv role playing api,

    shengxia, “Rwkv role playing api,” https://github.com/ shengxia/RWKV Role Playing API, 2023

    2023

  83. [92]

    Rwkv6 keras operator,

    RWKV6, “Rwkv6 keras operator,” https://github.com/ infiy-quine/RWKV6 Keras Operator, 2024. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 16

    2024

  84. [93]

    rwkv chatbot,

    H. Vanderbyl, “rwkv chatbot,” https://github.com/ harrisonvanderbyl/rwkv chatbot, 2023

    2023

  85. [94]

    nanorwkv,

    Hannibal046, “nanorwkv,” https://github.com/Hannibal046/ nanoRWKV, 2023

    2023

  86. [95]

    rwkv chat command line,

    az13js, “rwkv chat command line,” https://github.com/ az13js/rwkv chat command line, 2024

    2024

  87. [96]

    Rwkv-wechat-bot,

    averyyan, “Rwkv-wechat-bot,” https://github.com/averyyan/ RWKV-wechat-bot, 2024

    2024

  88. [97]

    Cryscan, “Eloise,” https://github.com/cryscan/eloise, 2023

    2023

  89. [98]

    Combining information retrieval and large language models for a chatbot that generates reliable, natural-style answers

    A. Lommatzsch, B. Llanque, V . S. Rosenberg, S. A. M. Tahir, H. D. Boyadzhiev, and M. Walny, “Combining information retrieval and large language models for a chatbot that generates reliable, natural-style answers.” in LWDA, 2023, pp. 298–310

    2023

  90. [99]

    Espritchatbot-rasa-rwkv,

    M. K. Akkari, “Espritchatbot-rasa-rwkv,” https://github.com/ kimou6055/Espritchatbot-RASA-RWKV, 2023

    2023

  91. [100]

    Meow-ai,

    Bilibini, “Meow-ai,” https://github.com/bilibini/Meow-AI, 2024

    2024

  92. [101]

    Chatrwkv-in-wechat-version-1,

    MrTom34, “Chatrwkv-in-wechat-version-1,” https://github. com/MrTom34/ChatRWKV-in-wechat-Version-1, 2023

    2023

  93. [102]

    Rwkv-rag,

    AIIRWKV , “Rwkv-rag,” https://github.com/AIIRWKV/ RWKV-RAG, 2024

    2024

  94. [103]

    Rwkv-drama,

    ms KuroNeko, “Rwkv-drama,” https://github.com/ ms-KuroNeko/RWKV-Drama, 2024

    2024

  95. [104]

    Infofusion,

    L. Lin, “Infofusion,” https://github.com/LeoLin4258/ Infofusion, 2024

    2024

  96. [105]

    Kosev, “notgpt,” https://github.com/spion/notgpt, 2023

    G. Kosev, “notgpt,” https://github.com/spion/notgpt, 2023

    2023

  97. [106]

    Rwkv role playing with gpt-sovits,

    Y. Liu, “Rwkv role playing with gpt-sovits,” https://github. com/v3ucn/RWKV Role Playing with GPT-SoVITS, 2024

    2024

  98. [107]

    Easy rwkv webui,

    No-22-Github, “Easy rwkv webui,” https://github.com/ No-22-Github/Easy RWKV webui, 2024

    2024

  99. [108]

    123summertime, “ykkz,” https://github.com/123summertime/ ykkz, 2023

    2023

  100. [109]

    lala rwkv chatbot 2.0,

    J. Zhang, “lala rwkv chatbot 2.0,” https://github.com/ samttoo22-MewCat/lala rwkv chatbot 2.0, 2024

    2024

  101. [110]

    Espitchatbot-rasa-raven,

    K. ALOULOU, “Espitchatbot-rasa-raven,” https://github.com/ karim-aloulou/Espitchatbot-RASA-RAVEN, 2023

    2023

  102. [111]

    Onlysportslm: Optimizing sports-domain language models with sota performance under billion parameters,

    Z. Chen, C. Li, X. Xie, and P . Dube, “Onlysportslm: Optimizing sports-domain language models with sota performance under billion parameters,” arXiv preprint arXiv:2409.00286, 2024

    2024 arXiv

  103. [112]

    pdf parsing,

    ck unifr, “pdf parsing,” https://github.com/ck-unifr/pdf parsing, 2023

    2023

  104. [113]

    novel-rwkv demo,

    jiawanfan yyds, “novel-rwkv demo,” https://github.com/ jiawanfan-yyds/novel-rwkv demo, 2023

    2023

  105. [114]

    Rwkv-classification,

    Y. Lin, “Rwkv-classification,” https://github.com/yynil/ RWKV-Classification, 2023

    2023

  106. [115]

    Ravenwhisperer,

    P . Seitz, “Ravenwhisperer,” https://github.com/seitzquest/ RavenWhisperer, 2023

    2023

  107. [116]

    ai-town-rwkv-proxy,

    R. AI, “ai-town-rwkv-proxy,” https://github.com/recursal/ ai-town-rwkv-proxy, 2023

    2023

  108. [117]

    Avatario,

    M. K. Azzabi, “Avatario,” https://github.com/khhaliil/ AVATARIO, 2023

    2023

  109. [118]

    wenda-rwkv,

    xiaol, “wenda-rwkv,” https://github.com/xiaol/wenda-RWKV, 2023

    2023

  110. [119]

    Contrastive learn- ing for clinical outcome prediction with partial data sources,

    M. Xia, J. Wilson, B. Goldstein, and R. Henao, “Contrastive learn- ing for clinical outcome prediction with partial data sources,” Proceedings of machine learning research, vol. 235, p. 54156, 2024

    2024

  111. [120]

    rwkv kg,

    Ojiyumm, “rwkv kg,” https://github.com/Ojiyumm/rwkv kg, 2024

    2024

  112. [121]

    Dft: A dual-branch framework of fluctuation and trend for stock price prediction,

    C. Dong, Z. Cao, S. K. Zhou, and J. Liu, “Dft: A dual-branch framework of fluctuation and trend for stock price prediction,” arXiv preprint arXiv:2411.06065, 2024

    2024 arXiv

  113. [122]

    Stock-prediction-using-rwkv,

    tomer9080, “Stock-prediction-using-rwkv,” https://github.com/ tomer9080/Stock-Prediction-Using-RWKV, 2023

    2023

  114. [123]

    A code completion model based rwkv with bimodal pretraining,

    L. Zhou and S. Li, “A code completion model based rwkv with bimodal pretraining,” 2023

    2023

  115. [124]

    Matcc: A novel approach for robust stock price prediction incorporating market trends and cross-time correlations,

    Z. Cao, J. Xu, C. Dong, P . Yu, and T. Bai, “Matcc: A novel approach for robust stock price prediction incorporating market trends and cross-time correlations,” inProceedings of the 33rd ACM International Conference on Information and Knowledge Management, 2024, pp. 187–196

    2024

  116. [125]

    Experi- mentation in content moderation using rwkv,

    U. Yildirim, R. Dutta, B. Yildirim, and A. Vaidya, “Experi- mentation in content moderation using rwkv,” arXiv preprint arXiv:2409.03939, 2024

    2024 arXiv

  117. [126]

    Rwkv-based encoder-decoder model for code completion,

    L. Zhou, Z. Xiao, and Z. Ning, “Rwkv-based encoder-decoder model for code completion,” in 2023 3rd International Conference on Electronic Information Engineering and Computer (EIECT). IEEE, 2023, pp. 425–428

    2023

  118. [127]

    Rwkv-ts: Beyond traditional recurrent neu- ral network for time series tasks,

    H. Hou and F. R. Yu, “Rwkv-ts: Beyond traditional recurrent neu- ral network for time series tasks,” arXiv preprint arXiv:2401.09093, 2024

    2024 arXiv

  119. [128]

    Goldfinch: High performance rwkv/transformer hybrid with linear pre-fill and extreme kv-cache compression,

    D. Goldstein, F. Obeid, E. Alcaide, G. Song, and E. Cheah, “Goldfinch: High performance rwkv/transformer hybrid with linear pre-fill and extreme kv-cache compression,” arXiv preprint arXiv:2407.12077, 2024

    2024 arXiv

  120. [129]

    Rrwkv: capturing long-range dependencies in rwkv,

    L. Wang, “Rrwkv: capturing long-range dependencies in rwkv,” arXiv preprint arXiv:2306.05176, 2023

    2023 arXiv

  121. [130]

    Temporal and interactive modeling for efficient human-human motion generation,

    Y. Wang, S. Wang, J. Zhang, K. Fan, J. Wu, Z. Jiang, and Y. Liu, “Temporal and interactive modeling for efficient human-human motion generation,” arXiv preprint arXiv:2408.17135, 2024

    2024

  122. [131]

    Optimizing robotic manipulation with decision-rwkv: A recurrent sequence modeling approach for lifelong learning,

    Y. Dong, T. Wu, and C. Song, “Optimizing robotic manipulation with decision-rwkv: A recurrent sequence modeling approach for lifelong learning,” arXiv preprint arXiv:2407.16306, 2024

    2024 arXiv

  123. [132]

    Lkpnr: Large language models and knowledge graph for per- sonalized news recommendation framework

    H. Chen, R. Xie, X. Cui, Z. Yan, X. Wang, Z. Xuan, and K. Zhang, “Lkpnr: Large language models and knowledge graph for per- sonalized news recommendation framework.” Computers, Mate- rials & Continua, vol. 79, no. 3, 2024

    2024

  124. [133]

    Spikegpt: Gener- ative pre-trained language model with spiking neural networks,

    R.-J. Zhu, Q. Zhao, G. Li, and J. K. Eshraghian, “Spikegpt: Gener- ative pre-trained language model with spiking neural networks,” arXiv preprint arXiv:2302.13939, 2023

    2023 arXiv

  125. [134]

    Prosg: Using prompt synthetic gradients to alleviate prompt forgetting of rnn- like language models,

    H. Luo, K. Wu, C. Dai, S. Ding, and X. Chen, “Prosg: Using prompt synthetic gradients to alleviate prompt forgetting of rnn- like language models,” arXiv preprint arXiv:2311.01981, 2023

    2023 arXiv

  126. [135]

    Enhancing trans- former rnns with multiple temporal perspectives,

    R.-G. Dumitru, D. Peteleaza, and M. Surdeanu, “Enhancing trans- former rnns with multiple temporal perspectives,” arXiv preprint arXiv:2402.02625, 2024

    2024 arXiv

  127. [136]

    General population projection model with census population data,

    T. Tsuruga, “General population projection model with census population data,” Electronic Theses, Projects, and Dissertations. , 2023

    2023

  128. [137]

    A transfer learning- based training approach for dga classification,

    A. Drichel, B. von Querfurth, and U. Meyer, “A transfer learning- based training approach for dga classification,” in International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. Springer, 2024, pp. 381–391

    2024

  129. [138]

    Sensorimotor attention and language- based regressions in shared latent variables for integrating robot motion learning and llm,

    K. Suzuki and T. Ogata, “Sensorimotor attention and language- based regressions in shared latent variables for integrating robot motion learning and llm,” arXiv preprint arXiv:2407.09044, 2024

    2024 arXiv

  130. [139]

    Restore-rwkv: Efficient and effective medical image restoration with rwkv,

    Z. Yang, H. Zhang, D. Zhao, B. Wei, and Y. Xu, “Restore-rwkv: Efficient and effective medical image restoration with rwkv,” arXiv preprint arXiv:2407.11087, 2024

    2024 arXiv

  131. [140]

    Bsbp-rwkv: Background suppression with boundary preservation for efficient medical image segmenta- tion,

    X. Zhou and T. Chen, “Bsbp-rwkv: Background suppression with boundary preservation for efficient medical image segmenta- tion,” in ACM Multimedia 2024, 2024

    2024

  132. [141]

    Lion: Linear group rnn for 3d object detection in point clouds,

    Z. Liu, J. Hou, X. Wang, X. Ye, J. Wang, H. Zhao, and X. Bai, “Lion: Linear group rnn for 3d object detection in point clouds,” arXiv preprint arXiv:2407.18232, 2024

    2024 arXiv

  133. [142]

    Rnn-based multiple instance learning for the classification of histopathology whole slide images,

    G. Ji and P . Liu, “Rnn-based multiple instance learning for the classification of histopathology whole slide images,” in Interna- tional Conference on Medical Imaging and Computer-Aided Diagnosis. Springer, 2023, pp. 329–339

    2023

  134. [143]

    Occrwkv: Rethinking efficient 3d semantic occupancy prediction with linear complexity,

    J. Wang, W. Yin, X. Long, X. Zhang, Z. Xing, X. Guo, and Q. Zhang, “Occrwkv: Rethinking efficient 3d semantic occupancy prediction with linear complexity,” arXiv preprint arXiv:2409.19987, 2024

    2024 arXiv

  135. [144]

    Pointr- wkv: Efficient rwkv-like model for hierarchical point cloud learn- ing,

    Q. He, J. Zhang, J. Peng, H. He, Y. Wang, and C. Wang, “Pointr- wkv: Efficient rwkv-like model for hierarchical point cloud learn- ing,” arXiv preprint arXiv:2405.15214, 2024

    2024 arXiv

  136. [145]

    From explicit rules to implicit reasoning in an interpretable violence monitoring system,

    W.-D. Jiang, C.-Y. Chang, H.-C. Chang, and D. S. Roy, “From explicit rules to implicit reasoning in an interpretable violence monitoring system,” arXiv preprint arXiv:2410.21991, 2024

    2024 arXiv

  137. [146]

    Tls-rwkv: Real-time online action detection with temporal label smoothing,

    Z. Zhu, W. Shao, and D. Jiao, “Tls-rwkv: Real-time online action detection with temporal label smoothing,” Neural Processing Let- ters, vol. 56, no. 2, p. 57, 2024

    2024

  138. [147]

    Mamba or rwkv: Exploring high-quality and high-efficiency segment anything model,

    H. Yuan, X. Li, L. Qi, T. Zhang, M.-H. Yang, S. Yan, and C. C. Loy, “Mamba or rwkv: Exploring high-quality and high-efficiency segment anything model,” arXiv preprint arXiv:2406.19369, 2024

    2024 arXiv

  139. [148]

    Hybrid recurrent-attentive neural network for onboard predictive hyperspectral image com- pression,

    D. Valsesia, T. Bianchi, and E. Magli, “Hybrid recurrent-attentive neural network for onboard predictive hyperspectral image com- pression,” in IGARSS 2024-2024 IEEE International Geoscience and Remote Sensing Symposium. IEEE, 2024, pp. 7898–7902

    2024

  140. [149]

    Visualrwkv-hd and uhd: Advancing high- resolution processing for visual language models,

    Z. Li and H. Hou, “Visualrwkv-hd and uhd: Advancing high- resolution processing for visual language models,” arXiv preprint arXiv:2410.11665, 2024

    2024 arXiv

  141. [150]

    Vision-rwkv: Efficient and scalable JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 17 visual perception with rwkv-like architectures,

    Y. Duan, W. Wang, Z. Chen, X. Zhu, L. Lu, T. Lu, Y. Qiao, H. Li, J. Dai, and W. Wang, “Vision-rwkv: Efficient and scalable JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 17 visual perception with rwkv-like architectures,” arXiv preprint arXiv:2403.02308, 2024

    2015 arXiv

  142. [151]

    Rwkv-clip: A robust vision-language representation learner,

    T. Gu, K. Yang, X. An, Z. Feng, D. Liu, W. Cai, and J. Deng, “Rwkv-clip: A robust vision-language representation learner,” arXiv preprint arXiv:2406.06973, 2024

    2024 arXiv

  143. [152]

    Video rwkv: Video action recognition based rwkv,

    Z. Yin, C. Li, and X. Dong, “Video rwkv: Video action recognition based rwkv,” arXiv preprint arXiv:2411.05636, 2024

    2024 arXiv

  144. [153]

    Social-cvae: Pedes- trian trajectory prediction using conditional variational auto- encoder,

    B. Xu, X. Wang, S. Li, J. Li, and C. Liu, “Social-cvae: Pedes- trian trajectory prediction using conditional variational auto- encoder,” in International Conference on Neural Information Process- ing. Springer, 2023, pp. 476–489

    2023

  145. [154]

    Sdit: Spiking dif- fusion model with transformer,

    S. Yang, H. Ma, C. Yu, A. Wang, and E.-P . Li, “Sdit: Spiking dif- fusion model with transformer,” arXiv preprint arXiv:2402.11588 , 2024

    2024 arXiv

  146. [155]

    Exploring real&synthetic dataset and linear attention in image restoration,

    Y. Du, T. Hu, J. Zhang, R. Y. C. Xu, X. Hu, K. Wu, D. Luo, Y. Wang, and L. Ma, “Exploring real&synthetic dataset and linear attention in image restoration,” 2024. [Online]. Available: https://arxiv.org/abs/2412.03814

    2024 arXiv

  147. [156]

    Diffusion-rwkv: Scal- ing rwkv-like architectures for diffusion models,

    Z. Fei, M. Fan, C. Yu, D. Li, and J. Huang, “Diffusion-rwkv: Scal- ing rwkv-like architectures for diffusion models,” arXiv preprint arXiv:2404.04478, 2024

    2024 arXiv

  148. [157]

    Facerwkv,

    L. Vierling, “Facerwkv,” https://github.com/lukasVierling/ FaceRWKV, 2023

    2023

  149. [158]

    Stylerwkv: High-quality and high-efficiency style transfer with rwkv-like architecture,

    M. Dai, Q. Zhou, and L. Ma, “Stylerwkv: High-quality and high-efficiency style transfer with rwkv-like architecture,” arXiv preprint arXiv:2412.19535, 2024

    2024 arXiv

  150. [159]

    Android-rwkv-midi,

    T. Zhang, “Android-rwkv-midi,” https://github.com/ ZTMIDGO/Android-RWKV-MIDI, 2023

    2023

  151. [160]

    rwkv-denoise,

    lll143653, “rwkv-denoise,” https://github.com/lll143653/ rwkv-denoise, 2024

    2024

  152. [161]

    Procedural-purgatory,

    agreene5, “Procedural-purgatory,” https://github.com/ agreene5/Procedural-Purgatory, 2024

    2024

  153. [162]

    Rwkv-v4-midi,

    Patchbanks, “Rwkv-v4-midi,” https://github.com/patchbanks/ RWKV-v4-MIDI, 2024

    2024

  154. [163]

    Advancing vad systems based on multi-task learning with improved model structures,

    L. Zuo, K. An, S. Zhang, and Z. Yan, “Advancing vad systems based on multi-task learning with improved model structures,” arXiv preprint arXiv:2312.14860, 2023

    2023 arXiv

  155. [164]

    Why perturbing symbolic mu- sic is necessary: Fitting the distribution of never-used notes through a joint probabilistic diffusion model,

    S. Liu, X. Fan, and G. Wu, “Why perturbing symbolic mu- sic is necessary: Fitting the distribution of never-used notes through a joint probabilistic diffusion model,” arXiv preprint arXiv:2408.01950, 2024

    2024 arXiv

  156. [165]

    Rwkv-asr,

    J. Jiang, “Rwkv-asr,” https://github.com/AGENDD/ RWKV-ASR, 2024

    2024

  157. [166]

    Exploring rwkv for memory efficient and low latency streaming asr,

    K. An and S. Zhang, “Exploring rwkv for memory efficient and low latency streaming asr,” arXiv preprint arXiv:2309.14758, 2023

    2023 arXiv

  158. [167]

    Rwkv-musicgenerator,

    A. Suller, “Rwkv-musicgenerator,” https://github.com/asuller/ RWKV-MusicGenerator, 2023

    2023

  159. [168]

    Audiorwkv,

    X. Du, “Audiorwkv,” https://github.com/diggerdu/ AudioRWKV, 2024

    2024

  160. [169]

    Rwkv-speechchat,

    J. Jiang, “Rwkv-speechchat,” https://github.com/AGENDD/ RWKV-SpeechChat, 2024

    2024

  161. [170]

    Music-genre-classification-rwkv,

    AverageJoe9, “Music-genre-classification-rwkv,” https://github. com/AverageJoe9/Music-Genre-Classification-RWKV, 2024

    2024

  162. [171]

    Rwkv 3b 7b webui gpt-sovits,

    Y. Liu, “Rwkv 3b 7b webui gpt-sovits,” https://github.com/ v3ucn/RWKV 3B 7B Webui GPT-SoVITS, 2024

    2024

  163. [172]

    Mozi-rwkv,

    YuChuXi, “Mozi-rwkv,” https://github.com/YuChuXi/ MoZi-RWKV, 2024

    2024

  164. [173]

    rwkv-v4-web,

    josephrocca, “rwkv-v4-web,” https://github.com/josephrocca/ rwkv-v4-web, 2023

    2023

  165. [174]

    ai00 server,

    Ai00-X, “ai00 server,” https://github.com/Ai00-X/ai00 server, 2024

    2024

  166. [175]

    Rwkv qqbot backend,

    yuunnn w, “Rwkv qqbot backend,” https://github.com/ yuunnn-w/RWKV QQBot BackEnd, 2024

    2024

  167. [176]

    Rwkv role playing,

    shengxia, “Rwkv role playing,” https://github.com/shengxia/ RWKV Role Playing, 2023

    2023

  168. [177]

    Webchatrwkvstic,

    H. Felix, “Webchatrwkvstic,” https://github.com/hizkifw/ WebChatRWKVstic, 2023

    2023

  169. [178]

    web-rwkv-axum,

    J. LI, “web-rwkv-axum,” https://github.com/Prunoideae/ web-rwkv-axum, 2024

    2024

  170. [179]

    Chatrwkv-flask-api,

    t4wefan, “Chatrwkv-flask-api,” https://github.com/t4wefan/ ChatRWKV-flask-api, 2023

    2023

  171. [180]

    chatux-server-rwkv,

    riversun, “chatux-server-rwkv,” https://github.com/riversun/ chatux-server-rwkv, 2023

    2023

  172. [181]

    Rwkv-api,

    RafaRed, “Rwkv-api,” https://github.com/RafaRed/ RWKV-api, 2023

    2023

  173. [182]

    Chatrwkv-webui,

    StarDreamAndFeng, “Chatrwkv-webui,” https://github.com/ StarDreamAndFeng/ChatRWKV-webui, 2023

    2023

  174. [183]

    Rwkv role playing ui,

    shengxia, “Rwkv role playing ui,” https://github.com/ shengxia/RWKV Role Playing UI, 2023

    2023

  175. [184]

    rwkv.cpp webui macos,

    Y. Liu, “rwkv.cpp webui macos,” https://github.com/v3ucn/ rwkv.cpp webui Macos, 2024

    2024

  176. [185]

    In-context language learning: Arhitectures and algorithms,

    E. Aky ¨urek, B. Wang, Y. Kim, and J. Andreas, “In-context language learning: Arhitectures and algorithms,” arXiv preprint arXiv:2401.12973, 2024

    2024 arXiv

  177. [186]

    rwkv.cpp webui gpt-sovits,

    Y. Liu, “rwkv.cpp webui gpt-sovits,” https://github.com/ v3ucn/rwkv.cpp webui GPT-SoVITS, 2024

    2024

  178. [187]

    Loogle: Can long- context language models understand long contexts?

    J. Li, M. Wang, Z. Zheng, and M. Zhang, “Loogle: Can long- context language models understand long contexts?” arXiv preprint arXiv:2311.04939, 2023

    2023 arXiv

  179. [188]

    Kv cache compres- sion, but what must we give in return? a comprehensive benchmark of long context capable approaches,

    J. Yuan, H. Liu, Y.-N. Chuang, S. Li, G. Wang, D. Le, H. Jin, V . Chaudhary, Z. Xu, Z. Liu et al. , “Kv cache compres- sion, but what must we give in return? a comprehensive benchmark of long context capable approaches,” arXiv preprint arXiv:2407.01527, 2024

    2024 arXiv

  180. [189]

    S3eval: A synthetic, scalable, systematic evaluation suite for large language models,

    F. Lei, Q. Liu, Y. Huang, S. He, J. Zhao, and K. Liu, “S3eval: A synthetic, scalable, systematic evaluation suite for large language models,” arXiv preprint arXiv:2310.15147, 2023

    2023 arXiv

  181. [190]

    Ruler: What’s the real context size of your long- context language models?

    C.-P . Hsieh, S. Sun, S. Kriman, S. Acharya, D. Rekesh, F. Jia, and B. Ginsburg, “Ruler: What’s the real context size of your long- context language models?” arXiv preprint arXiv:2404.06654, 2024

    2024 arXiv

  182. [191]

    Mango: A benchmark for evaluating mapping and navigation abilities of large language models,

    P . Ding, J. Fang, P . Li, K. Wang, X. Zhou, M. Yu, J. Li, M. R. Walter, and H. Mei, “Mango: A benchmark for evaluating mapping and navigation abilities of large language models,” arXiv preprint arXiv:2403.19913, 2024

    2024 arXiv

  183. [192]

    Mag- nifico: Evaluating the in-context learning ability of large language models to generalize to novel interpretations,

    A. Patel, S. Bhattamishra, S. Reddy, and D. Bahdanau, “Mag- nifico: Evaluating the in-context learning ability of large language models to generalize to novel interpretations,” arXiv preprint arXiv:2310.11634, 2023

    2023 arXiv

  184. [193]

    Zhujiu: A multi-dimensional, multi-faceted chinese benchmark for large language models,

    B. Zhang, H. Xie, P . Du, J. Chen, P . Cao, Y. Chen, S. Liu, K. Liu, and J. Zhao, “Zhujiu: A multi-dimensional, multi-faceted chinese benchmark for large language models,” arXiv preprint arXiv:2308.14353, 2023

    2023 arXiv

  185. [194]

    Superclue: A comprehensive chinese large language model benchmark,

    L. Xu, A. Li, L. Zhu, H. Xue, C. Zhu, K. Zhao, H. He, X. Zhang, Q. Kang, and Z. Lan, “Superclue: A comprehensive chinese large language model benchmark,” arXiv preprint arXiv:2307.15020 , 2023

    2023 arXiv

  186. [195]

    Benchmarking large language models on controllable generation under diversified instructions,

    Y. Chen, B. Xu, Q. Wang, Y. Liu, and Z. Mao, “Benchmarking large language models on controllable generation under diversified instructions,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 16, 2024, pp. 17 808–17 816

    2024

  187. [196]

    Benchmarking and defending against indirect prompt injection attacks on large language models,

    J. Yi, Y. Xie, B. Zhu, K. Hines, E. Kiciman, G. Sun, X. Xie, and F. Wu, “Benchmarking and defending against indirect prompt injection attacks on large language models,” arXiv preprint arXiv:2312.14197, 2023

    2023 arXiv

  188. [197]

    Cmath: Can your language model pass chinese elementary school math test?

    T. Wei, J. Luan, W. Liu, S. Dong, and B. Wang, “Cmath: Can your language model pass chinese elementary school math test?”arXiv preprint arXiv:2306.16636, 2023

    2023 arXiv

  189. [198]

    Evaluating agents using social choice theory,

    M. Lanctot, K. Larson, Y. Bachrach, L. Marris, Z. Li, A. Bhoopc- hand, T. Anthony, B. Tanner, and A. Koop, “Evaluating agents using social choice theory,” arXiv preprint arXiv:2312.03121, 2023

    2023 arXiv

  190. [199]

    Pre: A peer review based large language model evaluator,

    Z. Chu, Q. Ai, Y. Tu, H. Li, and Y. Liu, “Pre: A peer review based large language model evaluator,” arXiv preprint arXiv:2401.15641, 2024

    2024 arXiv

  191. [200]

    Head-to-tail: How knowledgeable are large language models (llm),

    K. Sun, Y. E. Xu, H. Zha, Y. Liu, and X. L. Dong, “Head-to-tail: How knowledgeable are large language models (llm),” AKA will llms replace knowledge graphs, 2023

    2023

  192. [201]

    Long-context llms struggle with long in-context learning,

    T. Li, G. Zhang, Q. D. Do, X. Yue, and W. Chen, “Long-context llms struggle with long in-context learning,” arXiv preprint arXiv:2404.02060, 2024

    2024 arXiv

  193. [202]

    Evaluating llms at eval- uating temporal generalization,

    C. Zhu, N. Chen, Y. Gao, and B. Wang, “Evaluating llms at eval- uating temporal generalization,” arXiv preprint arXiv:2405.08460, 2024

    2024 arXiv

  194. [203]

    rwkv-long-range-arena,

    X. Du, “rwkv-long-range-arena,” https://github.com/ diggerdu/rwkv-long-range-arena, 2023

    2023

  195. [204]

    Longbench rwkv,

    Ojiyumm, “Longbench rwkv,” https://github.com/Ojiyumm/ LongBench RWKV, 2024

    2024

  196. [205]

    Is attention required for icl? exploring the relationship between model architecture and JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 18 in-context learning ability,

    I. Lee, N. Jiang, and T. Berg-Kirkpatrick, “Is attention required for icl? exploring the relationship between model architecture and JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 18 in-context learning ability,” in The Twelfth International Conference on Learning R...

    2015

  197. [206]

    Mt bench rwkv,

    Ojiyumm, “Mt bench rwkv,” https://github.com/Ojiyumm/ MT BENCH RWKV, 2024

    2024

  198. [207]

    Veliki jezikovni modeli so strojni ucenci v casu sklepanja,

    K. Grm, “Veliki jezikovni modeli so strojni ucenci v casu sklepanja,” 2023

    2023

  199. [208]

    Revenge of the fallen? recurrent models match transformers at predict- ing human language comprehension metrics,

    J. A. Michaelov, C. Arnett, and B. K. Bergen, “Revenge of the fallen? recurrent models match transformers at predict- ing human language comprehension metrics,” arXiv preprint arXiv:2404.19178, 2024

    2024 arXiv

  200. [209]

    Does transformer in- terpretability transfer to rnns?

    G. Paulo, T. Marshall, and N. Belrose, “Does transformer in- terpretability transfer to rnns?” arXiv preprint arXiv:2404.05971 , 2024

    2024 arXiv

  201. [210]

    Benchmarking neural decoding backbones towards enhanced on-edge ibci applications,

    Z. Zhou, G. He, Z. Zhang, L. Leng, Q. Guo, J. Liao, X. Song, and R. Cheng, “Benchmarking neural decoding backbones towards enhanced on-edge ibci applications,” arXiv preprint arXiv:2406.06626, 2024

    2024 arXiv

  202. [211]

    How well can a long sequence model model long sequences? comparing architechtural inductive biases on long- context abilities,

    J. Huang, “How well can a long sequence model model long sequences? comparing architechtural inductive biases on long- context abilities,” arXiv preprint arXiv:2407.08112, 2024

    2024 arXiv

  203. [212]

    From words to numbers: Your large language model is secretly a ca- pable regressor when given in-context examples,

    R. Vacareanu, V .-A. Negru, V . Suciu, and M. Surdeanu, “From words to numbers: Your large language model is secretly a ca- pable regressor when given in-context examples,” arXiv preprint arXiv:2404.07544, 2024

    2024 arXiv

  204. [213]

    Spiking mixers for robust and energy-efficient vision-and-language learning,

    M. Chen and E. Li, “Spiking mixers for robust and energy-efficient vision-and-language learning,” 2024. [Online]. Available: https://openreview.net/forum?id=FyZaVdQLdJ

    2024

  205. [214]

    compare llms,

    ´Ad´am M ´ark Bir ´o, “compare llms,” https://github.com/ BiroAd/compare llms, 2024

    2024

  206. [215]

    Visualrwkv: Exploring recurrent neural networks for visual language models,

    H. Hou, P . Zeng, F. Ma, and F. R. Yu, “Visualrwkv: Exploring recurrent neural networks for visual language models,” arXiv preprint arXiv:2406.13362, 2024

    2024 arXiv

  207. [216]

    Minirwkv-4,

    StarRing, “Minirwkv-4,” https://github.com/StarRing2022/ MiniRWKV-4, 2023

    2023

  208. [217]

    Rwkv lm ext,

    Y. Lin, “Rwkv lm ext,” https://github.com/yynil/RWKV LM EXT, 2024

    2024

  209. [218]

    Dlip-rwkv,

    StarRing, “Dlip-rwkv,” https://github.com/StarRing2022/ Dlip-RWKV, 2023

    2023

  210. [219]

    Bot-ani-rwkv-twitter-bot-detection,

    Max-SF1, “Bot-ani-rwkv-twitter-bot-detection,” https://github. com/Max-SF1/Bot-Ani-RWKV-twitter-bot-detection, 2024

    2024

  211. [220]

    rwkv-reward-enhanced,

    I. Neon, “rwkv-reward-enhanced,” https://github.com/ Mazidad/rwkv-reward-enhanced, 2023

    2023

  212. [221]

    Rwkv-peft,

    JL-er, “Rwkv-peft,” https://github.com/JL-er/RWKV-PEFT, 2024

    2024

  213. [222]

    A unified implicit attention formulation for gated-linear recurrent sequence models,

    I. Zimerman, A. Ali, and L. Wolf, “A unified implicit attention formulation for gated-linear recurrent sequence models,” arXiv preprint arXiv:2405.16504, 2024

    2024 arXiv

  214. [223]

    Rwkv5-infctxlm,

    JL-er, “Rwkv5-infctxlm,” https://github.com/JL-er/ RWKV5-infctxLM, 2023

    2023

  215. [224]

    Rwkv-peft-simple,

    Seikaiju, “Rwkv-peft-simple,” https://github.com/Seikaijyu/ RWKV-PEFT-Simple, 2024

    2024

  216. [225]

    Rwkv-nonogram,

    LeC-Z, “Rwkv-nonogram,” https://github.com/LeC-Z/ RWKV-nonogram, 2024

    2024

  217. [226]

    Decisionrwkv,

    S. Design and L. Lab, “Decisionrwkv,” https://github.com/ ancorasir/DecisionRWKV, 2024

    2024

  218. [227]

    Trainchatgalrwkv,

    SynthiaDL, “Trainchatgalrwkv,” https://github.com/ SynthiaDL/TrainChatGalRWKV, 2023

    2023

  219. [228]

    Rwkv rul,

    S. Li, “Rwkv rul,” https://github.com/ShixiangLi/RWKV RUL, 2024

    2024

  220. [229]

    simple rwkv,

    Nintorac, “simple rwkv,” https://github.com/Nintorac/ simple rwkv, 2023

    2023

  221. [230]

    tinyrwkv,

    wozeparrot, “tinyrwkv,” https://github.com/wozeparrot/ tinyrwkv, 2023

    2023

  222. [231]

    llama-node,

    Atome-FE, “llama-node,” https://github.com/Atome-FE/ llama-node, 2023

    2023

  223. [232]

    Rwkv-runner,

    josStorer, “Rwkv-runner,” https://github.com/josStorer/ RWKV-Runner, 2024

    2024

  224. [233]

    Hf-for-rwkvraven-alpaca,

    StarRing, “Hf-for-rwkvraven-alpaca,” https://github.com/ StarRing2022/HF-For-RWKVRaven-Alpaca, 2023

    2023

  225. [234]

    godot-rwkv,

    H. Vanderbyl, “godot-rwkv,” https://github.com/ harrisonvanderbyl/godot-rwkv, 2024

    2024

  226. [235]

    Decision-rwkv,

    typoverflow, “Decision-rwkv,” https://github.com/ typoverflow/Decision-RWKV, 2023

    2023

  227. [236]

    rwkv reward,

    J. Kong, “rwkv reward,” https://github.com/jiamingkong/ rwkv reward, 2023

    2023

  228. [237]

    Rwkv-lm-state-4bit-orpo,

    ——, “Rwkv-lm-state-4bit-orpo,” https://github.com/ OpenMOSE/RWKV-LM-State-4bit-Orpo, 2024

    2024

  229. [238]

    Rwkv-lm-rlhf,

    OpenMOSE, “Rwkv-lm-rlhf,” https://github.com/OpenMOSE/ RWKV-LM-RLHF, 2024

    2024

  230. [239]

    Rwkv-finetune-script,

    D. Tarasov, “Rwkv-finetune-script,” https://github.com/ Durham/RWKV-finetune-script, 2023

    2023

  231. [240]

    Modern sequence models in context of multi-agent reinforcement learning,

    J. Thapa, “Modern sequence models in context of multi-agent reinforcement learning,” 2024

    2024

  232. [241]

    Rwkv-lm-lora-ja,

    if001, “Rwkv-lm-lora-ja,” https://github.com/if001/ RWKV-LM-LoRA-ja, 2023

    2023

  233. [242]

    Hf-for-rwkvworld-loraalpaca,

    StarRing, “Hf-for-rwkvworld-loraalpaca,” https://github.com/ StarRing2022/HF-For-RWKVWorld-LoraAlpaca, 2023

    2023

  234. [243]

    Rwkv-statetuning,

    Jellyfish042, “Rwkv-statetuning,” https://github.com/ Jellyfish042/RWKV-StateTuning, 2024

    2024

  235. [244]

    Rwkv-light-api,

    ssg qwq, “Rwkv-light-api,” https://github.com/ssg-qwq/ RWKV-Light-API, 2023

    2023

  236. [245]

    Rwkv-lm-interpretability- research,

    UnstoppableCurry, “Rwkv-lm-interpretability- research,” https://github.com/UnstoppableCurry/ RWKV-LM-Interpretability-Research, 2023

    2023

  237. [246]

    chunkrwkv6,

    00ffcc, “chunkrwkv6,” https://github.com/00ffcc/ chunkRWKV6, 2024

    2024

  238. [247]

    rwkv umap,

    J. LI, “rwkv umap,” https://github.com/Prunoideae/rwkv umap, 2023

    2023

  239. [248]

    web-rwkv-inspector,

    cryscan, “web-rwkv-inspector,” https://github.com/cryscan/ web-rwkv-inspector, 2024

    2024

  240. [249]

    Llmfarm,

    guinmoon, “Llmfarm,” https://github.com/guinmoon/ LLMFarm, 2023

    2023

  241. [250]

    rwkv-tensorrt,

    D. wanna M, “rwkv-tensorrt,” https://github.com/ Dan-wanna-M/rwkv-tensorrt, 2023

    2023

  242. [251]

    Rwkv-ouroboros-app,

    neromous, “Rwkv-ouroboros-app,” https://github.com/ neromous/RWKV-Ouroboros-app, 2023

    2023

  243. [252]

    Rwkv chains,

    J. Kong, “Rwkv chains,” https://github.com/jiamingkong/ RWKV chains, 2023

    2023

  244. [253]

    Rwkvinllama,

    Y. Lin, “Rwkvinllama,” https://github.com/yynil/ RWKVinLLAMA, 2024

    2024

  245. [254]

    Altaeraai,

    K. Wegrzyniak, “Altaeraai,” https://github.com/latestissue/ AltaeraAI, 2024

    2024

  246. [255]

    Rwkv-lm-amd-radeon-rocm-hip,

    Alic-Li, “Rwkv-lm-amd-radeon-rocm-hip,” https://github.com/ Alic-Li/RWKV-LM-AMD-Radeon-ROCm-hip, 2024

    2024

  247. [256]

    Gptq-for-rwkv,

    F. Mom, “Gptq-for-rwkv,” https://github.com/3outeille/ GPTQ-for-RWKV, 2023

    2023

  248. [257]

    Easychat-server,

    Ow1onp, “Easychat-server,” https://github.com/Ow1onp/ EasyChat-Server, 2024

    2024

  249. [258]

    Innerdnn,

    SingingRivulet, “Innerdnn,” https://github.com/ SingingRivulet/InnerDNN, 2023

    2023

  250. [259]

    Generative cal- ibration for in-context learning,

    Z. Jiang, Y. Zhang, C. Liu, J. Zhao, and K. Liu, “Generative cal- ibration for in-context learning,” arXiv preprint arXiv:2310.10266, 2023

    2023 arXiv

  251. [260]

    Chatrwkv pc,

    mosterwei13, “Chatrwkv pc,” https://github.com/ mosterwei13/ChatRWKV PC, 2023

    2023

  252. [261]

    Rwkv-edge: Deeply com- pressed rwkv for resource-constrained devices,

    W. Choe, Y. Ji, and F. Lin, “Rwkv-edge: Deeply com- pressed rwkv for resource-constrained devices,” arXiv preprint arXiv:2412.10856, 2024

    2024

  253. [262]

    Linear attention based channel estimation scheme for v2x communications,

    Q. Fu, H. Yuan, Y. Hou, and X. Shen, “Linear attention based channel estimation scheme for v2x communications,” in 2024 8th International Conference on Communication and Information Systems (ICCIS). IEEE, 2024, pp. 96–101

    2024

  254. [263]

    L3tc: Leveraging rwkv for learned lossless low- complexity text compression,

    J. Zhang, Z. Cheng, Y. Zhao, S. Wang, D. Zhou, G. Lu, and L. Song, “L3tc: Leveraging rwkv for learned lossless low- complexity text compression,” arXiv preprint arXiv:2412.16642 , 2024

    2024 arXiv

  255. [264]

    Pcf-rwkv: Product carbon footprint estimation system based on large language model,

    Z. Li, P . Tang, X. Wang, X. Liu, and P . Mou, “Pcf-rwkv: Product carbon footprint estimation system based on large language model,” 2024

    2024

  256. [265]

    Long short-term memory,

    A. Graves and A. Graves, “Long short-term memory,” Supervised sequence labelling with recurrent neural networks, pp. 37–45, 2012

    2012

  257. [266]

    Plantflowerdatasets,

    lovebull, “Plantflowerdatasets,” https://github.com/lovebull/ PlantFlowerDatasets, 2023

    2023

  258. [267]

    Bidirectional recurrent neural networks,

    M. Schuster and K. K. Paliwal, “Bidirectional recurrent neural networks,” IEEE transactions on Signal Processing , vol. 45, no. 11, pp. 2673–2681, 1997

    1997

  259. [268]

    Empirical evalua- tion of gated recurrent neural networks on sequence modeling,

    J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical evalua- tion of gated recurrent neural networks on sequence modeling,” arXiv preprint arXiv:1412.3555, 2014

    2014 arXiv

  260. [269]

    xlstm: Extended long short-term memory,

    M. Beck, K. P ¨oppel, M. Spanring, A. Auer, O. Prudnikova, M. Kopp, G. Klambauer, J. Brandstetter, and S. Hochreiter, “xlstm: Extended long short-term memory,” 2024. [Online]. Available: https://arxiv.org/abs/2405.04517 JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 19

    2024 arXiv

  261. [270]

    Were rnns all we needed?

    L. Feng, F. Tung, M. O. Ahmed, Y. Bengio, and H. Hajimirsadeghi, “Were rnns all we needed?” 2024. [Online]. Available: https://arxiv.org/abs/2410.01201

    2024 arXiv

  262. [271]

    Eagle and finch: Rwkv with matrix-valued states and dynamic recurrence,

    B. Peng, D. Goldstein, Q. Anthony, A. Albalak, E. Alcaide, S. Bi- derman, E. Cheah, X. Du, T. Ferdinan, H. Hou et al., “Eagle and finch: Rwkv with matrix-valued states and dynamic recurrence,” arXiv preprint arXiv:2404.05892, 2024

    2024 arXiv

  263. [272]

    An attention free transformer,

    S. Zhai, W. Talbott, N. Srivastava, C. Huang, H. Goh, R. Zhang, and J. Susskind, “An attention free transformer,” arXiv preprint arXiv:2105.14103, 2021

    2021 arXiv

  264. [273]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752, 2023

    2023 arXiv

  265. [274]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, Y. Shen, P . Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in International Conference on Learning Representations ,

  266. [275]

    Available: https://openreview.net/forum?id= nZeVKeeFYf9

    [Online]. Available: https://openreview.net/forum?id= nZeVKeeFYf9

  267. [277]

    Hyena hierarchy: Towards larger convolutional language models,

    M. Poli, S. Massaroli, E. Nguyen, D. Y. Fu, T. Dao, S. Baccus, Y. Bengio, S. Ermon, and C. R´e, “Hyena hierarchy: Towards larger convolutional language models,” in International Conference on Machine Learning. PMLR, 2023, pp. 28 043–28 078

    2023

  268. [278]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y. Shen, P . Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  269. [2022]

    Available: https://arxiv.org/abs/2101.03961

    [Online]. Available: https://arxiv.org/abs/2101.03961

    arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.