Pith. sign in

REVIEW 5 major objections 6 minor 30 references

Multimodal Item Parameter Estimation using Simulated Response Probabilitie

T0 review · 5 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fine-tuning a multimodal language model to imitate students at fixed ability levels lets it recover the response curves behind test items and predict held-out difficulty with a Pearson correlation of 0.85.

desk verdict A real multimodal extension of the simulated-respondent method with a plausible 0.85 difficulty correlation, but the margin over baselines is confounded by image input and the mechanism claim needs a vision-capable regression baseline. read the letter →

arxiv 2608.10154 v1 pith:2V2W4EVQ submitted 2026-08-10 cs.CL cs.AI

Christopher Ormerod , YoungKoung Kim This is my paper · ORCID
classification cs.CLcs.AI
keywords itemresponsetheorythree-parameterlogisticmodelmultiple-choicemultimodalLLMfine-tuningsimulatedstudentresponsesdifficultypredictionquadraticweightedkappaQwen3.5
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that item parameters—especially difficulty—can be estimated from a fine-tuned multimodal language model without collecting new student response data. The strategy is to fine-tune the model to act like a student at each of twenty discrete ability levels, so its next-token probabilities for the four answer options match empirically calibrated response curves. On a held-out set of items, the 9-billion-parameter model reaches a Pearson correlation of 0.85 and a quadratic weighted kappa of 0.835 for difficulty, exceeding the MathBERT (0.68) and MetaMath (0.75) regression baselines. If true, this would reduce the expense and security risk of field-testing by letting item parameters be read off the stimulus itself.

What carries the argument

The object that carries the argument is the discrete ability-level response-probability target: for item $i$, option $k$, and ability interval $I_j = (\gamma_{j-1}, \gamma_j]$, the model is trained to match $\rho_{ijk} = \frac{1}{w_j}\int_{\gamma_{j-1}}^{\gamma_j} f(\theta) P_{i\theta k}\,d\theta$, where $w_j$ is the normal-prior probability of that ability interval and $P_{i\theta k}$ is the option probability from the multiple-choice model. The 3PL is $P(X=1)=c+(1-c)\sigma(a(\theta-b))$, and the MCM sets option probabilities from $\exp(a_{ik}\theta+b_{ik})+d_{ik}$ normalized across options. The fine-tuning loss is the mean squared error between the LLM's four next-token option probabilities and the $\rho$ values; after training, the same predicted probabilities are used to refit 3PL and MCM parameters by weighted least squares. This bridges raw response data, item response curves, and LLM outputs.

What would settle it

Run the same simulated-respondent fine-tuning on a text-only version of the items that strips out every image; if the difficulty correlation stays near 0.85, the image channel is not the source of the gain, whereas a drop toward the 0.68–0.75 baseline range would show the comparison drove the result. As a second check, renormalize the four option probabilities over only the four option tokens before refitting the curves; large shifts in fitted b values would reveal that the full-vocabulary softmax assumption is load-bearing.

Watch

Extended reading notes

Core claim

The central claim is that training a multimodal LLM to replicate the systematic option-choice patterns of students across a labeled range of ability levels makes it implicitly capture the response probabilities encoded in the three-parameter logistic and multiple-choice models. The fine-tuned Qwen3.5-9B model scores 0.85 Pearson correlation and 0.835 QWK on held-out difficulty, with an RMSE of 0.55 after a linear correction, while recovering the guessing parameter at 0.48 and the discrimination parameter at only 0.31. The authors interpret the strong difficulty and guessing results as evidence that the simulated-respondent framing captures response behavior rather than surface stimulus features, and the weak discrimination recovery as a cost of discretizing ability.

Load-bearing premise

The load-bearing comparison assumes that text-only regression models are the proper baselines for a multimodal model, so the reported gain could come from seeing images rather than from the simulated-respondent training.

Editorial extensions

If this is right

  • Item difficulty can be obtained directly from a fine-tuned model's option probabilities, so routine field-testing for difficulty calibration could be reduced or supplemented by simulation.
  • The same setup handles items whose stimuli and options contain images, unifying text-only and image-plus-text items under one input representation.
  • The guessing parameter is recovered far better than by regression baselines (0.48 versus 0.25–0.28), suggesting distractor attraction is genuinely modeled.
  • Discrimination recovery is weak (0.31), and the paper attributes the loss to ability discretization, implying finer ability bins should improve slope estimates.
  • Under the five-band difficulty labels, predicted and calibrated labels agree at QWK 0.835, so the model supports the coarse labels educators actually receive.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A text-only variant of the simulated-respondent training would isolate how much of the improvement comes from image input; the paper's current design cannot separate image access from response simulation.
  • Because the option probabilities are read from a full-vocabulary softmax, renormalizing over the four option tokens is a natural robustness check that could change the fitted parameters.
  • The same prompting scheme could be extended to produce joint response patterns across a set of items, which would let test assemblers simulate form-level statistics before any field test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes and evaluates a method for estimating item parameters in educational assessments, specifically the difficulty (b), discrimination (a), and guessing (c) parameters of the three-parameter logistic (3PL) model and the option probabilities of the multiple-choice model (MCM). The method fine-tunes a multimodal LLM (Qwen3.5-4B/9B) to act as a simulated respondent: given an item (text plus a combined image), a labeled ability level, and the correct answer, the model is trained with an MSE loss to match empirically derived option choice probabilities for that ability bin. At inference, the model's four option-token probabilities are treated as response probabilities, and the 3PL/MCM parameters are recovered by weighted least-squares fitting to the resulting curves. The method is tested on a large corpus of mathematics items (4,848 items from 970 item models) and is reported to achieve Pearson correlation 0.85 and QWK 0.835 for difficulty, exceeding text-only regression baselines (MathBERT, MetaMath). The authors interpret this as evidence that the simulated-respondent framing recovers response behavior beyond surface features of the stimulus.

Significance. The paper addresses a practically important problem—estimating item parameters without costly field testing—and extends the emerging line of work on LLM-simulated respondents to a multimodal setting. Its strengths include the large real-response dataset, the clean train/dev/test split by item model that prevents item-level leakage, an explicit loss function and reconstruction objective, and the use of multiple metrics (Pearson, RMSE, QWK). If the method's advantage over baselines could be attributed to the simulated-respondent mechanism rather than to the mere addition of image input, it would be a genuine contribution. As it stands, the headline comparative claim is not supported by the experimental design; the absolute reconstruction results, however, remain plausible and worth further study.

major comments (5)
  1. [Section 3, Table 5] The comparison between the proposed Qwen3.5 models and the regression baselines is confounded by input modality: the baselines (MathBERT, MetaMath) are text-only, while the proposed models consume the combined stimulus-option image constructed in Figure 2. Since the corpus is described as "inherently visual," the reported margin (Pearson 0.85 vs. 0.75) is fully consistent with the alternative explanation that the advantage comes from image input alone, not from the simulated-respondent training objective in Eq. (4). This undermines the abstract's and Section 3's central claim that the simulated-respondent framing "recovers difficulty-relevant signal that direct stimulus-to-parameter regression does not capture." To establish the mechanism, the authors must add a vision-capable regression baseline (e.g., the same multimodal encoder with a linear head regressing directly to b) or an ablation of the proposed model with text-only input. Without such controls, the comparative headline is not supported.
  2. [Section 2.4.2, Eq. (4)] The training loss uses the raw softmax probabilities v_{\iota_k} for the four option tokens as response probabilities. Because the softmax is over the entire vocabulary, these token probabilities are not the probabilities of selecting each option; they are marginal token probabilities that can be influenced by the rest of the vocabulary. The same issue applies at inference, where the "predicted option probabilities" are read directly from the token logits. This is a load-bearing assumption: if the model assigns non-negligible probability to other tokens, the reconstructed IRT parameters will be systematically biased. The authors should either renormalize over the four option tokens or provide empirical evidence (e.g., the sum of the four option-token probabilities over a sample of prompts) that the remaining vocabulary mass is negligible.
  3. [Sections 2.3 and 3] The title and abstract promise reconstruction of "MCM curves," and Eq. (iv) defines an objective for recovering the MCM parameters {a_ik, b_ik, d_ik}, but Table 5 evaluates only the 3PL parameters (a, b, c). No results are reported for distractor-level parameters or for the MCM fit itself. As written, the paper does not support the stated claim of reconstructing MCM curves; at minimum, report the recovery accuracy for MCM parameters or explicitly narrow the contribution to 3PL difficulty and related parameters. This is essential for assessing whether the approach captures "the strength of distractor items" as discussed in Sections 2.2 and 4.
  4. [Section 3] The reported correlations treat test items as independent, but items are generated from only 146 test item models, so the effective sample size for evaluating item-level generalization is at most 146, not 730. Items within an item model share structure and the model's predictions are likely correlated, yet no account is taken of this clustering (e.g., bootstrapping by item model). Without a measure of uncertainty, the "clear margin" between 0.85 and 0.75 cannot be assessed; this issue is especially relevant to the claim of a marked improvement over the MetaMath baseline.
  5. [Section 2.3] The computation of the target probabilities \rho_{ijk} in Eq. (iii) requires integrating a normal density times the MCM probability over each ability interval, but the paper does not state how these integrals are evaluated (e.g., Gauss-Hermite quadrature, adaptive quadrature, or Monte Carlo). Without this detail, the method is not reproducible, and the choice of approximation could affect the targets and hence the trained model. Please specify the numerical integration scheme and any relevant convergence settings.
minor comments (6)
  1. [Table 1] Table 1 is difficult to read: the columns for item models, items, and responses are not clearly aligned, and response counts such as "363310.2\times 10^6" are confusingly formatted; please use plain numbers or clearly labeled units.
  2. [Section 2.4.2] The system prompt includes the correct answer in the instruction to the simulated student; please clarify whether this is intentional and how it interacts with the goal of reproducing students' systematic errors, since providing the correct answer could make the task of selecting wrong options artificially driven.
  3. [Section 2.5] No error bars, standard deviations, or multiple-seed results are reported for any model; for the "clear margin" claim in Section 3, at least a variance estimate across fine-tuning seeds or item-model resamples is needed.
  4. [Section 3] The RMSE of 0.55 for the difficulty parameter is reported only after the "linear development-set correction." Because Pearson correlation is invariant to affine transformations, the reported correlation is unaffected, but the raw RMSE before the correction should be reported to make the description of the procedure unambiguous.
  5. [Tables 2 and 4] The descriptive labels for ability bands (e.g., "Emerging foundations") and difficulty bands (e.g., "Very Hard") are nonstandard; please provide conventional labels or a reference for the band definitions, which are used in the QWK computation.
  6. [References] The paper cites "Ormerod, 2026" as prior work without a full bibliographic entry, making it impossible for readers to verify the relationship to the present manuscript; please add the arXiv identifier or venue.

Circularity Check

0 steps flagged · score 0.0 of 10

Held-out item-model split and fully specified equations make the difficulty predictions genuine generalizations; the modality confound and development-set correction are interpretation threats, not circular reductions.

full rationale

The derivation chain is: raw responses are calibrated into 3PL/MCM parameters (arrows i and ii); those parameters are transformed into binned expected option probabilities rho_ijk (eq. iii); the LLM is fine-tuned with the MSE loss in eq. 4 to output option-token probabilities matching rho for training items; at inference the LLM's outputs are fitted back through eqs. iv/v to obtain parameter estimates; and the resulting estimates are compared with calibrated parameters on a held-out test set. The paper states that 'we split the items into training, development, and test sets by item model,' so no test item's calibrated parameters enter the LLM's training labels or the rho targets used for training. The development-set linear correction is fitted only on development items and, as the paper explicitly says, 'does not change the overall Pearson correlations, but does affect the mean squared error'; therefore the headline Pearson 0.85 result does not depend on that fitted correction. The claim that the model 'implicitly captures the underlying response probabilities encoded in the 3PL and MCM curves' is literal for training items because those probabilities are the training targets, but for held-out items it is a genuine empirical generalization claim rather than a reduction to inputs. The main weakness in the comparative claim is that the proposed Qwen models consume the combined stimulus-and-option image while MathBERT and MetaMath are text-only; this is a confound affecting attribution of the mechanism, not a circularity in which a prediction is forced by construction. Self-citations to Ormerod (2026) supply the discrete binning and reconstruction framework, but the equations (iii)-(v) are restated in the present paper, and no load-bearing uniqueness theorem or ansatz is imported solely by citation. No exhibited step reduces a predicted quantity to its own fitted input, so the appropriate finding is no significant circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on five modeling choices and assumptions: the IRT models are true generating models; ability estimates are accurate; item-model splitting removes leakage; the LLM's truncated next-token distribution is a usable response distribution; and the normal prior with Table 2 bins is meaningful. None of these has independent evidence in the paper; each either is assumed or is calibrated on the development set.

free parameters (5)
  • IRT regularization strength lambda = 1e-4
    Used in the L-BFGS-B fits of the 3PL (i) and MCM (ii) targets; hand-chosen, and the resulting parameters serve as both training targets and evaluation ground truth.
  • Ability prior (mu, sigma) = not reported; Table 2 cutoffs imply standard normal
    Section 2.3 states the prior {theta_j} ~ N(mu, sigma^2) but does not give mu and sigma; the bin boundaries imply a standard normal, and this prior enters the rho_ijk definitions in (iii).
  • Ability bin boundaries gamma_l = 20 intervals, 0.3 width from -2.7 to 2.7
    Hand-chosen discretization; the fineness of the bins affects how much slope information survives and the rho_ijk training targets via (iii).
  • Development-set linear calibration coefficients = not reported
    Section 2.3: a linear mapping from raw predicted parameters to calibrated parameters is fit on the development set and applied before RMSE is reported; the coefficients are direct fits to target parameters.
  • LoRA fine-tuning hyperparameters = learning rate 1e-5, 10 epochs; rank and alpha not reported
    Training regime for the LLM in Section 2.4.1; these choices affect the quality of the learned response probabilities.
assumptions (5)
  • domain assumption The 3PL and MCM models adequately describe the empirical response data.
    Section 2.2 obtains the item parameters by L-BFGS-B maximum likelihood and then treats these fitted curves as ground truth for both training targets and evaluation.
  • domain assumption The respondent ability estimates theta_j are accurate enough to be used as conditioning labels.
    Section 2.2 states ability was independently derived from a larger calibrated dataset, but no uncertainty is propagated into the discretization or target probabilities.
  • domain assumption Splitting by item model prevents content leakage from training to test items.
    Section 2.1.1 splits by item model; items generated from the same model share structure, so item-model-level memorization could inflate held-out performance.
  • domain assumption The next-token distribution over the four option tokens is a valid response distribution.
    Section 2.4.2 reads option-token probabilities from a truncated assistant response and uses them directly in the MSE loss in Eq. (4); no renormalization over the option set is described.
  • domain assumption The population ability prior is normal and the Table 2 bin labels are meaningful to the LLM.
    Section 2.3 defines the prior and uses the labels in prompts; the weighted integrals in (iii)-(v) depend on this prior and on the chosen bin edges.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multimodal Item Parameter Estimation using Simulated Response Probabilitie." pith.science (2026). https://pith.science/paper/2V2W4EVQ

@misc{pith2026260810154,
  author       = {Pith},
  title        = {Pith review of: Multimodal Item Parameter Estimation using Simulated Response Probabilitie},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2V2W4EVQ}},
  note         = {Machine review of arXiv:2608.10154}
}
read the original abstract

We present results from reconstructing multiple-choice model (MCM) and three-parameter logistic (3PL) model curves using a fine-tuned multimodal large language model (LLM) based on Qwen3.5. The model is prompted and fine-tuned to replicate choice probabilities across a large training corpus of multiple-choice items containing both image and text stimuli, conditioned on a labeled set of student ability levels. By learning to reproduce the systematic error patterns of students across a discrete range of abilities, the LLM implicitly captures the underlying response probabilities encoded in the 3PL and MCM curves. This allows us to accurately approximate item difficulty on a held-out test set directly from the model's predicted option probabilities.

Figures

Figures reproduced from arXiv: 2608.10154 by the authors.

Figure 2
Figure 2. The way in which stimulus and option images [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 1
Figure 1. Diagram of the relationships between the data [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. The structure of the Gated DeltaNet layers. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The structure of the Gated Attention layers. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 9 canonical work pages

  1. [1]

    Payne, and Valentina Tamma

    Samah AlKhuzaey, Floriana Grasso, Terry R. Payne, and Valentina Tamma. 2024. https://doi.org/10.1007/s40593-023-00362-1 Text-based Question Difficulty Prediction : A Systematic Review of Automatic Approaches . International Journal of Artificial Intelligence in Education, 34(3):862--914

    doi:10.1007/s40593-023-00362-1 2024

  2. [2]

    Okan Bulut, Guher Gorgun, and Bin Tan. 2024. https://aclanthology.org/2024.bea-1.44/ Item Difficulty and Response Time Prediction with Large Language Models : An Empirical Analysis of USMLE Items . In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications ( BEA 2024) , pages 522--527, Mexico City, Mexico. Associati...

    work page 2024

  3. [3]

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, and 16 others. 2022. http://arxiv.org/abs/2210.11416 Scaling Instruction - Fi...

    arXiv 2022

  4. [4]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/hash/1feb87871436031bdc0f2beaa62a049b-Abstract-Conference.html QLoRA : Efficient Finetuning of Quantized LLMs . Advances in Neural Information Processing Systems, 36:10088--10115

    2023

  5. [5]

    Gierl and Thomas M

    Mark J. Gierl and Thomas M. Haladyna, editors. 2013. https://www.routledge.com/Automatic-Item-Generation-Theory-and-Practice/Gierl-Haladyna/p/book/9780415897518 Automatic Item Generation : Theory and Practice . Routledge

    arXiv 2013

  6. [6]

    Suhwa Han, Christopher Ormerod, and Frank Rijmen. 2026. https://osf.io/cdg5v_v1 Fine-tuning of Open -source Large Language Models to Align Items to Standards . Preprint, OSF/EdArXiv

    work page 2026

  7. [7]

    Domingue, Emma Brunskill, and Noah D

    Joy He-Yueya, Wanjing Anya Ma, Kanishk Gandhi, Benjamin W. Domingue, Emma Brunskill, and Noah D. Goodman. 2024. https://doi.org/10.48550/arXiv.2407.15645 Psychometric Alignment : Capturing Human Knowledge Distributions via Language Models . arXiv preprint. ArXiv:2407.15645 [cs.CL]

    arXiv doi:10.48550/arxiv.2407.15645 2024

  8. [8]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. https://doi.org/10.48550/arXiv.2106.09685 LoRA : Low - Rank Adaptation of Large Language Models . arXiv preprint

    arXiv doi:10.48550/arxiv.2106.09685 2021

Show all 30 references
  1. [9]

    Ming Li, Hong Jiao, Tianyi Zhou, Nan Zhang, Sydney Peters, and Robert W Lissitz. 2025. https://aclanthology.org/2025.aimecon-sessions.5/ Item Difficulty Modeling Using Fine - Tuned Small and Large Language Models . In Proceedings of the Artificial Intelligence in Measurement a...

    2025

  2. [10]

    Opher Lieber, Barak Lenz, Hofit Bata, Gal Cohen, Jhonathan Osin, Itay Dalmedigos, Erez Safahi, Shaked Meirom, Yonatan Belinkov, Shai Shalev-Shwartz, Omri Abend, Raz Alon, Tomer Asida, Amir Bergman, Roman Glozman, Michael Gokhman, Avashalom Manevich, Nir Ratner, Noam Rozen, and...

    2024 doi arXiv

  3. [11]

    Yunting Liu, Shreya Bhandari, and Zachary A. Pardos. 2025. https://doi.org/10.1111/bjet.13570 Leveraging LLM respondents for item evaluation: A psychometric analysis . British Journal of Educational Technology, 56(3):1028--1052. \_eprint: https://bera-journals.onlinelibrary.wi...

    2025 doi

  4. [12]

    Xinyi Lu and Xu Wang. 2024. https://doi.org/10.1145/3657604.3662031 Generative Students : Using LLM - Simulated Student Profiles to Support Question Item Evaluation . In Proceedings of the Eleventh ACM Conference on Learning @ Scale , pages 16--27. ArXiv:2405.11591 [cs.HC]

    2024

  5. [13]

    Hotaka Maeda. 2025. https://doi.org/10.1177/00131644241281053 Field- Testing Multiple - Choice Questions With AI Examinees : English Grammar Items . Educational and Psychological Measurement, 85(2):221--244

    2025 doi

  6. [14]

    Chris Ormerod and Alexander Kwako. 2024. https://doi.org/10.59863/OKUU1904 Automated Text Scoring in the Age of Generative AI for the GPU -poor . Chinese/English Journal of Educational Measurement and Evaluation, 5(3)

    2024 doi

  7. [15]

    Christopher Ormerod. 2026. https://arxiv.org/abs/2601.02580v1 Reconstructing Item Characteristic Curves using Fine - Tuned Large Language Models

    2026 arXiv

  8. [16]

    Shuai Peng, Ke Yuan, Liangcai Gao, and Zhi Tang. 2021. https://doi.org/10.48550/arXiv.2105.00377 MathBERT : A Pre - Trained Model for Mathematical Formula Understanding . arXiv preprint

    2021 doi arXiv

  9. [17]

    Sydney Peters, Nan Zhang, Hong Jiao, Ming Li, Tianyi Zhou, and Robert Lissitz. 2025. https://arxiv.org/abs/2509.23486v1 Text- Based Approaches to Item Difficulty Modeling in Large - Scale Assessments : A Systematic Review

    2025

  10. [18]

    Zihan Qiu, Zekun Wang, Bo Zheng, Zeyu Huang, Kaiyue Wen, Songlin Yang, Rui Men, Le Yu, Fei Huang, Suozhi Huang, Dayiheng Liu, Jingren Zhou, and Junyang Lin. 2025. https://doi.org/10.48550/arXiv.2505.06708 Gated Attention for Large Language Models : Non -linearity, Sparsity , a...

    2025 doi arXiv

  11. [19]

    Qwen Team . 2026. https://qwen.ai/blog?id=qwen3.5 Qwen3.5 : Towards Native Multimodal Agents

    2026

  12. [20]

    Fumiko Samejima. 1969. https://doi.org/10.1007/BF03372160 Estimation of Latent Ability Using a Response Pattern of Graded Scores . Psychometrika, 34(S1):1--97

    1969 doi

  13. [21]

    Alexander Scarlatos, Nigel Fernandez, Christopher Ormerod, Susan Lottridge, and Andrew Lan. 2025. https://doi.org/10.18653/v1/2025.emnlp-main.1274 SMART : Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction . In Proceedings of the 2025 Confe...

    2025 doi

  14. [22]

    David Thissen and Lynne Steinberg. 1984. https://doi.org/10.1007/BF02302588 A response model for multiple choice items . Psychometrika, 49(4):501--519

    1984 doi

  15. [23]

    David Thissen and Lynne Steinberg. 1986. https://doi.org/10.1007/BF02295596 A taxonomy of item response models . Psychometrika, 51(4):567--577

    1986 doi

  16. [24]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 doi arXiv

  17. [25]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html Attention is All you Need . In Advances in Neural In...

    2017

  18. [26]

    Shuhe Wang, Shengyu Zhang, Jie Zhang, Runyi Hu, Xiaoya Li, Tianwei Zhang, Jiwei Li, Fei Wu, Guoyin Wang, and Eduard Hovy. 2024. https://arxiv.org/abs/2412.10400v3 Reinforcement Learning Enhanced LLMs : A Survey

    2024 arXiv

  19. [27]

    Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022. https://arxiv.org/abs/2206.07682v2 Emerg...

    2022 arXiv

  20. [28]

    Lingling Xu, Haoran Xie, Si-Zhao Joe Qin, Xiaohui Tao, and Fu Lee Wang. 2023. https://doi.org/10.48550/arXiv.2312.12148 Parameter- Efficient Fine - Tuning Methods for Pretrained Language Models : A Critical Review and Assessment . arXiv preprint

    2023 doi arXiv

  21. [29]

    Songlin Yang, Jan Kautz, and Ali Hatamizadeh. 2025. https://doi.org/10.48550/arXiv.2412.06464 Gated Delta Networks : Improving Mamba2 with Delta Rule . arXiv preprint. ArXiv:2412.06464 [cs.CL]

    2025 doi arXiv

  22. [30]

    Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu

    Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T. Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2024. https://doi.org/10.48550/arXiv.2309.12284 MetaMath : Bootstrap Your Own Mathematical Questions for Large Language Models . arXiv preprint. ...

    2024 doi arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.