Pith Number

pith:RNZCDRQD

pith:2026:RNZCDRQDAW3LK2PM65GJEUARPA

not attested not anchored not stored refs resolved

Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators

Heejin Do, Mrinmaya Sachan, Shashank Sonkar

LLM student simulators correct answers at similar rates whether feedback targets the actual misconception or not.

arxiv:2605.12748 v1 · 2026-05-12 · cs.CL · cs.AI · cs.CY · cs.LG

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{RNZCDRQDAW3LK2PM65GJEUARPA}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Across seven LLMs (4B-120B), multiple datasets, and prompting strategies, simulators exhibit near-zero SFS, correcting their answers at similarly high rates regardless of feedback relevance.

C2weakest assumption

That the misconception-contrastive feedback protocol (targeted vs misaligned vs generic) cleanly isolates whether a simulator maintains a misconception-driven belief state rather than other response patterns.

C3one line summary

LLM simulators exhibit near-zero selective response to targeted misconception feedback and behave sycophantically, but SFT and SFS-aligned RL improve this property.

References

36 extracted · 36 resolved · 6 Pith anchors

[1] gpt-oss-120b & gpt-oss-20b Model Card 2025 · arXiv:2508.10925

[2] Cognitive tutors: Lessons learned.The journal of the learning sciences, 4(2):167–207, 1995 1995

[3] Order and equivalence of rational numbers: A clinical teaching experiment.Journal for Research in Mathematics Education, 15(5):323–341, 1984 1984

[4] Diagnostic models for procedural bugs in basic mathematical skills.Cognitive science, 2(2):155–192, 1978 1978

[5] The impact of high school life science teachers’ subject matter knowledge and knowledge of student misconceptions on students’ learning.CBE—Life Sciences Education, 19(1):ar9, 2020 2020

Receipt and verification

First computed	2026-05-18T03:09:48.896653Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

8b7221c60305b6b569ecf74c9250117817115970632101d712d1fe269813f704

Aliases

arxiv: 2605.12748 · arxiv_version: 2605.12748v1 · doi: 10.48550/arxiv.2605.12748 · pith_short_12: RNZCDRQDAW3L · pith_short_16: RNZCDRQDAW3LK2PM · pith_short_8: RNZCDRQD

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/RNZCDRQDAW3LK2PM65GJEUARPA \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 8b7221c60305b6b569ecf74c9250117817115970632101d712d1fe269813f704

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "ac932974a05c293286a5917cdcbd1004ad26b9f214004c95b7e5a49ec6597b1c",
    "cross_cats_sorted": [
      "cs.AI",
      "cs.CY",
      "cs.LG"
    ],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2026-05-12T20:55:23Z",
    "title_canon_sha256": "940dd0db7ec7362f5f266d0d72b242a9046d0c6539643c55202cd55b85ca3136"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.12748",
    "kind": "arxiv",
    "version": 1
  }
}