Pith Number

pith:XDRDHMN3

pith:2024:XDRDHMN3T3SQAVSFP2ZCKABITF

not attested not anchored not stored refs resolved

VoiceBench: Benchmarking LLM-Based Voice Assistants

Chen Zhang, Haizhou Li, Robby T. Tan, Xianghu Yue, Xiaoxue Gao, Yiming Chen

VoiceBench introduces the first benchmark to evaluate LLM-based voice assistants under real-world variations in speakers, environments, and content.

arxiv:2410.17196 v3 · 2024-10-22 · cs.CL · cs.AI · cs.SD · eess.AS

Open paper page JSON What is a Pith Number?

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

Claims

C1strongest claim

We introduce VoiceBench, the first benchmark designed to provide a multi-faceted evaluation of LLM-based voice assistants. VoiceBench also includes both real and synthetic spoken instructions that incorporate the above three key real-world variations.

C2weakest assumption

That the selected variations in speaker characteristics, environmental factors, and content factors adequately represent the intricate real-world scenarios that current evaluations neglect.

C3one line summary

VoiceBench is the first benchmark for multi-faceted evaluation of LLM voice assistants using real and synthetic spoken instructions with speaker, environmental, and content variations.

References

85 extracted · 85 resolved · 10 Pith anchors

[2] Advances in Neural Information Processing Systems , volume=

[3] The Twelfth International Conference on Learning Representations , year=

[7] The Twelfth International Conference on Learning Representations , year=

[13] Preliminaries to a theory of speech disfluencies , author=. 1994 , school= 1994

[14] Advances in neural information processing systems , volume=

Formal links

2 machine-checked theorem links

Cited by

20 papers in Pith

AudioRole: An Audio Dataset for Character Role-Playing in Large Language Models

Game-Time: Evaluating Temporal Dynamics in Spoken Language Models

MM-tau-p$^2$: Persona-Adaptive Prompting for Robust Multi-Modal Agent Evaluation in Dual-Control Settings

The Silent Thought: Modeling Internal Cognition in Full-Duplex Spoken Dialogue Models via Latent Reasoning

Boosting Omni-Modal Language Models: Staged Post-Training with Visually Debiased Evaluation

Receipt and verification

First computed	2026-05-17T23:38:46.069024Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

b8e233b1bb9ee50056457eb22500289973b4161e537385c49cc64f207385d600

Aliases

arxiv: 2410.17196 · arxiv_version: 2410.17196v3 · doi: 10.48550/arxiv.2410.17196 · pith_short_12: XDRDHMN3T3SQ · pith_short_16: XDRDHMN3T3SQAVSF · pith_short_8: XDRDHMN3

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/XDRDHMN3T3SQAVSFP2ZCKABITF \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: b8e233b1bb9ee50056457eb22500289973b4161e537385c49cc64f207385d600

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "30a1817c039a304e979ed3fd0961d078c6c729215c13cc633a1639054246f41c",
    "cross_cats_sorted": [
      "cs.AI",
      "cs.SD",
      "eess.AS"
    ],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2024-10-22T17:15:20Z",
    "title_canon_sha256": "89cc10e5f8b173057e5fbfa94b77714278acfadf0e45951cd819b631a771ec77"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2410.17196",
    "kind": "arxiv",
    "version": 3
  }
}