Pith Number

pith:NLNZMRHZ

pith:2026:NLNZMRHZ3Z24GQGYSU4ECKFPLK

not attested not anchored not stored refs resolved

OxyEcomBench: Benchmarking Multimodal Foundation Models across E-Commerce Ecosystems

Bing Bai, Guoqing Yang, Ke Zhang, Xiaoqiang Xu, Ximan Liu, Yan Li, Yong Liu, Zhen Chen

A new benchmark for e-commerce shows leading multimodal models achieve only modest performance because they lack domain-specific knowledge.

arxiv:2605.13173 v1 · 2026-05-13 · cs.DB

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{NLNZMRHZ3Z24GQGYSU4ECKFPLK}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Evaluations on 20 mainstream LLMs and MLLMs show that even the leading models attain modest performance and that performance gaps narrow on OxyEcomBench, suggesting that insufficient e-commerce-specific knowledge infusion mutes the advantages of advanced general-purpose models in this domain.

C2weakest assumption

That the 29 tasks and the four-level P0-P3 difficulty rubric, chosen with expert consensus and emphasis on visually salient cases, faithfully represent the full range of real-world e-commerce challenges without introducing selection bias.

C3one line summary

OxyEcomBench is a unified multimodal benchmark covering 6 capability areas and 29 tasks with authentic e-commerce data to measure how well foundation models handle real platform, merchant, and customer challenges.

References

44 extracted · 44 resolved · 7 Pith anchors

[1] GPT-4 Technical Report 2023 · arXiv:2303.08774

[2] Visual instruction tuning 2023

[3] Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond 2023 · arXiv:2308.12966

[4] Shopping MMLU: A massive multi-task online shopping benchmark for large language models 2024

[5] Haoxin Wang, Xianhan Peng, Huang Cheng, Yizhe Huang, Ming Gong, Chenghan Y ang, Y ang Liu, and Jiang Lin. ECom-Bench: Can LLM agent resolve real-world e-commerce customer support issues? In Proceeding 2025

Formal links

1 machine-checked theorem link

Receipt and verification

First computed	2026-05-18T03:08:56.509968Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

6adb9644f9de75c340d895384128af5aa427ef6b3a382160a451cbdc79ed5699

Aliases

arxiv: 2605.13173 · arxiv_version: 2605.13173v1 · doi: 10.48550/arxiv.2605.13173 · pith_short_12: NLNZMRHZ3Z24 · pith_short_16: NLNZMRHZ3Z24GQGY · pith_short_8: NLNZMRHZ

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/NLNZMRHZ3Z24GQGYSU4ECKFPLK \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 6adb9644f9de75c340d895384128af5aa427ef6b3a382160a451cbdc79ed5699

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "b530e8efa5109ab607d902c6b48559ac926fcb9f1530d42d17e19fc1ccd7825d",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.DB",
    "submitted_at": "2026-05-13T08:34:10Z",
    "title_canon_sha256": "30b9bedfd6a3f9ca453ea1d1c3e1ed59484cc81c99da32399ca6a4542839b898"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.13173",
    "kind": "arxiv",
    "version": 1
  }
}