Paper Citation Record · LEDGER

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA

As of 18 August 2026, this Paper Citation Record lists 31 of 31 outbound references and 1 inbound Pith citation observation for arXiv:2412.20677.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2412.20677 v2

Coverage vector

measured 31 of 31 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-10T23:19:23.112017Z

measured 32 of 32 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-18T06:34:40.430872+00:00

measured 1 of 1 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-16T00:59:42.128875Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: pith, observed 2026-08-16T00:59:42.230640Z

Reference resolution

31 of 31 outbound references displayed

verified exact0
verified fuzzy3
unresolved28
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 7151ea51-0361-4590-87fa-d8dec4883b12 · outbound

This paper cites BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:22.995212Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7fbde293-76e1-4a51-b55a-3fe3ccb88c84 · outbound

This paper cites an unresolved cited work.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA Unresolved cited work

Reference 6

Resolution: unresolved; raw_fallback, observed 2026-08-10T23:19:23.525027Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 7fe3f08a-d2e1-4909-a378-f717391ae2a3 · outbound

This paper cites LoRA: Low-Rank Adaptation of Large Language Models.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA LoRA: Low-Rank Adaptation of Large Language Models

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.014442Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 343c113a-3b63-4898-b0d8-2cb8df6d0ffa · outbound

This paper cites Mistral 7B.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA Mistral 7B

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.019925Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation acebbc93-a900-4ccf-87e8-8f88b18c52f5 · outbound

This paper cites BiLD: Bi-directional Logits Difference Loss for Large Language Model Distillation.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA BiLD: Bi-directional Logits Difference Loss for Large Language Model Distillation

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.024524Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a489ebb9-8328-4c56-8ea7-5a33c9091de7 · outbound

This paper cites DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.028493Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3b049843-b8fb-466d-a225-3f858b14dfa8 · outbound

This paper cites Learning Sparse Neural Networks through $L_0$ Regularization.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA Learning Sparse Neural Networks through $L_0$ Regularization

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.032946Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b7705666-0266-47cd-b997-559697d8ccb8 · outbound

This paper cites SocialIQA: Commonsense Reasoning about Social Interactions.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA SocialIQA: Commonsense Reasoning about Social Interactions

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.040939Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3f5afc6e-3c90-4e81-97b4-c96794f439e1 · outbound

This paper cites Recursive deep models for se- mantic compositionality over a sentiment treebank.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA Recursive deep models for se- mantic compositionality over a sentiment treebank

Reference 17

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T23:19:23.552556Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 45404514-2a8d-4b1b-9869-eafcc5300f46 · outbound

This paper cites A Simple and Effective Pruning Approach for Large Language Models.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA A Simple and Effective Pruning Approach for Large Language Models

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.053957Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d7f1e43b-321b-4635-a518-8a98c59e813f · outbound

This paper cites RazorAttention: Efficient KV Cache Compression Through Retrieval Heads.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA RazorAttention: Efficient KV Cache Compression Through Retrieval Heads

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.058774Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b62e7200-aa4a-4488-a2bc-c7cbe7f77dd2 · outbound

This paper cites Llama 2: Open Foundation and Fine-Tuned Chat Models.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA Llama 2: Open Foundation and Fine-Tuned Chat Models

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.063117Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a5f7f156-3956-4295-bb7e-95440dce08e5 · outbound

This paper cites Structured Pruning of Large Language Models.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA Structured Pruning of Large Language Models

Reference 21

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.067137Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 62655e4c-e53d-406b-8e09-a916ae36eb06 · outbound

This paper cites Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.075700Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 948d91bc-580f-4a81-9f94-579e967b72e8 · outbound

This paper cites Qwen2 Technical Report.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA Qwen2 Technical Report

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.080312Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e49414fa-1196-4d67-903c-894eef64356b · outbound

This paper cites Effectively Compress KV Heads for LLM.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA Effectively Compress KV Heads for LLM

Reference 25

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.084595Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 674a44b1-e114-4054-9095-f3990fc287f5 · outbound

This paper cites WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.089199Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 831db064-129f-4930-bd18-0ab67374e58c · outbound

This paper cites HellaSwag: Can a Machine Really Finish Your Sentence?.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA HellaSwag: Can a Machine Really Finish Your Sentence?

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.093364Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 780554e5-c090-4460-85f1-7df43cb64211 · outbound

This paper cites TinyLlama: An Open-Source Small Language Model.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA TinyLlama: An Open-Source Small Language Model

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.097981Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 61840526-4f67-4146-bf65-83deef55c602 · outbound

This paper cites During the pruning training process, the sparsity warm-up steps account for 30% of the total steps, during which the target size of the L0 masks decreases linearly to zero.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA During the pruning training process, the sparsity warm-up steps account for 30% of the total steps, during which the target size of the L0 masks decreases linearly to zero

Reference 29

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T23:19:23.539292Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation eac5dce5-d74b-4b33-85f6-e1199cb4fd14 · outbound

This paper cites an unresolved cited work.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA Unresolved cited work

Reference 31

Resolution: unresolved; raw_fallback, observed 2026-08-10T23:19:23.511078Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation ab5f951b-3fd8-45ef-87d6-b0e6d373263d · outbound

This paper cites Fast Transformer Decoding: One Write-Head is All You Need.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA Fast Transformer Decoding: One Write-Head is All You Need

Reference 1966

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.045321Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b3067072-352b-47bb-8ca1-7faa285b5a07 · outbound

This paper cites DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion

Reference 2013

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:22.990805Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9b327448-53b0-48e8-b877-aeb244b96897 · outbound

This paper cites Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering

Reference 2017

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.037009Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d5bbf37a-8162-4973-9119-fb744a724811 · outbound

This paper cites The Llama 3 Herd of Models.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA The Llama 3 Herd of Models

Reference 2018

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.004497Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2f2fc995-ba2a-41ca-8213-8f1f88cd251e · outbound

This paper cites Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge

Reference 2019

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:22.999949Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation cf5d0739-4783-4149-ab55-ad05197b6b36 · outbound

This paper cites Lan- guage models are few-shot learners.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA Lan- guage models are few-shot learners

Reference 2020

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T23:19:23.566901Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation ad30bf8b-f329-46cd-9743-077854a87204 · outbound

This paper cites Measuring Massive Multitask Language Understanding.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA Measuring Massive Multitask Language Understanding

Reference 2021

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.009235Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6aaea453-9a7a-4c1b-810d-ebbda686b8bd · outbound

This paper cites Structured Pruning Learns Compact and Accurate Models.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA Structured Pruning Learns Compact and Accurate Models

Reference 2022

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:23.071189Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a5f98dbc-0d0f-4192-a531-e065ae2d4a72 · outbound

This paper cites SliceGPT: Compress Large Language Models by Deleting Rows and Columns.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA SliceGPT: Compress Large Language Models by Deleting Rows and Columns

Reference 2023

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:22.982078Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2aca0bbe-26d1-4b46-a4b5-fbdb433eec82 · outbound

This paper cites GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints.

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints

Reference 2024

Resolution: unresolved; no resolver link, observed 2026-08-10T23:19:22.976195Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

Observation 6ad80293-41fd-4ec1-b430-229b314136f0 · inbound

Opt-GPTQ: An Optimized GPTQ Combining Sparse Attention and Quantization Techniques cites this paper.

Opt-GPTQ: An Optimized GPTQ Combining Sparse Attention and Quantization Techniques Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA

Reference 13

Resolution: verified exact; local_arxiv, observed 2026-08-16T00:59:42.239517Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.