A roofline-based model is used to assess bandwidth and latency needs for High Bandwidth Storage in 13B-parameter models with long contexts and the utility of bonded memory chiplets for 1B-parameter models to ease capacity and bandwidth constraints in on-device gen-AI inference.
Llm in a flash: Efficient large language model inference with limited memory
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
verdicts
UNVERDICTED 2roles
background 1polarities
background 1representative citing papers
This survey discusses key components and challenges for Personal LLM Agents and reviews solutions for their capability, efficiency, and security.
citing papers explorer
-
Technology solutions targeting the performance of gen-AI inference in resource constrained platforms
A roofline-based model is used to assess bandwidth and latency needs for High Bandwidth Storage in 13B-parameter models with long contexts and the utility of bonded memory chiplets for 1B-parameter models to ease capacity and bandwidth constraints in on-device gen-AI inference.
-
Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security
This survey discusses key components and challenges for Personal LLM Agents and reviews solutions for their capability, efficiency, and security.