New KV cache compaction technique cuts LLM memory 50x without accuracy loss

MIT researchers developed Attention Matching, a KV cache compaction technique that compresses LLM memory by 50x in seconds — ...
Nature

Cache Performance and Memory Hierarchy Optimization

The dynamic interplay between processor speed and memory access times has rendered cache performance a critical determinant of computing efficiency. As modern systems increasingly rely on hierarchical ...
EDN

Last-level cache has become a critical SoC design element

LLC, positioned between external memory and internal subsystems, stores frequently accessed data close to compute resources.
Electronic Design

Adding Cache to IPs and SoCs

Cache memory significantly reduces time and power consumption for memory access in systems-on-chip. Technologies like AMBA protocols facilitate cache coherence and efficient data management across CPU ...
Semiconductor Engineering

A Primer On Last-Level Cache Memory For SoC Designs

System-on-chip (SoC) architects have a new memory technology, last level cache (LLC), to help overcome the design obstacles of bandwidth, latency and power consumption in megachips for advanced driver ...
Semiconductor Engineering

Memory Evolution Drives Requirements For Design Technology Co-Optimization

As new technology nodes have become available, memory has been one of the most aggressive semiconductor applications to adopt advanced process technology. The relentless demand by users of electronic ...
Nature

Scratchpad Memory Optimization in Embedded Systems

Embedded systems demand high performance with minimal power consumption, and the optimisation of scratchpad memory (SPM) plays a critical role in meeting these stringent requirements. SPM, a small ...
TechRepublic

Design of Cache Memory with Cache Controller Using VHDL

The authors report on the design of efficient cache controller suitable for use in FPGA-based processors. Semiconductor memory which can operate at speeds comparable with the operation of the ...
Neowin

AMD's new patent suggests Ryzen 3D V-cache CPUs may get lot more powerful and faster

AMD recently published a new patent that reveals that the company is working on making its 3D V-cache tech even better. Back in early 2021, we started hearing the first whispers and murmurs of a new ...
EDN

MRAM technologies: from space applications to unified cache memory?

Magneto-resistive random access memory (MRAM) is a non-volatile memory technology that relies on the (relative) magnetization state of two ferromagnetic layers to store binary information. Throughout ...