Unit I — Memory Hierarchy

A computer uses multiple types of memory organised in a hierarchy. As you move up the hierarchy, memory becomes faster and more expensive, but smaller in capacity. As you move down, it becomes slower, cheaper, and larger.

---

The Memory Hierarchy (Top to Bottom)

---

Why a Hierarchy?

An ideal memory would be infinitely large, infinitely fast, and free. In reality, no single technology satisfies all three. The hierarchy exploits the principle of locality:

• Temporal Locality: Recently accessed data is likely to be accessed again soon (keep it in fast cache).
• Spatial Locality: Data near recently accessed locations is likely to be accessed soon (prefetch into cache).

---

Levels Explained

Registers

• Built into the CPU chip itself.
• Accessed in a single clock cycle (sub-nanosecond).
• Capacity: typically 32 or 64 registers of 32/64 bits each.

Cache Memory

• Acts as a buffer between the fast CPU and slower RAM.
• L1 Cache: Smallest and fastest; inside the CPU core.
• L2 Cache: Slightly larger; inside or near the CPU.
• L3 Cache: Shared among CPU cores; larger but slower than L1/L2.

Primary Memory (RAM)

• Main working memory; holds running programs and data.
• Volatile — contents lost when power is off.
• Access time: ~50–100 nanoseconds.

Secondary Memory

• Permanent storage (hard disks, SSDs, optical discs).
• Non-volatile — data persists without power.
• Access time: milliseconds (HDD) to microseconds (SSD).

Tertiary / Offline Storage

• Magnetic tapes, cloud storage.
• Used for archival and backup.
• Very slow access but virtually unlimited capacity.

---

Memory Access Time Comparison

Key Takeaway: The memory hierarchy is a trade-off between speed, cost, and capacity. Registers and cache are fast but small; secondary storage is large but slow. The CPU always tries to find data in the fastest (highest) level first before going lower.