Memory Hierarchy Study Notes for Computer Science Engineering Exams

By Mukesh Kumar|Updated : December 29th, 2021

Memory Hierarchy

Memory Hierarchy is a feature in computer system design that helps to organize memory such that access time is reduced. A processor, as well as a huge number of memory devices, were used in the computer system's architecture. However, the fundamental issue is that these components are costly. As a result, memory hierarchy can be used to organize the system's memory. It has many memory levels with varying performance rates. However, any of these can provide a precise objective, reducing the access time. The memory hierarchy was created in response to the program's activity. The memory hierarchy in computer architecture is discussed in this article.

  • The memory unit is used for storing programs and data. It fulfills the need for storage of information.
  • The additional storage with main memory capacity enhances the performance of the general-purpose computers and makes them efficient.
  • Only those programs and data, which are currently needed by the processor, reside in the main memory. Information can be transferred from auxiliary memory to main memory when needed.
Table of Content

Memory Hierarchy Levels

In a computer system, the memory hierarchy design primarily consists of several storage devices. The majority of computers came with extra storage to allow them to run faster than the main memory. Primary (internal) memory and secondary (external) memory are the two forms of the memory hierarchy that are designed. A hierarchical pyramid for computer memory is shown in the diagram below.



The register is usually a static RAM or SRAM in the computer's processor that holds the data word, which is normally 64 or 128 bits. The most essential register is the program counter register, which is found in all processors. A status word register and an accumulator are used by the majority of CPUs. A status word register is used for decision-making, and an accumulator is used to store data in the same way that a calculator is used to store numbers. Computers with complex instruction sets typically have a large number of registers for receiving main memory, whereas RISC-based computers have fewer registers.

Cache Memory

  • A small, fast storage memory is used to improve average access time Or We can say that cache is a very high-speed memory that is used to increase the speed of processing by making current programs and data available to the CPU at a rapid rate.
  • The cache is used for storing segments of programs currently being executed in the CPU and temporary data frequently needed in the present calculations.


Cache Performance

When the processor needs to read or write to a location in the main memory, it first checks whether a copy of that data is in the cache. If so, the processor immediately reads from or writes to the cache.

  • Cache hit If the processor immediately reads or writes the data in the cache line.
  • Cache miss If the processor does not find the required word in the cache, then a cache miss has occurred.
  • Hit ratio Percentage of memory accesses satisfied by the cache.
  • Miss ratio = 1– Hit ratio
  • Temporal Locality: The word referenced now is likely to be referenced again soon. Hence it is wise to keep the currently accessed word handy for a while.
  • Spatial Locality: Words near the currently referenced word are likely to be referenced soon. Hence it is wise to prefetch words near the currently referenced word and keep them handy for a while.
  • Write through writes the data to memory as well as to the cache.
  • Writeback: Don't write to memory now, do it later when this cache block is evicted.

Main Memory

  • The main memory refers to the physical memory and it is one central storage unit in a computer system.
  • The main memory is relatively large and fast memory used to store programs and data during computer operation.
  • The main memory in a general-purpose computer is made up of a RAM integrated circuit.

Latency: The latency is the time taken to transfer a block of data either from the main memory or caches.

  • As the CPU executes instructions, both the instructions themselves and the data they operate on must be brought into the registers, until the instruction/data is available, the CPU cannot proceed to execute it and must wait. The latency is thus the time the CPU waits to obtain the data.
  • The latency of the main memory directly influences the efficiency of the CPU.

Auxiliary Memory

In a computer system, auxiliary memory is the lowest-cost, highest-capacity, and slowest-access storage. Auxiliary memory is used to store programs and data that are stored for long periods or are not in use right away. Magnetic tapes and magnetic discs are the most common types of auxiliary memory.

Magnetic Disks

  • A magnetic disk, is a circular plate constructed of metal or plastic coated with magnetized material.
  • Often, both sides of the disk are used and several disks may be stacked on one spindle with read/write heads available on each surface.
  • All disks rotate together at high speed. Bits are stored in the magnetized surface in spots along concentric circles called tracks. The tracks are commonly divided into sections called sectors.

Magnetic Tapes

  • A magnetic tape is a medium of magnetic recording, made of a thin magnetizable coating on a long, narrow strip of plastic film.
  • Bits are recorded as magnetic spots on the tape along several tracks. Magnetic tapes can be stopped, started to move forward, or in reverse. Read/write heads are mounted one in each track, so that data can be recorded and read as a sequence of characters.

Memory Hierarchical Design

Level NameRegistersCache MemoryMain MemorySecondary memory
Typical Size<1KB<16MB<16GB>100GB
ImplementationCustomized MultiportSRAM (Flip-Flops)DRAM (Capacitor)Magnetic devices
Bandwidth (MB/s)(20,000-1,00,000)(5,000-10,000)(1,000-5,000)(20-150)
Access Time (ns)(0.25-0.5)(0.5-2.5)(80-250)5000000
Managed ByCompilerHardwareOSOS
Backed ByCache Memory Main MemorySecondary MemoryCompact Disk

Memory Organization

Based on the style of accessing the memory, it can be classified into two


  • Simultaneous Memory Access Organization.
  • Hierarchical Memory Access Organization.

Simultaneous Access Memory Organization

  • In this memory Organization, the CPU is directly connected to all the levels of memory but access is allowed in a sequence i.e., Whenever there is a miss in level 1 memory, data can be accessed directly from level 2 without copying it into level 1. 
  • Levels are designed in such a way that lower the level, higher the performance, lower the access time, and higher the level, lower the performance, and higher the access time.
  • Hit ratio of the last level must be 1 since it represents the whole system.


Average Memory Access time= H1T1+ (1-H1)H2T2+ (1-H1)(1-H2)H3T3+.....

Where, H1, H2, H3..are the hit ratio for level 1, level 2, Level3, and so on.

Hierarchical Memory Access Organization

  • In this memory organization, the CPU always accesses the data from the first level of memory. Whenever there is a miss in level 1, data is transferred from a higher level of memory to the lower level of memory. After the transfer of data, the CPU access the data from the lower level.
  • It is a better approach if the spatial locality is needed in the program.


Average Access time= H1T1+ (1-H1)H2(T2+ T1)+ (1-H1)(1-H2)H3(T3+ T2+T1) +...

Characteristics of Memory Hierarchy

The following are the main characteristics of memory hierarchy:


Initially, computer systems were designed without a memory hierarchy, and the speed gap between main memory and CPU registers grew as a result of the large differential in access time, resulting in lower system performance. As a result, enhancement was required. Because of the system's increased performance, this was enhanced in the memory hierarchy model.


The total quantity of data that the memory hierarchy can store is its capability. Because the memory hierarchy's capacity grows as we move from top to bottom.

Cost per bit

When we move from the bottom to the top of the memory hierarchy, the cost of each bit increases, implying that internal memory is more expensive than external memory.

Access Time

In the memory hierarchy, the access time is the time delay between data availability and requests to read or write. Because the access time increases as we move from the top to the bottom of the memory hierarchy.

Advantages of Memory Hierarchy

The following are some of the reasons why a memory hierarchy is necessary:

  • Memory distribution is easy and cost-effective.
  • External destruction is removed.
  • Data can be spread all over
  • Allows for pre-paging and demand paging.
  • Swapping will be a lot easier.

This concludes all about memory hierarchy. Finally, based on the given information, we can deduce that it is mostly used to reduce bit cost, access frequency, and enhance capacity, and access time. As a result, it is up to the designer to determine how important these features are for meeting the needs of their customers. We hope you find this information helpful.

Best Wishes

Also from us,

You can follow the detailed champion study plan for GATE CS 2021 from the following link:

Detailed GATE CSE 2021 Champion Study Plan

Candidates can also practice 110+ Mock tests for exams like GATE, NIELIT with BYJU'S Exam Prep Test Series check the following link:

Click Here to Avail GATE CSE Test Series! (100+ Mock Tests)

Get unlimited access to 21+ structured Live Courses all 112+ mock tests with Online Classroom Program for GATE CS & PSU Exams:

Click here to avail Online Classroom Program for Computer Science Engineering


write a comment


  • Memory Hierarchy is a characteristic in computer system design that helps to organize memory such that access time is reduced. The Memory Hierarchy was created using a programming technique called locality of references.

  • The access time grows as we progress along the Hierarchy from top to bottom. Performance: Due to the substantial disparity in access time between the CPU registers and Main Memory when the computer system was developed without a Memory Hierarchy design, the speed gap between the CPU registers and Main Memory increased.

  • If the data is needed again, the cache is examined first before proceeding to the main memory. In terms of the time it takes to search for and retrieve data, this structure is closer to the CPU than the main memory.

  • As the lowest level in the memory hierarchy, most modern computer systems use a hard drive composed of magnetic or solid-state storage.

Follow us for latest updates