Hardware Architectures: Flynn's Taxonomy

In 1966, Michael Flynn proposed a classification of computer architectures based on two dimensions: the number of instruction streams and the number of data streams being processed simultaneously. This taxonomy remains the standard framework for discussing parallel hardware.

The Four Categories

Comparison Table

SISD — Single Instruction, Single Data

The classic von Neumann architecture. One CPU fetches one instruction and operates on one data element at a time. Modern uniprocessors execute instructions out-of-order and use pipelining, but conceptually remain SISD. Example: A single-threaded Python script.

SIMD — Single Instruction, Multiple Data

One instruction is broadcast to many processing elements, each operating on its own data. This is data parallelism at the hardware level.

• Vector processors: Apply the same operation to entire arrays in one instruction (e.g., CPU AVX-512 adds 16 floats simultaneously)
• GPUs: NVIDIA A100 has 6,912 CUDA cores all executing the same shader instruction on different pixels/data points. This is why GPUs excel at deep learning — matrix multiplications are inherently SIMD.

MISD — Multiple Instruction, Single Data

Rarely used in practice. Multiple processors apply different instructions to the same data stream. Used in fault-tolerant systems where redundant processors cross-check each other (e.g., flight control computers) and in systolic arrays for signal processing.

MIMD — Multiple Instruction, Multiple Data

The most general and widely used category. Each processor fetches its own instructions and operates on its own data independently.

• Multicore CPUs: Intel Core i9 with 24 cores — each core runs a different thread on different data.
• Compute clusters: Thousands of nodes in an AWS HPC cluster, each running independent MPI ranks.
• Cloud VMs: Each virtual machine on a hypervisor is an independent MIMD unit.

Practical Hardware Organizations

Beyond Flynn's taxonomy, real systems are organized as:

Cloud Relevance

AWS P4d instances pack 8 NVIDIA A100 GPUs (SIMD/MIMD hybrid) per host, connected by NVLink. The instance itself is MIMD (8 independent GPUs), while each GPU executes SIMD across its CUDA cores. Understanding this lets you choose the right instance type for your workload.