Siksha Sarovar

Siksha Sarovar (sikshasarovar.com) is a free educational web application that helps students in India learn programming and prepare for academic and competitive exams. The platform offers structured coding courses (C, C++, Python, Java, HTML, CSS, PHP, Power BI, AI, Machine Learning, Data Science), complete university curriculum notes for BCA/MCA students with previous year question papers, Class 10 and Class 12 CBSE/HBSE school notes, and dedicated preparation material for SSC, UPSC, Banking, Railway and other government exams. Browsing the site is completely free and requires no account. Users may optionally sign in with Google solely to save their learning progress, quiz scores and personal preferences across devices.

Privacy Policy | Terms of Service | Contact Siksha Sarovar | About Siksha Sarovar

v4.0.9 · PWA
Siksha Sarovar logo
Siksha Sarovar
Your Learning Universe

Siksha Sarovar is a free e-learning platform for coding courses, BCA university notes and competitive exam preparation. Optional Google sign-in saves your learning progress across devices.

Initializing knowledge base…
Compiling modules 0%

Unit III: Overview - Parallel and Distributed Computing

Lesson 17 of 30 in the free Cloud Computing notes on Siksha Sarovar, written by Rohit Jangra.

Unit III: Principles of Parallel and Distributed Computing

This unit explores the foundational concepts that power modern cloud infrastructure. At its core, cloud computing is possible because of two complementary disciplines: parallel computing (doing many things at once on a single powerful machine) and distributed computing (coordinating work across many machines).

Why This Matters for Cloud

Every time you run a MapReduce job on AWS EMR, train a neural network on GPU clusters, or query a globally replicated database, you are relying on the principles covered in this unit. Understanding them lets you design systems that are faster, more scalable, and more resilient.

Unit Roadmap

Key Themes

  • Concurrency and Parallelism are related but distinct ideas — you can have one without the other.
  • Hardware architecture determines what kinds of parallelism are physically possible.
  • Programming models (OpenMP, MPI, CUDA) are abstractions over that hardware.
  • Theoretical limits (Amdahl's Law, Gustafson's Law) tell us what speedup we can realistically expect.

Quick Vocabulary

TermMeaning
ProcessAn independent unit of execution with its own memory
ThreadA lightweight execution unit sharing memory with siblings
NodeA single machine in a distributed cluster
LatencyTime to complete one operation
ThroughputOperations completed per unit time
ScalabilityAbility to handle growth (more data, more users)

Cloud Connection

AWS, Azure, and GCP are built on thousands of nodes working in concert. Services like AWS HPC (High Performance Computing), Google TPUs, and Azure Batch are direct applications of the material in this unit. By the end of Unit III, you will be able to reason about why a cloud provider chooses a particular architecture for a given workload.