1.4 Agile, Scrum & Extreme Programming (XP)

The Agile Manifesto (2001)

In 2001, seventeen software practitioners (Kent Beck, Robert C. Martin, Ken Schwaber, Jeff Sutherland, Martin Fowler and others) met at Snowbird, Utah and authored the Agile Manifesto:

"We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: - Individuals and interactions over processes and tools - Working software over comprehensive documentation - Customer collaboration over contract negotiation - Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more."*

The 12 Agile principles (abridged)

1. Satisfy the customer through early and continuous delivery
2. Welcome changing requirements, even late
3. Deliver working software frequently (weeks not months)
4. Business people and developers work together daily
5. Build projects around motivated individuals
6. Face-to-face conversation is the most efficient communication
7. Working software is the primary measure of progress
8. Sustainable development pace
9. Continuous attention to technical excellence
10. Simplicity — maximise the work not done
11. Self-organising teams produce the best designs
12. Regularly reflect and adjust (retrospectives)

Exam tip: When asked "What is Agile?" — quote the four value statements verbatim, then list at least 4 of the 12 principles. Mention that Agile is a philosophy, not a process — Scrum and XP are concrete frameworks that implement Agile.

---

The Agile process — generic view

Common to all Agile frameworks:

• Iteration / Sprint — fixed-length cycle (typically 2 weeks), produces working software
• Backlog — prioritised list of features
• Continuous integration — code is integrated daily
• Working software is the primary measure of progress

---

Scrum

Scrum (Ken Schwaber & Jeff Sutherland, mid-1990s) is the most popular Agile framework. Schwaber called it "lightweight" — only three roles, three artefacts, five events.

Roles

Scrum prescribes only three roles, each with a sharply different responsibility. The Product Owner owns the what — they curate the product backlog, decide priorities, write or accept user stories, and serve as the team's single point of contact with the business and the customer. The Scrum Master owns the how — they facilitate the Scrum events, coach the team on Scrum practices, and (most importantly) remove blockers that prevent the team from making progress. The Development Team is the 5–9 cross-functional members who actually build the increment; importantly, the Scrum Team is self-organising — no one outside the team tells them how to implement the work.

Artefacts

Scrum has three artefacts, each playing a distinct planning role. The Product Backlog is the master, prioritised list of everything the product needs — features, bugs, technical debt, spikes — maintained continuously by the Product Owner. The Sprint Backlog is the subset of the product backlog that the team commits to delivering in the current sprint, plus the tasks needed to deliver each item. The Increment is the working, tested software produced at the end of each sprint — Scrum's definition of "done" requires it to be potentially shippable, even if the business chooses to wait before releasing.

Events (the five Scrum ceremonies)

The Sprint

Day 1 ─── Sprint Planning ──┐
                            │
Day 2 ─── Daily Stand-up    │
Day 3 ─── Daily Stand-up    │  Sprint
...        ...              │  (2 weeks)
Day 13 ── Daily Stand-up    │
                            │
Day 14 ── Review + Retro ───┘

A sprint is timeboxed — it ends on the scheduled day regardless of whether all backlog items are done. Incomplete items return to the product backlog.

Famous adopter: Spotify popularised an extension called the Spotify Model (Squads, Tribes, Chapters, Guilds) which scales Scrum to large organisations.

---

Extreme Programming (XP)

XP (Kent Beck, 1996) takes specific software engineering practices and turns them up to "extreme" levels:

• Code reviews are good? → Do them continuously (pair programming).
• Testing is good? → Write the test before the code (TDD).
• Design is good? → Refactor continuously, no big up-front design.
• Simplicity is good? → Do the simplest thing that could possibly work (YAGNI).

The 12 XP practices

The XP cycle (the small loops)

XP — when to use

• Small to medium teams (4–12 people)
• Customer is available on-site
• Rapidly changing requirements
• High-risk projects where bugs are expensive

XP — disadvantages

• Pair programming requires cultural buy-in
• Demands disciplined developers (TDD, refactoring)
• Customer must be genuinely available
• Hard to scale beyond ~12 people

---

Scrum vs XP

Many teams use Scrum + XP together — Scrum's ceremonies and roles for the process, XP's TDD and pairing for the engineering.

---

Other agile frameworks (brief)

---

Agile vs Traditional (Waterfall) — exam-ready comparison

---

Key Terms — Lesson 1.4

The Agile world has its own dense vocabulary. These terms appear in every Agile-related PYQ and in almost every modern industry job description.

Agile — A philosophy of software development (codified in the 2001 Agile Manifesto) that values individuals and interactions over processes and tools, working software over comprehensive documentation, customer collaboration over contract negotiation, and responding to change over following a plan. Agile is not a specific process; Scrum, XP, Kanban, and SAFe are concrete frameworks that implement the Agile philosophy.

Agile Manifesto — The four-value, twelve-principle document authored by 17 software practitioners at Snowbird, Utah, in February 2001. The signatories included Kent Beck, Robert C. Martin, Ken Schwaber, Jeff Sutherland, Martin Fowler, Alistair Cockburn, Ward Cunningham, Jim Highsmith, and others. The manifesto crystallised a decade of work on lightweight processes (XP, Scrum, Crystal, FDD, DSDM) into a shared declaration.

Iteration / Sprint — A fixed-length cycle (typically 1–4 weeks; most commonly 2 weeks in modern Scrum) during which the team turns a slice of the backlog into a working increment. The sprint is timeboxed — it ends on the scheduled day regardless of whether all backlog items are complete. Incomplete items return to the backlog for re-prioritisation.

Backlog — A prioritised list of work to be done. Agile distinguishes between the product backlog (the long-running, evolving master list owned by the Product Owner) and the sprint backlog (the subset committed to the current sprint, owned by the development team).

User Story — A short, customer-centric description of a feature, written in the format: "As a [type of user], I want [capability] so that [benefit]." Example: "As a registered customer, I want to save items to a wishlist so that I can return to them later." User stories replace traditional functional specifications in Agile teams; acceptance criteria define how "done" is judged.

Acceptance Criteria — The specific, testable conditions that a user story must satisfy to be considered complete. Acceptance criteria are written in the same Given/When/Then form (or as bulleted checklist) and form the basis of acceptance tests. Without acceptance criteria, "done" is subjective.

Definition of Done (DoD) — The team's shared, written standard for what must be true before an increment is considered complete — code written, code reviewed, unit-tested, integration-tested, documented, deployed to staging, signed off by Product Owner. DoD is the team's quality bar; without it, "done" tends to drift downward under sprint pressure.

Scrum — The most widely used Agile framework, defined by Ken Schwaber and Jeff Sutherland in the mid-1990s and codified in the Scrum Guide. Scrum prescribes three roles (Product Owner, Scrum Master, Development Team), three artefacts (Product Backlog, Sprint Backlog, Increment), and five events (Sprint, Sprint Planning, Daily Stand-up, Sprint Review, Sprint Retrospective).

Product Owner (PO) — The Scrum role responsible for maximising the value of the product. The PO owns the product backlog, decides priorities, writes or accepts user stories, and represents the business and customer voice to the team. Crucially, the PO is one person, not a committee.

Scrum Master — The Scrum role responsible for enabling the team's effectiveness. The Scrum Master is not a project manager; they facilitate the Scrum events, coach the team on Scrum practices, and remove organisational blockers (a third-party API access, a security review delay, a misaligned stakeholder).

Sprint Planning — The Scrum event at the start of each sprint where the team and Product Owner agree on the sprint goal and select the user stories the team will commit to. Output: the sprint backlog.

Daily Stand-up (Daily Scrum) — A 15-minute time-boxed meeting held every working day where each team member answers three questions: What did I do yesterday? What will I do today? What is blocking me? The stand-up is for the team's own coordination, not status reporting to managers.

Sprint Review — The Scrum event at the end of each sprint where the team demonstrates the working increment to stakeholders and gathers feedback. Sprint Review is about the product (the what).

Sprint Retrospective — The Scrum event at the end of each sprint where the team reflects on its process — what went well, what did not, what to try differently next sprint. Sprint Retrospective is about the team (the how) and is the engine of continuous improvement.

Increment — The sum of all completed work during a sprint, in a state that is potentially shippable. The increment is the only mandatory output of a sprint that touches the product.

Velocity — The amount of work (measured in story points or completed user stories) a team completes in a sprint, averaged over recent sprints. Velocity is a planning aid for forecasting how much the team can commit to in future sprints. Velocity is not a performance metric across teams — it is team-specific and cannot be compared between teams.

Story Points — A relative estimation unit used to size user stories by complexity, effort, and uncertainty rather than calendar time. Often follows a Fibonacci scale (1, 2, 3, 5, 8, 13, 21) to discourage false precision. Story points let teams plan without committing to hour estimates that ignore unknowns.

Burndown Chart — A graph showing work remaining vs time over the course of a sprint or release. A healthy burndown declines toward zero by the end of the timebox. Burndowns expose schedule risk early — a flat or rising line signals trouble.

Extreme Programming (XP) — Kent Beck's 1996 framework that turns common-sense engineering practices "up to extreme": code review → pair programming; testing → test-driven development; design → continuous refactoring; simplicity → YAGNI (You Aren't Gonna Need It). XP focuses on engineering practices, while Scrum focuses on project management.

Test-Driven Development (TDD) — An XP practice (originated by Kent Beck) where the developer follows the red–green–refactor cycle: write a failing test (red), write the minimum code to make it pass (green), then refactor to improve structure without changing behaviour. TDD produces tightly-tested code and pressures the developer to keep designs simple.

Refactoring — Restructuring existing code without changing its external behaviour to improve readability, reduce duplication, or simplify design. Refactoring is supported by a comprehensive test suite — the tests guarantee that behaviour has not changed. Martin Fowler's book Refactoring (1999) is the canonical reference.

Pair Programming — An XP practice where two developers work together at one workstation — one as the "driver" (typing the code) and the other as the "navigator" (reviewing, thinking ahead). Pair programming roughly halves raw output but produces fewer defects, better design, and broader knowledge sharing. Some studies (Cockburn & Williams 2000) report net productivity comparable to solo programming.

Continuous Integration (CI) — The practice of integrating code changes into the shared main branch multiple times per day, with each integration triggering an automated build and test pipeline. CI catches integration bugs hours after they are introduced rather than days or weeks. Industry tools: GitHub Actions, GitLab CI, Jenkins, CircleCI.

Continuous Delivery / Continuous Deployment (CD) — The natural extension of CI: continuous delivery means the build pipeline can deploy to production at any time on a button press; continuous deployment means every passing build is automatically released to production. CD is the DevOps culmination of Agile's "deliver working software frequently" principle.

YAGNI (You Aren't Gonna Need It) — An XP design principle: do not implement functionality you don't currently need, even if you can foresee a need. Speculative work usually goes unused and adds maintenance burden. YAGNI is the partner of KISS ("Keep It Simple, Stupid") and DRY ("Don't Repeat Yourself") — the three principles every developer is expected to internalise.

Kanban — A pull-based Agile method (from Toyota's manufacturing system) that visualises work on a Kanban board with columns for workflow stages (To Do, In Progress, In Review, Done) and enforces Work-In-Progress (WIP) limits on each column. Kanban has no fixed sprints; the team continuously pulls new work as capacity allows. Popular for support and maintenance work, where Scrum's sprint structure is a poor fit.

SAFe (Scaled Agile Framework) — A framework for applying Agile at enterprise scale (hundreds or thousands of developers, multiple products). SAFe defines additional layers — Agile Release Trains, Program Increments, Portfolio management — on top of team-level Scrum. Controversial: SAFe is widely adopted but criticised for re-introducing waterfall planning overhead.

Spotify Model — An extension of Scrum popularised by Spotify, organising people into Squads (small Scrum teams), Tribes (collections of related squads), Chapters (cross-squad professional groups, e.g., "all the iOS engineers"), and Guilds (interest-based communities). The model has been influential well beyond Spotify.

Spike — A time-boxed exploratory task added to a sprint to answer a specific technical or research question before a real story can be estimated and committed — for example, "what is the actual latency of this third-party API under load?" Spikes are the modern Scrum equivalent of Spiral's risk-resolution prototypes.

Retrospective — Any structured team reflection on its own process — Sprint Retrospective in Scrum is the canonical case, but project retrospectives, release retrospectives, and incident retrospectives all follow the same pattern: what went well, what did not, what will we change. The retrospective is the engine of continuous improvement.

Self-Organising Team — A team that decides for itself how to do the work, rather than receiving task assignments from a manager. The team is given a goal and constraints; it decides the implementation, the tools, the breakdown of tasks. Self-organisation requires trust and clear shared understanding of goals; it is foundational to all Agile frameworks.

---

Study deep: misconceptions about Agile

1. "Agile means no documentation." False. The manifesto says "working software over comprehensive documentation" — value working software more, but documentation still matters. Most Agile teams still produce SRS-equivalents (user stories with acceptance criteria), architecture decision records (ADRs), and runbooks.

1. "Agile means no planning." False. Agile teams plan continuously — at the sprint level, at the release level, and at the product roadmap level. The plan is updated as learning happens, not abandoned.

1. "Agile is faster." Misleading. Agile is more adaptive, which usually means the right product is built faster, even if total dev hours are similar. The benefit is risk reduction, not raw speed.

1. "Agile only works for small teams." Partly true. Pure Scrum struggles past ~9 people per team, but frameworks like SAFe, LeSS, and Nexus scale Agile to hundreds of developers — albeit with overhead.

1. "Pair programming wastes resources — two people, one screen." Studies (Cockburn & Williams, 2000) show pairs produce ~15% slower but with ~15% fewer defects and better design — net productivity is comparable, and the quality is higher. Knowledge sharing is the bonus.

PYQ pattern: "What is Agile? Explain Scrum framework with its roles, artefacts and events." — Quote the 4 manifesto values, then table the 3 roles, 3 artefacts and 5 events.