4.2 Black-Box Testing Methods

What is Black-Box Testing?

Black-box testing treats the software as a sealed box — testers know what it should do (the inputs and expected outputs) but not how it does it internally. Also called:

• Functional testing
• Behavioural testing
• Specification-based testing
• Closed-box testing

   Input ──► [ Black Box ] ──► Output
                  ?

The test cases come from the SRS / requirements / use cases, not from looking at code.

When to use

• System testing
• Acceptance testing
• Most integration testing
• Functional sub-tests at any level

Advantages

• Tester does not need programming knowledge
• Same test cases work for any implementation
• Catches missing/extra functionality
• Realistic — uses the system the way users do

Disadvantages

• Cannot find code-level bugs that aren't user-visible
• May leave large portions of code untested
• Test redundancy is common (multiple inputs hit the same code path)

---

The five classical black-box techniques (IPU favourites)

Plus often listed: Error Guessing and Pairwise / Combinatorial Testing.

---

1. Equivalence Partitioning (EP)

Idea: input space is partitioned into classes where any single input from a class is "equivalent" — if one input from the class triggers a bug, others will too. So pick one representative per class.

Example: a function accepts age (1 to 120)

Partitions:

Pick one input from each: 0, 25, 999. That's 3 test cases instead of 122.

Rules for choosing partitions

• If input is a range — one valid + two invalid partitions
• If input is a specific value — one valid + two invalid partitions
• If input is a set of values — one valid + one invalid
• If input is boolean — one true, one false

---

2. Boundary Value Analysis (BVA)

Idea: defects cluster at boundaries. Test the edges of each partition, not just the middle.

For a range a–b, test:

• a − 1 (just below min)
• a (at min)
• a + 1 (just above min)
• b − 1 (just below max)
• b (at max)
• b + 1 (just above max)

Example: age range 1–120

So 6 BVA tests vs 3 EP tests. Most exam questions combine the two:

PYQ pattern: "Apply Equivalence Partitioning and Boundary Value Analysis on a function that accepts marks 0–100." — Define partitions: <0 invalid, 0–100 valid, >100 invalid. EP tests: -10, 50, 150. BVA tests: -1, 0, 1, 99, 100, 101.

---

3. Decision Table Testing

Idea: when behaviour depends on a combination of conditions, tabulate all combinations.

Example: Insurance discount

Rules:

• If applicant is a non-smoker AND age < 50 → 20% discount
• If applicant is a non-smoker AND age ≥ 50 → 10% discount
• If applicant is a smoker → no discount

Decision table:

4 test cases cover all combinations. For n binary conditions there are 2ⁿ rows.

Reducing the table

Adjacent columns with the same action and only one differing condition can be collapsed. T3 and T4 both give 0% regardless of age, so:

3 test cases instead of 4.

---

4. State Transition Testing

Idea: model the system as a state machine and design tests that traverse each state and transition at least once.

Example: ATM card states

[Inserted] ──PIN OK──► [Authenticated] ──Cash────► [Dispensing]
   │                          │
   │ wrong PIN x3             │ wrong PIN
   ▼                          ▼
[Blocked]                  [Inserted]

Test cases:

1. Insert + valid PIN → Authenticated ✓
2. Insert + 1 wrong PIN + correct → Authenticated ✓
3. Insert + 3 wrong PINs → Blocked ✓
4. Authenticated + cash withdraw → Dispensing ✓

State transition tables list (current state, event) → (new state, action) for every combination.

---

5. Cause-Effect Graphing

Idea: model relationships between causes (inputs / conditions) and effects (outputs / actions) as a logic graph using AND, OR, NOT gates. Then derive a decision table from the graph.

   cause1 ─── AND ─── effect1
   cause2 ──┘
   cause3 ─── OR ──── effect2
   cause4 ──┘

This is more rigorous than decision table testing but harder to draw. The IPU exam often just expects you to define it without drawing the full graph.

---

6. Error Guessing

Idea: experienced testers guess where defects are likely to live based on:

• Empty/null inputs
• Maximum/minimum values
• Special characters
• Concurrent operations
• Recently changed code

Error guessing is complementary to systematic techniques — it catches things specifications didn't anticipate.

---

7. Pairwise / Combinatorial Testing

For systems with many independent inputs:

• Pure exhaustive: 5 inputs × 4 values each = 1,024 combinations
• Pairwise: test every pair of values together → ~20 tests
• Tools: Microsoft PICT, all-pairs algorithms

Pairwise testing catches ~70% of combinatorial bugs at a fraction of the cost.

---

Black-box techniques — when to use each

---

Worked Exam Example

Question: Apply EP and BVA on a function that calculates discount on order amount:

• Order < ₹1,000 → no discount
• Order ₹1,000 to ₹5,000 → 5% discount
• Order > ₹5,000 → 10% discount

Partitions:

EP tests: 5 cases — 500 (P1), 3000 (P2), 10000 (P3), -100 (P4).

BVA tests around each boundary:

~9 BVA tests vs ~5 EP tests. Together they give strong coverage.

---

Key Terms — Lesson 4.2

The vocabulary below covers the seven canonical black-box testing techniques and their working concepts.

Black-Box Testing — A testing approach where the internal structure of the code is unknown to the tester. Test cases are derived from specifications, requirements, or use cases, not from inspecting code. Also called functional testing, behavioural testing, specification-based testing, or closed-box testing.

Functional Testing — Synonymous with black-box testing in most usage — verifying that each function specified in the SRS produces correct outputs for given inputs.

Specification-Based Testing — The formal name for testing whose test cases are systematically derived from a specification document (SRS, design document, use cases) rather than from code or intuition.

Test Case Derivation — The process of turning a specification into testable cases. Black-box techniques (EP, BVA, decision tables) are systematic methods for test-case derivation.

Equivalence Partitioning (EP) — A technique that divides the input domain into equivalence classes — sets of inputs that the system treats the same way. One representative per class is sufficient to test the class. EP dramatically reduces the number of test cases needed to cover the input space.

Equivalence Class / Equivalence Partition — A subset of the input domain where any element will exercise the same code path. If one input from the class reveals a bug, the others would too — picking one representative is enough.

Boundary Value Analysis (BVA) — A technique that specifically tests the edges of equivalence partitions — just below the minimum, at the minimum, just above; just below the maximum, at the maximum, just above. Justified by the empirical observation that defects cluster at boundaries (off-by-one errors, edge conditions, type overflow).

Off-by-One Error — The classic defect class that motivates BVA — looping one iteration too many or too few, using < instead of <=, reading past array end. Boundary value tests are designed specifically to catch off-by-one errors.

Decision Table Testing — A technique that tabulates all combinations of conditions and the corresponding actions the system should take. Each column of the table is a distinct test case. For n binary conditions, the table has up to 2ⁿ columns.

Cause-Effect Graph — A diagram showing causes (inputs / conditions) and effects (outputs / actions) connected by logic gates (AND, OR, NOT). The graph is then transformed into a decision table for test case derivation. More rigorous than direct decision tables but harder to construct.

State Transition Testing — A technique for systems with significant state (ATMs, traffic lights, login flows). The system is modelled as a finite state machine; test cases are derived to exercise each state and each transition at least once.

State Transition Diagram / State Machine — A graphical model of states (boxes) connected by transitions (labelled arrows). Standard for stateful systems and the basis of state transition testing.

Error Guessing — A heuristic, experience-based testing technique where a senior tester deliberately probes places defects are most likely to hide — empty inputs, null values, max/min boundaries, special characters, concurrent operations, recently changed code. Complements systematic techniques.

Exploratory Testing — A test-design technique where the tester simultaneously designs, executes, and learns — there is no upfront test case list; instead, the tester explores the system, forms hypotheses, and follows defects opportunistically. Common in Agile teams alongside automated regression.

Pairwise / All-Pairs / Combinatorial Testing — A technique for systems with many independent inputs, each with several values. Instead of testing all combinations (exponential), test every pair of values together. Pairwise testing catches ~70% of combinatorial bugs at a fraction of the cost. Tools: Microsoft PICT, AllPairs, ACTS.

Property-Based Testing — A modern generalisation of black-box testing where the tester defines invariant properties that should hold for any input ("reversing a list twice gives the original list"), and the tool generates hundreds or thousands of random inputs to try to falsify the property. Tools: QuickCheck (Haskell), Hypothesis (Python), fast-check (JS), jqwik (Java).

Test Oracle — The mechanism that decides whether a test passed or failed — usually the expected output, but can also be a property, a reference implementation, or a stored prior result. Test oracles are sometimes the hardest part of testing — knowing "what should the output be?" is non-trivial for ML models, graphical output, randomised algorithms.

Decision Coverage / Branch Coverage (Black-Box Context) — Although strictly a white-box criterion, decision tables aim for the analogous goal in black-box terms: every documented decision path is exercised.

Use Case Testing — A black-box technique where the test cases are derived from use cases in the SRS — main success scenario, alternative flows, exception flows. Useful for end-to-end and system testing.

Scenario Testing — Similar to use case testing but at a coarser grain — testing end-to-end user journeys through the system. Often used for customer demonstrations and acceptance testing.

Risk-Based Testing — A black-box test-prioritisation strategy where higher-risk areas (frequently used features, recently changed code, high-impact failure modes) get proportionally more test cases. Common when complete testing is impractical.

---

Study deep

1. Black-box and white-box are complementary, not competing. Mature test suites use both: white-box for unit tests (developer-side), black-box for integration / system / acceptance (tester-side).

1. Specifications are imperfect. Black-box testing finds bugs given the spec. If the spec itself is wrong, black-box testing will not catch it. This is why reviews of the SRS matter — they fix bugs that no amount of testing can.

1. Equivalence partitioning assumes homogeneity within a class. Sometimes a class contains hidden boundaries (e.g. valid age 1–120 — but 18 is the legal voting age, 60 is the senior-citizen boundary). Domain knowledge guides which boundaries deserve tests.

1. Decision tables grow quickly. With n conditions you get up to 2ⁿ rows. Tools like CTE-XL help manage large decision-table testing.

1. Modern equivalents. Property-based testing (QuickCheck, Hypothesis) is a generalisation — define properties that should hold for any input, then have the tool generate hundreds of inputs.

PYQ pattern: "Explain black-box testing. Discuss equivalence partitioning and boundary value analysis with an example." — Define black-box; pick a numeric range, apply EP (1 valid + 2 invalid partitions), apply BVA (test at boundaries ±1). Show test cases in a table.