Day 27: Architecture Evaluation—SAAM, ATAM, CBAM, Sensitivity & Trade‑off Points

Why Architecture Evaluation Matters

After a design is completed, simply assuming the solution is good is insufficient; the architecture must be examined against concrete quality‑attribute scenarios before large coding and deployment costs are incurred. The goal is to discover risks and conflicts between decisions and quality goals early.

Four Types of Input

Business goals : why the system is built (e.g., support online registration for many hospitals).

Quality‑attribute goals : how well the system must perform (e.g., millions of concurrent users, 2 s response, fault recovery).

Quality‑attribute scenarios : concrete conditions and measurable targets (e.g., 10 000 patients register simultaneously at 8 am, 95 % of requests finish within 2 s, error rate ≤ 0.1 %).

Architecture description and decisions : how the system intends to meet the goals (caching, multi‑instance deployment, adapters, encryption, etc.).

Evaluation Process

Choose high‑priority scenarios
↓
Map scenarios to architectural components and decisions
↓
Analyze system response when the stimulus occurs
↓
Determine whether the response meets the metric goals

The output is not a single score but a set of actionable findings: sensitivity points, trade‑off points, risks, non‑risks, risk themes, and improvement suggestions.

Quality‑Attribute Scenarios as “Exam Questions”

Instead of vague statements like “the system must be fast,” a scenario specifies who, when, what, how, and the success threshold. Example:

During the weekday 8 am registration peak, 10 000 patients simultaneously submit requests; the system must perform rate‑limiting, query available slots, and create registration records, with 95 % of requests returning within 2 s and an error rate below 0.1 %.

This concrete “question” guides the evaluator to probe the architecture (e.g., cache consistency, single‑point Redis, adapter isolation, encryption overhead).

SAAM, ATAM, CBAM

SAAM (Software Architecture Analysis Method) focuses on scenario‑driven analysis of modifiability. It asks which components must change for a new scenario and how many interfaces are affected.

ATAM (Architecture Tradeoff Analysis Method) examines multiple quality attributes and their interactions. It reveals trade‑offs such as increased security causing higher CPU usage and slower response.

CBAM (Cost‑Benefit Analysis Method) extends quality‑attribute analysis by adding cost, benefit, and ROI considerations. Example: comparing two solutions that both improve availability but differ in investment.

Mnemonic: “Scenario changes architecture → SAAM; attributes trade off → ATAM; calculate economics → CBAM.”

Utility Tree

ATAM uses a utility tree to decompose abstract quality goals into four layers: utility → quality attribute → attribute refinement → concrete scenario → priority. Stakeholders prioritize based on business importance and implementation difficulty.

ATAM Nine‑Step Workflow (Compressed)

Explain business and architecture
↓
Create utility tree and perform first analysis
↓
Gather more stakeholders, refine scenarios, re‑analyze, present results

The three compressed phases help remember the steps while still recognizing all nine in formal assessments.

Key Concepts: Sensitivity, Trade‑off, Risk, Non‑risk

Sensitivity point : a parameter or decision that, when altered, significantly impacts a single quality attribute. Example: database connection‑pool size dramatically affects peak‑period response time.

Trade‑off point : a decision that simultaneously influences multiple quality attributes, requiring a balance. Example: extending Redis cache TTL improves performance but raises data‑staleness risk.

Risk : a decision that may cause a quality‑attribute scenario to fail. Example: all registration services depend on a single Redis instance; Redis failure would break availability.

Non‑risk : after analysis, a decision is shown to support the current scenario. Example: using adapters isolates hospital‑interface changes, so adding a new hospital does not affect core registration logic.

Concrete Internet‑Hospital Evaluation

Quality requirements :

10 000 concurrent registrations, 95 % within 2 s.

Single‑instance failure recovered within 30 s.

New hospital integration within 5 person‑days, affecting ≤ 3 modules.

Patient data must be encrypted, authenticated, and audited.

Current decisions : multiple registration instances behind a gateway, Redis cache for hot slots, all instances share a single Redis, adapters for hospital APIs, stronger encryption.

Mapping scenarios to decisions (summarized from the original table):

High‑concurrency scenario → cache, rate‑limiting, load‑balancing, connection‑pool.

Single‑instance failure → multi‑instance, health‑check, fail‑over.

New hospital integration → adapters, stable interfaces, module boundaries.

Privacy data → encryption, authz, audit.

Findings :

Sensitivity point : connection‑pool size heavily influences response time during peak load.

Trade‑off point : stronger encryption improves security but adds CPU overhead and latency.

Risk : single Redis instance creates a availability risk; its failure would block all registration instances.

Non‑risk : adapters successfully isolate hospital‑interface changes, supporting modifiability.

Recommended actions after the evaluation:

Add high‑availability and fail‑over for Redis.

Run performance tests to tune connection‑pool and encryption parameters.

Validate the actual module changes and effort when adding a new hospital.

Conduct load, fault‑injection, and security testing for high‑priority scenarios.

Answering an ATAM Assessment Question

Collect business goals, quality‑attribute needs, stakeholder concerns, and architecture description; refine requirements into measurable scenarios; build a utility tree; map high‑priority scenarios to architectural decisions; analyze responses; identify sensitivity, trade‑off, risk, and non‑risk; summarize results; propose improvements; and verify with tests.

When describing a specific risk, follow the pattern: scenario → decision → cause → consequence → conclusion → improvement & verification . Example given for the single‑Redis risk.

High‑Frequency Keywords and Their Associated Methods

Modifiability, candidate architecture, scenario analysis → SAAM

Multiple quality attributes, trade‑off, sensitivity, risk → ATAM

Cost, benefit, budget, ROI → CBAM

Utility‑tree layers → utility → quality attribute → refinement → scenario

Common Pitfalls

Sensitivity ≠ risk: a parameter may be sensitive but not yet a failure.

Trade‑off ≠ defect: a trade‑off is a deliberate balance, not necessarily a flaw.

Non‑risk ≠ permanent safety: it only applies to the current analyzed scenario.

ATAM does not replace performance testing; it identifies where testing is needed.