Mastering Software Architecture Evaluation: ATAM, ARID, and Practical Steps

Software Architecture Evaluation Overview

Software architecture evaluation is essential for the success of large projects. It involves reviewing a set of architectural decisions to ensure they meet desired quality‑attribute goals. The most widely used methods are ATAM (Architecture Trade‑off Analysis Method) and ARID (Active Reviews for Intermediate Design).

Key Concepts

Architecture Method: A collection of coordinated architectural decisions aimed at achieving quality‑attribute goals.

Architecture Evaluation: The analysis and review of those decisions.

ATAM: A method that uses utility trees and scenario discussions to uncover risks, trade‑offs, and sensitivity points.

ATAM Method Steps

The ATAM process consists of four major parts and nine activities:

Presentation: The evaluation lead introduces ATAM, explains the steps, and answers questions.

Business Motivation Presentation: The project sponsor (project manager or client) describes commercial goals, key functionality, constraints, and risk owners.

Architecture Presentation: The architect describes technical constraints, required interactions, architectural methods, and key use‑case scenarios.

Investigation & Analysis: Identify the architectural methods, generate a quality‑attribute utility tree, and analyze methods against high‑priority scenarios.

Testing: Conduct collective discussions to prioritize scenarios, then re‑analyze the architecture using those scenarios.

Report Formation: The evaluation team documents the utility tree, identified risks, sensitivity points, trade‑offs, and other findings.

ATAM Evaluation Phases

Phase 0 – Form the evaluation team and establish collaboration.

Phase 1 – Gather and analyze architectural information.

Phase 2 – Validate results with risk owners.

Phase 3 – Produce the final report and plan follow‑up activities.

Roles and Responsibilities

Evaluation Team Leader: Prepares the evaluation, coordinates with the client, and ensures delivery of the final report.

Evaluation Lead: Manages the evaluation, drives scenario identification, and facilitates discussions.

Scenario Recorder: Writes scenarios on a board or whiteboard using agreed terminology.

Progress Recorder: Captures progress, scenarios, questions, and architectural solutions electronically.

Time Keeper: Monitors time spent on each scenario.

Process Observer: Notes process improvements and reports lessons learned.

Process Supervisor: Reminds the team of ATAM steps.

Questioner: Raises unexpected architectural questions.

Quality‑Attribute Goals

Maintainability (M1‑M3): Change a subsystem without affecting others; minimize deployment requirements; reduce regression test time.

Reliability (R1, R2, R6): Prevent long‑lasting resource blockage; isolate erroneous data; limit downtime to one hour.

Operability (O10, O14): Re‑prioritize 1,000 orders within 20 minutes; handle 1,000 concurrent requests without manual intervention.

Scalability (S2, S3): Support 50 stations and 100 data sources.

Performance (P1): Increase Landsat L‑7 query speed fivefold.

Utility Tree Example (First Phase)

The utility tree is organized into four levels: Utility → Quality Attribute → Refined Attribute → Scenario. Each scenario is assigned an importance and difficulty score, which are summed to prioritize analysis.

Scenario Analysis Example

Scenario M1.1: Deploy the next version of the scientific data server within eight hours without affecting other subsystems, search, browse, or reservation functions. The associated architectural decisions include backward‑compatible interfaces, static client placeholders, single‑copy critical databases, and distributed object APIs.

ARID Evaluation Method

ARID (Active Reviews for Intermediate Design) is suited for evaluating incomplete or evolving architectures. It focuses on suitability and uses scenario‑based discussions to assess design decisions when detailed documentation is lacking.

Comparison of ATAM, SAAM, and ARID

ATAM: Broad quality‑attribute focus (modifiability, security, reliability, performance); analyzes architecture methods and styles.

SAAM: Emphasizes modifiability and functionality; reviews logical or module views.

ARID: Concentrates on design suitability; reviews component interface specifications.

Common Risk Signals

Architecture misaligned with organizational structure.

More than 25 top‑level components, indicating excessive complexity.

Over‑emphasis on a single quality attribute (e.g., availability) at the expense of others.

Reliance on mutable operating‑system parts.

Use of proprietary components where standard ones exist.

Hardware‑centric component definitions.

Unnecessary redundancy beyond reliability requirements.

Unclear ownership of the overall architecture.

Key Management Issues Threatening Project Success

Unidentified risk owners.

Lack of domain experts.

Unsecured project funding.

No designated project manager.

Missing project plan or schedule.

Unrealistic deployment dates.

No clear criteria for evaluating alternatives.

Absence of an architectural selection process.

No overall architecture responsibility.

No comprehensive architecture plan.

Performance‑Related Concerns

End users have not defined performance requirements.

Missing performance data collection.

No identified performance overhead.

Unverified communication rates.

Lack of mechanisms to measure processing time or throughput.

No performance model.

Stakeholder (Risk Owner) List

Software Architecture Designer – balances competing quality requirements.

Developers – focus on clear architecture description, cohesion, and coupling.

Maintainers – concerned with maintainability and impact of changes.

Integrators – share concerns with developers.

Testers – need consistent error handling, limited coupling, and high cohesion.

Standards Experts – ensure compliance and interoperability.

Performance Engineers – evaluate performance, reliability, and understandability.

Security Experts – address security requirements.

Project Manager – allocates resources, tracks schedule, and manages budget.

Product Line Manager – looks at reusability and flexibility.

Customers – care about progress, budget, and usefulness.

End Users – focus on functionality and usability.

Application Developers – need clear, simple architecture for reuse.

Task Experts/Planners – consider functionality, usability, and flexibility.

System Administrators – need easy fault identification.

Network Administrators – monitor network performance and predictability.

Service Representatives – concerned with usability, serviceability, and customizability.

Domain Representatives – care about interoperability.

System Architecture Designer – responsible for hardware‑software trade‑offs, portability, and performance.

Device Experts – focus on maintainability and performance.

Software architecture evaluation quality largely depends on the competence of the assembled risk owners.