SmartCall: Building an AI‑Powered Voice Customer Service Center

Project Overview

SmartCall is an open‑source AI customer‑service call‑center project targeting enterprise scenarios. It is built with Java 17, Spring Boot 3.5, Spring Cloud 2025, MyBatis‑Flex, Asterisk, Spring AI, DashScope, DeepSeek, Alibaba Cloud NLS, MaxKB and supports both microservice and monolithic deployment modes.

Key Functional Chains

Phone access and call control via Asterisk and PJSIP.

IVR workflow orchestration through visual nodes.

Real‑time speech recognition (ASR) converting voice to text.

Text‑to‑speech (TTS) broadcasting system replies and prompts.

Large‑model intent recognition beyond simple DTMF routing.

Knowledge‑base Q&A powered by MaxKB for multi‑turn answers.

Emotion analysis and information extraction (e.g., name, address, order ID).

Enterprise‑grade capabilities: gateway, authentication, multi‑tenant, logging, API docs, monitoring.

Technical Architecture

SmartCall adopts a "microservice + monolith dual‑mode" design. In microservice mode each module is independently deployed and registered via Spring Cloud for service discovery, gateway routing, unified authentication and inter‑service calls. In monolithic mode the smart-boot artifact launches the entire system, reducing dependencies on Nacos, Gateway, etc.

Core modules include: smart-aster: core call module handling Asterisk AMI/AGI integration, call control, IVR execution, ASR/TTS integration and AI intent detection. smart-maxkb: adapter for MaxKB, managing agent applications, API keys, session handling and knowledge‑base Q&A. smart-gateway: API gateway for routing, authentication, CORS, access logs and dynamic routing. smart-auth: unified OAuth2 authentication center. smart-upms: user‑permission management (systems, users, resources). smart-ops: operations monitoring, audit logs, API doc aggregation, Spring Boot Admin. smart-api: Feign API definitions for inter‑service calls. smart-common: shared constants, enums, validation annotations and utilities.

The call flow starts when a user dials in via SIP or carrier line, entering the Asterisk PBX. Asterisk triggers the IVR via AGI, handing control to smart-aster. The engine plays TTS prompts, listens to user speech, runs ASR, identifies intent, executes conditional branches, calls external HTTP services, optionally engages the MaxKB agent or transfers to a human seat.

AI is embedded in workflow nodes, enabling intent recognition, information collection, emotion judgment, knowledge‑base answering and hand‑off decisions.

Core Capabilities

SmartCall transforms traditional IVR into an extensible, observable, and scalable business process. Nodes support voice playback, speech listening, DTMF capture, intent detection (regex + large‑model dual engine), agent dialogue, conditional branching, data extraction, HTTP calls, script execution, human transfer, sub‑process invocation, variable assignment and hang‑up.

Intent detection uses a fast regex engine for high‑frequency, stable queries and a large‑model engine for complex natural‑language inputs, balancing speed and understanding.

Voice capabilities integrate multiple ASR/TTS providers (Alibaba Cloud NLS, DashScope, telecom services) managed through the VoiceModelEnum enumeration, allowing per‑node selection of model, voice, and sample rate.

Large‑model integration relies on Spring AI abstractions, supporting models such as Tongyi Qianwen and DeepSeek, facilitating seamless provider switching.

The MaxKB adapter ( smart-maxkb) enables enterprise knowledge‑base integration; the plugin architecture can extend to Dify, Coze, FastGPT, etc.

Deployment Scenarios

SmartCall is suited for industries where phone contact remains critical but human cost and service pressure are high:

E‑commerce after‑sales: voice‑driven order queries, refunds, logistics, with automatic escalation for complaints or negative sentiment.

Financial collection and notifications: batch outbound calls, compliance‑driven scripts, identity verification, structured recording of repayment intent.

Education enrollment and follow‑up: lead qualification, course inquiries, reminder calls, satisfaction surveys, with AI handling high‑frequency interactions.

Enterprise hotlines: replacing traditional IVR with natural‑language intent routing, backend queries and agent hand‑off.

Government and public services: policy consultation, progress checks, appointment reminders, leveraging knowledge‑base answers.

Private‑cloud deployment options address data‑security and on‑premises requirements.

Future Extensions

Enhanced seat‑operation features: status management, call hold, monitoring, barge‑in, transfer, scheduling, quality scoring.

Advanced outbound task system: sharding, deduplication, retry, concurrency control, risk detection, grouping, result analytics.

Productized call‑data analytics: dashboards for call duration, hand‑off rate, bot resolution rate, emotion distribution, intent distribution, seat efficiency, call loss metrics.

AI evaluation framework: measuring intent‑recognition accuracy, knowledge‑base hit rate, compliance, hallucination, hand‑off strategy rationality.

Broader agent platform support: integrating Dify, Coze, FastGPT or custom agents.

Deepened private and domestic deployment: on‑premise voice engines, large‑model inference, databases, OS and middleware for finance, government, energy, manufacturing.

Open‑Source Repository

https://gitee.com/gdzWork/SmartCall