How to Build an Agent‑Powered Financial Q&A System with RAG and SQL

Preface

In the previous chapter we completed the basic RAG workflow; this chapter reviews the Agent framework and shows how to combine RAG data with SQL queries to answer financial questions autonomously.

Core Question

Before introducing the Agent framework we must answer the core problem:

How to automatically decide whether a user question should be answered via SQL query or RAG retrieval?

Intent Recognition Solution

We previously proposed a solution:

Use an intent‑recognition module to determine whether the question requires SQL or RAG.

The module sends the question to a large model for classification.

We fine‑tuned the model (few‑shot) so it knows which questions belong to SQL and which to RAG, e.g.:

<code>examples = [
    {
        "inn": "我想知道东方阿尔法优势产业混合C基金，在2021年年度报告中，前10大重仓股中，有多少只股票在报告期内取得正收益。",
        "out": "rag_question***我想知道东方阿尔法优势产业混合C基金，在2021年年度报告中，前10大重仓股中，有多少只股票在报告期内取得正收益。"
    },
    {
        "inn": "森赫电梯股份有限公司产品生产材料是什么？",
        "out": "rag_question***森赫电梯股份有限公司产品生产材料是什么？"
    },
    {
        "inn": "20210930日，一级行业为机械的股票的成交金额合计是多少？取整。",
        "out": "agent_question***20210930日，一级行业为机械的股票的成交金额合计是多少？取整。"
    }
    ...
]
</code>

After intent recognition, the code routes the request with an

if…else

block:

<code>if intent == "rag_question":
    # RAG handling
    result = self.rag.get_result(question=question)
    return result
elif intent == "agent_question":
    result, result_list = self.agent.get_result(input=question)
    return result
else:
    result = self.chat.invoke(input=question).content
    return result
</code>

Problems with Intent Recognition

If the intent is misidentified, the subsequent processing fails and the answer is wrong.

Adding correction strategies (e.g., fallback from RAG to SQL) makes the logic exponentially more complex.

This approach feels unintelligent and follows a traditional development mindset where the developer controls every step and the LLM is merely an auxiliary tool.

Inspired by the autonomous reasoning of Agents, we switch to using an Agent to make the decision and solve the problem.

Agent Framework

From the earlier summary article we learned the Agent framework (illustrated below):

By giving the Agent appropriate tools, it can plan and act autonomously. The first step is to provide those tools.

1. Provide Tools to the Agent

1.1 Create Agent Management Class

Code file:

app/finance_bot_ex.py

<code>import logging
import datetime
from langgraph.prebuilt import create_react_agent
from langchain.tools import retriever
from langchain.tools import create_retriever_tool
from langchain_community.utilities import SQLDatabase
from langchain_community.agent_toolkits import SQLDatabaseToolkit
from rag.rag import RagManager
import settings

class FinanceBotEx:
    def __init__(self, llm=settings.llm, chat=settings.chat, embed=settings.embed):
        self.llm = llm
        self.chat = chat
        self.tools = []
        self.rag = RagManager(llm=llm, embed=embed)
</code>

1.2 Create SQL Tool

Add a method to connect to a SQLite database and return the toolkit tools.

<code>def init_sql_tool(self, path):
    # Connect database
    db = SQLDatabase.from_uri(f"sqlite:///{path}")
    toolkit = SQLDatabaseToolkit(db=db, llm=self.llm)
    sql_tools = toolkit.get_tools()  # tools
    return sql_tools
</code>

1.3 Create RAG Retrieval Tool

Wrap the RAG retriever as a LangChain tool.

<code>def init_rag_tools(self):
    retriever = self.rag.get_retriever()
    retriever_tool = create_retriever_tool(
        retriever=retriever,
        name="rag_search",
        description="Search relevant documents for prospectus‑type questions."
    )
    return retriever_tool
</code>

2. Prompt for Agent Logic

<code>def create_prompt():
    system_prompt = """You are a financial assistant that can query both a database and a RAG store. Use tools to answer the user question in up to three concise sentences.\n\n# SQL tool rules\n1. Generate correct SQLite queries based on the question.\n2. Limit results to at most 5 rows unless the user requests otherwise.\n3. Do not query all columns; only request columns relevant to the question.\n4. Never execute DML statements.\n5. If a query fails, rewrite it and retry.\n\n# RAG tool rules\n1. Use the retriever to fetch relevant context and answer from it.\n2. If you do not know the answer, say so.\n\n# Thought‑Action loop format\nQuestion: <user input>\nThought: ...\nAction: <tool name>\nAction Input: ...\nObservation: ...\n...\nFinal Answer: ...\n\nBegin!"""
    return system_prompt
</code>

3. Create Agent

<code>def init_agent(self):
    # Initialize RAG tool
    retriever_tool = self.init_rag_tools()
    # Initialize SQL tools
    sql_tools = self.init_sql_tool(settings.SQLDATABASE_URI)
    # System prompt
    system_prompt = self.create_prompt()
    # Create Agent
    agent_executor = create_react_agent(
        self.chat,
        tools=[retriever_tool] + sql_tools,
        state_modifier=system_prompt
    )
    return agent_executor
</code>

4. Run Agent

<code>def handle_query(self, example_query):
    events = self.agent_executor.stream(
        {"messages": [("user", example_query)]},
        stream_mode="values",
    )
    for event in events:
        event["messages"][-1].pretty_print()
    return event["messages"][-1].content
</code>

Test code in

test_framework.py

demonstrates two scenarios: a SQL‑based query about stock volume and a RAG‑based query about prospectus profit margins. Logs show the Agent first inspects the database schema, generates the appropriate SQL, and returns the answer; for the prospectus question it directly invokes the

rag_search

tool.

Summary

The Agent approach differs from traditional step‑by‑step development; the prompt encodes the rules and the LLM decides actions autonomously.

Implementation steps: create the necessary tools, craft a detailed system prompt, instantiate the Agent with tools and prompt, and stream queries through the Agent.

Appendix

Other articles in the series:

Project Overview

RAG Retrieval Module Initial Build

Agent Framework Construction

Front‑Back End Integration

Code Refactoring

ElasticSearch Enhancement for RAG

Context Re‑ranking and Compression for RAG