Competitive Agent Ensembles: Boosting Reliability in Agentic AI

Competitive Agent Ensembles

Combining multiple AI agents that have distinct biases, strengths, and weaknesses reduces the risk of sub‑optimal or faulty outputs. The ensemble acts as a “second opinion” for AI, improving resilience and output quality.

LLM Initialization

from langchain_huggingface import HuggingFacePipeline
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
from langchain_google_vertexai import ChatVertexAI
import torch

# LLM 1: Llama 3 8B Instruct (open‑source, local)
model_id = "meta-llama/Meta-Llama-3-8B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
hf_model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    load_in_4bit=True,
)
pipe = pipeline(
    "text-generation",
    model=hf_model,
    tokenizer=tokenizer,
    max_new_tokens=1024,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
)
llama3_llm = HuggingFacePipeline(pipeline=pipe)

# LLM 2: Claude 3.5 Sonnet on Vertex AI (proprietary, cloud‑based)
claude_sonnet_llm = ChatVertexAI(model_name="claude-3-5-sonnet@001", temperature=0.7)
print("LLMs Initialized: Llama 3 and Claude 3.5 Sonnet are ready to compete.")

Structured Output Models

from langchain_core.pydantic_v1 import BaseModel, Field
from typing import List

class ProductDescription(BaseModel):
    """A structured product description with a headline and body."""
    headline: str = Field(description="A catchy, attention‑grabbing headline for the product.")
    body: str = Field(description="A short paragraph (2‑3 sentences) detailing the product's benefits and features.")

class FinalEvaluation(BaseModel):
    """Evaluator output containing the winning description and a detailed critique."""
    best_description: ProductDescription = Field(description="The winning product description chosen by the judge.")
    critique: str = Field(description="Point‑by‑point explanation of why this description was selected.")
    winning_agent: str = Field(description="Name of the agent that produced the winning description.")

Graph State and Helper Functions

from langchain_core.pydantic_v1 import BaseModel, Field
from typing import TypedDict, Annotated, Dict, List
import operator, time

class GraphState(TypedDict):
    product_name: str
    product_category: str
    features: str
    competitor_results: Annotated[Dict[str, ProductDescription], operator.update]
    final_evaluation: FinalEvaluation
    performance_log: Annotated[List[str], operator.add]

def create_competitor_node(agent_name: str, llm, prompt):
    """Factory that returns a LangGraph node for a single competitor."""
    chain = prompt | llm.with_structured_output(ProductDescription)
    def competitor_node(state: GraphState):
        print(f"--- [COMPETITOR: {agent_name}] Starting generation... ---")
        start_time = time.time()
        result = chain.invoke({
            "product_name": state['product_name'],
            "product_category": state['product_category'],
            "features": state['features'],
        })
        execution_time = time.time() - start_time
        log = f"[{agent_name}] Completed in {execution_time:.2f}s."
        print(log)
        return {"competitor_results": {agent_name: result}, "performance_log": [log]}
    return competitor_node

Competitor and Judge Nodes

# Create three competitor nodes with distinct models and prompts
claude_creative_node = create_competitor_node(
    "Claude_Sonnet_Creative", claude_sonnet_llm, claude_creative_prompt)
llama3_direct_node = create_competitor_node(
    "Llama3_Direct", llama3_llm, llama3_direct_prompt)
llama3_luxury_node = create_competitor_node(
    "Llama3_Luxury", llama3_llm, llama3_luxury_prompt)

def judge_node(state: GraphState):
    """Evaluates all competitor outputs and selects the best one."""
    print("--- [JUDGE] Evaluating competing descriptions... ---")
    start_time = time.time()
    descriptions_to_evaluate = ""
    for name, desc in state['competitor_results'].items():
        descriptions_to_evaluate += (
            f"--- Option from {name} ---
"
            f"Headline: {desc.headline}
"
            f"Body: {desc.body}

"
        )
    judge_chain = judge_prompt | llm.with_structured_output(FinalEvaluation)
    evaluation = judge_chain.invoke({
        "product_name": state['product_name'],
        "descriptions_to_evaluate": descriptions_to_evaluate,
    })
    execution_time = time.time() - start_time
    log = f"[Judge] Completed evaluation in {execution_time:.2f}s."
    print(log)
    return {"final_evaluation": evaluation, "performance_log": [log]}

Workflow Assembly

from langgraph.graph import StateGraph, END

workflow = StateGraph(GraphState)
# Add competitor nodes
workflow.add_node("claude_creative", claude_creative_node)
workflow.add_node("llama3_direct", llama3_direct_node)
workflow.add_node("llama3_luxury", llama3_luxury_node)
# Add judge node
workflow.add_node("judge", judge_node)
# Parallel entry point
workflow.set_entry_point(["claude_creative", "llama3_direct", "llama3_luxury"])
# Fan‑in to judge after all competitors finish
workflow.add_edge(["claude_creative", "llama3_direct", "llama3_luxury"], "judge")
workflow.add_edge("judge", END)
app = workflow.compile()

Execution Output

============================================================
            THE COMPETING PRODUCT DESCRIPTIONS
============================================================

--- [Claude_Sonnet_Creative] ---
Headline: Your Life, Unlocked. Your Wellness, Understood.
Body: The Aura Smart Ring is more than a tracker; its your silent wellness partner. Crafted from durable titanium, it deciphers your body signals‑sleep, activity, and heart rate‑translating them into insights that empower your every day. With a 7‑day battery, its always on, always learning, always you.

--- [Llama3_Direct] ---
Headline: Track Everything. Wear Nothing.
Body: Meet the Aura Smart Ring. Get elite sleep and activity tracking, 24/7 heart rate monitoring, and a 7‑day battery. Built from tough titanium, it delivers powerful health insights without the bulk of a watch.

--- [Llama3_Luxury] ---
Headline: Master Your Narrative.
Body: For the discerning individual, the Aura Smart Ring is an emblem of effortlessly engineered from aerospace‑grade titanium, it provides a seamless interface to your personal biometrics. Command your well‑being with seven days of uninterrupted power and unparalleled insight.

============================================================
            THE JUDGES FINAL VERDICT
============================================================
Winning Agent: Claude_Sonnet_Creative
Winning Description:
 - Headline: Your Life, Unlocked. Your Wellness, Understood.
 - Body: The Aura Smart Ring is more than a tracker; its your silent wellness partner. Crafted from durable titanium, it deciphers your body signals‑sleep, activity, and heart rate‑translating them into insights that empower your every day. With a 7‑day battery, its always on, always learning, always you.

------------------------------------------------------------
            PERFORMANCE ANALYSIS
------------------------------------------------------------
Total Execution Time: 16.24 seconds

Analysis of Advantages

Higher quality through diversity + evaluation : The three agents produce markedly different outputs (Claude Creative, Llama3 Direct, Llama3 Luxury). The judge’s transparent critique shows that quality stems from the competitive‑plus‑evaluation process rather than any single model.

Higher performance via parallelism : Running all agents in parallel reduces wall‑clock time by about 63 % compared with sequential execution, achieving the same diversity with only the slowest agent’s latency.

References

[1] Building the 14 Key Pillars of Agentic AI – https://levelup.gitconnected.com/building-the-14-key-pillars-of-agentic-ai-229e50f65986

[2] Speculative execution – https://en.wikipedia.org/wiki/Speculative_execution

[3] Redundant execution – https://developer.arm.com/community/arm-community-blogs/b/embedded-and-microcontrollers-blog/posts/comparing-lock-step-redundant-execution-versus-split-lock-technologies

[4] 🤖 Agentic Parallelism: A Practical Guide – https://github.com/FareedKhan-dev/agentic-parallelism