Why AI Feels Useless to Some but Indispensable to Others: Karpathy's 4-Layer Funnel

Introduction: The AI Experience Gap

The author describes a conversation with a teacher friend who finds AI unhelpful for tasks like making PPTs, while the author is excited by new AI capabilities. This discrepancy reflects a broader divide: some people see AI as a clumsy chatbot, while others have handed complex work to agents. Andrej Karpathy's framework, updated on October 12, explains this split through a four-layer funnel of AI experience.

Karpathy's Four-Layer Funnel of AI Experience

Karpathy emphasizes these are rough estimates ("napkin math"), not survey statistics.

Layer 1: No Exposure (~6 billion, 75%)

Approximately 6 billion people, about 75% of the global population, have almost never interacted with a large language model.

Layer 2: Occasional Free Chat Users (~1-2 billion, 20%)

Roughly 1 to 2 billion people (20%) occasionally use free chat products. For them, AI acts as a better search entry or writing assistant. Karpathy notes that many non-technical friends see little use, which is understandable because much of their work time involves human communication; understanding technical progress does not guarantee direct workflow impact.

Layer 3: Professional Frontier Model Users (~20 million, 0.2%)

About 20 million people (0.2%) professionally use frontier models in domains like mathematics and coding. Examples include creating, replicating, or translating applications, and reverse-engineering programs from binaries. Karpathy observes that projects previously taking weeks or months can now be delegated to agents via prompting. He personally shifted from writing code line-by-line with occasional tab completion to relying on agents. However, this is Karpathy's observation, not a guaranteed success rate for all projects; requirement completeness, result quality, and repair time must be factored in.

Layer 4: Internal Frontier System Users (~5,000, 0.00006%)

Only about 5,000 people (0.00006%) access internal frontier systems. Karpathy describes thousands of agents collaborating continuously for weeks on large software, security, or research problems, raising concerns that human review and understanding cannot keep pace. Deployment scale and success rates of these internal systems cannot be confirmed from his description alone.

Karpathy finds it strange that these wildly different experiences — clumsy chatbots, limited productivity gains, and sudden capability cliffs — coexist at the same moment.

Three Reasons for the Divergence

1. Impact Varies with Task Ambition, Scale, and Time Span

Asking a question that needs only a paragraph answer cannot fully utilize a system's capabilities. To see a clear difference, one must assign a batch of important, large, and difficult problems. Professional users may feel a stronger shock because a whole segment of work — researching, coding, debugging, verifying — can be offloaded, saving time across multiple stages. However, larger tasks do not guarantee easier success; errors propagate in long pipelines, and vague goals can produce seemingly complete but unusable results.

2. Capability Skews Toward Verifiable, Digital, High-Value Domains

Code runs and tests pass or fail; mathematical derivations can be checked step by step. In contrast, evaluating "does this article move the reader" or "will this advice close a deal" lacks a fixed correct/incorrect signal. Karpathy attributes this gap to two mechanisms: verifiable outcomes provide clearer reward signals for reinforcement learning training, and companies prioritize investing in tasks with revenue potential. This is his mechanistic explanation; the original post does not provide experiments quantifying each factor's contribution. This framework allows both experiences to be true: math and coding users see rapid progress, while other roles get limited help. Passing tests does not equal meeting requirements, nor does it imply AI can reliably handle interpersonal coordination or aesthetic judgment.

3. Access Conditions: Free, Paid, Internal Systems

The original post explicitly lists three access tiers: free, paid, and internal systems. When comparing experiences, one must specify which version, tools, and tasks were used; merely saying "I've used AI" makes comparison impossible. Paying does not automatically grant internal-system capabilities , nor does it compensate for missing data or unclear goals. Conversely, a free tool completing a simple task does not prove complex work lacks value. These three factors interact: upgrading the model while only asking routine questions may not change the experience; switching an agent to a hard-to-verify, data-poor task can quickly hit a wall.

Differences Within the Funnel: Speed vs. New Capabilities

Commenters note that even within Layer 3, some users merely complete existing work faster, while others start doing previously impossible work. "How much faster" and "what new things become possible" should be measured separately. Two people using the same frontier agent may have completely different feelings.

Kartik offers a counterpoint: he gave his non-programmer father a Codex subscription, and the father built a directory website with payment for his business. Kartik emphasizes the importance of teaching concrete use cases to ordinary users so they can see how AI helps. This anecdote (website quality and long-term usage unobserved) reminds us that professional labels do not determine one's layer; besides model and access, having a concrete need, a demonstration, and willingness to learn can shift the experience.

A Practical Framework for Choosing AI Tasks

Based on this analysis, the author suggests starting with a task where you can clearly state what was saved after completion. Example: unify a batch of differently formatted order files into a single spreadsheet, check for duplicate records and amount discrepancies, output an exception list, and keep original files. This is more evaluable than "help me improve efficiency."

When selecting a problem, ask four questions:

Is the goal complex yet clear? The order task involves reading, organizing, and checking across multiple steps, but the deliverable is explicitly a unified spreadsheet and exception list.

Can the result be verified? Record counts, total amounts can be cross-checked; spot-check original rows; confirm exceptions have evidence.

Are tools and data accessible? The agent must read files and have spreadsheet-processing tools; a one-sentence description is insufficient.

Is failure recoverable? Run on a copy first, keep originals and intermediate results, so mistakes can be retried.

This task does not guarantee the agent succeeds on the first try. If the bottleneck is negotiating vague terms with a client, faster AI organization won't unblock the work. After the trial, record time invested, verification time, and rework time. Only by accounting for these can you know how much work was truly offloaded.

Conclusion

Finding AI not very useful does not mean you are falling behind; seeing others unable to return to old workflows does not mean they are exaggerating. Take a real task, specify the environment, examine the deliverable, and count the rework — only then can the two experiences be meaningfully compared.