Andrew Ng on Building Agentic AI Systems: Tools, MCP, and Practical Insights

Andrew Ng recently discussed with Harrison Chase the current opportunities and challenges in AI, focusing on agentic systems, modular toolkits, and emerging standards such as MCP.

Original video link: https://www.youtube.com/watch?v=4pYzYmSdSH4

1. Agentic Architecture: Task Decomposition and Workflow Orchestration

Harrison Chase: You suggest we stop debating whether an application is an "Agent" and instead focus on how "Agentic" it is. Can you elaborate?

Andrew Ng: The debate over what counts as an Agent distracts us; we should view "Agenticness" as a spectrum, from low to high autonomy, and call such systems "Agentic systems" while concentrating on building them.

He notes that many real‑world workflows are linear and simple, offering abundant opportunities for automation, but designing granular, evaluable micro‑steps remains a scarce skill.

2. From Tools to a Modular Era

Ng emphasizes that AI toolkits (LangGraph, RAG, etc.) act like Lego bricks; experienced developers can quickly assemble solutions, while novices may waste months.

He warns that rapid changes—such as increasing LLM context windows—can render past best practices obsolete, and that MCP, though still "wild", fills a critical integration gap.

He also highlights the difficulty of building robust evaluation pipelines and the need for quick, lightweight evals to catch regressions.

3. Voice Interaction and Latency

Ng points out that voice interfaces demand sub‑second response times, unlike typical Agentic workflows that tolerate several seconds, and describes techniques like pre‑responses and background audio to mask latency.

4. MCP and Early‑Stage Agent Communication

He praises MCP for standardizing data‑to‑Agent integration, reducing the n × m integration effort to n + m, but notes current implementations are immature, with authentication and token issues.

Agent‑to‑Agent communication is even earlier in its development; achieving reliable collaboration between agents from different teams remains a major challenge.

5. "Vibe Coding" and the Future of Programming

Ng observes that AI‑assisted coding reduces the need to read code line‑by‑line, yet warns that "vibe coding" can be exhausting and should not be mistaken for coding without rigor.

He argues that lowering programming barriers consistently expands the pool of developers, and that the essential skill will be clearly instructing computers to act on one’s behalf.

6. Speed and Technical Ability as Success Predictors

Ng explains that the AI Fund invests only in companies co‑founded with the fund, and that speed of execution and deep technical understanding are the primary predictors of startup success, outweighing traditional business skills.