From Traditional Data Science to AI Workflows: My QoderWork Skills Journey

The author recounts a shift from manual data‑science tasks to an AI‑enabled workflow by developing QoderWork Skills, which encapsulate domain knowledge, standard processes, and anti‑pitfall guides into reusable AI agents.

Why Skills?

Repeated, non‑standardized analysis tasks (e.g., user‑insight queries, AB‑experiment checks, report formatting) consume time without adding value. By turning these routines into "digital assistants," the team aims to boost efficiency and output stability.

Four‑Layer Architecture

A Skill consists of four separated layers:

Orchestration layer (SKILL.md) : defines when the Skill is triggered, the step‑by‑step workflow, and which script functions to call. It contains no large code blocks.

Parameter layer (config.yaml) : serves as a template, using placeholders such as auto or empty lists [] that are filled at runtime, rather than hard‑coded values.

Implementation layer (scripts/) : holds the concrete, deterministic logic (e.g., field auto‑detection, statistical tests, deployment scripts) ensuring consistent results.

Knowledge layer (references/) : provides progressive disclosure of detailed explanations, statistical method rationales, and auxiliary resources.

SKILL.md Design Guidelines

Only orchestrate; embed no large Python code.

Specify trigger conditions, step sequence, script delegation, and exception handling.

Limit length (ideally <200 lines) to keep focus.

Use "NON‑NEGOTIABLE" tags and explicit negative constraints to counteract LLM pattern collapse.

Provide a concise description that acts as the Skill’s soul, clearly stating WHAT and WHEN.

Config.yaml Philosophy

Configuration should be a decision template, not a filled‑in form. Placeholders like auto or [] indicate values that will be detected or supplied during execution, preserving flexibility across different tasks.

Implementation (scripts/)

Complex logic—such as statistical test selection, field mapping, or deployment steps—belongs here. For example, the AB‑experiment Skill automatically chooses Welch’s t‑test, Mann‑Whitney U, or Z‑test based on metric type, normality, and sample size, and enforces SRM checks before reporting.

Knowledge (references/)

Since SKILL.md cannot hold full methodological explanations, the references directory offers progressive disclosure. A user can request deeper rationale (e.g., why Welch’s t‑test is used) and the agent will fetch the appropriate markdown file.

Concrete Skill Examples

Follow Builders : an AI‑driven information aggregator that tracks product‑focused creators, summarises updates, and pushes daily digests.

Frontend Slides : generates zero‑dependency HTML slide decks via a visual preview flow, with strict engineering boundaries to protect data.

User‑Insight Report : automates audience segmentation, RFM layering, and PIA insight framing, outputting business‑oriented conclusions.

AB‑Experiment Analysis : standardises experiment validation, enforces SRM, auto‑selects statistical tests, applies multiple‑comparison correction, and produces actionable recommendations.

Development Tips

Write clear, structured descriptions; they act as the Skill’s trigger and purpose.

Prefer templates over free‑form code in SKILL.md; use negative constraints to guide the LLM.

Keep the parameter file minimal; let runtime dialogue fill in specifics.

Separate complex logic into scripts to guarantee deterministic behaviour.

Use progressive disclosure in references to keep SKILL.md concise while still offering depth.

Key Takeaways

Test‑driven development with realistic mock data is essential; it can dominate 70‑80% of effort.

Report formats must serve both data‑science and business audiences; raw p‑values are less useful than clear business impact statements.

Engineering thinking—layered architecture, modular scripts, and template‑driven configs—greatly improves maintainability.

Token consumption can be high; optimisation is needed for cost‑effective deployment.

AI’s ceiling is set by the quality of injected domain knowledge, not just model size.

In summary, QoderWork Skills provides a systematic way to productise data‑science expertise into AI agents, turning repetitive analysis into standardized, controllable, and reusable digital assistants.