Big Data 10 min read

From Three Months to Two Weeks: DataWorks Data Agent Revamps Xinfēi Technology’s Multi‑Country Data‑Warehouse Delivery

Xinfēi Technology, a fintech data‑intelligence provider, cut its multi‑country data‑warehouse rollout from three months to about two weeks by adopting Alibaba Cloud’s DataWorks Data Agent, which unified data ingestion, task development, quality configuration, lineage analysis and release verification, introduced Skills and an enterprise semantic layer, and achieved up to six‑fold efficiency gains across ODS, DWD, DWS, DQC and reporting.

DataFunTalk
DataFunTalk
DataFunTalk
From Three Months to Two Weeks: DataWorks Data Agent Revamps Xinfēi Technology’s Multi‑Country Data‑Warehouse Delivery

Business Background: Replicate, Not Copy

Xinfēi Technology, a fintech data‑intelligence service provider, expanded overseas to Southeast Asia and Latin America. Each new country has different regulations, time zones, data sources and business parameters, making a straight copy of existing data‑warehouse code impossible and driving up manual effort.

Three‑Stage Evolution

Initially the team relied on fully manual migration, taking 2–3 months per country. They then introduced a private Dify deployment to generate code via LLMs, but the bottleneck remained in post‑generation testing, validation and publishing. Recognizing that the “last mile” could not be solved by LLMs alone, they switched to DataWorks Data Agent, which brings the entire task lifecycle—generation, modification, trial run, release, lineage analysis and result checking—inside the platform.

Task Classification and Skills

Tasks were split into two paths:

Generic tasks : data ingestion, task publishing, lineage, simple DWD, DQC configuration, slow‑task analysis, full‑load merges. Implemented with prompt + model and packaged as reusable Skills .

Business‑semantic tasks : metric queries, definition exploration, data‑issue analysis, report generation, development based on business language. Implemented with prompt + enterprise semantic layer + model.

Practice 1: ODS Batch Ingestion Skill

For a new country, the first batch may involve >100 tables. Manual ODS setup required ~2 hours per table (≈200 hours total). The team wrapped table‑structure reading, type mapping, task generation, validation and publishing into a Skill. Because a single Agent call cannot handle >100 tables, they split the batch into ~10‑table chunks, reducing configuration time to about 5 minutes per batch.

Practice 2: Building an Enterprise Semantic Layer

The semantic layer addresses “does the model understand the company’s business language?” rather than generic language understanding. Real‑world examples include:

Business users ask for “golden position” (金刚位) data; the model knows it refers to a UI resource but not the specific table/field.

The term “new customer” (新客) means different things in risk (withdrawal history) vs. marketing (registration status).

The team collected raw requests from chats, tickets, docs and meeting notes to populate the knowledge base, iterating it twice over two months. They focused on three problem types: blind spots (uncovered terminology), breakpoints (Agent stalls mid‑analysis), and limits (context length, timeout).

Practice 3: Extending Governance to Existing Countries

DataWorks Data Agent was also applied to slow, invalid or orphan tasks and storage governance. Governance cost dropped sharply, and the Agent, combined with lineage and task code, helped pinpoint root causes without manual table‑by‑table tracing.

Efficiency Results: From Single‑Point Speedup to Country‑Level Acceleration

ODS ingestion: 2 h / table → 5 min / batch (≈200 h → 5 min for 100 tables).

DWD modeling: 4 h / table → 15 min / table (AI‑generated + manual review).

DWS wide‑table creation: 1–2 days → ~2 h (AI‑generated + review).

DQC coverage: <40 % → >95 % (auto‑config during task generation).

Demand delivery: 3–5 days → hours.

Issue tracing: hours → minutes.

Report development: 1–3 days → ~30 minutes.

Overall new‑country rollout: 2–3 months → ~2 weeks (≈6× faster), validated by two consecutive country launches.

Future Outlook

The team will keep expanding the enterprise knowledge base across business domains (marketing, collection, risk, operations, asset insurance) and continue classifying warehouse tasks to create more Skills. On the quality side, they plan to add scheduled inspections, code reviews and asset‑freshness mechanisms, with AI first summarizing changes and risks before human confirmation.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

Big DataEfficiencyAutomationData WarehouseDataWorksData AgentEnterprise Semantic Layer
DataFunTalk
Written by

DataFunTalk

Dedicated to sharing and discussing big data and AI technology applications, aiming to empower a million data scientists. Regularly hosts live tech talks and curates articles on big data, recommendation/search algorithms, advertising algorithms, NLP, intelligent risk control, autonomous driving, and machine learning/deep learning.

0 followers
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.