2023 Data Development Trends and Outlook for 2024

At the end of a busy year, the author reflects on how data development continued its rapid evolution in 2023, with new concepts and technologies emerging constantly.

2023 proved to be the year when many companies implemented large‑scale new technologies, moving from theoretical discussions to practical best‑practice implementations across organizations of all sizes.

The author shares three key observations about industry developments and future directions.

1. Data system iteration accelerates; offline computing remains stable and efficient, with future focus on stability

After more than a decade of iteration, offline computing platforms are now well‑established and highly stable across companies of all sizes. Standardized practices and surrounding capabilities have become widespread, largely thanks to Alibaba’s OneData methodology, enabling even small firms to build robust data warehouses without many missteps. Many midsize and small enterprises now directly purchase cloud‑based products that match the capabilities of leading firms, making stability the primary goal for offline computing, which remains the foundation for most business scenarios.

Because offline computing has relatively low entry barriers and a closed loop, its market competitiveness is modest; professionals seeking future growth are advised to allocate more time to emerging areas to enhance their market value.

2. Full embrace of real‑time computing; data‑lake solutions deployed at leading companies, industry standards beginning to emerge

The hottest topics in recent years are real‑time computing and lake‑warehouse architectures. Currently, there is no unified industry standard for real‑time computing, leading to a diverse landscape of architectures tailored to each company’s capabilities and business needs. While this flexibility accelerates feature delivery, it also risks future instability and costly rewrites, making skilled real‑time architects highly sought after.

Alibaba has contributed many best‑practice guidelines during the early “data development wilderness,” but companies such as ByteDance and Kuaishou are poised to set de‑facto standards due to their massive real‑time workloads and tight platform‑business collaboration. Over the next 1‑3 years, industry‑wide best practices are expected to crystallize, reshaping the market.

3. Technical layering becomes more pronounced; upward career channels narrow, traditional firms form distinct paths

Large internet firms benefit from stronger technical depth and richer business scenarios, leading to a clear divide in technology stacks between big and smaller companies. Simple solutions that work for small‑scale projects may not be favored in interviews at larger enterprises, raising the bar for job seekers.

Technical leads (TLs) must stay engaged with cutting‑edge technologies to avoid becoming detached from production realities. Even data‑governance or project‑management professionals need to maintain strong coding awareness, as their technical relevance can erode quickly.

Traditional enterprises such as state‑owned companies, banks, and energy firms develop their own data‑development standards and high industry barriers, creating unique career paths that value a blend of business understanding, development processes, and data governance over deep technical specialization.

In summary, these reflections capture the author’s view of 2023’s developments and expectations for 2024, inviting readers to follow the evolving landscape.

300万字！全网最全大数据学习面试社区等你来！

If this article helped you, don’t forget to watch , like and bookmark – the classic three‑step engagement!