BestHub
Sign in
Tag

DataPipeline

0 views collected around this technical thread.

Tencent Cloud Developer
Tencent Cloud Developer
Mar 1, 2023 · Big Data

We Analysis User Profiling System: Architecture and Technical Implementation

We Analysis, the official data‑analysis platform for WeChat mini‑program providers, delivers a zero‑learning‑curve user‑profiling system that combines basic tag analysis and flexible, rule‑based segmentation, using an ETL pipeline to store pre‑computed data in TDSQL and online bitmap‑optimized queries in ClickHouse with RoaringBitmap, ensuring low‑latency, stable, and comprehensive analytics.

BigDataClickHouseDataPipeline
0 likes · 20 min read
We Analysis User Profiling System: Architecture and Technical Implementation
JD Tech
JD Tech
Jan 31, 2023 · Backend Development

Architecture and Configuration Design of the Transaction Fulfillment Order Center

This article details the architecture, data model, configurable workflow, and implementation techniques of JD Tech's Transaction Fulfillment Order Center, illustrating how a modular, high‑throughput backend system supports billions of annual orders across multiple business scenarios through configurable templates, expression engines, and MQ‑driven data flows.

BackendConfigurationDataPipeline
0 likes · 10 min read
Architecture and Configuration Design of the Transaction Fulfillment Order Center
Top Architect
Top Architect
Jan 7, 2023 · Big Data

Real‑time Data Processing with ElasticSearch, Kibana and Logstash: Installation, CRUD, Bulk Import, and Data Transformation

This tutorial walks through building a real‑time data processing pipeline using ElasticSearch, Kibana and Logstash, covering core concepts such as data volume, velocity, variety and accuracy, detailed installation steps, CRUD operations, bulk data import, Java‑based data conversion, and Logstash pipeline configuration with filters and date parsing.

BigDataBulkImportDataPipeline
0 likes · 31 min read
Real‑time Data Processing with ElasticSearch, Kibana and Logstash: Installation, CRUD, Bulk Import, and Data Transformation