BestHub
Sign in
Tagged articles
2 articles
Page 1 of 1
Architect
Architect
Dec 30, 2023 · Big Data

Designing a Scalable Log Collection Agent: Lessons from Vivo’s Bees‑Agent

This article details the end‑to‑end design of Vivo’s custom log‑collection agent, covering file discovery with inotify, unique file identification using inode and content hash, real‑time reading via RandomAccessFile, checkpointing, Kafka integration, offline HDFS ingestion, resource throttling, and platform‑wide management, while comparing it with open‑source alternatives.

Agent DesignBig DataKafka
0 likes · 26 min read
Designing a Scalable Log Collection Agent: Lessons from Vivo’s Bees‑Agent
dbaplus Community
dbaplus Community
Jan 11, 2021 · Databases

Why eBay Switched Its Ad Analytics from Druid to ClickHouse – A Deep Dive

eBay’s ad data platform, originally built on a custom SQL engine and later migrated to Druid, was re‑engineered to use ClickHouse, highlighting challenges such as massive data volume, atomic offline replacements, schema design, compression, and operational simplifications, and demonstrating performance and scalability gains for advertisers.

Ad AnalyticsBig DataClickHouse
0 likes · 18 min read
Why eBay Switched Its Ad Analytics from Druid to ClickHouse – A Deep Dive