Comprehensive Guide to User Profiling: Concepts, Data Sources, Tagging System, Architecture, and Implementation

User profiling (user portrait) is a label‑based abstract model built from user attributes, preferences, habits, and behaviors, enabling concise, computable representations of real‑world users.

The profile includes five aspects: goals (understanding users), methods (formal data‑driven vs. informal textual descriptions), organization (structured vs. unstructured), standards (knowledge‑driven labeling), and verification (fact‑based, testable).

In early product stages, profiling helps product teams grasp user needs, imagine usage scenarios, and reduce complexity by focusing on a few representative personas.

Key applications in internet and e‑commerce include precise marketing, user statistics, data mining for recommendation/search/advertising, product service improvement, and industry research.

Typical data dimensions for profiling encompass demographic attributes, interest features, consumption traits, location signals, device characteristics, behavioral logs, and social data. An example from Qunar shows a multi‑dimensional data warehouse covering RFM, route information, etc.

Tagging systems are organized into leaf tags (specific user features) and parent tags (aggregated categories). Tags can be classified as basic attribute tags or behavior attribute tags, and further structured as fact tags, model tags, and prediction tags.

Tag attributes include inherent, derived, behavior, attitude, and test properties, often overlapping (e.g., zodiac as both inherent and derived).

Tag hierarchy levels: raw input layer (raw user logs), fact layer (verified user attributes), model prediction layer (statistical/ML models), marketing model prediction layer (segmentation, churn, loyalty), and business layer (presentation).

Tag taxonomy can be structured, semi‑structured, or unstructured, depending on the use case (e.g., advertising targeting vs. search keywords).

Typical profiling workflow: define direction, collect data, and build tag models. Data collection may involve batch pipelines (Sqoop to HDFS), ETL with Hive UDFs, and storage in Hive, MongoDB, Redis, or MySQL.

Technical stack often includes Hadoop HDFS for storage, Spark (batch processing, SparkSQL) and RHadoop for computation, MongoDB for real‑time user queries, and MySQL for metadata and UI data.

Implementation challenges include handling billion‑scale user data, ensuring low‑latency updates, and maintaining tag assignability despite data gaps or modeling failures.

Finally, the article encourages engagement (likes, shares) and provides visual illustrations throughout.