Hotel Search Ranking: Problem Definition, Model Construction, Feature Engineering, and Offline Evaluation

Speaker Tan Naiqiang, a graduate of Hunan University with experience in computational advertising at Alibaba and Pinyou, currently works at eLong focusing on hotel search ranking.

Ranking is a mature research area in both academia and industry; in information retrieval (IR) and online travel agency (OTA) domains, competitions have been held by Yahoo, Yandex, and Expedia.

Problem Definition : From a machine learning perspective, hotel ranking can be treated either as a ranking problem (optimizing order) or as a probability estimation problem (predicting conversion probability). Since the goal is revenue improvement, the article treats it as a probability estimation task.

Model Building : Position bias, which heavily influences conversion, is addressed using three approaches: assuming all displayed hotels are viewed, applying click models (e.g., COEC, Cascade, DBN), or directly modeling position bias within the learning algorithm. The third approach is adopted.

Features : Hotel search differs from sponsored search because the query is a geographic coordinate rather than a textual string, leading to weaker textual relevance but richer structured data and abundant user history. Features are organized into Contextual, Hotel, Query, and User categories, and include pairwise features that capture relative superiority between nearby hotels, exploiting locality to limit feature explosion.

Model : Due to the high dimensionality and discrete nature of features, a linear boosting model named AnyBoost is used instead of GBDT. AnyBoost supports multiple loss functions, regularization, columnar storage, Spark parallelism, and can assess feature usefulness without retraining.

Offline Evaluation : Offline metrics (AUC, MLE) guide model and feature selection, but AUC alone does not reflect online impact because it ignores click model characteristics. A custom offline evaluation method is developed to better align offline scores with online conversion improvements, especially considering mobile UI constraints (e.g., only three hotels displayed per screen).

Future Work : Plans include incorporating additional features to further enhance ranking performance.

Online Results : The deployed solution achieved a 4.73% increase in CVR conversion and a 4.63% uplift in overall user conversion in A/B testing.