Intelligent Freight Matching and Pricing: Building a Logistics Big Data Platform with DeepFM and LSTM Models

Introduction : Over recent years, Shiqiao Group has created an intelligent, high‑efficiency logistics platform that digitizes the entire freight ecosystem, enabling every participant—logistics firms, fleets, drivers—to improve efficiency and cut costs.

Logistics Big Data Applications : The platform aggregates massive business data from vehicle movement, ticketing, and financial settlement, providing real‑time process linkage and supporting use cases such as cargo‑driver recommendation, intelligent pricing, OCR, anti‑spam, smart客服, face recognition, and risk control.

Platform Architecture :

Top layer: Application layer (sales management, smart dispatch, cargo recommendation, image audit, etc.)

Middle layer: Algorithm platform offering rich models and algorithms.

Data layer: Data warehouse storing all business data for model training.

Bottom layer: Big‑data infrastructure (CDH/Hadoop/Spark/Impala, Doris cluster, monitoring) delivering storage and second‑level batch analytics.

Application Cases :

Vehicle in‑transit tracking via GPS devices reporting coordinates every 30 seconds.

Real‑time dispatch center aggregating order volume, active drivers, merchants, and transaction amount for business decisions.

Smart Cargo‑Driver Matching Algorithm :

Problem is framed as a click‑through‑rate (CTR) prediction where a driver’s click on a cargo listing indicates interest. The matching function is expressed as y = F(X_i, X_u, X_c) (item, user, context).

Model used: DeepFM , which combines Factorization Machines (low‑order feature interactions) with a deep neural network (high‑order interactions) to predict click probability.

Key components:

FM layer: captures pairwise feature combinations.

DNN layer: learns higher‑order interactions.

Shared embedding for both FM and DNN.

Evaluation: offline AUC, Top‑10 CTR, overall CTR, CVR, and order volume; best model is promoted to online A/B testing.

Driver Tagging System :

Tags include origin, destination, vehicle type, length, and cargo type. Cold‑start uses rule‑based tagging from driver clicks, weighted by a decay function inspired by Newton’s cooling law (older actions receive lower weight).

When sufficient data is collected, a multi‑label LSTM model predicts driver tags. The LSTM processes variable‑length behavior sequences (clicks, calls, etc.) and outputs binary predictions for each tag. Model performance is measured by precision and recall.

Freight Price Prediction :

Price is decomposed into linear cost (distance, tolls, fuel), periodic cost (weather, season), and time‑series cost (driver labor, dynamic market factors). Linear costs use linear regression; periodic costs rely on seasonal models; time‑series costs are forecast with LSTM. Rule‑based adjustments handle sudden market changes.

Evaluation metric: SMAPE, achieving ~85 % accuracy within ±10 % price deviation.

Conclusion : By leveraging a big‑data platform, the company built a DeepFM‑based cargo‑driver matching system, an LSTM‑driven driver‑tag model, and a cost‑aware freight‑price predictor, creating an efficient, data‑centric logistics ecosystem.

Q&A Highlights :

Pairwise models can be explored for driver‑cargo preference.

LSTM‑generated tags are kept separate from the CTR model to reduce complexity and improve generalization.

Cold‑start assumptions treat every click as interest; later data refines this via learning.

GPS distance, start‑time proximity, and current load affect driver click probability.

Market price fluctuations are monitored; major anomalies trigger manual model adjustments.

Thank you for attending the session.