Multi-Agent Cooperative Bidding Game Framework for Multi-Objective Optimization in Online Advertising

This paper proposes a multi-objective cooperative bidding optimization framework called Multi-Agent Cooperative Bidding Game (MACG). The framework introduces a global objective to optimize the overall interests of all advertisers, encouraging better cooperation among advertisers and indirectly protecting the interests of self-bidding advertisers, leading to fairer traffic allocation. Through theoretical analysis, the paper provides a functional form of the optimal bidding formula and designs a strategy network to explore the optimal parameters within this formula. An efficient multi-agent evolutionary strategy search algorithm is developed to find these optimal parameters. The solution has been tested both offline and through online A/B testing on the Taobao search advertising platform, showing significant improvements. The work has been published in KDD 2021.

The paper addresses the challenge of achieving collective rational bidding in a non-cooperative game environment where individual rational bidding often fails to maximize overall efficiency. It tackles issues such as advertiser collusion and the instability of existing multi-agent reinforcement learning approaches. The framework is designed to handle multiple advertiser objectives simultaneously, including maximizing clicks under cost constraints, maximizing GMV under budget constraints, and maximizing cart additions under budget constraints.

The theoretical analysis derives optimal bidding formulas for both individual advertiser objectives and global objectives. The model consists of three networks: a private network for individual advertiser objectives, a shared network for global objectives, and a fusion network to combine these objectives. The evolutionary strategy optimization approach is used to update network parameters, showing strong convergence properties even with millions of advertisers and billions of bidding data points.

Experimental results demonstrate significant improvements over baseline methods, with 5% or more increases in key metrics. The online deployment shows stable performance over 15 consecutive days, validating the effectiveness and robustness of the MACG model in real-world applications.