Applying Deep Reinforcement Learning (DQN) to the 2048 Game: Experiments and Insights

The author revisits reinforcement learning by using the 2048 game as a benchmark, starting with random baseline evaluations and then building a DQN model with a simple dense network (reshape, dense layers) and specific hyperparameters (memory size 1,000,000, learning rate 0.001, gamma 1, e‑greedy decay to 0.1).

Initial experiments on the 4×4 board showed limited performance (max tile 256, avg score ~700, max score ~2k). To better understand algorithmic limits, the author applied classical RL methods—value iteration, policy iteration, Monte Carlo, and Q‑learning—on a reduced 2×2 board, where they converged quickly.

Further DQN trials on the 2×2 board revealed a gap between learned and optimal policies, prompting network redesigns: replacing the dense network with one‑hot encoding plus CNN, embedding layers, and incorporating next‑step information inspired by AlphaGo Zero.

After fixing bugs (exploration, reward scaling) and switching to an embedding‑CNN architecture, the model achieved max tile 1024 and max score 6000 on the 4×4 board. Subsequent experiments with larger memory (6000), discrete learning‑rate decay, and adjusted e‑greedy yielded max tile 2048, max score 34k, and average score 10k.

The author also explored Monte Carlo Tree Search (MCTS), which reached 2048 with average scores above 20k using 100–200 simulations per move, highlighting the importance of monitoring loss, reward design, and exploration in RL tasks.

Concluding remarks note that while many questions remain, the study provides a solid foundation for applying deep RL to 2048 and suggests future work on policy‑gradient methods.