Prevent Remote Code Execution: Securing Java Serialization with a Simple Class Whitelist

Background

Java serialization converts data structures into a byte stream for storage or transmission, while deserialization reconstructs the original objects. Since at least 2010, security flaws in this process have been known, and recent discoveries show that many Java environments still contain exploitable serialization bugs.

Problem

These vulnerabilities are severe because an attacker can craft a malicious serialized object that, when deserialized, triggers remote command execution. Any program that accepts serialized data can be hijacked once it calls readObject(), similar to the impact of XXE attacks. The core difficulty is that the application often cannot determine the exact class types being deserialized before the process occurs.

Typical Attack Scenario

An attacker sends a serialized payload containing a class that, when instantiated, runs arbitrary code on the host. Because the deserialization mechanism automatically loads classes from the payload, the victim program may execute the attacker’s code without any additional code changes.

Mitigation Strategy

A practical defense is to restrict which classes may be instantiated during deserialization. By overriding ObjectInputStream 's resolveClass() method, developers can implement a whitelist (or blacklist) check: if the class being loaded is not on the approved list, the method throws a security exception and aborts the request.

This approach does not require rewriting the application’s business logic; it only adds a security layer around the deserialization process.

Implementation Example

The following steps illustrate the solution:

Identify all locations in the codebase where deserialization occurs (e.g., calls to readObject()).

Create a custom subclass of ObjectInputStream that overrides resolveClass().

Inside the overridden method, compare the incoming class name against a predefined whitelist.

If the class is not authorized, throw a SecurityException to stop processing.

Replace standard deserialization streams with the custom subclass throughout the application.

The diagram below shows a typical Java serialization flow and where the custom resolveClass() check is inserted.

Another illustration demonstrates the whitelist configuration and the exception handling when an unauthorized class is encountered.

Finally, the overridden resolveClass() method adds the security checks without modifying the underlying application code, providing a lightweight yet effective protection.

Conclusion

By enforcing a class whitelist through a custom ObjectInputStream, developers can mitigate the risk of remote code execution caused by unsafe Java deserialization. This solution requires only minimal code changes, preserves existing functionality, and can be deployed across vulnerable Java applications without extensive rewrites.