Design and Implementation of Redis Application Layer Protocol (RESP) and Its Python Clients

Redis uses the RESP (Redis Serialization Protocol) as its application‑layer protocol, which defines five data types—Simple Strings, Errors, Integers, Bulk Strings, and Arrays. Each type is identified by a leading character (+, -, :, $, *) and is illustrated with concrete examples such as *2\r\n$3\r\nGET\r\n$1\r\nA\r\n for a simple GET command.

The protocol was designed with pipeline support in mind; multiple commands can be concatenated and sent in a single TCP packet. The article shows raw socket interactions in Python, for example sending sock.sendall(b"*2\r\n$6\r\nEXISTS\r\n$3\r\nAAA\r\n") and receiving :0\r\n as an integer reply.

In the popular redis‑py client, the PythonParser.read_response method parses server replies. The implementation reads a line, determines the response type, and processes each type accordingly. Key excerpts include: def read_response(self): response = self._buffer.readline() if not response: raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR) byte, response = byte_to_chr(response[0]), response[1:] if byte not in ('-', '+', ':', '$', '*'): raise InvalidResponse("Protocol Error: %s, %s" % (str(byte), str(response))) if byte == '-': response = nativestr(response) error = self.parse_error(response) return error elif byte == '+': pass elif byte == ':': response = long(response) elif byte == '$': length = int(response) if length == -1: return None response = self._buffer.read(length) elif byte == '*': length = int(response) if length == -1: return None response = [self.read_response() for i in xrange(length)] if isinstance(response, bytes): response = self.encoder.decode(response) return response

The same client builds commands with Connection.pack_command , which assembles arguments into an array header and bulk strings. A simplified excerpt: def pack_command(self, *args): output = [] command = args[0] if ' ' in command: args = tuple(Token.get_token(s) for s in command.split()) + args[1:] else: args = (Token.get_token(command),) + args[1:] buff = SYM_EMPTY.join((SYM_STAR, str(len(args)).encode(), SYM_CRLF)) for arg in imap(self.encoder.encode, args): if len(buff) > buffer_cutoff or len(arg) > buffer_cutoff: buff = SYM_EMPTY.join((buff, SYM_DOLLAR, str(len(arg)).encode(), SYM_CRLF)) output.append(buff) output.append(arg) buff = SYM_CRLF else: buff = SYM_EMPTY.join((buff, SYM_DOLLAR, str(len(arg)).encode(), SYM_CRLF, arg, SYM_CRLF)) output.append(buff) return output

When using redis‑py‑cluster , the pipeline implementation discards transaction support and distributes commands across cluster nodes. The core method send_cluster_commands sorts commands, determines the slot for each, groups them by node, writes all commands, then reads responses, handling retries for errors. A representative fragment: def send_cluster_commands(self, stack, raise_on_error=True, allow_redirections=True): attempt = sorted(stack, key=lambda x: x.position) nodes = {} for c in attempt: slot = self._determine_slot(*c.args) node = self.connection_pool.get_node_by_slot(slot) node_name = node['name'] if node_name not in nodes: nodes[node_name] = NodeCommands(self.parse_response, self.connection_pool.get_connection_by_node(node)) nodes[node_name].append(c) for n in nodes.values(): n.write() for n in nodes.values(): n.read() # retry logic omitted for brevity response = [c.result for c in sorted(stack, key=lambda x: x.position)] if raise_on_error: self.raise_first_error(stack) return response

The article also looks ahead to RESP3, which adds floating‑point and big‑integer types, a dedicated NULL bulk string ( $-1\r\n ), empty array ( *-1\r\n ), and a binary‑safe error type ( !21\r\nSYNTAX invalid syntax\r\n ). These changes could enable richer statistical commands, more precise error handling, and larger string values, potentially turning Redis into a more general‑purpose cache.

In conclusion, while RESP2 is simple and effective, its limitations are being addressed by newer client implementations and the upcoming RESP3 specification, which together promise improved performance, safety, and functionality for Redis deployments.