Rabin cryptosystem

The Rabin cryptosystem is a family of public-key encryption schemes based on a trapdoor function whose security, like that of RSA, is related to the difficulty of integer factorization.

The Rabin trapdoor function has the advantage that inverting it has been mathematically proven to be as hard as factoring integers, while there is no such proof known for the RSA trapdoor function. It has the disadvantage that each output of the Rabin function can be generated by any of four possible inputs; if each output is a ciphertext, extra complexity is required on decryption to identify which of the four possible inputs was the true plaintext. Naive attempts to work around this often either enable a chosen-ciphertext attack to recover the secret key or, by encoding redundancy in the plaintext space, invalidate the proof of security relative to factoring.

Public-key encryption schemes based on the Rabin trapdoor function are used mainly for examples in textbooks. In contrast, RSA is the basis of standard public-key encryption schemes such as RSAES-PKCS1-v1_5 and RSAES-OAEP that are used widely in practice.

History
The Rabin trapdoor function was first published as part of the Rabin signature scheme in 1978 by Michael O. Rabin. The Rabin signature scheme was the first digital signature scheme where forging a signature could be proven to be as hard as factoring.

The trapdoor function was later repurposed in textbooks as an example of a public-key encryption scheme, which came to be known as the Rabin cryptosystem even though Rabin never published it as an encryption scheme.

Encryption Algorithm
Like all asymmetric cryptosystems, the Rabin system uses a key pair: a public key for encryption and a private key for decryption. The public key is published for anyone to use, while the private key remains known only to the recipient of the message.

Key generation
The keys for the Rabin cryptosystem are generated as follows:

Then $$n$$ is the public key and the pair $$(p,q)$$ is the private key.
 * 1) Choose two large distinct prime numbers $$p$$ and $$q$$ such that $$p \equiv 3 \bmod{4}$$ and $$q \equiv 3 \bmod{4}$$.
 * 2) Compute $$n = p q$$.

Encryption
A message $$M$$ can be encrypted by first converting it to a number $$m < n$$ using a reversible mapping, then computing $$c = m^2 \bmod{n}$$. The ciphertext is $$c$$.

Decryption
The message $$m$$ can be recovered from the ciphertext $$c$$ by taking its square root modulo $$n$$ as follows.

m_p &= c^{\frac{1}{4}(p+1)} \bmod{p} \\ m_q &= c^{\frac{1}{4}(q+1)} \bmod{q} \end{align}$$ r_1 &= \left( y_p \cdot p \cdot m_q + y_q \cdot q \cdot m_p \right) \bmod{n} \\ r_2 &= n - r_1 \\ r_3 &= \left( y_p \cdot p \cdot m_q - y_q \cdot q \cdot m_p \right) \bmod{n} \\ r_4 &= n - r_3 \end{align}$$
 * 1) Compute the square root of $$c$$ modulo $$p$$ and $$q$$ using these formulas:
 * $$\begin{align}
 * 1) Use the extended Euclidean algorithm to find $$y_p$$ and $$y_q$$ such that $$y_p \cdot p + y_q \cdot q = 1$$.
 * 2) Use the Chinese remainder theorem to find the four square roots of $$c$$ modulo $$n$$:
 * $$\begin{align}

One of these four values is the original plaintext $$m$$, although which of the four is the correct one cannot be determined without additional information.

Computing square roots
We can show that the formulas in step 1 above actually produce the square roots of $$c$$ as follows. For the first formula, we want to prove that $$m_p^2 \equiv c \bmod{p}$$. Since $$p \equiv 3 \bmod{4},$$ the exponent $\frac{1}{4}(p+1)$ is an integer. The proof is trivial if $$c \equiv 0 \bmod{p}$$, so we may assume that $$p$$ does not divide $$c$$. Note that $$c \equiv m^2 \bmod{pq}$$ implies that $$c \equiv m^2 \bmod{p}$$, so c is a quadratic residue modulo $$p$$. Then
 * $$m_p^2 \equiv c^{\frac{1}{2}(p+1)} \equiv c\cdot c^{\frac{1}{2}(p-1)} \equiv c \cdot 1 \mod p$$

The last step is justified by Euler's criterion.

Example
As an example, take $$p = 7$$ and $$q = 11$$, then $$n=77$$. Take $$m = 20$$ as our plaintext. The ciphertext is thus $$c = m^2 \bmod{n} = 400 \bmod{77} = 15$$.

Decryption proceeds as follows: r_1 &= (-3 \cdot 7 \cdot 9 + 2 \cdot 11 \cdot 1) \bmod{77} = 64 \\ r_2 &= 77 - 64 = 13 \\ r_3 &= (-3 \cdot 7 \cdot 9 - 2 \cdot 11 \cdot 1) \bmod{77} = \mathbf{20} \\ r_4 &= 77 - 20 = 57 \end{align}$$
 * 1) Compute $$m_p = c^{\frac{1}{4}(p+1)} \bmod{p} = 15^2 \bmod{7} = 1$$ and $$m_q = c^{\frac{1}{4}(q+1)} \bmod{q} = 15^3 \bmod{11} = 9$$.
 * 2) Use the extended Euclidean algorithm to compute $$y_p = -3$$ and $$y_q = 2$$. We can confirm that $$y_p \cdot p + y_q \cdot q =  (-3 \cdot 7) + (2 \cdot 11) = 1$$.
 * 3) Compute the four plaintext candidates:
 * $$\begin{align}

and we see that $$r_3$$ is the desired plaintext. Note that all four candidates are square roots of 15 mod 77. That is, for each candidate, $$r_i^2 \bmod{77} = 15$$, so each $$r_i$$ encrypts to the same value, 15.

Digital Signature Algorithm
The Rabin cryptosystem can be used to create and verify digital signatures. Creating a signature requires the private key $$(p,q)$$. Verifying a signature requires the public key $$n$$.

Signing
A message $$m$$ can be signed with a private key $$(p,q)$$ as follows.
 * 1) Generate a random value $$u$$.
 * 2) Use a cryptographic hash function $$H$$ to compute $$c = H(m\mathbin\Vert u)$$, where the double-bar denotes concatenation. $$c$$ should be an integer less than $$n$$.
 * 3) Find a square root of $$c$$ using the private key $$(p,q)$$. This will produce the usual four results, $$r_1, r_2, r_3, r_4$$.
 * 4) One might expect that squaring each $$r_i$$ would produce $$c$$. However, this will be true only if $$c$$ happens to be a quadratic residue modulo $$p$$ and $$q$$. To determine if this is the case, square $$r_1$$. If this does not yield $$c$$, repeat this algorithm with a new random $$u$$. The expected number of times this algorithm needs to be repeated before finding a suitable $$c$$ is 4.
 * 5) Having found a square root $$r_1$$ of $$c$$, the signature is $$(r_1,u)$$.

Verifying a signature
A signature $$(r,u)$$ for a message $$m$$ can be verified using the public key $$n$$ as follows.
 * 1) Compute $$c = H(m\mathbin\Vert u)$$.
 * 2) Compute $$r' = c^2 \bmod n$$
 * 3) The signature is valid if $$r' \mathrel{\stackrel?=} r$$ and a forgery otherwise.

Effectiveness
Decrypting produces three false results in addition to the correct one, so that the correct result must be guessed. This is the major disadvantage of the Rabin cryptosystem and one of the factors which have prevented it from finding widespread practical use.

If the plaintext is intended to represent a text message, guessing is not difficult; however, if the plaintext is intended to represent a numerical value, this issue becomes a problem that must be resolved by some kind of disambiguation scheme. It is possible to choose plaintexts with special structures, or to add padding, to eliminate this problem. A way of removing the ambiguity of inversion was suggested by Blum and Williams: the two primes used are restricted to primes congruent to 3 modulo 4 and the domain of the squaring is restricted to the set of quadratic residues. These restrictions make the squaring function into a trapdoor permutation, eliminating the ambiguity.

Efficiency
For encryption, a square modulo n must be calculated. This is more efficient than RSA, which requires the calculation of at least a cube.

For decryption, the Chinese remainder theorem is applied, along with two modular exponentiations. Here the efficiency is comparable to RSA.

Security
It has been proven that any algorithm which finds one of the possible plaintexts for every Rabin-encrypted ciphertext can be used to factor the modulus $$n$$. Thus, Rabin decryption for random plaintext is at least as hard as the integer factorization problem, something that has not been proven for RSA. It is generally believed that there is no polynomial-time algorithm for factoring, which implies that there is no efficient algorithm for decrypting a random Rabin-encrypted value without the private key $$(p,q)$$.

The Rabin cryptosystem does not provide indistinguishability against chosen plaintext attacks since the process of encryption is deterministic. An adversary, given a ciphertext and a candidate message, can easily determine whether or not the ciphertext encodes the candidate message (by simply checking whether encrypting the candidate message yields the given ciphertext).

The Rabin cryptosystem is insecure against a chosen ciphertext attack (even when challenge messages are chosen uniformly at random from the message space). By adding redundancies, for example, the repetition of the last 64 bits, the system can be made to produce a single root. This thwarts this specific chosen-ciphertext attack, since the decryption algorithm then only produces the root that the attacker already knows. If this technique is applied, the proof of the equivalence with the factorization problem fails, so it is uncertain as of 2004 if this variant is secure. The Handbook of Applied Cryptography by Menezes, Oorschot and Vanstone considers this equivalence probable, however, as long as the finding of the roots remains a two-part process (1. roots $$\bmod{p}$$ and $$\bmod{q}$$ and 2. application of the Chinese remainder theorem).