User:Ramakanth.ch/sandbox

Introduction
For the given [n,k,d] codeword, MDS(Maximum Distance Separable) is classified as one of the pivotal class in error correcting codes that meet the Singleton bound $$d=n-k+1 $$ where,

$$d$$ is the distance, that is the minimum number of positions in which any 2 codewords differ.

$$n$$ is the block length or the length of the codeword, in short it can be viewed as the number of symbols/bits in a code.

$$k$$ is the dimension of the codeword.

MDS codes are defined over a Finite field $$\mathbb{F}$$ or Galois field $$GF$$. These codes have been under study extensively due to their error correcting ability, which is discussed extensively in the applications section. The codes close to this bound (Singleton bound), but failing to fulfill it, are given minimal importance in the following discussion but these codes also have their own importance in coding theory.

MDS Codes
The following section mainly deals with some propositions of MDS codes. In short, will be dealing with the dual codes and prove if a linear code $$C$$ is MDS code then its dual $$C^{\perp}$$ is also an MDS code and few claims such as a code $$[n,n]$$ has minimum non zero weight as 1. A quick overview is also provided on a linear code with $$n-k$$ columns linearly independent in a parity check matrix is an MDS code.

For a better understanding of claim 1, consider the following example. Consider a code $$[n,n]$$. As the original message and encrypted message are different, this implies that $$ k \neq n $$. This clearly states that every codeword is different and therefore the distance $$d \geq 1 $$. Lets take an example. Assume $$C_{\mathrm{1}} $$is a code word with all zero's. As $$C_{\mathrm{1}} \neq C_{\mathrm{2}}$$ and $$ C1,C2 \in C$$, so $$C_{\mathrm{2}} $$ should have at least one bit which is nonzero.

Proposition 1
For a Linear code, q-ary $$[n,k]$$ to be an MDS code, it is necessary and sufficient that minimum non zero weight of any codeword is $$n-k+1$$.

The proof of this proposition can be found in Linear codes as the proposition follows the property of Linear codes and definition of MDS codes.

Claim 1: A code $$[n,n]$$ for any $$n$$ and $$q$$ is an MDS code if and only if minimum non zero weight is 1.

Claim 2: A code $$[n,1]$$ for any $$n$$ and $$q$$ is an MDS code if and only if minimum weight is n.

Illustration for the claim 2: Consider a Linear code $$[n,1]$$ for any $$n$$ and $$q$$. So here 1 bit message is encoded to n bits. We generate this using a cyclic code by all 1's vector. Therefore the minimum non zero weight in this case is $$n$$.

Consider a Binary Hamming code of order $$r$$. Hamming codes with order $$r \geq 2$$ have block length as $$ 2^{r}-1$$ and message length equals to $$2^{r}-r-1$$. The hamming code $$[2^{r}-1,2^{r}-r-1]$$ has minimum distance 3. Lets consider $$r=2$$ and when substituted in the equation, it results in $$(2^{2}-1,2^{2}-2-1) = (3,1) $$. Now, lets add a parity check bit by extending the codeword by one bit, resulting a codeword that has even weight by making number of 0's or 1's even. The latter code is also called as Extended Hamming Code. So now, the above equation is reduced to $$(2^{r}-1+1,2^{r}-r-1)$$. Here only the block length is changed and the message length remains the same. Now we have (4,1) code and from claim 2, we can conclude that the minimum distance is 4.

Therefore we can conclude by stating that $$(3,1)$$ hamming code and $$(4,1)$$ extended hamming codes are both MDS.

Proposition 2
A q-ary $$[n,k]$$ linear code is an MDS code precisely if the parity check matrix $$H$$ has every set of $$n-k$$ columns linearly independent.

Proof: To show, the minimum nonzero weight of the code word is $$n-k$$ linearly independent columns. For this proof, will show that there are k linearly dependent columns which in turn implies there are $$n-k$$ linearly independent columns for the parity check matrix. This being one of the properties of Linear codes, the proof can be found in Linear codes.

Proposition 3
The code $$ C^{\perp}$$ dual to $$C$$ is a linear MDS code if $$C$$ itself is a linear MDS code.

Proof: Let $$C$$ be a linear code $$[n, k, n-k-1]$$ then $$C^{\perp}$$ is a dual code $$[n, n-k, d]$$ (from the definition of dual of a code). In order to prove that $$C^{\perp}$$ is MDS, we need to show $$d \leq k+1 $$ and $$d \geq k+1 $$. For some integer $$d \geq 0 $$ this follows the bound at $$d \leq k+1$$ from Singleton bound.

For the other inequality, for the sake of contradiction suppose $$c \in C^{\perp}$$ and $$ wt(c) \leq k$$. Assume that $$i \leq k$$ elements in $$c$$ are non-zero and consider $$b \in C$$. We know that product of Parity check matrix Transpose with Generator matrix results zero, i.e $$G.H^{T}=0$$. This implies $$\sum_{j=1}^k b_jc_j = 0 $$. Now project codewords of $$C$$ into $$k$$ co-ordinates which are non zero positions in $$C$$, where $$C$$ is a sub space of $$\mathbb{F}_{q}^{k}$$. In such case, there exists two vectors $$c_1$$ and $$c_2$$ which have same projections. So $$\Delta (c_1,c_2) \leq n-k $$, which disputes with minimum distance $$n-k+1$$. So $$ d \geq k+1 $$, Which concludes the proof.

Reed Solomon Code is an MDS code
If $$C$$ is an MDS code of integral dimension $$k$$ i.e. $$C \subseteq \sum^n$$, then we have $$ |C_S|=\sum^k$$ for all $$S \subseteq [n]$$ such that $$ |S| = k$$, where $$S$$ is any subset of indices and $$C_S$$ is the set of all codewords in $$C$$ that are projected over the indices in $$S$$.

Proof: First, lets consider only the special case of Reed Solomon code for time being and in later part will look into the more generalized proof.

Let us consider $$S$$ be any subset of indices of size $$k$$, $$S \subseteq [n] $$ and choose any arbitrary $$v$$ from $$ \mathbb{F}_q^k$$ such that $$ v = (v_1,v_2,....v_k) \in F_q^k$$ and fix it. Reed Solomon codes are evaluated over $$ \alpha_{1}, \alpha_{2},...., \alpha_{k} \in \mathbb{F}_q $$ points over a polynomial of $$k-1$$ degree. So, we need to prove that codeword $$ c \in RS$$. Let the polynomial be $$ P(x)=\sum_{i=0}^{k-1} P_ix^i$$, we also need to prove $$P(\alpha_i)=v_i$$ for all $$i \in S $$ for $$P(x)$$. For ease, let us assume $$S = [k]$$. We will solve this using matrix multiplication to get to the relation $$P(\alpha_i)=v_i $$

The above matrix seems to have the behavior of geometric progression in each column which suggests it to be a Vandermonde matrix and it has a full rank. This clearly suggests that there exists only a unique solution. Hence $$c \in RS$$.

Consider a $$ |C| \times n $$ matrix, where each row is a codeword in $$C$$ and the number of columns is equal to $$n$$, which is the size of block. Given in the statement, $$ C $$ is a Maximum Distance Separable code and therefore $$ d=n-k+1 $$.

Let the size of $$ S \subseteq [n]$$, be $$k$$. Consider $$ c^i \neq c^j \in C$$, Then we claim that their projections $$ c_S^i , c_S^j \in C_S $$ are different. For the sake of contradiction assume that $$\Delta (c^i,c^j) \leq d-1 $$. This is not possible as minimum distance is d for the given code $$ C$$. Given code $$ C $$ is an MDS code. From the definition of MDS code the distance between any two code words is $$d$$. If the distance is $$d-1$$ then the code is not an MDS code. This implies, every codeword is mapped to unique codeword in $$ C_S $$. This implies $$ |C_S|=|C|=|\sum|^k$$. Using this result and also the fact that $$C_S \subseteq \sum^k$$, implies that $$C_S =     \sum^k$$.

Weight Distribution of MDS codes
Let C be a $$[n,k,d]$$ MDS code. The Weight Distribution of $$C$$ is given by $$ A_i=(_i^n) \sum{_{j=0}^{i-d}}  (-1)^j \binom{i}{j} (q^{i+1-d-j} -1)$$, where $$A_i$$ is the number of codewords of hamming weight $$i$$ in the code $$C$$, for $$d \leq i \leq n$$ and $$d=n-k+1$$.

Proof: Consider a code $$ C$$ of length $$n$$. Assume that, this code is now shortened by $$ T$$ bits. So the resulting code is $$ [n-T,k-T]$$ as the number of bits reduced in code $$C$$ is by a factor of $$T$$. This code is also an MDS code, if $$ |T| < k$$ and the code is an empty or a zero code, if $$|T| \geq k $$. Lets move on and pick $$ C(T) $$ to be the subcode of $$C$$ which is zero on $$T$$. So $$ C(T) $$ can be represented as

$$ |C(T)| = \begin{cases} q^{k-t} & t = |T| < k \\ 1 & t=|T| \geq k \quad \text{---(i)} \end{cases} $$

Define $$ N_t$$ to be the number of codewords of $$ C$$ whose weights are $$ n-t $$ or less. So $$ N_t $$ is sum of all the codewords$$^{[9]}$$. $$C(T)$$ is picked as the set of codewords of C which is zero on $$T$$, so we can define$$ N_t = \sum_{|T|=t} |C(T)| $$

Lemma: $$ N_t = \sum_{|T|=t} |C(T)| $$

Proof

Let's claim that the number of elements of $$C$$ not in any of its subsets is

$$ N-N_1+N_2-N_3+....+(-1)^nN_n$$

This equation is equal to

$$1- \binom{k}{1} + \binom{k}{2} - ..... +(-1)^k \binom{k}{k}=(1-1)^k=0 $$

where $$k$$ is the dimension of the codeword.

Now getting back to the original proof and substitute this value in the equation $$(i)$$. This results in

$$ |C(T)| = \begin{cases} \binom{n}{t} q^{k-t} & t = |T| < k \\ \binom{n}{t} & t=|T| \geq k \quad \text{---(ii)} \end{cases} $$

Now by inclusion and exclusion property on the give condition $$ d \leq i \leq n $$

$$ A_i = \sum_{j=0}^i (-1)^j \binom{n-i+j}{j} N_{n-i+j} \quad \text{-(iii)} $$

Lemma

$$ \sum_{i=0}^n (-1)^i \binom{n}{i} \binom{m+n-i}{k-i} = \begin{cases} \binom{m}{k} & m \geq  k \\ 0 & m < k \geq k \quad \text{---(iv)} \end{cases} $$

Proof

To prove this using inclusion and exclusion principle lets assume that $$ E_i$$ be the set that is to be excluded from a $$n$$ set. As $$ i$$ has been chosen, now will choose $$ k-i $$ elements from the set of size $$ m+n-i $$

Now consider a set $$ Z=X \bigcup Y $$ where $$ X$$ is an $$n$$-set of white balls and $$ Y$$ is an $$m$$-set of black balls. How many $$k$$-subsets contain only black balls? The answer is the equation on the right hand side of the lemma. To proceed further will use the following expression

$$ \sum_{j=0}^{\infty} \binom{a+j}{j} x^j=(1-x)^{-a-1}$$

$$ (-1)^i \binom{n}{i}$$ is the coefficient of $$ x^i$$ in $$ (1-x)^n$$ from the above equation. Similarly the coefficient of $$x^{k-i}$$ is $$ \binom{m+n-i}{k-i}$$ in $$ (1-x)^{k-m-n-1}$$. So the left hand side of the equation $$(iv)$$ is the coefficient of $$x^k$$ in $$(1-x)^{k-m-1} $$. If $$ m \leq k-1 $$ its obviously $$0$$ and $$ \binom{m}{k}$$ if $$ m \geq k $$, hence Proved

Now from equation $$(ii)$$ we can substitute these inequalities in the above equation $$(iii)$$.

$$ A_i = \sum^{k+i-n-1}_{j=0} (-1)^j \binom{n-i+j}{j} \binom{n}{n-i+j} q^{k-(n-i+j)} + \sum_{j=k+i-n}^i (-1)^j \binom{n-i+j}{j} \binom{n}{n-i+j}$$

The sum is divided into two parts so that the final equation is deduced into a known term $$d$$. This equation can be further simplified. Lets look at the intermediate results for better understanding.

Lemma:

$$ \binom{n-i+j}{j} \binom{n}{n-i+j} = \binom{n}{i} \binom{i}{j} $$

Proof

The binomial coefficients can be represented as follows:

$$ \frac{n!}{k!(n-k)!} = \binom{n}{k} $$

Using this representation,

$$ \frac{(n-i+j)!}{j!(n-i)!} = \binom{n-i+j}{j} $$

$$ \frac{n!}{(n-i+j)!(i-j)!} = \binom{n}{n-i+j} $$

$$ \frac{(n-i+j)!}{j!(n-i)!} \frac{n!}{(n-i+j)!(i-j)!} = \binom{n}{i} \binom{i}{j} $$

Now replace $$n-k+1=d$$, the equation is further more simplified

$$ A_i = \binom{n}{i} [ \sum^{i-d}_{j=0} (-1)^j \binom{i}{j} q^{i+1-d-j} + \sum_{j=i-d+1}^i (-1)^j \binom{i}{j}] $$

Lemma:

$$ \sum_{j=i-d+1}^i (-1)^j \binom{i}{j} = - \sum_{j=0}^{i-d}(-1)^j \binom{i}{j} $$

Proof

Binomial coefficients have a property

$$ \sum_{j=0}^i(-1)^j \binom{n}{j}=0$$

now the above equation can be splited as

$$ \sum_{j=0}^{i-d}(-1)^j \binom{n}{j} + \sum_{i-d+1}^i (-1)^j \binom{n}{j} = 0 $$

Therefore $$ \sum_{j=i-d+1}^i (-1)^j \binom{i}{j} = - \sum_{j=0}^{i-d}(-1)^j \binom{i}{j} $$

So The weight distribution is given by

$$ A_i=(_i^n) \sum{_{j=0}^{i-d}} (-1)^j \binom{i}{j} (q^{i+1-d-j} -1)$$, as desired.

Applications Of MDS Codes
In the present day scenario, storage systems have amplified to an extent where fault tolerance is a major concern. It isn't astonishing to present that MDS codes are the ones which are being deployed for the fault tolerance. The computer world is moving in to a new era of cloud storage, and companies like Google, Microsoft and Amazon$$^{[6]}$$ fore runners in the cloud-world are adopting MDS codes to overcome these erasures.

MDS codes are extensively used in RAID for fault tolerance. In early 90's RAID-4 and RAID-5 were used extensively. These storage systems had only 1 disk for encoding and it contained the parity of data disks.The parity is done by the XOR operation. In early 2000, RAID-4 and RAID-5 failed to ensure protection against failures. Then RAID-6 was developed where the number of encoding disks is increased to 2. This provided more of a commercial success and desired fault tolerance was achieved.

Now lets move on and see how actually MDS codes are used to recover lost information. An MDS with $$r$$ redundant symbols is capable of recovering original message if no more than $$r$$ symbols are lost. In order to achieve this, MDS array codes are used. An array code is a 2 dimensional array, where each column has symbols in it. In the following discussion each entry or symbol is referred as an element and columns as nodes.

Suppose if a node is erased, the original information can be rebuilt using the available nodes. So the fraction of information used to retrieve the lost information from the surviving nodes is called as rebuilding ratio. Say if $$r$$ nodes are lost then the rebuilding ratio is $$1$$. For better understanding lets deal with an example.

In the above figure, four elements which are present in two nodes are encoded in to four nodes of which two nodes are used for providing redundancy. Assume that $$1^{st}$$ node $$(a,b)$$ is lost. In order to retrieve the lost information, the elements in the surviving nodes are used. For recovering $$a$$, elements $$c$$ and $$a \oplus c$$ are used. Similarly for recovering $$b$$ ,elements $$c$$ and $$c \oplus b$$ are used. This clearly shows that out of the $$6$$ surviving elements, only $$3$$ are used to recover lost information. So the rebuilding ratio is equal to $$3/6=1/2$$.

Generally in an $$(n,k)$$ MDS code, $$n$$ is the total number of nodes of which $$k$$ is the number of nodes containing information and $$r=n-k$$ nodes as redundancy.