Channel Coding

Channel Coding is an encoding process that is done to add redundancy to the data stream (block) before modulation or transmission. This redundancy is added just for one specific reason: To make the reception of the transmitted data at the receiver reliable (error free). So to sum it all up we can say that Channel Coding is done to reduce or remove the error from the received data at the receiver which gets added to the transmitted message during transmission in the channel.

Difference between Channel Coding and Source Coding:

As mentioned earlier that Channel Coding is used to make the transmitted data more reliable by adding redundancy to the actual data to be transmitted. Whereas Source Coding is done to accomplish another important aspect as far as the data transmission point of view is concerned that is Data Compression, its basic function is to reduce that size of the data to be transmitted by the use of various Encoding Algorithms. So as seen from the above given explanation that Channel Coding and Source Coding are two completely different processes in the Digital Communication System Block Diagram. Note: Always Source Coding is done prior to Channel Coding is almost every Digital Communication system.

Need for Channel Coding:

A general objective, however, is often to achieve maximum data transfer, in a minimum bandwidth while maintaining an acceptable quality of transmission. The quality of transmission, in the context of digital communications, is essentially concerned with the probability of bit error (Pe) at the receiver.

The Shannon Hartley Law (equation 1) demonstrates two basic things: Firstly it shows (quantitatively) how bandwidth (B) and signal power (S/N) may be traded in an ideal system. Secondly it gives a theoretical limit for the transmission rate of (reliable, i.e. error free) data (Rmax) from a transmitter of given power, over a channel with a given bandwidth, operating in a given noise environment. Equation 1:

Rmax = B . log2. (1 + S: N) bits/sec

Where: B = Bandwidth S:N = Signal to Noise Ratio Rmax = Maximum Information that can be transmitted So for the communication system to operate at the Optimum Level, means to send maximum amount of information in minimum possible bandwidth and least transmitter power, the Signal to Noise Ratio should be reduced considerably and made almost negligible, this can be done by the elimination of errors in transmitted data for which Channel Coding is a must.

Classification of Channel Coding:

Channel Coding can be broadly classified into 2 main categories: 1. Block Coding: In this type of channel coding the data to be transmitted is grouped into blocks and the redundancy is added separately to each data block to form a codeword which is then transmitted. 2. Convolution Coding: In this type of channel coding the data to be transmitted is split in to a bit stream and redundancy is added to each bit and is then transmitted.

These 2 categories are further classified into sub groups and finally into different techniques:

Block Coding:
This main category can be divided into two sub groups: RECC (Reverse Error Correcting Codes) FECC (Forward Error Correcting Codes) The proper classification of Channel Coding is given in the following Tree Diagram:

Tree Diagram:

So the above tree shows the classifications of the different coding techniques used under the Channel Coding Block.

RECC (Reverse Error Correcting Codes):

In this type of Error Correcting Code, the redundancy added to the actual data can only help in detection of transmission errors or errors due to the propagation through the channel of transmission. These detected errors are corrected by sending a signal on a reverse channel to the Transmitter to resend the corresponding block of data. Hence they are called Reverse Error Correction Codes. Popular Examples: Parity Check Code Checksum CRC (Cyclic Redundancy Check)

FECC (Forward Error Correcting Codes):

In this type of Error Correcting Code, the redundancy added not only helps to detect errors due to transmission or propagation through the channel but also helps to correct them at the Receiver without the need for any resending of data and eliminating the need of a Reverse channel to the Transmitter, hence making them more reliable and efficient. Since the data transmission is only one way, (Transmitter to Receiver) they are called Forward Error Correcting Codes. Popular Examples: 3 Repetition Code Hamming Code LDPC (Low Density Parity Check Codes) / Gallager Code Reed Solomon Code

Important Abbreviations: http://www.telecomvt.org/documents/acronyms-glossary-SCG.pdf

Channel Coding Techniques:

RECC (Reverse Error Correcting Code):
Parity Check: In this type of Error Correcting Code, the no. of 1s is counted and an additional parity bit is added to define the count. There are two kinds of Parity Checks: Even Parity: The additional parity bit is 0 for even no. of 1s and is 1 for odd no. of 1s. Odd Parity: The additional parity bit is 1 for even no. of 1s and is 0 for odd no. of 1s. During data transmission, the additional parity bit is sent along with the corresponding Data block and at the Receiver the received data block is matched with its parity bit, mismatch will occur due to any transmission errors then the Receiver sends a signal to the Transmitter on a Reverse Channel asking for retransmission of the corresponding data block.

Checksum: In this type of Error Correction Code, the data block is split into individual bits and each of the bits is EXORed (Modulo - 2 Addition) to get a final resultant bit which is transmitted along with the corresponding block of data. Example: Data Block = 101011, then 1 0 1 0 1 1 = 0, therefore the Checksum of the above given data block = 0 so the codeword transmitted will be 1010110 instead of 101011. At the Receiver the received data block is matched with its Checksum, if a mismatch occurs due to any transmission errors then the Receiver sends a signal to the Transmitter on a Reverse Channel asking for retransmission of the corresponding data block.

CRC (Cyclic Redundancy Check): In this type of Error Correcting Code, for each data block a short finite length binary sequence called as the CRC is generated and sent along with the corresponding Data block, this CRC is generated based on a Polynomial Long Division method in which the quotient is discarded and the remainder is considered to be the CRC. Example:
11010011101100 1011 01100011101100 1011 00111011101100 1011 00010111101100 1011 00000001101100 1011 00000000110100 1011 00000000011000 1011 00000000001110 1011 -------------00000000000101 <--<--<--<--input divisor result divisor...

<---remainder (3 bits) = CRC

So the Data block consists of 11 01 00 11 10 11 00 and the Generator Polynomial is 1011 and so by Long Division Process, the CRC obtained is 0101. Then instead of 11 01 00 11 10 11 00 the Codeword transmitted is given by 11 01 00 11 10 11 00 01 01. At the Receiver the received data block is matched with its CRC, if a mismatch occurs due to any transmission errors then the Receiver sends a signal to the Transmitter on a Reverse Channel asking for retransmission of the corresponding data block. Note: There are many different kinds of CRC based on the Polynomial function used to generate them.

FECC (Forward Error Correcting Code):

Repetition Code: In this type of Error Correcting Code, the data bits to be transmitted over the channel are repeated and then sent. This code is specified using the following specification. (n,m) Repetition Code, where n = total no. of bits (size of the codeword) and m = no. of message bits (data bits). k (redundancy) = n (codeword) m (data) Consider n = 3 & m = 1, therefore the code can be given by (3,1) Repetition Code or simply by 3 Repetition Code. So this can be described by saying that each bit to be transmitted is repeated 3 times and transmitted. Example: If 100110 is to be transmitted then 111 000 000 111 111 000 the codeword will be sent instead of the actual data, so as can be seen from the codeword that each data bit is repeated 3 times. This Code can detect errors equivalent up to k Hamming Distance (k = 2) and can correct errors up to m Hamming Distance only (m = 1). Hamming Distance:

** Red path defines a Hamming Distance of 3 whereas the Blue path defines that of 2 **

Definition of 1 Hamming Distance: Its the distance from one point in the lattice to another point in the lattice allowing only one of three bits to change.

Error Correcting Property:

Consider 10110 is transferred over a channel in the form of 111 000 111 111 000, suppose at the Receiver the codeword is received along with some errors in it. Suppose for the above given codeword, a code word corresponding to 110 000 111 101 010 is received then the receiver corrects the data all by itself by the use of a Majority Vote algorithm, in which 110 has majority of 1s so it must have been 111 at the transmitter which corresponds to 1. Similarly for 010 = 000 = 0. Note that the errors detected and corrected here are at 1 Hamming Distance. Now for Errors more than 1 Hamming Distance, like for example 1 = 111 was transmitted and received as 001, then the Receiver will detect an Error and the following assumptions will be made at the Receiver, 001 = 000 = 0, so the error will be detected but Not Corrected at the Receiver. To increase the Error correction capability, no. of repetitions should be more which will compromise on the Code Rate. Code Rate: Code Rate can be defined as the Ratio of the Actual Data Bits to the Codeword Transmitted. Lesser the Code Rate, lesser is the Efficiency of the Code. Code Rate = no. of data bits Codeword bits = m n = 1 3 = 0.33334

So the Code Rate for 3 Repetition Code = 0.333334

Advantages of Repetition Codes

Disadvantages of Repetition Codes

Can Detect & Correct errors up to 1 Can Detect but cannot Correct Errors Hamming Distance of >1 Hamming Distance Easy to Implement Inefficient due to Low Code Rate

Hamming Code: In this type of Error Correction Code, the data message bits (m) to be sent over the channel is added with an additional check bits (k) to form a codeword of (n) bits, which is then transmitted over the channel using various modulation techniques. This added redundancy at the transmitter helps in detection and correction of the errors at the receiver and to obtain the original data bits (m) from the Codeword (n), the redundancy bits (k) are removed. Hamming Code is specified in a similar manner: (n,m) Hamming Code. Generally (7,4) Hamming Code is used for small scale operations. Generation of Hamming Code: The Hamming Code (X) is generated using multiplication of two Matrices, the first matrix is that of the Message bit stream (m bits) and the second is the Generator Matrix which is nothing but the concatenation of the Identity Matrix and the Parity Check Matrix. Order of Matrices: Message Matrix(M) (1 x m) | m = 4 Generator Matrix(G) (m x n) | n = 7 & k = 3 Hamming Code Matrix(X) (1 x n) Therefore it can be written as: X = M x G

Message Matrix(M): Its nothing but the actual data bit stream to be transmitted. Parity Check Matrix: It is a k bit number having a Hamming Distance minimum of 2 arranged in any order but not repeated. Hamming Code Matrix(X): Its the multiplication of the Message Matrix(M) and Generator Matrix(G). Assuming the above given data, the Hamming Matrix is calculated as follows using Matrix Multiplication and Mod-2 Addition: (X) = [0110011]1x7, this is the Hamming Code for Data = 0110 in a (7,4) Hamming Code. Error Detection & Correction using Hamming Codes:

So at the Receiver, Codeword is received along with the Error signal, this is decoded using a Syndrome Detection & Correction Technique. Let Y (Received Codeword) = X(Codeword) + E(Error) Syndrome(S) = HT x Y = HT.X + HT.E If Syndrome(S) = 0 then there is no error but if (S) is not equal to 0 then Error exists. Where HT = Transpose of the concatenation of Parity Check Matrix / Identity Matrix.

Process at the Receiver:

At the Receiver, the Syndrome is calculated first using the relations given above and then depending on the Syndrome, most likely possible Error pattern is selected from the Look-Up Table, hence the error is detected and the Most likely error pattern is EXORed with the Received codeword to get the Corrected Data Block. Analysis: Since n = 7 and k = 3; Possible Error Patterns = 27 = 128 Possible Syndromes = 23 = 8 So: Possible Error Patterns Possible Syndromes = 128 8 = 16

Therefore 16 different Error patterns can Generate 1 Syndrome so many errors are undetected and hence uncorrected. This can be overcome by increasing the no. of check bits in the Coding, which not only helps to improve the error detection and correction capacity but also improves the Code Rate. How increasing k increases the Code Rate? Code Rate = m = nk = 2k 1 k n n 2k 1 So as k increase, Code Rate tends to be equal to 1, which makes this Code highly efficient.

Advantages of Hamming Code Very Reliable Code High Code Rate Cheap to Implement

Disadvantages of Hamming Code Complexity in Algorithms Not all Errors can be detected Look-Up Table required at Receiver

LDPC (Low Density Parity Check Code): In this kind of FECC, these codes are highly efficient since they work at almost the channel capacities of various different channels and also at the same time have a huge error detection and correction capacity. Representation of LDPC: They can be represented in two ways: 1. Matrix Method 2. Graphical Method Matrix Method Representation: To know this method lets look at the below given example of a (8,4) Parity Check Matrix. Let the dimensions of the matrix be n = 8, m = 4 respectively. [The no. of 1s in each row = Wr & the no. of 1s in each column = Wc ] For this to be a LDPC matrix, Wr << 4(m) and also Wc << 8(n).

The Graphical method consists of a Bi-partite graph where v_nodes are Variable Nodes and c_nodes are Fixed Nodes also the Points Co to C7 are variable as shown above and also f0 to f3 are fixed. Note: Wr = 2 & Wc = 4 Relation between the Matrix form and Graphical form is simple that is the Rows in the Matrix represent the points f0 to f3 and the Columns of the matrix are nothing but the points Co to C7 . All those co- ordinates in the matrix where there is a 1 are the lines joining those two corresponding nodes in the Graph. Performance & Complexity of LDPC: To improve the error detection and correction capacity the length of the data block (bit stream) has to be large which also serves the purpose of increasing the Code Rate and hence the efficiency but requires a huge bandwidth and to design such an Algorithm is very complex as well as the involving complexities at the Decoder, so as a compromise the length of the data block is always kept fixed and an appropriate length is taken for optimum performance. Example: Within 0.004dB of the Shannons Limit, with an error probability of 10-6 can be achieved by taking block length of 107 bits.

Decoding LDPC: The decoding of this code can be done using various algorithms, some of the popular algorithms are: Belief Propagation Algorithm Message Passing Algorithm Sum-Product Algorithm These are really complex, so to demonstrate the actual decoding of the LDPC, we use the Hard Decoding Technique. But first assume that the data transmitted is c = [1 0 0 1 0 1 0 1] and the received codeword c = [1 1 0 1 0 1 0 1] where we can see that due to the error in the channel, the bit at C1 got flipped from 0 to 1. Now to detect and correct the received codeword involves a series of steps based on the Algorithm used. Step1: In this step the Variable nodes send the message bits to their corresponding Fixed nodes as shown below.

Step2: Here individual responses are calculated by the Fixed node on the basis of the information sent to them by 3 Variable node and it decides the value of the fourth Variable node to satisfy the parity check matrix and sends it to that particular Variable node this is again shown in the table above. Step3: In this step the Variable node decides whether the bit it received initially was correct or not based on the majority vote of 3 inputs to it, 2 by Fixed Nodes and 1 by Input itself, as shown below if found to be incorrect it is corrected and the process for that particular Variable node is terminated.

Step4: This jumps to Step 2 and repeats steps 2 & 3 for all the given Variable nodes. Once done the entire process terminates.

Advantages of LDPC High Code Rate High Efficiency Highly Reliable Highly Compatible

Disadvantages of LDPC Very Complex to Implement Needs large Bandwidth Complex Algorithms